SVD - Singular Value Decomposition Example [[1,0,1,0],[0,1,0,1]]

Find Singular Value Decomposition (SVD) of a Matrix ...
[10100101] $[[1,0,1,0],[0,1,0,1]]$

Solution:

A= $A =$

	1 $1$	0 $0$	1 $1$	0 $0$
	0 $0$	1 $1$	0 $0$	1 $1$

A⋅A′ $A * A'$

AT $A^T$

	1 $1$	0 $0$	1 $1$	0 $0$
	0 $0$	1 $1$	0 $0$	1 $1$

	1 $1$	0 $0$
	0 $0$	1 $1$
	1 $1$	0 $0$
	0 $0$	1 $1$

A×(AT) $A×(A^T)$

	1 $1$	0 $0$	1 $1$	0 $0$
	0 $0$	1 $1$	0 $0$	1 $1$

	1 $1$	0 $0$
	0 $0$	1 $1$
	1 $1$	0 $0$
	0 $0$	1 $1$

	1×1+0×0+1×1+0×0 $1×1+0×0+1×1+0×0$	1×0+0×1+1×0+0×1 $1×0+0×1+1×0+0×1$
	0×1+1×0+0×1+1×0 $0×1+1×0+0×1+1×0$	0×0+1×1+0×0+1×1 $0×0+1×1+0×0+1×1$

	1+0+1+0 $1+0+1+0$	0+0+0+0 $0+0+0+0$
	0+0+0+0 $0+0+0+0$	0+1+0+1 $0+1+0+1$

	2 $2$	0 $0$
	0 $0$	2 $2$

∴A⋅A′= $:. A * A' =$

	2 $2$	0 $0$
	0 $0$	2 $2$

Find Eigen vector for

A⋅A′ $A * A'$

|A⋅A′-λI|=0 $|A * A'-lamdaI|=0$

	(2-λ) $(2-lamda)$	0 $0$
	0 $0$	(2-λ) $(2-lamda)$

= 0

∴(2-λ)×(2-λ)-0×0=0 $:.(2-lamda) × (2-lamda) - 0 × 0=0$

∴(4-4λ+λ2)-0=0 $:.(4-4lamda+lamda^2)-0=0$

∴(λ2-4λ+4)=0 $:.(lamda^2-4lamda+4)=0$

∴(λ-2)(λ-2)=0 $:.(lamda-2)(lamda-2)=0$

∴(λ-2)=0or(λ-2)=0 $:.(lamda-2)=0 or(lamda-2)=0$

∴ $:.$ The eigenvalues of the matrix A are given by

λ=2,2 $lamda=2,2$ ,

1. Eigenvectors for

λ=2 $lamda=2$

1. Eigenvectors for

λ=2 $lamda=2$

A⋅A′-λI= $A * A'-lamdaI =$

	2	0
	0	2

2 $2$

	1	0
	0	1

	2	0
	0	2

	2	0
	0	2

	0 $0$	0 $0$
	0 $0$	0 $0$

Now, reduce this matrix
The system associated with the eigenvalue

λ=2 $lamda=2$

(A⋅A′-2I) $(A * A'-2I)$

	x1 $x_1$
	x2 $x_2$

	0 $0$	0 $0$
	0 $0$	0 $0$

	x1 $x_1$
	x2 $x_2$

	0 $0$
	0 $0$

⇒ $=>$

∴ $:.$ eigenvectors corresponding to the eigenvalue

λ=2 $lamda=2$ is

v= $v=$

	x1 $x_1$
	x2 $x_2$

Let

x1=1,x2=0 $x_1=1,x_2=0$

v1= $v_1=$

	1 $1$
	0 $0$

Let

x1=0,x2=1 $x_1=0,x_2=1$

v2= $v_2=$

	0 $0$
	1 $1$

For Eigenvector-1

(1,0) $(1,0)$ , Length L =

√12+02=1 $sqrt(1^2+0^2)=1$

So, normalizing gives

u1=(11,01)=(1,0) $u_1=(1/1,0/1)=(1,0)$

For Eigenvector-2

(0,1) $(0,1)$ , Length L =

√02+12=1 $sqrt(0^2+1^2)=1$

So, normalizing gives

u2=(01,11)=(0,1) $u_2=(0/1,1/1)=(0,1)$

A′⋅A $A' * A$

AT $A^T$

	1 $1$	0 $0$	1 $1$	0 $0$
	0 $0$	1 $1$	0 $0$	1 $1$

	1 $1$	0 $0$
	0 $0$	1 $1$
	1 $1$	0 $0$
	0 $0$	1 $1$

(AT)×A $(A^T)×A$

	1 $1$	0 $0$
	0 $0$	1 $1$
	1 $1$	0 $0$
	0 $0$	1 $1$

	1 $1$	0 $0$	1 $1$	0 $0$
	0 $0$	1 $1$	0 $0$	1 $1$

1×1+0×0 $1×1+0×0$	1×0+0×1 $1×0+0×1$	1×1+0×0 $1×1+0×0$	1×0+0×1 $1×0+0×1$
0×1+1×0 $0×1+1×0$	0×0+1×1 $0×0+1×1$	0×1+1×0 $0×1+1×0$	0×0+1×1 $0×0+1×1$
1×1+0×0 $1×1+0×0$	1×0+0×1 $1×0+0×1$	1×1+0×0 $1×1+0×0$	1×0+0×1 $1×0+0×1$
0×1+1×0 $0×1+1×0$	0×0+1×1 $0×0+1×1$	0×1+1×0 $0×1+1×0$	0×0+1×1 $0×0+1×1$

1+0 $1+0$	0+0 $0+0$	1+0 $1+0$	0+0 $0+0$
0+0 $0+0$	0+1 $0+1$	0+0 $0+0$	0+1 $0+1$
1+0 $1+0$	0+0 $0+0$	1+0 $1+0$	0+0 $0+0$
0+0 $0+0$	0+1 $0+1$	0+0 $0+0$	0+1 $0+1$

1 $1$	0 $0$	1 $1$	0 $0$
0 $0$	1 $1$	0 $0$	1 $1$
1 $1$	0 $0$	1 $1$	0 $0$
0 $0$	1 $1$	0 $0$	1 $1$

∴A′⋅A= $:. A' * A =$

1 $1$	0 $0$	1 $1$	0 $0$
0 $0$	1 $1$	0 $0$	1 $1$
1 $1$	0 $0$	1 $1$	0 $0$
0 $0$	1 $1$	0 $0$	1 $1$

Find Eigen vector for

A′⋅A $A' * A$

|A′⋅A-λI|=0 $|A' * A-lamdaI|=0$

(1-λ) $(1-lamda)$	0 $0$	1 $1$	0 $0$
0 $0$	(1-λ) $(1-lamda)$	0 $0$	1 $1$
1 $1$	0 $0$	(1-λ) $(1-lamda)$	0 $0$
0 $0$	1 $1$	0 $0$	(1-λ) $(1-lamda)$

= 0

∴(1-λ)×|(1-λ)010(1-λ)010(1-λ)|+0×|0011(1-λ)000(1-λ)|+1×|0(1-λ)110001(1-λ)|+0×|0(1-λ)010(1-λ)010|=0 $:.(1-lamda) × |[(1-lamda),0,1],[0,(1-lamda),0],[1,0,(1-lamda)]|+0 × |[0,0,1],[1,(1-lamda),0],[0,0,(1-lamda)]|+1 × |[0,(1-lamda),1],[1,0,0],[0,1,(1-lamda)]|+0 × |[0,(1-lamda),0],[1,0,(1-lamda)],[0,1,0]|=0$

∴(1-λ)×(-2λ+3λ2-λ3)+0×(0)+1×(2λ-λ2)+0×(0)=0 $:.(1-lamda) × (-2lamda+3lamda^2-lamda^3)+0 × (0)+1 × (2lamda-lamda^2)+0 × (0)=0$

∴(-2λ+5λ2-4λ3+λ4)+0+(2λ-λ2)+0=0 $:.(-2lamda+5lamda^2-4lamda^3+lamda^4)+0+(2lamda-lamda^2)+0=0$

∴(λ4-4λ3+4λ2)=0 $:.(lamda^4-4lamda^3+4lamda^2)=0$

∴λ2(λ-2)(λ-2)=0 $:.lamda^2(lamda-2)(lamda-2)=0$

∴λ2=0or(λ-2)=0or(λ-2)=0 $:.lamda^2=0 or(lamda-2)=0 or(lamda-2)=0$

∴ $:.$ The eigenvalues of the matrix A are given by

λ=0,0,2,2 $lamda=0,0,2,2$ ,

1. Eigenvectors for

λ=2 $lamda=2$

1. Eigenvectors for

λ=2 $lamda=2$

A′⋅A-λI= $A' * A-lamdaI =$

1	0	1	0
0	1	0	1
1	0	1	0
0	1	0	1

2 $2$

1	0	0	0
0	1	0	0
0	0	1	0
0	0	0	1

1	0	1	0
0	1	0	1
1	0	1	0
0	1	0	1

2	0	0	0
0	2	0	0
0	0	2	0
0	0	0	2

-1 $-1$	0 $0$	1 $1$	0 $0$
0 $0$	-1 $-1$	0 $0$	1 $1$
1 $1$	0 $0$	-1 $-1$	0 $0$
0 $0$	1 $1$	0 $0$	-1 $-1$

Now, reduce this matrix

R1←R1÷-1 $R_1 larr R_1-:-1$

1 $1$ 1=-1÷-1 $1=-1-:-1$ R1←R1÷-1 $R_1 larr R_1-:-1$	0 $0$ 0=0÷-1 $0=0-:-1$ R1←R1÷-1 $R_1 larr R_1-:-1$	-1 $-1$ -1=1÷-1 $-1=1-:-1$ R1←R1÷-1 $R_1 larr R_1-:-1$	0 $0$ 0=0÷-1 $0=0-:-1$ R1←R1÷-1 $R_1 larr R_1-:-1$
0 $0$	-1 $-1$	0 $0$	1 $1$
1 $1$	0 $0$	-1 $-1$	0 $0$
0 $0$	1 $1$	0 $0$	-1 $-1$

R3←R3-R1 $R_3 larr R_3- R_1$

1 $1$	0 $0$	-1 $-1$	0 $0$
0 $0$	-1 $-1$	0 $0$	1 $1$
0 $0$ 0=1-1 $0=1-1$ R3←R3-R1 $R_3 larr R_3- R_1$	0 $0$ 0=0-0 $0=0-0$ R3←R3-R1 $R_3 larr R_3- R_1$	0 $0$ 0=-1--1 $0=-1--1$ R3←R3-R1 $R_3 larr R_3- R_1$	0 $0$ 0=0-0 $0=0-0$ R3←R3-R1 $R_3 larr R_3- R_1$
0 $0$	1 $1$	0 $0$	-1 $-1$

R2←R2÷-1 $R_2 larr R_2-:-1$

1 $1$	0 $0$	-1 $-1$	0 $0$
0 $0$ 0=0÷-1 $0=0-:-1$ R2←R2÷-1 $R_2 larr R_2-:-1$	1 $1$ 1=-1÷-1 $1=-1-:-1$ R2←R2÷-1 $R_2 larr R_2-:-1$	0 $0$ 0=0÷-1 $0=0-:-1$ R2←R2÷-1 $R_2 larr R_2-:-1$	-1 $-1$ -1=1÷-1 $-1=1-:-1$ R2←R2÷-1 $R_2 larr R_2-:-1$
0 $0$	0 $0$	0 $0$	0 $0$
0 $0$	1 $1$	0 $0$	-1 $-1$

R4←R4-R2 $R_4 larr R_4- R_2$

1 $1$	0 $0$	-1 $-1$	0 $0$
0 $0$	1 $1$	0 $0$	-1 $-1$
0 $0$	0 $0$	0 $0$	0 $0$
0 $0$ 0=0-0 $0=0-0$ R4←R4-R2 $R_4 larr R_4- R_2$	0 $0$ 0=1-1 $0=1-1$ R4←R4-R2 $R_4 larr R_4- R_2$	0 $0$ 0=0-0 $0=0-0$ R4←R4-R2 $R_4 larr R_4- R_2$	0 $0$ 0=-1--1 $0=-1--1$ R4←R4-R2 $R_4 larr R_4- R_2$

The system associated with the eigenvalue

λ=2 $lamda=2$

(A′⋅A-2I) $(A' * A-2I)$

	x1 $x_1$
	x2 $x_2$
	x3 $x_3$
	x4 $x_4$

1 $1$	0 $0$	-1 $-1$	0 $0$
0 $0$	1 $1$	0 $0$	-1 $-1$
0 $0$	0 $0$	0 $0$	0 $0$
0 $0$	0 $0$	0 $0$	0 $0$

	x1 $x_1$
	x2 $x_2$
	x3 $x_3$
	x4 $x_4$

	0 $0$
	0 $0$
	0 $0$
	0 $0$

⇒x1-x3=0,x2-x4=0 $=>x_1-x_3=0,x_2-x_4=0$

⇒x1=x3,x2=x4 $=>x_1=x_3,x_2=x_4$

∴ $:.$ eigenvectors corresponding to the eigenvalue

λ=2 $lamda=2$ is

v= $v=$

	x3 $x_3$
	x4 $x_4$
	x3 $x_3$
	x4 $x_4$

Let

x3=1,x4=0 $x_3=1,x_4=0$

v1= $v_1=$

	1 $1$
	0 $0$
	1 $1$
	0 $0$

Let

x3=0,x4=1 $x_3=0,x_4=1$

v2= $v_2=$

	0 $0$
	1 $1$
	0 $0$
	1 $1$

3. Eigenvectors for

λ=0 $lamda=0$

3. Eigenvectors for

λ=0 $lamda=0$

A′⋅A-λI= $A' * A-lamdaI =$

1	0	1	0
0	1	0	1
1	0	1	0
0	1	0	1

0 $0$

1	0	0	0
0	1	0	0
0	0	1	0
0	0	0	1

1 $1$	0 $0$	1 $1$	0 $0$
0 $0$	1 $1$	0 $0$	1 $1$
1 $1$	0 $0$	1 $1$	0 $0$
0 $0$	1 $1$	0 $0$	1 $1$

Now, reduce this matrix

R3←R3-R1 $R_3 larr R_3- R_1$

1 $1$	0 $0$	1 $1$	0 $0$
0 $0$	1 $1$	0 $0$	1 $1$
0 $0$ 0=1-1 $0=1-1$ R3←R3-R1 $R_3 larr R_3- R_1$	0 $0$ 0=0-0 $0=0-0$ R3←R3-R1 $R_3 larr R_3- R_1$	0 $0$ 0=1-1 $0=1-1$ R3←R3-R1 $R_3 larr R_3- R_1$	0 $0$ 0=0-0 $0=0-0$ R3←R3-R1 $R_3 larr R_3- R_1$
0 $0$	1 $1$	0 $0$	1 $1$

R4←R4-R2 $R_4 larr R_4- R_2$

1 $1$	0 $0$	1 $1$	0 $0$
0 $0$	1 $1$	0 $0$	1 $1$
0 $0$	0 $0$	0 $0$	0 $0$
0 $0$ 0=0-0 $0=0-0$ R4←R4-R2 $R_4 larr R_4- R_2$	0 $0$ 0=1-1 $0=1-1$ R4←R4-R2 $R_4 larr R_4- R_2$	0 $0$ 0=0-0 $0=0-0$ R4←R4-R2 $R_4 larr R_4- R_2$	0 $0$ 0=1-1 $0=1-1$ R4←R4-R2 $R_4 larr R_4- R_2$

The system associated with the eigenvalue

λ=0 $lamda=0$

(A′⋅A-0I) $(A' * A-0I)$

	x1 $x_1$
	x2 $x_2$
	x3 $x_3$
	x4 $x_4$

1 $1$	0 $0$	1 $1$	0 $0$
0 $0$	1 $1$	0 $0$	1 $1$
0 $0$	0 $0$	0 $0$	0 $0$
0 $0$	0 $0$	0 $0$	0 $0$

	x1 $x_1$
	x2 $x_2$
	x3 $x_3$
	x4 $x_4$

	0 $0$
	0 $0$
	0 $0$
	0 $0$

⇒x1+x3=0,x2+x4=0 $=>x_1+x_3=0,x_2+x_4=0$

⇒x1=-x3,x2=-x4 $=>x_1=-x_3,x_2=-x_4$

∴ $:.$ eigenvectors corresponding to the eigenvalue

λ=0 $lamda=0$ is

v= $v=$

	-x3 $-x_3$
	-x4 $-x_4$
	x3 $x_3$
	x4 $x_4$

Let

x3=1,x4=0 $x_3=1,x_4=0$

v3= $v_3=$

	-1 $-1$
	0 $0$
	1 $1$
	0 $0$

Let

x3=0,x4=1 $x_3=0,x_4=1$

v4= $v_4=$

	0 $0$
	-1 $-1$
	0 $0$
	1 $1$

For Eigenvector-1

(1,0,1,0) $(1,0,1,0)$ , Length L =

√12+02+12+02=1.41421 $sqrt(1^2+0^2+1^2+0^2)=1.41421$

So, normalizing gives

v1=(11.41421,01.41421,11.41421,01.41421)=(0.7071,0,0.7071,0) $v_1=(1/1.41421,0/1.41421,1/1.41421,0/1.41421)=(0.7071,0,0.7071,0)$

For Eigenvector-2

(0,1,0,1) $(0,1,0,1)$ , Length L =

√02+12+02+12=1.41421 $sqrt(0^2+1^2+0^2+1^2)=1.41421$

So, normalizing gives

v2=(01.41421,11.41421,01.41421,11.41421)=(0,0.7071,0,0.7071) $v_2=(0/1.41421,1/1.41421,0/1.41421,1/1.41421)=(0,0.7071,0,0.7071)$

For Eigenvector-3

(-1,0,1,0) $(-1,0,1,0)$ , Length L =

√(-1)2+02+12+02=1.41421 $sqrt((-1)^2+0^2+1^2+0^2)=1.41421$

So, normalizing gives

v3=(-11.41421,01.41421,11.41421,01.41421)=(-0.7071,0,0.7071,0) $v_3=((-1)/1.41421,0/1.41421,1/1.41421,0/1.41421)=(-0.7071,0,0.7071,0)$

For Eigenvector-4

(0,-1,0,1) $(0,-1,0,1)$ , Length L =

√02+(-1)2+02+12=1.41421 $sqrt(0^2+(-1)^2+0^2+1^2)=1.41421$

So, normalizing gives

v4=(01.41421,-11.41421,01.41421,11.41421)=(0,-0.7071,0,0.7071) $v_4=(0/1.41421,(-1)/1.41421,0/1.41421,1/1.41421)=(0,-0.7071,0,0.7071)$

1st $1^"st"$ Solution

∴Σ= $:. Sigma =$

	$sqrt(2)$	$0$	$0$	$0$
	$0$	$sqrt(2)$	$0$	$0$

$=$

	$1.41421$	$0$	$0$	$0$
	$0$	$1.41421$	$0$	$0$

$:. U =$

$[u_1,u_2]$

$=$

	$1$	$0$
	$0$	$1$

$V$ is found using formula

$v_i=1/sigma_i A^T*u_i$

$:. V =$

$0.70711$	$0$	$-0.70711$	$0$
$0$	$0.70711$	$0$	$-0.70711$
$0.70711$	$0$	$0.70711$	$0$
$0$	$0.70711$	$0$	$0.70711$

$2^"nd"$ Solution

$:. Sigma =$

	$sqrt(2)$	$0$	$0$	$0$
	$0$	$sqrt(2)$	$0$	$0$

$=$

	$1.41421$	$0$	$0$	$0$
	$0$	$1.41421$	$0$	$0$

$:. V =$

$[v_1,v_2,v_3,v_4]$

$=$

$0.70711$	$0$	$-0.70711$	$0$
$0$	$0.70711$	$0$	$-0.70711$
$0.70711$	$0$	$0.70711$	$0$
$0$	$0.70711$	$0$	$0.70711$

$U$ is found using formula

$u_i=1/sigma_i A*v_i$

$:. U =$

	$1$	$0$
	$0$	$1$

Verify

$1^"st"$ Solution

$A = U Sigma V^T$

$U×Sigma$

	$1$	$0$
	$0$	$1$

	$1.4142$	$0$	$0$	$0$
	$0$	$1.4142$	$0$	$0$

	$1×1.4142+0×0$	$1×0+0×1.4142$	$1×0+0×0$	$1×0+0×0$
	$0×1.4142+1×0$	$0×0+1×1.4142$	$0×0+1×0$	$0×0+1×0$

	$1.4142+0$	$0+0$	$0+0$	$0+0$
	$0+0$	$0+1.4142$	$0+0$	$0+0$

	$1.4142$	$0$	$0$	$0$
	$0$	$1.4142$	$0$	$0$

$(U × Sigma)×(V^T)$

	$1.4142$	$0$	$0$	$0$
	$0$	$1.4142$	$0$	$0$

$0.7071$	$0$	$0.7071$	$0$
$0$	$0.7071$	$0$	$0.7071$
$-0.7071$	$0$	$0.7071$	$0$
$0$	$-0.7071$	$0$	$0.7071$

	$1.4142×0.7071+0×0+0×-0.7071+0×0$	$1.4142×0+0×0.7071+0×0+0×-0.7071$	$1.4142×0.7071+0×0+0×0.7071+0×0$	$1.4142×0+0×0.7071+0×0+0×0.7071$
	$0×0.7071+1.4142×0+0×-0.7071+0×0$	$0×0+1.4142×0.7071+0×0+0×-0.7071$	$0×0.7071+1.4142×0+0×0.7071+0×0$	$0×0+1.4142×0.7071+0×0+0×0.7071$

	$1+0+0+0$	$0+0+0+0$	$1+0+0+0$	$0+0+0+0$
	$0+0+0+0$	$0+1+0+0$	$0+0+0+0$	$0+1+0+0$

	$1$	$0$	$1$	$0$
	$0$	$1$	$0$	$1$

Verify

$2^"nd"$ Solution

$A = U Sigma V^T$

$U×Sigma$

	$1$	$0$
	$0$	$1$

	$1.4142$	$0$	$0$	$0$
	$0$	$1.4142$	$0$	$0$

	$1×1.4142+0×0$	$1×0+0×1.4142$	$1×0+0×0$	$1×0+0×0$
	$0×1.4142+1×0$	$0×0+1×1.4142$	$0×0+1×0$	$0×0+1×0$

	$1.4142+0$	$0+0$	$0+0$	$0+0$
	$0+0$	$0+1.4142$	$0+0$	$0+0$

	$1.4142$	$0$	$0$	$0$
	$0$	$1.4142$	$0$	$0$

$(U × Sigma)×(V^T)$

	$1.4142$	$0$	$0$	$0$
	$0$	$1.4142$	$0$	$0$

$0.7071$	$0$	$0.7071$	$0$
$0$	$0.7071$	$0$	$0.7071$
$-0.7071$	$0$	$0.7071$	$0$
$0$	$-0.7071$	$0$	$0.7071$

	$1.4142×0.7071+0×0+0×-0.7071+0×0$	$1.4142×0+0×0.7071+0×0+0×-0.7071$	$1.4142×0.7071+0×0+0×0.7071+0×0$	$1.4142×0+0×0.7071+0×0+0×0.7071$
	$0×0.7071+1.4142×0+0×-0.7071+0×0$	$0×0+1.4142×0.7071+0×0+0×-0.7071$	$0×0.7071+1.4142×0+0×0.7071+0×0$	$0×0+1.4142×0.7071+0×0+0×0.7071$

	$1+0+0+0$	$0+0+0+0$	$1+0+0+0$	$0+0+0+0$
	$0+0+0+0$	$0+1+0+0$	$0+0+0+0$	$0+1+0+0$

	$1$	$0$	$1$	$0$
	$0$	$1$	$0$	$1$

This material is intended as a summary. Use your textbook for detail explanation.
Any bug, improvement, feedback then Submit Here

2. Example [10100101][[1,0,1,0],[0,1,0,1]]

2. Example [10100101] $[[1,0,1,0],[0,1,0,1]]$