Engineering Math - Matrix

  

 

 

 

SVD (Singular Value Decomposition) - Application - Orthogonal Projection

 

In this application, I will use SVD to find the principal components for a data set. Principal components means the axis (a vector) that represents the direction along which the data is most widely distributed, i.e the axis with the largest variance. In this example, I will use a 200 x 2 matrix which represents the 200 data points as plotted below.

 

 

First I assinged the D matrix to Matrix M. There is no mathmatical reason to do this. I just did this because I wanted to use the matrix M for all the matlab examples for this page.

 

Then I did SVD as follows.

 

D in the following plots represents the data set in scatter plot. You would intuitively find an axis along which the data are spread most widely. tr(V) shows two row vectors in transpose(V) vector that came from SVD. You see the red arrow is pointing the princial components, i.e the major axis. The third plot shows the transpose(V) vectors overlayed onto the data set. (Matlab  source code for this example is in < List 1 >)
The result of SVD is as follows.

M = D (data points)

Smajor

Sminor

  

200 x 2 Matrix (Data Points)

200 x 2 Matrix

------------------------

 14.6873         0

         0         0

.

.

all zeros

.

  200 x 2 Matrix

------------------------

0         0

  0    4.9745

.

.

all zeros

.

  

 

< List 1 >

 

    clear all;

     

    N = 200;

    rng(1);

    t = rand(1,N);

    rng(1);

    r = randn(1,N);

    x1 = 0.5 .* r .* cos(2*pi*t);

    x2 =1.5 .* r .* sin(2*pi*t);

     

    M = [x1; x2];

    tm = [cos(pi/6) -sin(pi/6);sin(pi/6) cos(pi/6)]

    D = (tm * M)';

    x = D(:,1);

    y = D(:,2);

    M = D;

     

    [U,S,V]=svd(M);

     

    vT = V';

    vT1 = vT(1,:);

    vT2 = vT(2,:);

     

    sigma1 = S(1,1);

    sigma2 = S(2,2);

     

    Smajor = S;

    Smajor(2,2) = 0;

    Sminor = S;

    Sminor(1,1) = 0;

     

    DrMajor = U * Smajor * V';

    DrMinor = U * Sminor * V';

     

    subplot(1,3,1);

    plot(x,y,'ko','MarkerFaceColor',[0 0 0],'MarkerSize',2);

    axis([-5 5 -5 5]);

    hold on;

    quiver(0,0,vT1(1),vT1(2),'r','MaxHeadSize',1.5,'linewidth',2);

    axis([-5 5 -5 5]);

    hold on;

    quiver(0,0,vT2(1),vT2(2),'b','MaxHeadSize',1.5,'linewidth',2);

    axis([-5 5 -5 5]);

    hold off;

    title('D');

     

    subplot(1,3,2);

    plot(DrMajor(:,1),DrMajor(:,2),'ro','MarkerFaceColor',[1 0 0],'MarkerSize',2);

    axis([-5 5 -5 5]);

    hold on;

    plot(DrMinor(:,1),DrMinor(:,2),'bo','MarkerFaceColor',[0 0 1],'MarkerSize',2);

    axis([-5 5 -5 5]);

    title('Projected D');

    hold off;

     

    subplot(1,3,3);

    plot(x,y,'ko','MarkerFaceColor',[0 0 0],'MarkerSize',2);

    axis([-5 5 -5 5]);

    hold on;

    plot(DrMajor(:,1),DrMajor(:,2),'ro','MarkerFaceColor',[1 0 0],'MarkerSize',2);

    axis([-5 5 -5 5]);

    hold on;

    plot(DrMinor(:,1),DrMinor(:,2),'bo','MarkerFaceColor',[0 0 1],'MarkerSize',2);

    axis([-5 5 -5 5]);

    title('Projected D');

    hold off;

    title('D and Projected D');;