Introduction

Singular Value Decomposition (SVD) is based on a theorem from linear algebra that says a rectangular matrix $A$ can be represented as a product of three matrices:

  1. an orthogonal matrix $U$ (i.e., $UU^T = I$)
  2. a diagonal matrix $\Sigma$ and
  3. the transpose of an orthogonal matrix $V$ (i.e., $V^TV = I$)
Formally, $$A_{mxn} = U_{mXm} \Sigma_{mxn} V^T_{nxn}$$ The columns of $U_{mxn}$ are eigenvectors of $AA^T$, the columns of $V_{nxn}$ are eigenvectors of $A^TA$ and the elements $\sigma_1, ... \sigma_r$ of the diagonal matrix $\Sigma$ are the square roots of the non-zero eigenvalues of both $AA^T$ and $A^TA$. They are the eigenvalues or singular values of $A$.

An eigenvector is a non-zero vector that satisfies the equation: $$A\vec{v} = \lambda\vec{v}$$ where $A$ is a square matrix, the scalar \lambda\ is called the eigenvalue.

Computing SVD of a 2x2 matrix

You can use MATLAB or Octave or program using LAPACK directly to compute SVD but let's see how it's done. First, here's the Octave script