Linear Algebra

Going through Gilbert Strang’s Linear Algebra MIT course.

Lecture 1: The geometry of linear equations

Linear equations as matrices

\[ 2x - y = 0 \\ -x + 2y = 3 \\ \begin{bmatrix} 2 & -1 \\ -1 & 2 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 0 \\ 3 \end{bmatrix} \\ Ax = b \]

Row picture

• Each row in \(Ax = b\) is a line.
• Intersection of the lines is the solution.

Column picture

Instead of \[ \begin{bmatrix} 2 & -1 \\ -1 & 2 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 0 \\ 3 \end{bmatrix} \\ \] we think of \[ x \begin{bmatrix} 2 \\ -1 \end{bmatrix} + y \begin{bmatrix} -1 \\ 2 \end{bmatrix} = \begin{bmatrix} 0 \\ 3 \end{bmatrix} \]

We want to find a linear weighting of \([2, -1]^T\) and \([-1, 2]^T\) that reaches \([0, 3]^T\).

Lecture 2: Elimination with Matrices

Identity matrix

\[ I = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{bmatrix} \]

Permutation matrix

\[ P_{1,2} = \begin{bmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \\ \end{bmatrix} \] #### Rows v. columns - \(P_{1,2}A\) will swap the 1st and 2nd rows of \(A\). - \(AP_{1,2}\) will swap the 1st and 2nd columns of \(A\).

Elimination algorithm

Give an \(n \times n\) matrix:

• Take the first pivot \((1, 1)\).
• Set \((1, 2), ..., (1, n)\) to zero by subtracting the first row.
• Repeat with the next pivots \((2, 2), ..., (n, n)\)

The resulting matrix is the upper triangular matrix \(U\).

Elimination matrices

We can define the subtractions in the elimination algorithm with elimination matrices.

For example, to subtract 4 row 1s from 2, we use: \[ E_{1,2} = \begin{bmatrix} 1 & 0 & 0 \\ -4 & 1 & 0 \\ 0 & 0 & 1 \\ \end{bmatrix} \]

Thus, the elimination algorithm is repeating this step: \[ E_{2,3}(E_{1,3}(E_{1,2}A)) = U \\ EA = U \]