Linear Algebra - Matrices

Linear Algebra - Matrices#

import numpy as np

What is a matrix?#

A matrix \( A\) is a ractangular array of elements arranged in rows \(m\) and columns \(n\).

\[\begin{split} \large A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ a_{31} & a_{32} & \cdots & a_{3n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \\ \end{bmatrix} \end{split}\]

A square matrix is when the number of rows is the same of the number of columns.

Matrix operations#

All the following operations are applicable to any vector in \(\large \mathbb{R}^{m \times n}\).

Matrix addition#

The addition of two matrices \(\large M_{3\times2}\) and \(\large N_{3\times2}\) is done by the sum of their correspondent components, resulting in another matrix.

\[\begin{split} \large M+N = \begin{bmatrix} m_{11} & m_{12} \\ m_{21} & m_{22} \\ m_{31} & m_{32} \\ \end{bmatrix} + \begin{bmatrix} n_{11} & n_{12} \\ n_{21} & n_{22} \\ n_{31} & n_{32} \\ \end{bmatrix} = \begin{bmatrix} m_{11}+n_{11} & m_{12}+n_{12} \\ m_{21}+n_{21} & m_{22}+n_{22} \\ m_{31}+n_{31} & m_{32}+n_{32} \\ \end{bmatrix} \end{split}\]

For example:

\[\begin{split} \large M = \begin{bmatrix} 2 & 5 \\ 3 & 7 \\ 8 & 6 \end{bmatrix} \quad , \quad N = \begin{bmatrix} 1 & 7 \\ 5 & 6 \\ 2 & 9 \end{bmatrix} \end{split}\]
\[\begin{split} \large M + N = \begin{bmatrix} 2+1 & 5+7 \\ 3+5 & 7+6 \\ 8+2 & 6+9 \end{bmatrix} = \begin{bmatrix} 3 & 12 \\ 8 & 13 \\ 10 & 15 \end{bmatrix} \end{split}\]
M = np.array([
    [2, 5],
    [3, 7],
    [8, 6],
])
N = np.array([
    [1, 7],
    [5, 6],
    [2, 9],
])
print(M, end=" = M\n\n")
print(N, end=" = N\n\n")
print(M + N, end=" = M + N")

Properties of matrix addition#

  1. \(M + (N + P) = (M + N) + P\)

  2. \(M + 0 = 0 + M = M\)

  3. \(M + N = N + M\)

Matrix subtraction#

Similarly to addition, the subtraction of two matrices \(\large M\) and \(\large N\) is done by the subtraction of their correspondent components, resulting in another matrix.

\[\begin{split} \large M-N = \begin{bmatrix} m_{11} & m_{12} \\ m_{21} & m_{22} \\ m_{31} & m_{32} \\ \end{bmatrix} - \begin{bmatrix} n_{11} & n_{12} \\ n_{21} & n_{22} \\ n_{31} & n_{32} \\ \end{bmatrix} = \begin{bmatrix} m_{11}-n_{11} & m_{12}-n_{12} \\ m_{21}-n_{21} & m_{22}-n_{22} \\ m_{31}-n_{31} & m_{32}-n_{32} \\ \end{bmatrix} \end{split}\]

For example:

\[\begin{split} \large M = \begin{bmatrix} 2 & 5 \\ 3 & 7 \\ 8 & 6 \end{bmatrix} \quad , \quad N = \begin{bmatrix} 1 & 7 \\ 5 & 6 \\ 2 & 9 \end{bmatrix} \end{split}\]
\[\begin{split} \large M - N = \begin{bmatrix} 2-1 & 5-7 \\ 3-5 & 7-6 \\ 8-2 & 6-9 \end{bmatrix} = \begin{bmatrix} 1 & -2 \\ -2 & 1 \\ 6 & -3 \end{bmatrix} \end{split}\]
M = np.array([
    [2, 5],
    [3, 7],
    [8, 6],
])
N = np.array([
    [1, 7],
    [5, 6],
    [2, 9],
])
print(M, end=" = M\n\n")
print(N, end=" = N\n\n")
print(M - N, end=" = M - N")

Properties of matrix subtraction#

  1. \(-(-M) = M\)

  2. \(-M + M = M - M = 0\)

  3. \(M - N \neq N - M\)

Scalar multiplication#

The scalar multiplication is the elementwise multiplication by a scalar number \(\large \alpha\). The same rule can be applied to divisions.

\[\begin{split} \large \alpha M = \alpha \cdot \begin{bmatrix} m_{11} & m_{12} & \cdots & m_{1n} \\ m_{21} & m_{22} & \cdots & m_{2n} \\ m_{31} & m_{32} & \cdots & m_{3n} \\ \vdots & \vdots & \ddots & \vdots \\ m_{m1} & m_{m2} & \cdots & m_{mn} \end{bmatrix} = \begin{bmatrix} \alpha \cdot m_{11} & \alpha \cdot m_{12} & \cdots & \alpha \cdot m_{1n} \\ \alpha \cdot m_{21} & \alpha \cdot m_{22} & \cdots & \alpha \cdot m_{2n} \\ \alpha \cdot m_{31} & \alpha \cdot m_{32} & \cdots & \alpha \cdot m_{3n} \\ \vdots & \vdots & \ddots & \vdots \\ \alpha \cdot m_{m1} & \alpha \cdot m_{m2} & \cdots & \alpha \cdot m_{mn} \end{bmatrix} \end{split}\]

For example:

\[\begin{split} \large \alpha = 2 \quad , \quad M = \begin{bmatrix} 2 & 5 \\ 3 & 7 \\ 8 & 6 \end{bmatrix} \end{split}\]
\[\begin{split} \large \alpha \cdot M = \begin{bmatrix} 2 \cdot 2 & 2 \cdot 5 \\ 2 \cdot 3 & 2 \cdot 7 \\ 2 \cdot 8 & 2 \cdot 6 \end{bmatrix} = \begin{bmatrix} 4 & 10 \\ 6 & 14 \\ 16 & 12 \end{bmatrix} \end{split}\]
a = 2
M = np.array([
    [2, 5],
    [3, 7],
    [8, 6],
])
print(a, end=" = a\n\n")
print(M, end=" = M\n\n")
print(a*M, end=" = a.M")

Properties of scalar multiplication#

  1. \((\alpha \beta) M = \alpha (\beta M)\)

  2. \((\alpha + \beta) M = \alpha M + \beta M\)

  3. \(\alpha(M + N) = \alpha M + \alpha N\)

  4. \(1M = M\)

Matrix multiplication#

The multiplication of two matrices \(\large M_{m \times n}\) and \(\large N_{n \times p}\) is defined under the rule that the number of columns of the first matrix (\({m \times n}\)) must be equal to the number of rows of the second matrix (\({n \times p}\)), resulting in another matrix (\({m \times p}\)).

\[ \large [MN]_{ij}=m_{i1} \cdot n_{1j}+m_{i2} \cdot n_{2j}+ \cdots + m_{in} \cdot n_{nj}=\sum_1^n m_{in} \cdot m_{nj} \]

For example:

\[\begin{split} \large M = \begin{bmatrix} 2 & 3 & 8 \\ 5 & 7 & 6 \end{bmatrix} \quad , \quad N = \begin{bmatrix} 1 & 7 \\ 5 & 6 \\ 2 & 9 \end{bmatrix} \end{split}\]
\[\begin{split} \large M \times N = \begin{bmatrix} 2 \cdot 1 + 3 \cdot 5 + 8 \cdot 2 & 2 \cdot 7 + 3 \cdot 6 + 8 \cdot 9 \\ 5 \cdot 1 + 7 \cdot 5 + 6 \cdot 2 & 5 \cdot 7 + 7 \cdot 6 + 6 \cdot 9 \end{bmatrix} = \begin{bmatrix} 33 & 104 \\ 52 & 131 \end{bmatrix} \end{split}\]
M = np.array([
    [2, 3, 8],
    [5, 7, 6]
])
N = np.array([
    [1, 7],
    [5, 6],
    [2, 9],
])
print(M, end=" = M\n\n")
print(N, end=" = N\n\n")
print(np.dot(M, N), end=" = M x N")

Properties of matrix multiplication#

  1. \(M_{m \times n} N_{n \times m} \neq N_{n \times m} M_{m \times n}\)

  2. \((M_{m \times n} N_{n \times p}) P_{p \times q} = M_{m \times n}(N_{n \times p} P_{p \times q})\)

  3. \((M_{m \times n} + N_{m \times n}) P_{n \times p} = M_{m \times n}P_{n \times p} + N_{m \times n}P_{n \times p}\)

  4. \(P_{m \times n} (M_{n \times p} + N_{n \times p}) = P_{m \times n} M_{n \times p} + P_{m \times n} N_{n \times p}\)

  5. \((\alpha M_{m \times n}) N_{n \times p} = M_{m \times n} (\alpha N_{n \times p}) = \alpha (M_{m \times n} N_{n \times p})\)

Matrix transpose#

The transposition of a matrix is the operation which converts rows into columns and vice versa, so \(M_{m \times n}^T = M_{n \times m}\).

For example:

\[\begin{split} \large \begin{bmatrix} 1 & 7 \\ 5 & 6 \\ 2 & 9 \end{bmatrix}^T = \begin{bmatrix} 1 & 5 & 2 \\ 7 & 6 & 9 \end{bmatrix} \end{split}\]
M = np.array([
    [1, 7],
    [5, 6],
    [2, 9],
])
print(M, end=" = M\n\n")
print(M.T, end=" = MT")

Properties of matrix transpose#

  1. \((M + N)^T = M^T + N^T\)

  2. \((\alpha M)^T = \alpha M^T\)

  3. \((M^T)^T = M\)

  4. \((M_{m \times n} N_{n \times p})^T = N_{n \times p}^T M_{m \times n}^T\)

Matrix determinant#

The determinant of a square matrix (\(\det(M)\) or \(|M|\)) is an operation which encodes certain properties of the linear transformation described by the matrix, resulting in a real number.

\[\begin{split} \large \det (M_{2 \times 2}) = \det \begin{bmatrix} m_{11} & m_{12} \\ m_{21} & m_{22} \end{bmatrix} = m_{11} m_{22} - m_{12} m_{21} \end{split}\]

The standard method for a \(3 \times 3\) matrix (or more) is based in a recursive process that gets the first row elements and multiply by the determinant of the \(2 \times 2\) matrix (or more) that is not in the elements’s row or column. Another pattern is the alternance of the operators \(+ -\).

\[\begin{split} \large \begin{aligned} |M_{3 \times 3}| & = \det \begin{bmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \end{bmatrix} \\ |M_{3 \times 3}| & = + m_{11} \cdot \det \begin{bmatrix} m_{22} & m_{23} \\ m_{32} & m_{33} \end{bmatrix} - m_{12} \cdot \det \begin{bmatrix} m_{21} & m_{23} \\ m_{31} & m_{33} \end{bmatrix} + m_{13} \cdot \det \begin{bmatrix} m_{21} & m_{22} \\ m_{31} & m_{32} \end{bmatrix} \\ |M_{3 \times 3}| & = m_{11} \cdot (m_{22} \cdot m_{33} - m_{23} \cdot m_{32}) - m_{12} \cdot (m_{21} \cdot m_{33} - m_{23} \cdot m_{31}) + m_{13} \cdot (m_{21} \cdot m_{32} - m_{22} \cdot m_{31}) \\ |M_{3 \times 3}| & = (m_{11} m_{22} m_{33} + m_{12} m_{23} m_{31} + m_{13} m_{21} m_{32}) - (m_{13} m_{22} m_{31} + m_{12} m_{21} m_{33} + m_{11} m_{23} m_{32}) \end{aligned} \end{split}\]

For example:

\[\begin{split} \large \begin{aligned} M & = \begin{bmatrix} 1 & 7 & 3 \\ 5 & 6 & 8 \\ 2 & 9 & 4 \end{bmatrix} \\ |M| & = 1 \cdot \det \begin{bmatrix} 6 & 8 \\ 9 & 4 \end{bmatrix} - 7 \cdot \det \begin{bmatrix} 5 & 8 \\ 2 & 4 \end{bmatrix} + 3 \cdot \det \begin{bmatrix} 5 & 6 \\ 2 & 9 \end{bmatrix} \\ |M| & = 1 \cdot (6 \cdot 4 - 8 \cdot 9) - 7 \cdot (5 \cdot 4 - 8 \cdot 2) + 3 \cdot (5 \cdot 9 - 6 \cdot 2) \\ |M| & = - 48 - 28 + 99 \\ |M| & = 23 \end{aligned} \end{split}\]

There is a shortcut method that facilitates the computation and consists of expanding the elements horizontally and multiply the diagonals like in the matrix \(2 \times 2\):

\[\begin{split} \large \begin{aligned} |M_{3 \times 3}| & = \det \begin{bmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \end{bmatrix} \begin{matrix} m_{11} & m_{12} \\ m_{21} & m_{22} \\ m_{31} & m_{32} \end{matrix} \\ |M_{3 \times 3}| & = (m_{11} m_{22} m_{33} + m_{12} m_{23} m_{31} + m_{13} m_{21} m_{32}) - (m_{13} m_{22} m_{31} + m_{12} m_{21} m_{33} + m_{11} m_{23} m_{32}) \end{aligned} \end{split}\]
M = np.array([
    [1, 7, 3],
    [5, 6, 8],
    [2, 9, 4],
])
print(M, end=" = M\n\n")
print(round(np.linalg.det(M)), end=" = |M|")

Properties of matrix determinant#

  1. \(|M^T| = |M|\)

  2. \(|MN| = |M| |N|\)

Identity matrix#

The identity matrix \(I_n\) is a square matrix which the elements of the main diagonal is equal to 1 and all other elements is equal to 0.

\[\begin{split} \large I_1 = \begin{bmatrix} 1 \end{bmatrix} \quad , \quad I_2 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \quad , \quad I_3 = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \quad , \quad I_n = \begin{bmatrix} 1 & 0 & \cdots & 0 \\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 1 \end{bmatrix} \end{split}\]
I2 = np.eye(2)
I3 = np.eye(3)
print(I2, end=" = I2\n\n")
print(I3, end=" = I3\n\n")

Properties of identity matrix#

  1. \(M_{m \times n} I_n = I_m M_{m \times n} = M\)

  2. \(I_n^T = I_n\)

Matrix inverse#

A square matrix \(M\) is invertible if it is non-singular and exists another square matrix \(N\) which satisfies the following conditions:

  1. \(MN = NM = I\)

  2. Non-singular matrix is when \(|M| \neq 0\)

\(N\) is inverse the inverse of \(M\) and is represented by \(M^{-1}\), what it means that \(M M^{-1} = I\).

For example:

if:

\[\begin{split} \large M = \begin{bmatrix} 2 & 1 & 0 \\ 0 & 1 & 0 \\ 1 & 2 & 1 \end{bmatrix} \end{split}\]

so:

\[\begin{split} \large M M^{-1} = I_3 \quad \Rightarrow \quad \begin{bmatrix} 2 & 1 & 0 \\ 0 & 1 & 0 \\ 1 & 2 & 1 \end{bmatrix} \begin{bmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \end{split}\]
\[\begin{split} \large \begin{bmatrix} 2 m_{11} + m_{21} & 2 m_{12} + m_{22} & 2 m_{13} + m_{23} \\ m_{21} & m_{22} & m_{23} \\ m_{11} + 2 m_{21} + m_{31} & m_{12} + 2 m_{22} + m_{32} & m_{13} + 2 m_{23} + m_{33} \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \end{split}\]

After solving this system of equations (we can solve this by using Cramer’s rule, for example), we have:

\[\begin{split} \large M^{-1} = \begin{bmatrix} \frac{1}{2} & - \frac{1}{2} & 0 \\ 0 & 1 & 0 \\ - \frac{1}{2} & - \frac{3}{2} & 1 \end{bmatrix} \end{split}\]
M = np.array([
    [2, 1, 0],
    [0, 1, 0],
    [1, 2, 1],
])
print(M, end=" = M\n\n")
print(np.linalg.inv(M), end=" = |M|")
Contents