[[#Week 2]]
[[#Week 3 - matrix arithmetic]]
	[[#Bringing all this together]]
[[#Week 4 - Eigy!!]]
[[#Week 5 - multivariate calc]]
[[#Week 6 - multivariate calc 2]]
[[#Week 7 - integration and hyperbolic functions]]
[[#Week 8 - ODEs]]
[[#Week 9 - more odes]]
### Week 2
- Gaussian elimination with back substitution
	- Basically, elimination but algorithm
	- Take two equations, say $$\begin{cases} y = -2x + 10 \\ y = 4x+4 \end{cases}$$
	- Rewrite these into the form $ax + by = c$ e.g. $$\begin{eqnarray}2&x&&+&\space&y&&=&10 \\ -4&x&&+&&y&&=&4\end{eqnarray}$$
	- Now, work down row by row, adding some multiple of the top row to the current row to eliminate the leftmost term e.g. add $2 \cdot \text{equation } 1$ to equation 2 to make $$\begin{eqnarray}2x\space+\space&y&&=&10 \\ &y&&=&8\end{eqnarray}$$Rinse and repeat if more than 2 eqn.
	- Then, the back substitution is going back up the list and subbing back in the known values as you get them (the last will have a value or parameter, and then the next will have one more var etc.)
- A linear system of m equations in n unknowns is a set of equations that take the form $$\begin{eqnarray}a_{11}x_{1}+a_{12}x_{2}+...+a_{1n}x&=&b_1 \\ a_{21}x_1+a_{22}x_2+...+a_{2n}x&=&b_2 \\ ... \\ a_{m1}x_1 + a_{m2}x_2 + ...+a_{mn}x&=&b_m\end{eqnarray}$$ ....where $a_{ij}$ and $b_i$ are known values and $x_j$ are unknown with $1 \leq i \leq m$ and $1\leq j \leq n$ 
	- Three possible outcomes: 
		- one solution 
		- infinite solutions
		both **consistent systems**, and
		- no solutions - an **inconsistent system** (some contradiction somewhere)
	- The set of all possible solutions is the solution space
- When solving using gaussian elimination, there are three "elementary row operations" that you can do that will not affect the results you get (like the RI/II/III moves)
	1. multiply any row by a non-zero scalar
	2. switch any two rows
	3. add/subtract some row multiplied by a scalar to/from another (e.g. (3) becomes (3) - 2\*(2))
##### Matrices!
$$\begin{cases}2x &+& 3y &=& 5 \\ -2x &+& y &=& 3\end{cases}$$
...becomes $$\begin{pmatrix}2 & 3 \\ -2 & 1\end{pmatrix}\begin{pmatrix}x \\ y\end{pmatrix} = \begin{pmatrix}5 \\ 3 \end{pmatrix}$$
(you can get the original equations back through matrix multiplication, and then matching up the rows)
The above is often written as an augmented matrix e.g.
   $$ \left(
\begin{array}{cc|c}
  2&3&5\\
  -2&1&3
\end{array}
\right) $$
As above, now matrix'd:
$$
\left(
\begin{array}{ccc|c}
3&7&-2&0 \\
6&16&-3&-1 \\
3&9&3&3
\end{array}
\right)
$$
Apply the same operations as with the equations:
$$
R_3 \rightarrow R_3-R_1 \ ,\ R_2 \rightarrow R_2 - 2R_1
$$
$$
\left(
\begin{array}{ccc|c}
3&7&-2&0 \\
0&2&1&-1 \\
0&2&5&3
\end{array}
\right)
$$
$$R_3 \rightarrow R_3 - R_2$$
$$
\left(
\begin{array}{ccc|c}
3&7&-2&0 \\
0&2&1&-1 \\
0&0&4&4
\end{array}
\right)
$$
This is **row echelon form** (diagonal of 0's. always going in at least 1 with each down). This form is not unique
1. divide final row by four
$$R_3 \rightarrow \frac{1}{4}R_3$$$$
\left(
\begin{array}{ccc|c}
3&7&-2&0 \\
0&2&1&-1 \\
0&0&1&1
\end{array}
\right)
$$
 2. sub row 3 in to row 2, then divide by two
$$R_2 \rightarrow \frac{1}{2}(R_2 - R_3)$$
$$
\left(
\begin{array}{ccc|c}
3&7&-2&0 \\
0&1&0&-1 \\
0&0&1&1
\end{array}
\right)
$$
 3. use rows 2 and 3 to substitute in to 1
 $$ R_1 \rightarrow R_1 + 2R_2 $$
$$
\left(
\begin{array}{ccc|c}
3&7&0&2 \\
0&1&0&-1 \\
0&0&1&1
\end{array}
\right)
$$
$$R_1 \rightarrow R_1 - 7R_2$$
$$
\left(
\begin{array}{ccc|c}
3&0&0&9 \\
0&1&0&-1 \\
0&0&1&1
\end{array}
\right)
$$
$$R_1 \rightarrow \frac{1}{3}R_1$$
$$
\left(
\begin{array}{ccc|c}
1&0&0&3 \\
0&1&0&-1 \\
0&0&1&1
\end{array}
\right)
$$
That's an identity matrix, and **reduced row echelon form**
Reading off, you get $x=3,y=-1,z=1$
This is gauss-jordan elimination.
**Algorithm:**
- Work from top row to bottom row
- use the current row to eliminate entries on the left of all rows below
- if this is not possible (e.g. a 0 in the position) the current row with a row that doesn't have a zero there
**Pivots**
- Pivots are the leftmost non-zero elements in each row (when in row echelon form)
- For some system of equations in row echelon form, where the pivots are located is unique (will never change)
- If there is ever a pivot in the final column (the augment bit), then that implies that $0x_1+0x_2+\cdots\neq0$, which is a contradiction. Thus, any matrix in REF that has this property is an inconsistent system
**Reduced row echelon form** (rref)
- is row echelon form (ref)
- ...with all pivot entries 1
- ...and all entries above pivots 0
- This form **is unique** (any given system has only one rref) *(what if inconsistent - then 0?)*
**Free and basic variables**
- Take this matrix in reduced row echelon form (works fine in ref, as the pivots don't move)
$$
\left(
\begin{array}{ccccc|c}
\color{red}1&0&-2&0&-1&1 \\
0&\color{red}1&1&0&1&0 \\
0&0&0&\color{red}1&1&2
\end{array}
\right)
$$
- As there are no pivots in the third and fifth columns, $x_3$ and $x_5$ are **free variables**, and should be substituted for parameters. 
- As there are pivots in the first, second, and fourth columns, $x_1$, $x_2$, and $x_4$ are **basic variables**, and will be in terms of the parameters
If there are any free variables, there are infinitely many solutions, and if there aren't, there is a unique solution.
**Rank**
- This is the number of pivots in a matrix
- For the coefficient matrix, this is $\text{rank}(A)$, and for the augmented matrix this is $\text{rank}(A|b)$
- if $\text{rank}(A)\neq\text{rank}(A|b)$ then inconsistent, and no solutions
- if $\text{rank}(A)=\text{rank}(A|b)$ and $\text{rank}(A)=\text{number of columns/unknowns}$ then consistent with one solution
- if $\text{rank}(A)=\text{rank}(A|b)$ and $\text{rank}(A)\neq\text{number of columns/unknowns}$ then consistent with infinite solutions
**Over- and underdetermined systems**
- A system is **overdetermined** if it has more equations than unknowns, and while it can have none, one, or infinitely many solutions, they normally have none (as any other option requires the equations to "overlap")
- A system is **underdetermined** if it has fewer equations than unknowns, and can only have no solutions, or an infinite number of solutions.
### Week 3 - matrix arithmetic
- An m\*n matrix is an array of numbers with m rows and n columns
- We use capital letters to represent matrices
$$
A =
\left(
\begin{array}{cccc}
a_{11}&a_{12}&\cdots&a_{1n} \\
a_{21}&a_{22}&\cdots&a_{2n} \\
\vdots&\vdots&\ddots&\vdots \\
a_{m1}&a_{m2}&\cdots&a_{mn}
\end{array}
\right)
$$
- An **element** of a matrix in the $i$th row and $j$th column is is denoted $a_{ij}$ 
- if $m=n$ then a matrix is square
**Operations**
- Two matrices are equal iff all their elements are equal
- Addition is elementwise
- Scalar multiplication applies the scalar to all elements
- Transpose - flip the rows and columns, thus $\forall i,j \space\space a_{ij} \rightarrow a_{ji}$, denoted $A^T$
	- note $(A^T)^T=A$
- Multiplication
	- For the element in the first row and first column, take the dot product of the first row of the first matrix, and the first column of the second matrix. This means that the number of columns in the first equals the number of rows in the second
	- The output matrix is going to have the rows of the first and the columns of the second
$$
A = \left(
\begin{array}{cc}
2&3 \\
4&1 \\
\end{array}
\right), \space\space
B = \left(
\begin{array}{cc}
1&7 \\
0&2 \\
\end{array}
\right) \implies
AB=
\left(
\begin{array}{cc}
2&3 \\
4&1 \\
\end{array}
\right)
\left(
\begin{array}{cc}
1&7 \\
0&2 \\
\end{array}
\right)
$$
$$
\left(
\begin{array}{cc}
2&3 \\
4&1 \\
\end{array}
\right)
\left(
\begin{array}{cc}
1&7 \\
0&2 \\
\end{array}
\right)
=
\left(
\begin{array}{cc}
2\times1+3\times0 && 2\times7 + 3\times2 \\
4\times1+1\times0 && 4\times7 + 1\times2
\end{array}
\right)
=
\left(
\begin{array}{cc}
2 && 20 \\
4 && 30
\end{array}
\right)
$$
	- Note, that $BA \neq AB$ (matrix multiplication is **not** commutative)
$$
A = \left(\begin{array}{cc}2 & 3 \\ 4 & 1\end{array}\right), \space\space
C = \left(\begin{array}{cc}1 & 7 \\ 0 & 2 \\ 5 & 1\end{array}\right)
$$
$AC$ is undefined, but $CA$ isn't
$$
CA = \left(\begin{array}{cc}1 & 7 \\ 0 & 2 \\ 5 & 1\end{array}\right)\left(\begin{array}{cc}2 & 3 \\ 4 & 1\end{array}\right)
$$
$$
\left(\begin{array}{cc}1 & 7 \\ 0 & 2 \\ 5 & 1\end{array}\right)\left(\begin{array}{cc}2 & 3 \\ 4 & 1\end{array}\right)
=
\left(\begin{array}{cc}
1\times2+7\times4 &1\times3 + 7\times1 \\
0\times2+2\times4 &0\times3+2\times1 \\
5\times2+1\times4 & 5\times3+1\times1 \\
\end{array}\right)
=
\left(\begin{array}{cc}
30 & 10 \\
8 &2 \\
14 & 16 \\
\end{array}\right)
$$
- If $A$ is $m\times n$ and $B$ is $p\times q$, then
	- $AB$ only exists if $n=p$
	- $AB$ is $m \times q$
	- $(AB)_{ij} = \sum_{k=1}^n A_{ik}B_{kj}$
- More properties
	- Associative i.e. $(AB)C=A(BC)$
	- $(AB)^T=B^TA^T$
**special matrices**
- Zero matrix $\left(\begin{array}{cccc}0&0&\cdots&0 \\ 0&0&\cdots&0 \\ \vdots&\vdots&\ddots&\vdots \\ 0&0&\cdots&0\end{array}\right)$
- Identity matrix $I=\left(\begin{array}{cccc}1&0&0&\cdots \\ 0&1&0&\cdots \\ 0&0&1 \\ \vdots&\vdots&&\ddots\end{array}\right)$, $IA=AI=A$
	- Always square
	- diagonal of 1s, rest 0s
- Symmetric matrices $A=A^T$ e.g. $\left(\begin{array}{ccc}1&2&5 \\ 2&3&-4 \\ 5&-4&-1\end{array}\right)$ (nth row = nth column)
- anti/skew symmetric matrices $A=-A^T$ e.g. $\left(\begin{array}{ccc}0&2&5\\-2&0&-4\\-5&4&0\end{array}\right)$
- upper triangular matrix e.g. $\left(\begin{array}{ccc}1&1&1\\0&1&1\\0&0&1\end{array}\right)$ (all numbers below the diagonal are 0)
- lower triangular matrix, all numbers above the diagonal are 0
- diagonal matrices, zeros on all except the diagonal
**Matrices as transformations**
$T=\left(\begin{array}{cc}a&b\\c&d\end{array}\right)\space, T\left(\begin{array}{c} x \\ y \end{array}\right)=\left(\begin{array}{c}x' \\ y' \end{array}\right)$
This is the definition for a linear transformation - if you can write it in this matrix, then it's linear
- Rotation by $\alpha$ anticlockwise about origin
	- $T=\left(\begin{array}{cc}\cos\alpha & -\sin\alpha \\ \sin\alpha & \cos\alpha \end{array}\right)$
- Reflections
	- Reflect in y axis: $T=\left(\begin{array}{cc}-1&0\\0&1\end{array}\right)$
- Dilations
	- $x'=\lambda x$
	- $y' = \mu y$
	- $\left(\begin{array}{c}x' \\ y' \end{array}\right)=\left(\begin{array}{cc}\lambda&0 \\ 0& \mu \end{array}\right)\left(\begin{array}{c}x\\y\end{array}\right)$
	- If $\lambda=\mu$, then pure dilation
	- If $\lambda = \frac{1}{\mu}$, then squeeze (area invariant)
- Shears
	- Simple shear - square to parallelogram along x axis
	- $\left(\begin{array}{c}x' \\ y' \end{array}\right)=\left(\begin{array}{cc}1&\lambda \\ 0& 1 \end{array}\right)\left(\begin{array}{c}x\\y\end{array}\right)$
- Projections
	- ![[Pasted image 20260315135931.png]]
	- $a_1=(a\cdot \hat b)\hat b$
	- Now, if we want to project a space on to a line
	- For some line with unit vector $\hat b = \left(\begin{array}{c}\hat b_1 \\ \hat b_2\end{array}\right)$, the projection matrix is $\left(\begin{array}{cc}\hat b_1^2 & \hat b_1 \hat b_2 \\ \hat b_1 \hat b_2 & \hat b_2^2 \end{array}\right)$
	- ![[Pasted image 20260315141206.png]]
	- This gets rid of the dimension perpendicular to the line, and this can't be undone by another transformation
**Determinants**
If we take a unit square, and transform it, then the area of the unit square (or what remains of it) after transformation is the **determinant** of the transformation matrix. For a 2x2 matrix, this is $ad-bc$, and is written as $\det A$ or $\left|\begin{array}{cc}a&b\\c&d\end{array}\right|$
- magnitude returns the "area scale factor" (as above)
- sign says whether it's been flipped
If we take the determinant of a projection matrix, we get $\hat b_1^2 \times \hat b^2_2 - \hat b_1 \hat b_2 \times \hat b_1 \hat b_2 = \hat b_1^2 \hat b^2_2 - \hat b_1^2 \hat b^2_2 = 0$ 

In 3D:
- Rather than area, we work with volume
- Takes a unit cube and make a ***parallelepiped***
- The determinant of a matrix $A$, $\det A$ is equal to the volume of the parallelepiped defined by the vectors $a, b$ and $c$ where $a,b,c =$ columns 1,2 and 3 of $A$. The ("signed") volume is also equal to the triple product of $a,b,$ and $c$, which is $a\cdot (b \times c)$
- Magnitude gives volume scale factor
- Sign indicates reflection (i.e. higher dimensional shenanigans)
- You know how to calculate 3D determinants - it's the cross product shenanigans OR
	- $\left|\begin{array}{ccc}a_{11}&a_{12}&a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32}& a_{33} \end{array}\right|=a_{11}(a_{22}a_{33}-a_{23}a_{32})-a_{12}(a_{21}a_{33}-a_{31}a_{23})+a_{13}(a_{21}a_{32}-a_{31}a_{22})$

Shortcuts for 3D
- as $\det A = a\cdot(b \times c)$
	- If one column is a multiple of another, then the determinant is equal to zero
	- if we add a multiple of one column to another, the determinant doesn't change
	- if we swap two columns, the determinant swaps sign
	- if we multiply a column by a scalar, then the determinant is multiplied by that same scalar
- Each of these properties also hold for rows, as $\det A = \det A^T$
- We can expand the determinant along any row or column, adding a sign flip if you pick the middle row/column
	- $A=\left(\begin{array}{ccc}2&1&3\\-3&5&6\\0&0&2\end{array}\right)$
	- Top row: $\det A = 2\times(5\times2-0\times6)-1\times(-3\times2-0\times6)+3\times(-3\times0-0\times5)$
	- First column: $\det A=2\times(5\times2-0\times6)-(-3)\times(1\times2-0\times3)+0\times(1\times6-5\times3)$
	- Second column (middle so add -ve): $\det A =-\left[1\times(-3\times2-0\times6)-5\times(2\times2-0\times3)+0\times(2\times6-(-3)\times3)\right]$
Overall example:
$$
\left|\begin{array}{ccc}1&2&3\\3&2&2\\0&9&8\end{array}\right|
$$
$C_2 \rightarrow C_2-C_3$
$$
\left|\begin{array}{ccc}1&-1&3\\3&0&2\\0&1&8\end{array}\right|
$$
along bottom row
$$
\det = 0 - 1\times(1\times2-3\times3) + 8\times(0-(-3))
$$
$$
=-1(-7)+24
$$
$$
=31
$$
**Inverses**
- When can we find the inverse of a transformation? (and thus, a matrix)
- Can definitely do for rotations (rotate in opposite direction)
- Can also do for shear, reflection and dilation.
- **Can't** do for projection (as it removes the perpendicular axis completely)
- Thus, a matrix $T$ is invertible (that is, has some $T^{-1}$ such that $TT^{-1}=T^{-1}T=I$, where $I$ is the identity matrix) if and only if $\det T \neq 0$
	- This also means that if a matrix is invertible, it creates a bijective mapping between its input and image spaces
- If a matrix is not invertible, it is also called singular
- For a $2\times2$ matrix $A=\left(\begin{array}{cc}a&b\\c&d\end{array}\right)$, $A^{-1}={\huge \frac{1}{ad-bc}}\left(\begin{array}{cc}d&-b\\-c&a\end{array}\right)$
3x3 matrices are much more annoying to invert, and you have two options:
- Use gaussian elimination
- Use cramer's rule
**Gaussian elimination for inverting 3x3 matrices**
For the matrix $A=\left(\begin{array}{ccc}2&3&-1\\1&1&3\\1&2&-1\end{array}\right)$
$$
\left(\begin{array}{ccc|ccc}2&3&-1&1&0&0\\1&1&3&0&1&0\\1&2&-1&0&0&1\end{array}\right)
$$
​$R_1 \leftrightarrow R_2$
$$
\left(\begin{array}{ccc|ccc}1&1&3&0&1&0\\2&3&-1&1&0&0\\1&2&-1&0&0&1\end{array}\right)
$$
$R_2 \rightarrow R_2 - 2R_1, \quad R_3 \rightarrow R_3 - R_1$
$$
\left(\begin{array}{ccc|ccc}1&1&3&0&1&0\\0&1&-7&1&-2&0\\0&1&-4&0&-1&1\end{array}\right)
$$
$R_1 \rightarrow R_1 - R_3, \quad R_2 \rightarrow R_2 - R_3$
$$
\left(\begin{array}{ccc|ccc}1&0&7&0&2&-1\\0&0&-3&1&-1&-1\\0&1&-4&0&-1&1\end{array}\right)
$$
$R_2 \leftrightarrow R_3$
$$
\left(\begin{array}{ccc|ccc}1&0&7&0&2&-1\\0&1&-4&0&-1&1\\0&0&-3&1&-1&-1\end{array}\right)
$$
$R_3 \rightarrow -\frac{1}{3}R_3$
$$
\left(\begin{array}{ccc|ccc}1&0&7&0&2&-1\\0&1&-4&0&-1&1\\0&0&1&-\frac{1}{3}&\frac{1}{3}&\frac{1}{3}\end{array}\right)
$$
$R_1 \rightarrow R_1 - 7R_3, \quad R_2 \rightarrow R_2 + 4R_3$
$$
{\large \left(\begin{array}{ccc|ccc}1&0&0&\frac{7}{3}&-\frac{1}{3}&-\frac{10}{3}\\[6pt]0&1&0&-\frac{4}{3}&\frac{1}{3}&\frac{7}{3}\\[6pt]0&0&1&-\frac{1}{3}&\frac{1}{3}&\frac{1}{3}\end{array}\right)}
$$
**Cramer's rule**
say we have $A=\left(\begin{array}{ccc}a_{11}&a_{12}&a_{13}\\a_{21}&a_{22}&a_{23}\\a_{31}&a_{32}&a_{33}\end{array}\right)$, and let $\large S_{ij}$ be $A$ with the $i$th row and $j$th column removed

then $\large (A^{-1})_{ij}=\frac{1}{\det A}(-1)^{i+j}\det S_{ji}$
In steps:
- Find $\det A$
- make a matrix $M$ with entries $m_{ij}=\det S_{ \large ij}$
- Make $C$ with entries $\large c_{ij}=(-1)^{i+j}m_{ij}$ . In practice, take $M$ and multiply ***elementwise*** with $\left(\begin{array}{ccc}1&-1&1\\-1&1&-1\\1&-1&1\end{array}\right)$
- then your answer, $A^{-1}$, is $\large \frac{1}{\det A}C^T$
With the previous matrix, here's cramer's rule
$A=\left(\begin{array}{ccc}2&3&-1\\1&1&3\\1&2&-1\end{array}\right)$


**det(A):**
$$\det(A) = 2\det\begin{pmatrix}1&3\\2&-1\end{pmatrix} - 3\det\begin{pmatrix}1&3\\1&-1\end{pmatrix} + (-1)\det\begin{pmatrix}1&1\\1&2\end{pmatrix}
$$
$$
= 2(-7) - 3(-4) - 1(1) = -14 + 12 - 1 = -3
$$
**Matrix of minors M:**
$$
M = \begin{pmatrix}-7&-4&1\\-1&-3&1\\10&7&-1\end{pmatrix}
$$
**Cofactor matrix C:**
$$
C = \begin{pmatrix}-7&4&1\\1&-3&-1\\10&-7&-1\end{pmatrix}
$$
**Inverse:**
$$
A^{-1} = \frac{1}{\det(A)}C^T = \frac{1}{-3}\begin{pmatrix}-7&1&10\\4&-3&-7\\1&-1&-1\end{pmatrix} = {\large\begin{pmatrix}\frac{7}{3}&-\frac{1}{3}&-\frac{10}{3}\\-\frac{4}{3}&1&\frac{7}{3}\\-\frac{1}{3}&\frac{1}{3}&\frac{1}{3}\end{pmatrix}}
$$
#### Bringing all this together
We have some equations:
$$\begin{eqnarray}a_{11}x_{1}+a_{12}x_{2}&=&b_1 \\ a_{21}x_1+a_{22}x_2&=&b_2\end{eqnarray}$$
In matrix form:
$$
A\begin{pmatrix}x_1\\x_2\end{pmatrix}=\begin{pmatrix}b_1\\b_2\end{pmatrix}
$$
where $A=\begin{pmatrix}a_{11}&a_{12}\\a_{21}&a_{22}\end{pmatrix}$ (the coefficient matrix)
- Solving this system means finding $\underline{x}$ such that it becomes $\underline{b}$ when transformed by $A$ 
- Rearranging, we see that $\underline{x}=A^{-1}\underline{b}$, and as such if $A$ is invertible, then there is a unique solution for $\underline{x}$. 
- If $A$ is _not_ invertible, then there is either:
	- a line of points corresponding to point $\underline{b}$ on the projection line and thus there are infinite solutions. 
	- $\underline{b}$ is not on the projection line of $A$, and thus there is no solution.

### Week 4 - Eigy!!
- For a set of vectors $\left\{ \underline{v}_1, \underline{v}_2 ,\ldots, \underline{v}_k \right\}$, a linear combination of those vectors is $\sum k_i\underline{v}_i$ where $k_i\in \mathbb{R}$
- A set of vectors is linearly dependent if one or more can be written as a linear combination of the others, and is linearly *independent* if this is not possible.
**Definitions**
An **eigenvector** $\underline{v}\neq\underline{0}$ of a matrix $A$ is one that does not change direction (although it may change "sense" (sign)) under that matrix i.e. $A\underline{v}=\lambda \underline{v}$ where $\lambda \in \mathbb{R}$. For this vector, there is a corresponding **eigenvalue** - this is $\lambda$.

Some common eigenvectors:
- For reflections, along the reflecting axis and orthogonal to it
- For shears, the axis along the shear (bottom of the parallelogram)
- For dilations, along the dilating axis and orthogonal to it
- Note: rotations will never have *real* eigenvectors - they do have complex ones though!

**Solving for them**
So you want an eigen-thingy.
We know $A\underline{v}=\lambda \underline{v}$, so lets do some math:
- $A\underline{v} - \lambda \underline{v}=0$
- Want to get only one V, so have to do some shenanigans
- Use the identity matrix - won't change the vector $A\underline{v} - \lambda I\underline{v}=0$
- $(A-\lambda I)\underline{v}=0$ - this is legit because matrix multiplication is distributive
- Now, for this to have a solution that isn't $\underline{v}=0$, then it must be impossible to get $(A-\lambda I)$ to the other side (because you'd multiply it by 0 and get 0) , **hence it must be singular** (not invertible)
**Characteristic equation**
> $\det(A-\lambda I)=0$

For $A=\begin{pmatrix}1&\frac{1}{2}\\1&\frac{3}{2}\end{pmatrix}$ , $A-\lambda I=\begin{pmatrix}1&\frac{1}{2}\\1&\frac{3}{2}\end{pmatrix}-\lambda \begin{pmatrix}1&0\\0&1\end{pmatrix}=\begin{pmatrix}1-\lambda & \frac{1}{2} \\ 1 & \frac{3}{2}-\lambda\end{pmatrix}$
Hence, $\det(A-\lambda I) = (1-\lambda)(\frac{3}{2}-\lambda)-\frac{1}{2}=\frac{3}{2}-\lambda-\frac{3}{2}\lambda+\lambda^2-\frac{1}{2}=\lambda^2-\frac{5}{2}\lambda+1$
Thus, $\lambda=2,\frac{1}{2}$
Find eigenvectors:
- sub in $A, \lambda$ in to $(A-\lambda I)\underline{v}=0$, and then solve using GE
let $\lambda=2$
$$
\left(\begin{pmatrix}1&\frac{1}{2}\\1&\frac{3}{2}\end{pmatrix}-2\begin{pmatrix}1&0\\0&1\end{pmatrix}\right)\begin{pmatrix}a\\b\end{pmatrix}=0
$$
$$
\begin{pmatrix}-1&\frac{1}{2}\\1&-\frac{1}{2}\end{pmatrix}\begin{pmatrix}a\\b\end{pmatrix}=0
$$
$$
\begin{pmatrix}-a+\frac{1}{2}b\\a-\frac{1}{2}b\end{pmatrix}=0
$$
$$
\left(\begin{array}{cc|c}-1&\frac{1}{2}&0\\1&-\frac{1}{2}&0\end{array}\right)
$$
$R_1\rightarrow R_1+R_2,\quad R_1 \leftrightarrow R_2$  
$$
\left(\begin{array}{cc|c}1&-\frac{1}{2}&0\\0&0&0\end{array}\right)
$$
col 2 is free, thus add a variable: $v_1-\frac{1}{2}v_2=0$, $2v_1=v_2$
$\underline{v} = t\begin{pmatrix}2\\1\end{pmatrix}$
**Repeated eigenvalues**
Ones so far have had two unique eigenvalues, each with an eigenvector
Shear: only has one eigenvector (direction) with a repeated eigenvalue (characteristic equation is a perfect square, thus one eigenvalue with algebraic multiplicity 2)
**Complex eigenvalues**
These come from rotations (or transformations with a rotation component)
let $A=\begin{pmatrix}\cos(30\degree)&-\sin(30\degree)\\ \sin(30\degree) & \cos(30\degree)\end{pmatrix}=\begin{pmatrix}\frac{\sqrt{3}}{2}&-\frac{1}{2}\\ \frac{1}{2} & \frac{\sqrt{3}}{2}\end{pmatrix}$
$$
\det(A-\lambda I)=\begin{vmatrix}\frac{\sqrt{3}}{2}-\lambda & -\frac{1}{2} \\ \frac{1}{2} & \frac{\sqrt{3}}{2}-\lambda\end{vmatrix}=0
$$
...
$$
\lambda^2-\sqrt{3}\lambda+1=0
$$
$$
\lambda=\text{cis}(\pm30\degree)
$$
let $\lambda = \text{cis}(30\degree)$
$$
\begin{pmatrix}
\cos(30\degree)-\text{cis}(30\degree) & -\sin(30\degree) \\
\sin(30\degree) & \cos(30\degree)-\text{cis}(30\degree)
\end{pmatrix}
\begin{pmatrix}
v_1 \\ v_2
\end{pmatrix}
=0
$$
$$
\left(
\begin{array}{cc|c}
-i\sin(30\degree) & -\sin(30\degree) & 0 \\
\sin(30\degree) & -i\sin(30\degree) & 0
\end{array}
\right)
$$
$$
\left(
\begin{array}{cc|c}
-\frac{1}{2}i & -\frac{1}{2} & 0 \\
\frac{1}{2} & -\frac{1}{2}i & 0
\end{array}
\right)
$$
$$
\left(
\begin{array}{cc|c}
1 & -i & 0 \\
0 & 0 & 0
\end{array}
\right)
$$
$$
\therefore \underline{v}=\begin{pmatrix}i\\1\end{pmatrix}t\space\space |\space t\in\mathbb{R}
$$
**For 2x2 matrices**
(with real entries)
there might be:
- two real, distinct eigenvalues, each with a different eigenvector direction
- one (thus repeated) eigenvalue with a single eigenvector direction
- one (thus repeated) eigenvalue for *all* directions (every vector is an eigenvector) e.g. a pure dilation
- a pair of complex conjugate eigenvalues with complex conjugate eigenvectors

**For nxn matrices**
- The characteristic polynomial ($\det(A-\lambda I)$) is a polynomial with degree n and thus n roots
- hence n eigenvalues (but some repeated). The number of repeats of an eigenvalue is its ***algebraic*** multiplicity
- For non-repeated eigenvalues, there is a single eigenvector direction associated with them
- For repeated eigenvalues, they can have a single direction or infinitely many - "infinite" isn't useful so we say the number of linearly independent eigenvectors - this is the ***geometric*** multiplicity
	- e.g. 2 would be a plane, 3 a 3D space etc
- If the matrix A is real, then eigenvalues are either real or in complex conjugate pairs, with their corresponding eigenvectors also conjugate pairs (same as the polynomial rules)

**diagonalisation**
Let $A=\begin{pmatrix}1 & \frac{1}{2} \\ 1 & \frac{3}{2}\end{pmatrix}$
Has eigenvalue 2 with eigenvector $\begin{pmatrix}\frac{1}{2} \\ 1\end{pmatrix}$ and eigenvalue $\frac{1}{2}$ with eigenvector $\begin{pmatrix}-1 \\ 1\end{pmatrix}$.
We make a special matrix $V$, which is the eigenvectors one after another, so here $V=\begin{pmatrix}\frac{1}{2} & -1 \\ 1 & 1\end{pmatrix}$ , and a diagonal matrix $D=\begin{pmatrix}2&0\\0&\frac{1}{2}\end{pmatrix}$ , whose entries are the corresponding eigenvalues to $V$.

It can be shown that $AV=VD$ and thus $A=VDV^{-1}$
Hence, for an $n\times n$ matrix with $n$ **linearly independent** eigenvectors (or equivalently, eigenvalues all with algebraic multiplicity 1), the matrix can be expressed in this form.
Important notes:
- A diagonalisation is **not** unique - you can have multiple pairs of $V$ and $D$ for the same input matrix
- Matrices with all distinct eigenvalues can *always* be diagonalised
	- algebraic multiplicity must also equal geometric multiplicity
- Real, symmetric matrices ($A=A^T$, all entries real) have orthogonal eigenvectors for distinct eigenvalues
	- For an $n\times n$ matrix that fits the above criteria
		- there are n eigenvectors
		- you can guarantee that you can pick them to be all perpendicular/orthogonal to each other
	- possible to choose all eigenvectors to be orthogonal and have length 1 $\therefore$ $V^{-1}=V^T$
Why is this useful? Matrix exponentiation.
$A^{50}$ can be written as $VD^{50}V^{-1}$ (kind of like a telescoping series - the neighboring $V$ and $V^{-1}$ in the middle cancel)
### Week 5 - multivariate calc
**Note - partial derivatives can't be flipped i.e.** $\frac{\partial z}{\partial x} \neq \frac{\partial x}{\partial z}$
Can do a function $y=f(x)$ easy enough.
What if we want to do stuff in 3D? What if we add time? For this we need to be able to write our gradients and derivatives in bits, and combine them in to vectors.
Hence, the partial derivative, $\frac{\partial}{\partial x}$. Treat everything that's not in terms of x as a constant e.g. $$
\frac{\partial}{\partial x}(xe^y)=e^y
$$
($e^y$ acts as a constant here)
Now, that's great. How do we combine?
Let $z=x^2+y^2$.
If we wanted to take a derivative only caring about x and z, we can do that fairly easily using the partial. If we want to get a picture of the whole thing, we need derivatives in all directions. Hence, $\nabla$ (nabla)
This goober is basically a derivative symbol but for multiple dimensions - if you want to take derivatives for a function $f$, you write $\nabla f$.
$$
\nabla f=\begin{bmatrix}\frac{\partial f}{\partial x} \\ \frac{\partial f}{\partial y} \\ \frac{\partial f}{\partial z}\end{bmatrix}
$$
What this gives you is the rates of change of the function with respect to each axis, so something like $\nabla f = \begin{bmatrix}1\\1\\1\end{bmatrix}$ would be a 45$\degree$ angle with everything (pretty much)
If you want to take the derivative in a particular direction (i.e. if you projected the graph on to a particular plane), you take the nabla thingy and project it in the direction of what you want to diff.
Hence, if you had $f(x,y)=e^{x^2}\sin(y)-y$, and the direction you wanted was $\underline{u}\begin{pmatrix}1\\1\end{pmatrix}$:
$$
f(x,y)=e^{x^2}\sin(y)-y 
$$
$$
\nabla f = \begin{bmatrix}
\sin(y)\cdot 2xe^{x^2} \\
e^{x^2}\cos(y)-1
\end{bmatrix}
$$
Now, project our gradient vector in the direction we want to evaluate
$$
\text{want unit vector} \therefore \hat u=\begin{pmatrix}\frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}}\end{pmatrix}
$$
$$
\frac{\partial f}{\partial u} = 
\underline{\hat u}\space \cdot \nabla f =

\frac{1}{\sqrt{2}}\cdot2xe^{x^2}\sin(y) +
\frac{1}{\sqrt{2}}\cdot e^{x^2}\cos(y)-\frac{1}{\sqrt{2}}
=
\sqrt{2}xe^{x^2}\sin(y) +
\frac{1}{\sqrt{2}}\cdot e^{x^2}\cos(y)-\frac{1}{\sqrt{2}}
$$
If we wanted to evaluate this at a point, we can sub in the point as soon as we have $\nabla f$, or can wait if you want (like here). Let's pick $\underline x = (0, \frac{\pi}{2})$
$$
\frac{\partial f}{\partial u}=\sqrt{2}(0)e^{0^2}\sin(\frac{\pi}{2})+\frac{1}{\sqrt{2}}e^{0^2}\cos(\frac{\pi}{2})-\frac{1}{\sqrt{2}} = 0 + 0 - \frac{1}{\sqrt{2}}=-\frac{1}{\sqrt{2}}
$$
(note we've picked nice numbers so there's lots of 0's in there)
##### Approximation
So we can calculate $f(x,y)=e^{x^2}\sin(y)-y$ with $x=0, y=\frac{\pi}{2}$ easy enough - just sub in the values for x and y:
$$
f(0,\frac{\pi}{2})=e^{0^2}\cdot \sin(\frac{\pi}{2})-\frac{\pi}{2}
=1-\frac{\pi}{2}
$$
What if I ask you for $f(0.1, \frac{5\pi}{12})$?  (ew)
What we can do is make a plane that is tangent to our surface at the nice point, and then use that plane as an approximation for the curve near the nice point (like using tangents to approximate curves in 2D).
To make this plane you will need:
- a point (that shouldn't be too ugly)
- a function
- ... and its gradient ($\nabla f$)
Let's work in 3D, with our previous example.
We know a function, its gradient vector, and a nice point
$$
f(x,y)=e^{x^2}\sin(y)-y ,\space
\nabla f = \begin{bmatrix}
\sin(y)\cdot 2xe^{x^2} \\
e^{x^2}\cos(y)-1
\end{bmatrix},\space
f(0,\frac{\pi}{2})=1-\frac{\pi}{2}, \space \text{let this be} \space\underline p
$$
A plane tangent at the point will
- contain the point of tangency $\underline p$
- contain the gradient vector at $\underline p$
so, let's get writing.
For some vector $\underline x = \begin{pmatrix}x\\y\end{pmatrix}$, for which you are trying to find the $z$ value for,
$$
z(x,y)=f(\underline p) + (\underline x - \underline p)\cdot\nabla f(\underline p)
$$
$z(x,y)$ - easier in this case to write as being in cartesian form rather than with a normal (how would you do it with the normal??? anyway)
$\underline p$ is the reference x and y, so $f(\underline p)$ is our reference height
$\underline x - \underline p$ is the vector between the reference point and the point being evaluated in the $xy$ plane
we know the rate of change for $z$ against $x$ or $y$, and hence can dot each delta with its gradient to get the change in z. 
### Week 6 - multivariate 2
Y'know how we looked at minima in last week's workshop questions? Lets think a bit more about that. 
We had that if $\frac{dy}{dx}=0$, that it was a point of interest for 2d functions - similar idea here. Wherever $\nabla f = \underline 0$, then that point is called a **critical point**
$$
g(x,y) = xye^{-x^2-y^2}
$$
$$
\nabla g = \begin{bmatrix}
ye^{-x^2-y^2}+xy(-2x)e^{-x^2-y^2} \\
xe^{-x^2-y^2} + xy(-2y)e^{-x^2-y^2}
\end{bmatrix}
$$
$$
\nabla g = e^{-x^2-y^2}
\begin{bmatrix}
y(1-2x^2) \\
x(1-2y^2)
\end{bmatrix}
$$

thus, for $\frac{\partial g}{\partial x}=0$, $x=\pm\frac{1}{\sqrt{2}} \text{ or } y=0$, and for $\frac{\partial g}{\partial y}=0$, $y=\pm\frac{1}{\sqrt{2}} \text{ or } x=0$
Enumerate:
- Let $y = 0$, hence $\frac{\partial g}{\partial x}=0$
	- thus for $\frac{\partial g}{\partial y}=0$, must have $x=0$ (as $y\neq \pm\frac{1}{\sqrt{2}}$ because it's zero)
- let $x=\pm\frac{1}{\sqrt{2}}$, hence $\frac{\partial g}{\partial x}=0$
	- thus for $\frac{\partial g}{\partial y}=0$, must have $y=\pm\frac{1}{\sqrt{2}}$ (as $x\neq0$ because it's $\pm\frac{1}{\sqrt{2}}$)
- thus, five critical points:
	- $(0,0)$
	- $(\pm\frac{1}{\sqrt{2}}, \pm\frac{1}{\sqrt{2}})$ in all four combinations

#### Categorising critical points
We know that in a plane if the second derivative is negative ($\Large{\frac{d^2f}{dx^2}}<0$), then the critical point will be a maximum (gradient was going down). Similarly, for a minimum, the second derivative is positive.
In 3D, we have to consider the second derivative in *all* directions from a point if we want to state that it is a minimum or maximum - it could be a minimum in some directions and a max in others (think of a pringles chip and its saddle shape)

**Maximum**
- let $F(u)=f(\underline p + u\hat{\underline u})$, the function value at some amount of $\hat{\underline u}$ from our point of interest $\underline p$
- for $F(t)$ to be a maximum it must satisfy $\frac{d}{du}\left(\frac{dF}{du}\right)<0$ for all directions ($\forall \hat{\underline u}$) at $u=0$ (this is taking the directional derivative)
	- This includes $\frac{\partial^2 f}{\partial x^2}<0$ and $\frac{\partial^2 f}{\partial y^2}<0$, but is a more restrictive condition ($\frac{d}{du}\left(\frac{dF}{du}\right)<0 \implies \frac{\partial^2 f}{\partial x^2}<0 \text{ and }\frac{\partial^2 f}{\partial y^2}<0$ at $u=0$, but the converse is not true) 
**Minimum**
Do the above but make it $\gt$.

**Saddle**
Minimum from some perspective, maximum from another
- let $F(u)=f(\underline p + u\hat{\underline u})$, the function value at some amount of $\hat{\underline u}$ from our point of interest $\underline p$
- For $\underline p$ to be a saddle point, the following must be satisfied
	- There exists $\hat{\underline u}$ such that $\frac{d}{du}\left(\frac{dF}{du}\right)<0,\space u=0$, *and*
	- There exists $\hat{\underline u}$ such that $\frac{d}{du}\left(\frac{dF}{du}\right)>0,\space u=0$
	- Basically, it's a minimum in at least one direction, and a maximum in another.

##### Funky matrix
- We know $\frac{d^2F}{du^2}=\frac{d}{du}\left(\frac{dF}{du}\right)$
- the definition of a direction derivative means that this is the same as $\frac{d}{du}\left(\hat{\underline u}\cdot\nabla f\right)$
- expanding, $\frac{d}{du}\left(\begin{pmatrix}\hat{\underline u_1} \\ \hat{\underline u_2}\end{pmatrix}\cdot\begin{pmatrix}\frac{\partial f}{\partial x} \\ \frac{\partial f}{\partial y}\end{pmatrix}\right)=\frac{d}{du}\left(\hat{\underline u_1}\frac{\partial f}{\partial x} + \hat{\underline u_2}\frac{\partial f}{\partial y}\right)$
- ... and again
- $\hat{\underline u}\cdot \nabla\left(\hat{\underline u_1}\frac{\partial f}{\partial x} + \hat{\underline u_2}\frac{\partial f}{\partial y}\right)$
- expand:
$$
\begin{pmatrix}\hat{\underline u_1} \\ \hat{\underline u_2}\end{pmatrix} \cdot
\begin{pmatrix}
\hat{\underline u_1}\frac{\partial^2 f}{\partial x^2}
+ \hat{\underline u_2}\frac{\partial^2 f}{\partial x\partial y} \\
\hat{\underline u_1}\frac{\partial^2 f}{\partial y\partial x}
+ \hat{\underline u_2}\frac{\partial^2 f}{\partial y^2}
\end{pmatrix}
$$
After some algebra:
$$
\frac{d^2F}{du^2}=\begin{pmatrix}\hat{\underline u_1} & \hat{\underline u_2}\end{pmatrix}
\begin{pmatrix}
\frac{\partial^2f}{\partial x^2} &
\frac{\partial^2f}{\partial x\partial y} \\

\frac{\partial^2f}{\partial y\partial x} &
\frac{\partial^2f}{\partial y^2}
\end{pmatrix}

\begin{pmatrix}
\hat{\underline u_1} \\
\hat{\underline u_2}
\end{pmatrix}
$$
This middle bit is the **Hessian matrix**, $H$, and as such we can write $\frac{d^2F}{du^2}=\hat{\underline u}^TH\hat{\underline u}$. It can make it easier to find when we have specific types of critical points.
**Maximums**
- $\hat{\underline u}^TH\hat{\underline u}<0 \space\forall \underline{\hat u}$, hence all eigenvalues of $H$ must be negative (using theory we haven't covered)
- This ends up requiring that $\det H>0$ and $\frac{\partial^2f}{\partial x^2}, \frac{\partial^2f}{\partial y^2}<0$
**Minimums**
- $\hat{\underline u}^TH\hat{\underline u}>0 \space\forall \underline{\hat u}$, hence all eigenvalues of $H$ must be positive (using theory we haven't covered)
- This ends up requiring that $\det H>0$ and $\frac{\partial^2f}{\partial x^2}, \frac{\partial^2f}{\partial y^2}>0$
	- As both eigenvalues have the same sign, then the determinant must be positive. If we have a negative determinant, then that means that there is a mirror, and that the eigenvector in the direction of the mirror will be flipped, whereas the one that is perpendicular to it will be lengthened.
	- Note, if we have two flips, then they cancel out (acting as a 180 rotation), and the $\det$ is still positive

Note that both the above have $\det H>0$
If $\det H<0$, then you have a **saddle point** (one positive eigenvalue, and one negative one)
$$
\det H = \frac{\partial^2f}{\partial x^2}\frac{\partial^2f}{\partial y^2}-\left(\frac{\partial^2f}{\partial x \partial y}\right)^2 \text{ as }\frac{\partial^2f}{\partial x \partial y}=\frac{\partial^2f}{\partial y \partial x}
$$
#### Absolute minima and maxima
- If, within your domain, you have no maximums, then the maximum must be on the boundary
	- Same for minima
- Saddle points (unless possibly if they are on the boundary) will never be absolute maxima or minima
- When checking boundaries, look for normal 2d stationary points on the face of them - they could well be the limits
- Also check the "corners", where it might be an absolute max/min but not be a SP
More generally, take your domain restriction and sub it in to get a function with one variable, and then use the methods from methods!

#### Lagrange multiplier
- for finding max/min with a constraint
- for finding absolute max/min with less work (usually)

say we have $f(x,y)$ with $g(x,y)=c$ as a restriction
There will exist a point where a contour of $f$ and $g(x,y)=c$ touch (tangent), and this point is a min or max of $f$ along $g(x,y)$. Also, $\nabla f$ will be perpendicular (as it's perpendicular to the contours), and as such $\nabla f = \lambda \nabla g$ (where $\lambda$ is the Lagrange multiplier)

If we want to find the max/min, we need this condition, but to also follow the contour, thus:
$$
\begin{eqnarray}\nabla f &=& \lambda \nabla g \\ g(x,y)&=&\text{const}\end{eqnarray}
$$
... which will return the maxima and minima of $f$ subject to the constraint
**Example**
$f(x,y)=x^2+y^2$, $g(x,y)=x+2y=1$
Solve $\nabla f = \lambda \nabla g$
$\begin{pmatrix}2x\\2y\end{pmatrix}=\lambda\begin{pmatrix}1\\2\end{pmatrix}\implies x=\frac{1}{2}\lambda, \space y=\lambda$
as we have $x+2y=1$, $\lambda=\frac{2}{5}$
hence $x=\frac{1}{5}, y=\frac{2}{5}$
Min is $f(\frac{1}{5}, \frac{2}{5})=\frac{1}{5}$

**More complex example**
$f(x,y)=-(x^2-1)(y^2-1)$, $x^2+y^2 \leq 4 \therefore g(x,y)=x^2+y^2=4$
Check the critical points of $f$
Local min at $f(0,0)=1$
Not a max!
Now, check the circular edge for SPs
$$
\nabla f  = \lambda \nabla g
$$
$$
\begin{pmatrix}
-2x(y^2-1)\\
-2y(x^2-1)
\end{pmatrix}
=
\lambda
\begin{pmatrix}
2x \\
2y
\end{pmatrix}
\implies
\begin{eqnarray}
0&=&2x(\lambda+y^2-1) \\
0&=&2y(\lambda+x^2-1)
\end{eqnarray}
\implies
$$
$$\begin{eqnarray}
&(1)& \space x=0 \text{ or } 1-y^2=\lambda \\
&(2)&\space y=0 \text{ or } 1-x^2=\lambda \\
&(3)&\space x^2+y^2=4
\end{eqnarray}$$
Let $x=0$:
- $\lambda = 1$ by $(2)$
- $y=\pm 2$ by $(3)$
- $f(0, \pm 2)=3$
let $y=0$:
- $\lambda = 1$ by $(1)$
- $x=\pm 2$ by $(3)$
- $f(\pm2, 0)=3$
let $1-y^2=\lambda$, $1-x^2=\lambda$
- $1-y^2=1-x^2$
- $x^2=y^2$
- $x=\pm\sqrt 2,\space y=\pm\sqrt 2$
- $f(\pm\sqrt2,\pm\sqrt2)=-1$
hence abs min is -1
#### Tangent vectors
vector
tangent to curve
at a point
*wow.*
let the curve be $\underline x(t)$
then the tangent vector is $\frac{d\underline x}{dt}$

**Funky domain notation**
$-\infty < t < \infty, -\infty < s < \infty \iff (s, t) \in (-\infty, \infty) \times (-\infty, \infty) \iff s\in(-\infty,\infty),\space t\in(-\infty,\infty)$
### Week 7 - integration and hyperbolic functions
Reference:
- $\int x^n dx = \frac{1}{n+1}x^{n+1} + C, \space n \neq -1$
- $\int \frac{1}{x} dx = \log x + C$
- $\int e^xdx = e^x + C$
- $\int \sin x dx = -\cos x+C$
- $\int \cos x dx = \sin x + C$

##### Integration by substitution (u-sub)
$$
\int e^{2x+1}dx
$$
^ That looks ugly - I can integrate $e^x$ though!
let $u=2x+1$
then $du = 2dx$, $\frac{du}{2}=dx$
$$
\int e^u \cdot \frac{du}{2}
$$
$$
\frac{1}{2}e^u {+C}
$$
... sub back in u
$$
\int e^{2x+1}dx = \frac{1}{2}e^{2x+1}+C
$$
Now, something harder:
$$
\int \frac{1}{\sqrt{1-x^2}}dx
$$
that looks trig-ish!
let $x=\sin u$, $dx = \cos u \space du$, hence
$$
\int \frac{1}{\sqrt{1-\sin^2u}}\cdot \cos u\space du
$$
$$
\int \frac{1}{\cos u}\cdot \cos u \space du
$$
$$
\int 1\cdot du
$$
$$
u+C
$$
... sub back in
$$
\sin^{-1}x+C=\int\frac{1}{\sqrt{1-x^2}}dx
$$
(which you should know from the spesh formula sheet)

##### Integration by parts
$$
\int f(x)\frac{dg}{dx}\cdot dx = f(x)g(x)-\int\frac{df}{dx}g(x)\cdot dx
$$
equivalently
$$
\int uv dx = u\int vdx-\int\int vdx\frac{du}{dx}dx
$$
For trig functions this will often chain, and might sometimes loop (in which case you try and subtract the loop from itself to get an expression for some value)
**Priority for things to be u**
- *first*
- **I**nverse trig
- **L**og
- **A**lgebraic (polynomials etc.)
- **T**rig
- **E**xponential (e.g. $e^x$)
- *last*
So:
$$
\int\log x dx=\int 1\cdot\log x dx=\log x\int1dx-\int\int1dx\frac{d}{dx}(\log x)dx
$$
$$
x\log x-\int x\cdot\frac{1}{x}dx
$$
$$
x\log x - x+C
$$
(**don't forget the +C!**)
These methods can be combined, often using u-sub to get rid of ugly things before using by parts.
##### Hyperbolic functions
Logical consequence of representing $\sin$ and $\cos$ as combinations of complex exponentials
$e^{ix}=\cos x + i\sin x$ thus:
- $\cos x = \frac{1}{2}(e^{ix}+e^{-ix})$ (add euler's identity to itself with $-x$ subbed in)
- $\sin x = \frac{1}{2i}(e^{ix}-e^{-ix})$ (subtract the $-x$ variant)
I hate $i$, so let's get rid of it:
- $\cosh x = \frac{1}{2}(e^x+e^{-x})$
- $\sinh x = \frac{1}{2}(e^x-e^{-x})$
![[Pasted image 20260422154342.png]]
- These have analogues to the derivatives of the "normal" trig functions, e.g.
![[Pasted image 20260422154435.png]]
Note that $\tanh x$ is a logistic curve shifted down a bit, such that the bottom asymptote isn't on the $x$ axis anymore.
##### Slightly modified identities
- $\sinh 2x = 2\cosh x \sinh x$ (unchanged)
- $\cosh^2x-\sinh^2x=1$
- $\cosh^2x+\sinh^2x=\cosh 2x$
- above two have been "flipped" from how they would be for the normal trig functions
for combinations of a normal trig function and its inverse we have the magic triangle of doom:
![[Pasted image 20260422154929.png|white]]
If you're a masochist, you might try and express these functions using euler's identity. I hate you, but you can do it:
$$
\begin{eqnarray}
\arcsin(z)=i\ln(\sqrt{1-z^2}-iz) \\
\arccos(z)=i\ln(z-i\sqrt{1-z^2})
\end{eqnarray}
$$
However, reiterating earlier that $i$ is terrible and fake news, the inverse forms for $\sinh$ and $\cosh$ are **nice** and **real** and ***not fake:***
$$
\begin{eqnarray}
\text{arcsinh}\ x=\ln(x+\sqrt{x^2+1}) \\
\text{arccosh}\ x=\ln(x+\sqrt{x^2-1}) \\
\text{arctanh}\ x=\frac{1}{2}\ln\frac{1+x}{1-x}
\end{eqnarray}
$$
(note that these are *nice* because they are quadratic rational functions with $e^x$, and thus can be solved with the quadratic formula and then a log)
$$
\begin{eqnarray}
\text{Let }y=\cosh x=\frac{1}{2}(e^x+e^{-x}) \\
\implies 2y=e^x+e^{-x}\\
\implies e^{2x}-2ye^x+1=0 \\
\implies e^x = \frac{2y\pm \sqrt{(2y)^2-4}}{2} \\
\implies e^x = y \pm \sqrt{y^2-1} \\
\implies x = \ln(y \pm \sqrt{y^2-1}) \\
\implies x=\pm\ln(y+\sqrt{y^2-1}) \text{ (mult - by conj.)}
\end{eqnarray}
$$
##### Derivatives of hyperbolic functions
$$
\begin{eqnarray}
\frac{d}{dx}(\cosh x)&=&\sinh x \text{ (Note - no negative!)}\\
\frac{d}{dx}(\sinh x)&=&\cosh x \\
\frac{d}{dx}(\tanh x)&=&\text{sech}^2\ x \\
\frac{d}{dx}(\text{sech}\ x)&=&-\text{sech}\ x\tanh x \\
\frac{d}{dx}(\text{cosech}\ x)&=&-\text{cosech}\ x\coth x\\
\frac{d}{dx}(\coth x)&=&-\text{cosech}^2\ x \\
\end{eqnarray}
$$
The signs for some of the others have probably also flipped but I can't rmb that.
##### Integration using hyperbolic substitution
We had $\large\int \frac{1}{\sqrt{1-x^2}}dx$ before, but what if we have
$$
\int\frac{1}{\sqrt{x^2-1}}dx\quad x\gt 1 \text{ (otherwise discontinuous)}
$$?
We have hyperbolic functions!
Let $x=\cosh u$ (note, previously used $\sin$)
$dx=\sinh u\cdot du$
$$
\begin{eqnarray}
\large{\int\frac{1}{\sqrt{x^2-1}}dx} \\
\text{let }x=\cosh u,\ dx=\sinh u\cdot du\text{ (note, previously used}\sin\text{)} \\
=\int\frac{1}{\sqrt{\cosh^2u-1}}\cdot \sinh u du\\
=\int\frac{1}{\sqrt{\sinh^2u}}\cdot\sinh u du\\
=\int1\cdot du \\
=u+C\\
=\text{arccosh}\ x+C
\end{eqnarray}
$$
Worth noting that, just like $\arccos(\sin(x))=\sqrt{1-x^2}$, a similar thing exists for hyperbolic functions (and found in the same way)
$$
\begin{eqnarray}
\cosh(\text{arcsinh}(x))=y \\
\implies\cosh^2(\text{arcsinh}(x))=y^2 \\
\text{as }\cosh^2x=1+\sinh^2x \\
\implies1+\sinh^2(\text{arcsinh}(x))=y^2 \\
\implies y=\pm\sqrt{1+x^2}\\
\end{eqnarray}
$$
### Week 8 - ODEs
And ODE is an equation involving a dependent variable y that is a function of one independent variable x and one or more of its derivatives e.g. $\frac{dy}{dx}$, $\frac{d^2y}{dx^2}$
$$
\frac{d^2y}{dx^2}+x\frac{dy}{dx}=-ye^{x+y}
$$
A solution is any $y(x)$ that satisfies (obeys) the ODE
Often when solving you will get a general solution with some arbitrary constants. You can set these using the extra information in the question to get a particular solution (e.g. from an initial condition)
- An **Initial value problem** is an ODE with ICs (initial conditions)
	- e.g. a value for $y$. $x$ and $y'$
- A **boundary value problem** is an ODE with BCs (boundary conditions)
	- e.g. two $(x,y)$ points

**Classification**
- order
	- highest derivative
- linear
	- $$c_n(x)\frac{d^n y}{dx^n} + c_{n-1}(x)\frac{d^{n-1} y}{dx^{n-1}} + \dots + c_2(x)\frac{d^2 y}{dx^2} + c_1(x)\frac{dy}{dx} + c_0(x)\, y = r(x)$$
	- so $$\frac{d^2x}{dt^2}=-g$$ is linear
	- but$$ \frac{dy}{dx}+\frac{x}{y}=0$$ isn't
	- a linear ODE is **homogeneous** if $r(x)=0$
- first order separable
	- $\frac{dy}{dx}=f(x)g(x)$
	- as in spesh

**first order linear ODEs**
this isn't nice and separable. well shit.
$$
\begin{eqnarray}
c_1(x)\frac{dy}{dx}+c_0(x)y=r(x) \\
\implies \frac{dy}{dx}+p(x)y=q(x)
\end{eqnarray}
$$
the left bit has two y's, which will be annoying to deal with - want to get the left side to collapse using the product rule
hence:
- mult everything by $\mu (x)$
$$
\mu(x)\frac{dy}{dx}+\mu(x)p(x)y=\mu(x)q(x)
$$
- for the left to product rule, need
$$
\begin{eqnarray}
\mu(x)p(x)=\mu'(x) \\
\text{rewrite as } \mu\cdot p(x)=\frac{d\mu}{dx} \\
p(x)dx=\frac{d\mu}{\mu}\\

\end{eqnarray}
$$
- and we can solve this one!
$$
\begin{eqnarray}
\int p(x)dx=\int\frac{1}{\mu}d\mu \\
\int p(x)dx=\ln\mu \\
\large\mu=e^{\int p(x)dx}
\end{eqnarray}
$$
- hence
$$
\begin{eqnarray}
\frac{d}{dx}\left[\mu y\right]=\mu q(x) \\
\implies\mu y=\int\mu q(x)dx+C \\
\implies y=\frac{1}{\mu}\int\mu q(x)dx+\frac{C}{\mu}
\end{eqnarray}
$$
**what if p(x) is a constant?**
$$
\begin{eqnarray}
\frac{dy}{dx}&+&2y=x^2 \\
I(x) &=& e^{\int p(x)dx}=e^{2x} \\
y(x)&=&\frac{1}{I(x)}\int I(x)q(x)dx+\frac{C}{I(x)} \\
&=&e^{-2x}\int e^{2x}x^2dx+Ce^{-2x} \\
&=&e^{-2x}\cdot e^{2x}(\frac{1}{2}x^2-\frac{1}{2}x+\frac{1}{4})+Ce^{-2x}\\
&=&\frac{1}{2}x^2-\frac{1}{2}x+\frac{1}{4}+Ce^{-2x} \\
\end{eqnarray}
$$
The first bit ($\frac{1}{2}x^2-\frac{1}{2}x+\frac{1}{4}$) will depend on $q(x)$ (here $x^2$), but the second bit ($Ce^{-2x}$) will always be an exponential that would satisfy the ODE if $q(x)=0$.

Consider $\frac{dy}{dx}+py=r(x)$:
set $y(x)=y_h(x)+y_p(x)$ where $y_h(x)$ is the solution to the homogeneous equation ($r(x)=0$) and $y_p(x)$ is a particular solution that satisfies the full ODE.
then we have $y_h=Ae^{\lambda x}$

try $p=2$, and let $y=y_h+y_p$:
$\lambda=-2$
let's guess that $y_p=ax^2+bx+c$
then $2ax+b+2(ax^2+bx+c)=x^2$
equate coefs
get $y_p=\frac{1}{2}x^2-\frac{1}{2}x+\frac{1}{4}$
![[Pasted image 20260426185659.png]]
### Week 9 - more odes
**Principle of superposition**
If $y_1$ solves a homogenous linear ODE, and so does $y_2$, then so does any linear combination of $y_1$ and $y_2$.

**Second-order, linear constant-coefficient ODEs**
Occur in things like newton's law, and as such you will run in to them from time to time.
- Set $y(x)=y_h(x)+y_p(x)$
- $y_h(x)$ is the homogenous solution - when you sub it in you get 0 (and as such you can add as much zero to any solution to get another one!)
- To get $y_p(x)$, we make an educated guess about the family the solution might be in.

**Let's find $y_h$ for this family:**
- For things like the above, we use $y_h=Ae^{\lambda x}$, which when subbed in to the ODE will give some number of solutions for $\lambda$, and leave $A$ as a constant that can be changed
	- If there are two solutions, we use the general solution $\large y=Ae^{\lambda_1 x}+Be^{\lambda_2 x}$
	- If there is only one unique solution for $\lambda$, then we need to add something else (as second order ODEs will have two constants of integration)
		- The true solution is not $\large Ae^{\lambda x}+Be^{\lambda x}$, as that smushes down into $Ce^{\lambda x}$, where $C=A+B$, which has only one constant. What we do instead is use $\large Ae^{\lambda x}+Bxe^{\lambda x}$
- If you end up with complex roots ($\lambda$), then there's probably something trig-ish going on: if you have $y=Ae^{3i}+Be^{-3ix}$ (where $A$ and $B$ complex) then you also have $\large y=\tilde A \cos(3x) + \tilde B \sin(3x)$ , where $\tilde A$ and $\tilde B$ are **real**.![[Pasted image 20260506224220.png]]
	- general rule for this (can prove using euler's formula with the $e^{ix}$):![[Pasted image 20260506224245.png]]
**Finding $y_p$**
If we have an ODE like above, we know that $y_p$ will be a polynomial of the same order as $r$ (if $r(x)\neq0$ , not homogeneous)![[Pasted image 20260506224654.png]]
If we have an ODE where $r(x)$ is a polynomial times an $e^{kx}$, then $y_p$ will also be a polynomial times $e^{kx}$ of the same order (same constant for the $e$)
- However, if $y_h=e^{kx}$, then we come to the issue mentioned earlier, and we make $y_p$ a polynomial times $xe^{kx}$ instead
For a linear combination of $\cos(mx)$ and $\sin(mx)$ times an $e^{kx}$, $y_p$ is also a combination of those times by an $e^{kx}$
![[Pasted image 20260506230859.png]]
If you're not sure, then have a guess at multiplying bits together and testing if it works!~~~~