From f25dd5313025796474652720af272aeb0787453d Mon Sep 17 00:00:00 2001 From: SmashingBumpkin <66125276+SmashingBumpkin@users.noreply.github.com> Date: Mon, 3 Jun 2024 12:26:32 +0200 Subject: [PATCH 1/2] Equation, notation + exam dates fixed --- .../14_neural_nets_backprop/14_neural_nets_backprop.ipynb | 8 ++++++-- .../15_backprop_jacobians/15_backprop_jacobians.ipynb | 6 +++--- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/AA2324/course/14_neural_nets_backprop/14_neural_nets_backprop.ipynb b/AA2324/course/14_neural_nets_backprop/14_neural_nets_backprop.ipynb index 69c88b1..5de0852 100644 --- a/AA2324/course/14_neural_nets_backprop/14_neural_nets_backprop.ipynb +++ b/AA2324/course/14_neural_nets_backprop/14_neural_nets_backprop.ipynb @@ -634,7 +634,11 @@ "\n", "\n", "1. **Initialization - Very Important if the function is not strictly convex** \n", - "$\\bmf{\\theta} \\sim \\mathcal{N}(\\cdot)~~~\\text{omit details for now}$$ With NN random initialization from a distribution (There are different methods). **We do not set them all to zero**\n", + "\n", + "$$\\bmf{\\theta} \\sim \\mathcal{N}(\\cdot)~~~\\text{omit details for now}$$\n", + "\n", + "With NN random initialization from a distribution (There are different methods). **We do not set them all to zero**\n", + "\n", "2. Repeat until **convergence**:\n", " - Compute the gradient of the loss wrt the parameters $\\bmf{\\theta}$ given **the mini-batch**\n", " - Take a small step in the opposite direction of steepest ascent **(so steepest descent).**

\n", @@ -1775,7 +1779,7 @@ "source": [ "# Universal Approximation Theorem [Informal]\n", "\n", - "Given a continuous function $\\mbf{y}=f(\\mbf{x})$ where $\\mbf{x} \\in \\mathbb{R}^d$ and $\\mbf{y} \\in \\mathbb{R}^k$, considering only a bounded region of $\\mbf{x}$, **there exists** a single-hidden-layer NN$_\\theta$ with a **finite number of neurons/units in the hidden layer**, such that:\n", + "Given a continuous function $\\mbf{y}=f(\\mbf{x})$ where $\\mbf{x} \\in \\mathbb{R}^d$ and $\\mbf{y} \\in \\mathbb{R}^k$, considering only a bounded region of $\\mbf{x}$, **there exists** a single-hidden-layer $NN_\\theta$ with a **finite number of neurons/units in the hidden layer**, such that:\n", "\n", "$$\\vert f(\\mbf{x}) - NN_\\theta(\\mbf{x}) \\vert \\le \\epsilon $$\n", "

\n", diff --git a/AA2324/course/15_backprop_jacobians/15_backprop_jacobians.ipynb b/AA2324/course/15_backprop_jacobians/15_backprop_jacobians.ipynb index a7d0316..22e3d46 100644 --- a/AA2324/course/15_backprop_jacobians/15_backprop_jacobians.ipynb +++ b/AA2324/course/15_backprop_jacobians/15_backprop_jacobians.ipynb @@ -3179,11 +3179,11 @@ }, "source": [ "# 🏁 END of the LINE 🏁\n", - "### Soon it will be your turn on 13 June 2023\n", + "### Soon it will be your turn on 13 June 2024\n", "\n", "## Do not worry there are also\n", - "- Exam session on **6 July 2023**\n", - "- Exam session on **14 September 2023**\n", + "- Exam session on **16 July 2024**\n", + "- Exam session on **18 September 2024**\n", "\n", "[Million Dollar 🤑 link](https://iacopomasi.github.io/AI-ML-Unit-2/AA2122/exams.html)" ] From 48ce28e81ee3b134cd8fb5f59723d02879aeab93 Mon Sep 17 00:00:00 2001 From: SmashingBumpkin <66125276+SmashingBumpkin@users.noreply.github.com> Date: Tue, 4 Jun 2024 10:29:04 +0200 Subject: [PATCH 2/2] Minor changes allowing 5 equations to be parsed correctly Previously unable to parse bold equations, and some incorrect formatting breaking inline equations. --- .../06_clustering_gaussian_MLE.ipynb | 2 +- .../11_regression_lsq_poly/11_regression_lsq_poly.ipynb | 9 +++++---- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/AA2324/course/06_clustering_gaussian_MLE/06_clustering_gaussian_MLE.ipynb b/AA2324/course/06_clustering_gaussian_MLE/06_clustering_gaussian_MLE.ipynb index 20f14bf..3722962 100644 --- a/AA2324/course/06_clustering_gaussian_MLE/06_clustering_gaussian_MLE.ipynb +++ b/AA2324/course/06_clustering_gaussian_MLE/06_clustering_gaussian_MLE.ipynb @@ -828,7 +828,7 @@ "source": [ "# The Maximum Likelihood Principle\n", "\n", - "This has a Bayesian interpretation which can be helpful to think about. Suppose that we have a model with parameters $\\boldsymbol{\\theta}\\doteq\\mu,\\Sigma$ and a collection of data examples $X=\\{\\mbf{x}_1,\\ldots,\\mbf{x}_N \\}$. \n", + "This has a Bayesian interpretation which can be helpful to think about. Suppose that we have a model with parameters $\\boldsymbol{\\theta}\\doteq\\mu,\\Sigma$ and a collection of data examples $X=\\{\\mbf{x}_1,\\ldots,\\mbf{x}_N \\}$.\n", "\n", "If we want to find the **most likely value for the parameters of our model, given the data**, that means we want to find\n", "\n", diff --git a/AA2324/course/11_regression_lsq_poly/11_regression_lsq_poly.ipynb b/AA2324/course/11_regression_lsq_poly/11_regression_lsq_poly.ipynb index 581f5db..3393e13 100644 --- a/AA2324/course/11_regression_lsq_poly/11_regression_lsq_poly.ipynb +++ b/AA2324/course/11_regression_lsq_poly/11_regression_lsq_poly.ipynb @@ -2152,7 +2152,8 @@ "# Gradient Descent and [Stochastic] GD\n", "\n", "1. **Initialization - Very Important if the function is not strictly convex** \n", - "$$\\bmf{\\theta} \\doteq \\mbf{0}^T$$ Set it to all zeros or random initialization from a distribution.\n", + "$$\\bmf{\\theta} \\doteq \\mbf{0}^T$$\n", + "Set it to all zeros or random initialization from a distribution.\n", "2. Repeat until **convergence**:\n", " - Compute the gradient of the loss wrt the parameters $\\bmf{\\theta}$ given **all the training set**\n", " - Take a small step in the opposite direction of steepest ascent **(so steepest descent).**

\n", @@ -2750,7 +2751,7 @@ "# Now we can still solve it with LS but $m=2$\n", "\n", "\n", - "We can have another dimensionality $m$ instead of $d$ by using **Basis Functions $\\bmf{\\phi}(\\mbf{x})$**.\n", + "We can have another dimensionality $m$ instead of $d$ by using **Basis Functions** $\\bmf{\\phi}(\\mbf{x})$ .\n", "\n", "With $\\bmf{\\phi}(\\mbf{x} = [1,\\phi(x_1),\\ldots,\\phi(x_m)]$ and $\\mbf{\\theta} = [\\theta_0,\\theta_1,\\ldots,\\theta_m]$, we have:\n", "\n", @@ -2837,7 +2838,7 @@ "# Now we can still solve it with LS but $m=3$\n", "\n", "\n", - "We can have another dimensionality $m$ instead of $d$ by using **Basis Functions $\\bmf{\\phi}(\\mbf{x})$**.\n", + "We can have another dimensionality $m$ instead of $d$ by using **Basis Functions** $\\bmf{\\phi}(\\mbf{x})$ .\n", "\n", "With $\\bmf{\\phi}(\\mbf{x} = [1,\\phi(x_1),\\ldots,\\phi(x_m)]$ and $\\mbf{\\theta} = [\\theta_0,\\theta_1,\\ldots,\\theta_m]$, we have:\n", "\n", @@ -2924,7 +2925,7 @@ "# We can analyze what happens in function of $m$\n", "\n", "\n", - "We can have another dimensionality $m$ instead of $d$ by using **Basis Functions $\\bmf{\\phi}(\\mbf{x})$**.\n", + "We can have another dimensionality $m$ instead of $d$ by using **Basis Functions** $\\bmf{\\phi}(\\mbf{x})$.\n", "\n", "With $\\bmf{\\phi}(\\mbf{x} = [1,\\phi(x_1),\\ldots,\\phi(x_m)]$ and $\\mbf{\\theta} = [\\theta_0,\\theta_1,\\ldots,\\theta_m]$, we have:\n", "\n",