PMF_Recommender_Systems.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "PMF_Recommender_Systems.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sE0imG0HLDgf",
        "colab_type": "text"
      },
      "source": [
        "# PMF for Recommender Systems\n",
        "\n",
        "This is an implementation for Probabilistic Matrix Factorisation. We have made use of a subset of the IMDB dataset for training purposes."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8-65Kyq2Kv8R",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "from sklearn.metrics import mean_squared_error"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pDzg9SMk_iW8",
        "colab_type": "text"
      },
      "source": [
        "Let's now download our dataset. We will also unzip it for further use."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iUB7uFWdQzF7",
        "colab_type": "code",
        "outputId": "32574479-ff4f-4e26-9fc0-305b20e19fd5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 334
        }
      },
      "source": [
        "!wget http://files.grouplens.org/datasets/movielens/ml-latest-small.zip\n",
        "!unzip ml-latest-small.zip"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2020-05-05 16:44:24--  http://files.grouplens.org/datasets/movielens/ml-latest-small.zip\n",
            "Resolving files.grouplens.org (files.grouplens.org)... 128.101.65.152\n",
            "Connecting to files.grouplens.org (files.grouplens.org)|128.101.65.152|:80... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 978202 (955K) [application/zip]\n",
            "Saving to: ‘ml-latest-small.zip’\n",
            "\n",
            "ml-latest-small.zip 100%[===================>] 955.28K   974KB/s    in 1.0s    \n",
            "\n",
            "2020-05-05 16:44:25 (974 KB/s) - ‘ml-latest-small.zip’ saved [978202/978202]\n",
            "\n",
            "Archive:  ml-latest-small.zip\n",
            "   creating: ml-latest-small/\n",
            "  inflating: ml-latest-small/links.csv  \n",
            "  inflating: ml-latest-small/tags.csv  \n",
            "  inflating: ml-latest-small/ratings.csv  \n",
            "  inflating: ml-latest-small/README.txt  \n",
            "  inflating: ml-latest-small/movies.csv  \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PCubprYO_zIS",
        "colab_type": "text"
      },
      "source": [
        "Let's take a look at part of the ratings file to get an idea of the columns available."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dbWjyrOEC5G1",
        "colab_type": "code",
        "outputId": "25cce28a-4357-4c26-f687-dd49d5f4fa9b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 195
        }
      },
      "source": [
        "df = pd.read_csv('ml-latest-small/ratings.csv')\n",
        "df.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>userId</th>\n",
              "      <th>movieId</th>\n",
              "      <th>rating</th>\n",
              "      <th>timestamp</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>964982703</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>4.0</td>\n",
              "      <td>964981247</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>6</td>\n",
              "      <td>4.0</td>\n",
              "      <td>964982224</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>47</td>\n",
              "      <td>5.0</td>\n",
              "      <td>964983815</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1</td>\n",
              "      <td>50</td>\n",
              "      <td>5.0</td>\n",
              "      <td>964982931</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   userId  movieId  rating  timestamp\n",
              "0       1        1     4.0  964982703\n",
              "1       1        3     4.0  964981247\n",
              "2       1        6     4.0  964982224\n",
              "3       1       47     5.0  964983815\n",
              "4       1       50     5.0  964982931"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XfASJGv4AWHY",
        "colab_type": "text"
      },
      "source": [
        "As part of the dataset, there is also another file that contains movie information, which includes the movie id, the title, the genre, among others. We can INNER JOIN the ratings and movies datasets to gain access to all data we will need for our analysis."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RQqW0Mzh_-Vd",
        "colab_type": "code",
        "outputId": "9a0cb934-2540-47db-8da0-5f16f4e927c5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 195
        }
      },
      "source": [
        "df_movies = pd.read_csv('ml-latest-small/movies.csv')\n",
        "df_join = pd.merge(df_movies, df, how='inner', on='movieId')\n",
        "df_join.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>movieId</th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "      <th>userId</th>\n",
              "      <th>rating</th>\n",
              "      <th>timestamp</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "      <td>1</td>\n",
              "      <td>4.0</td>\n",
              "      <td>964982703</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "      <td>5</td>\n",
              "      <td>4.0</td>\n",
              "      <td>847434962</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "      <td>7</td>\n",
              "      <td>4.5</td>\n",
              "      <td>1106635946</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "      <td>15</td>\n",
              "      <td>2.5</td>\n",
              "      <td>1510577970</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1</td>\n",
              "      <td>Toy Story (1995)</td>\n",
              "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
              "      <td>17</td>\n",
              "      <td>4.5</td>\n",
              "      <td>1305696483</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   movieId             title  ... rating   timestamp\n",
              "0        1  Toy Story (1995)  ...    4.0   964982703\n",
              "1        1  Toy Story (1995)  ...    4.0   847434962\n",
              "2        1  Toy Story (1995)  ...    4.5  1106635946\n",
              "3        1  Toy Story (1995)  ...    2.5  1510577970\n",
              "4        1  Toy Story (1995)  ...    4.5  1305696483\n",
              "\n",
              "[5 rows x 6 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "STnXs_BhAnsm",
        "colab_type": "text"
      },
      "source": [
        "Let's now prepare our sparse review matrix. For this purpose, we first need to find out how many users and movies we have in our dataset. Additionally, we will set S to 5 dimensions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BDoGa6i7OFy_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def get_ratings_matrix(df, train_size=0.75):\n",
        "    user_to_row = {}\n",
        "    movie_to_column = {}\n",
        "    df_values = df.values\n",
        "    n_dims = 10\n",
        "    parameters = {}\n",
        "    \n",
        "    uniq_users = np.unique(df_values[:, 3])\n",
        "    uniq_movies = np.unique(df_values[:, 0])\n",
        "\n",
        "    for i, user_id in enumerate(uniq_users):\n",
        "        user_to_row[user_id] = i\n",
        "\n",
        "    for j, movie_id in enumerate(uniq_movies):\n",
        "        movie_to_column[movie_id] = j\n",
        "    \n",
        "    n_users = len(uniq_users)\n",
        "    n_movies = len(uniq_movies)\n",
        "    \n",
        "    R = np.zeros((n_users, n_movies))\n",
        "    \n",
        "    df_copy = df.copy()\n",
        "    train_set = df_copy.sample(frac=train_size, random_state=0)\n",
        "    test_set = df_copy.drop(train_set.index)\n",
        "    \n",
        "    for index, row in train_set.iterrows():\n",
        "        i = user_to_row[row.userId]\n",
        "        j = movie_to_column[row.movieId]\n",
        "        R[i, j] = row.rating\n",
        "\n",
        "    return R, train_set, test_set, n_dims, n_users, n_movies, user_to_row, movie_to_column"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bKIg5iNe5SKn",
        "colab_type": "text"
      },
      "source": [
        "Let's now call this function and retrieve all necessary parameters."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HBjrbs8VD6_2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "R, train_set, test_set, n_dims, n_users, n_movies, user_to_row, movie_to_column = get_ratings_matrix(df_join, 0.8)\n",
        "parameters = {}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2C8nnfseBTOM",
        "colab_type": "text"
      },
      "source": [
        "We will now initialise our parameters. First the $V$ matrix can be initialised randomly using the following distribution:\n",
        "\n",
        "\\begin{equation}\n",
        "\\large\n",
        "V \\sim \\mathcal N\\left(0, \\frac {1} {\\lambda_V}\\right)\n",
        "\\end{equation}\n",
        "\n",
        "Let's remember that:\n",
        "\n",
        "\\begin{equation}\n",
        "\\large\n",
        "U \\in \\mathbb R^{D\\times N}, \\qquad V \\in \\mathbb R^{D\\times M}\n",
        "\\end{equation}\n",
        "\n",
        "Where $N$ is __n_users__, $M$ is __n_movies__ and $D$ is __n_dims__."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HaM_fnR0Nd_i",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def initialize_parameters(lambda_U, lambda_V):\n",
        "    U = np.zeros((n_dims, n_users), dtype=np.float64)\n",
        "    V = np.random.normal(0.0, 1.0 / lambda_V, (n_dims, n_movies))\n",
        "    \n",
        "    parameters['U'] = U\n",
        "    parameters['V'] = V\n",
        "    parameters['lambda_U'] = lambda_U\n",
        "    parameters['lambda_V'] = lambda_V"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ntiPL_EnCLe-",
        "colab_type": "text"
      },
      "source": [
        "Let's now implement the function that updates U and V. The elements of both matrices can be updated using the following expressions:\n",
        "\n",
        "\\begin{equation}\n",
        "\\large\n",
        "U_i=\\left[\\left(V_jV_j^T\\right)_{j\\in\\Omega_{U_i}}+\\lambda_UI\\right]^{-1}\\left(R_{ij}V_j^T\\right)_{j\\in\\Omega_{U_i}}\n",
        "\\end{equation}\n",
        "\n",
        "\\begin{equation}\n",
        "\\large\n",
        "V_j=\\left[\\left(U_iU_i^T\\right)_{i\\in\\Omega_{V_j}}+\\lambda_VI\\right]^{-1}\\left(R_{ij}U_i^T\\right)_{i\\in\\Omega_{V_j}}\n",
        "\\end{equation}"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DjKB4TzFkoJx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def update_parameters():\n",
        "    U = parameters['U']\n",
        "    V = parameters['V']\n",
        "    lambda_U = parameters['lambda_U']\n",
        "    lambda_V = parameters['lambda_V']\n",
        "    \n",
        "    for i in range(n_users):\n",
        "        V_j = V[:, R[i, :] > 0]\n",
        "        U[:, i] = np.dot(np.linalg.inv(np.dot(V_j, V_j.T) + lambda_U * np.identity(n_dims)), np.dot(R[i, R[i, :] > 0], V_j.T))\n",
        "        \n",
        "    for j in range(n_movies):\n",
        "        U_i = U[:, R[:, j] > 0]\n",
        "        V[:, j] = np.dot(np.linalg.inv(np.dot(U_i, U_i.T) + lambda_V * np.identity(n_dims)), np.dot(R[R[:, j] > 0, j], U_i.T))\n",
        "        \n",
        "    parameters['U'] = U\n",
        "    parameters['V'] = V"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hQ5in2inJApx",
        "colab_type": "text"
      },
      "source": [
        "Now let's implement the Log-a posteriori:\n",
        "\n",
        "\\begin{equation}\n",
        "\\large\n",
        "L=-\\frac 1 2 \\left(\\sum_{i=1}^N\\sum_{j=1}^M(R_{ij}-U_i^TV_j)_{(i,j) \\in \\Omega_{R_{ij}}}^2+\\lambda_U\\sum_{i=1}^N\\|U_i\\|_{Fro}^2+\\lambda_V\\sum_{j=1}^M\\|V_j\\|_{Fro}^2\\right)\n",
        "\\end{equation}"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5PUgHUp2MeZI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def log_a_posteriori():\n",
        "    lambda_U = parameters['lambda_U']\n",
        "    lambda_V = parameters['lambda_V']\n",
        "    U = parameters['U']\n",
        "    V = parameters['V']\n",
        "    \n",
        "    UV = np.dot(U.T, V)\n",
        "    R_UV = (R[R > 0] - UV[R > 0])\n",
        "    \n",
        "    return -0.5 * (np.sum(np.dot(R_UV, R_UV.T)) + lambda_U * np.sum(np.dot(U, U.T)) + lambda_V * np.sum(np.dot(V, V.T)))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "68fmuPQrPRnU",
        "colab_type": "text"
      },
      "source": [
        "The __predict__ function allows us to predict the rating value given the __user_id__ and the __movie_id__ parameters. The value has been scaled within the range 0-5"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "neTN8ejbQ5dg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def predict(user_id, movie_id):\n",
        "    U = parameters['U']\n",
        "    V = parameters['V']\n",
        "    \n",
        "    r_ij = U[:, user_to_row[user_id]].T.reshape(1, -1) @ V[:, movie_to_column[movie_id]].reshape(-1, 1)\n",
        "\n",
        "    max_rating = parameters['max_rating']\n",
        "    min_rating = parameters['min_rating']\n",
        "\n",
        "    return 0 if max_rating == min_rating else ((r_ij[0][0] - min_rating) / (max_rating - min_rating)) * 5.0"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7Rt2ol1j7OJ0",
        "colab_type": "text"
      },
      "source": [
        "The __evaluate__ function will calculate the __RMSE__ of the model given a dataset (train or test)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jnNGbogWh6OR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def evaluate(dataset):\n",
        "    ground_truths = []\n",
        "    predictions = []\n",
        "    \n",
        "    for index, row in dataset.iterrows():\n",
        "        ground_truths.append(row.loc['rating'])\n",
        "        predictions.append(predict(row.loc['userId'], row.loc['movieId']))\n",
        "    \n",
        "    return mean_squared_error(ground_truths, predictions, squared=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6R1ePeME7qSW",
        "colab_type": "text"
      },
      "source": [
        "For the purposes of scaling, we need the maximum and minimum rating values."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YNIlELvx0M0W",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def update_max_min_ratings():\n",
        "    U = parameters['U']\n",
        "    V = parameters['V']\n",
        "\n",
        "    R = U.T @ V\n",
        "    min_rating = np.min(R)\n",
        "    max_rating = np.max(R)\n",
        "\n",
        "    parameters['min_rating'] = min_rating\n",
        "    parameters['max_rating'] = max_rating"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AZGq6J9pDl5O",
        "colab_type": "text"
      },
      "source": [
        "The __train__ function implements the code necessary for training the model as well as recording the __RMSE__ values on the training and testing sets."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VYwTD_nBNO5F",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def train(n_epochs):\n",
        "    initialize_parameters(0.3, 0.3)\n",
        "    log_aps = []\n",
        "    rmse_train = []\n",
        "    rmse_test = []\n",
        "\n",
        "    update_max_min_ratings()\n",
        "    rmse_train.append(evaluate(train_set))\n",
        "    rmse_test.append(evaluate(test_set))\n",
        "    \n",
        "    for k in range(n_epochs):\n",
        "        update_parameters()\n",
        "        log_ap = log_a_posteriori()\n",
        "        log_aps.append(log_ap)\n",
        "\n",
        "        if (k + 1) % 10 == 0:\n",
        "            update_max_min_ratings()\n",
        "\n",
        "            rmse_train.append(evaluate(train_set))\n",
        "            rmse_test.append(evaluate(test_set))\n",
        "            print('Log p a-posteriori at iteration', k + 1, ':', log_ap)\n",
        "\n",
        "    update_max_min_ratings()\n",
        "\n",
        "    return log_aps, rmse_train, rmse_test"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SJbozBbQPk9A",
        "colab_type": "text"
      },
      "source": [
        "Let's train our model!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XeH5sRNp7eHA",
        "colab_type": "code",
        "outputId": "8e0cb783-9b32-41aa-9a4a-908aac9d613d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 281
        }
      },
      "source": [
        "log_ps, rmse_train, rmse_test = train(150)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Log p a-posteriori at iteration 10 : -47315.01695753759\n",
            "Log p a-posteriori at iteration 20 : -26049.43854684034\n",
            "Log p a-posteriori at iteration 30 : -18661.49545924417\n",
            "Log p a-posteriori at iteration 40 : -15071.180100170577\n",
            "Log p a-posteriori at iteration 50 : -13079.762498741024\n",
            "Log p a-posteriori at iteration 60 : -11893.64153224053\n",
            "Log p a-posteriori at iteration 70 : -11160.963307065578\n",
            "Log p a-posteriori at iteration 80 : -10704.998944575167\n",
            "Log p a-posteriori at iteration 90 : -10404.727479137038\n",
            "Log p a-posteriori at iteration 100 : -10201.562115083421\n",
            "Log p a-posteriori at iteration 110 : -10056.94029740754\n",
            "Log p a-posteriori at iteration 120 : -9947.130125510244\n",
            "Log p a-posteriori at iteration 130 : -9861.603527161862\n",
            "Log p a-posteriori at iteration 140 : -9797.336507154727\n",
            "Log p a-posteriori at iteration 150 : -9750.831386678909\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TKzp4hukECUE",
        "colab_type": "text"
      },
      "source": [
        "Now let's take a look at some graphs"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yRr_wko07e37",
        "colab_type": "code",
        "outputId": "46313b0b-5d4a-4448-8638-12ef81baf4c2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "_, (ax1, ax2) = plt.subplots(1, 2, figsize=(20, 10))\n",
        "plt.title('Training results')\n",
        "ax1.plot(np.arange(len(log_ps)), log_ps, label='MAP')\n",
        "ax1.legend()\n",
        "\n",
        "ax2.plot(np.arange(len(rmse_train)), rmse_train, label='RMSE train')\n",
        "ax2.plot(np.arange(len(rmse_test)), rmse_test, label='RMSE test')\n",
        "ax2.legend()\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABJ8AAAJOCAYAAAAZP6bBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzde3xcdZ3/8fdnbplLMknTpjdaaORmaSkBKhQVdQEBERZ2RcXFtVLcelnEHz/97bLiivoT1wv8/C0/6ipeWWULCLqyqFxXFHRFWy2ltHQpEOyNNk2bmVxmJnP5/v6YkzRJE3qdOZPk9Xw84pzzPd9zzmeCf5y+8/1+jznnBAAAAAAAAFRCwO8CAAAAAAAAMHERPgEAAAAAAKBiCJ8AAAAAAABQMYRPAAAAAAAAqBjCJwAAAAAAAFQM4RMAAAAAAAAqhvAJAAAAACYJM/u5mS090n1rkZnNMzNnZiG/awEmO3PO+V0DAAAAAGAMZtYzZDcuKSep6O1/0Dl3Z/Wrqn1mNk/SS5LCzrmCmT0u6QfOuW/5WRcwGZEAAwAAAEANc87VD2ybWbukDzjnHh3Zz8xCzrlCNWs7EsZr3QAOHNPuAAAAAGAcMrO3mNkWM/t7M3tF0nfNbIqZPWBmHWa2x9ueM+Scx83sA972+83sSTO72ev7kpm97RD7tprZr8ys28weNbMVZvaDg6g7YGbXm9kLZtZpZveYWbPXP2pmP/Dau8zs92Y2wzvWbmbnDbn2Z0a7r5ndJOlsSbeZWY+Z3WZlXzWznWaWNrNnzGzh4f1XATAawicAAAAAGL9mSmqWdIyk5Sr/G++73v7RkjKSbnuV88+UtFHSNElflvRtM7ND6Ptvkn4naaqkz0j664Os+6OSLpP0ZkmzJe2RtMLru1RSo6S53vU/5H2vA+acu0HSE5Kucc7VO+eukXS+pDdJOsG7/rskdR7MdQEcGMInAAAAABi/SpJudM7lnHMZ51ync+4+51yfc65b0k0qBzpjedk5903nXFHSHZJmSZpxMH3N7GhJr5P0aedcv3PuSUn3H0zdKgdKNzjntjjncioHWJd7i4XnVQ6djnPOFZ1zq51z6f3/avYrL6lB0mtVXg95g3Nu+xG4LoARCJ8AAAAAYPzqcM5lB3bMLG5m3zCzl80sLelXkprMLDjG+a8MbDjn+rzN+oPsO1vS7iFtkrT5YOpWeQTUj71pdV2SNqi8qPoMSd+X9JCku8xsm5l92czC+7n+fjnn/lPlUWErJO00s9vNLHm41wWwL8InAAAAABi/Rr6+/OOSTpR0pnMuqfK0MkkaayrdkbBdUrOZxYe0zd3POSPr3izpbc65piE/UefcVudc3jn3WefcSZJeL+liSe/zzutV+Q2AA2YexD3lnLvVOXe6pJNUnn73v/ZTN4BDQPgEAAAAABNHg8rrIXV5C3bfWOkbOudelrRK0mfMLGJmZ0m65CAv83VJN5nZMZJkZi1mdqm3/WdmdrI3eiut8nS5knfeGklXmFnYzBZLuvxV7rFD0msGdszsdWZ2pjeKqldSdsh1ARxBhE8AAAAAMHH8X0kxSbsk/VbSg1W675WSzlJ5we7PS7pbUu4gzv9nldeJetjMulWu/Uzv2ExJ96ocPG2Q9EuVp+JJ0j9KOlblBco/q/LC5692j8u9t/XdKikp6ZveuS97tX/lIGoGcIDMuX1GHgIAAAAAcMjM7G5JzznnKj7yCkDtY+QTAAAAAOCweFPYjjWzgJldKOlSSf/ud10AakPI7wIAAAAAAOPeTEk/kjRV0hZJH3bO/dHfkgDUCqbdAQAAAAAAoGKYdgcAAAAAAICKmXTT7qZNm+bmzZvndxkAAKCCVq9evcs51+J3HdiLZzAAACa2V3v+mnTh07x587Rq1Sq/ywAAABVkZi/7XQOG4xkMAICJ7dWev5h2BwAAAAAAgIohfAIAAAAAAEDFED4BAAAAAACgYibdmk8AAEwW+XxeW7ZsUTab9buUiolGo5ozZ47C4bDfpQAAgBowGZ5//HYoz1+ETwAATFBbtmxRQ0OD5s2bJzPzu5wjzjmnzs5ObdmyRa2trX6XAwAAasBEf/7x26E+fzHtDgCACSqbzWrq1KkT9sHLzDR16lT+sgkAAAZN9Ocfvx3q8xfhEwAAE9hEf/Ca6N8PAAAcPJ4PKutQfr+ETwAAAAAAAKiYCRE+mdmFZrbRzDaZ2fV+1wMAAMrMTO9973sH9wuFglpaWnTxxRcP63fZZZdpyZIlw9o+85nP6KijjlJbW5sWLlyo+++/vyo1AwAAHI5gMDj4/HLJJZeoq6tLktTe3i4z06c+9anBvrt27VI4HNY111wjSdq4caPe8pa3qK2tTfPnz9fy5cslSY8//rgaGxvV1tY2+PPoo4/uc+8vfOELh1TzBz7wAa1fv/6Qzj0Q4z58MrOgpBWS3ibpJEnvMbOT/K0KAABIUiKR0Lp165TJZCRJjzzyiI466qhhfbq6urR69WqlUim9+OKLw45dd911WrNmjX74wx9q2bJlKpVKVasdAADgUMRiMa1Zs0br1q1Tc3OzVqxYMXistbVVP/3pTwf3f/jDH2rBggWD+9dee+3g88+GDRv00Y9+dPDY2WefrTVr1gz+nHfeefvce6zwyTn3qs9R3/rWt3TSSZWLUsZ9+CTpDEmbnHMvOuf6Jd0l6VKfawIAAJ6LLrpo8CFr5cqVes973jPs+I9+9CNdcskluuKKK3TXXXeNeo358+crFApp165dFa8XAADgSDnrrLO0devWwf14PK758+dr1apVkqS7775b73rXuwaPb9++XXPmzBncP/nkkw/4Xtdff70ymYza2tp05ZVXqr29XSeeeKLe9773aeHChdq8ebM+/OEPa/HixVqwYIFuvPHGwXPf8pa3DNZUX1+vG264QaeccoqWLFmiHTt2HPL3HxA67Cv47yhJm4fsb5F05tAOZrZc0nJJOvroo6tXGQAANeKz//Gs1m9LH9FrnjQ7qRsvWbDffldccYU+97nP6eKLL9batWu1bNkyPfHEE4PHV65cqU9/+tOaMWOG3vGOd+iTn/zkPtd46qmnFAgE1NLSckS/AwAAmLj8fP6RpGKxqMcee0xXX331sPaBP7jNmDFDwWBQs2fP1rZt2ySVR32fc845ev3rX6/zzz9fV111lZqamiRJTzzxhNra2gavc9999+nYY48d3P/iF7+o2267TWvWrJFUnub3/PPP64477hhc3uCmm25Sc3OzisWizj33XK1du1aLFi0aVl9vb6+WLFmim266SX/3d3+nb37zm8OmCh6KiTDyab+cc7c75xY75xbz0AoAQHUtWrRI7e3tWrlypS666KJhx3bs2KHnn39eb3zjG3XCCScoHA5r3bp1g8e/+tWvqq2tTZ/4xCd099138/YaAABQ8wZGH82cOVM7duzQW9/61mHHL7zwQj3yyCO666679O53v3vYsauuukobNmzQO9/5Tj3++ONasmSJcrmcpH2n3Q0NnsZyzDHHDFtX85577tFpp52mU089Vc8+++yo6zxFIpHB9TlPP/10tbe3H+yvYB8TYeTTVklzh+zP8doAAIDnQP9CVyl//ud/rk984hN6/PHH1dnZOdh+zz33aM+ePWptbZUkpdNprVy5UjfddJOk8l//PvGJT/hSMwAAGN/8ev4ZWPOpr69PF1xwgVasWKFrr7128HgkEtHpp5+uW265RevXr9/npSqzZ8/WsmXLtGzZMi1cuHDYH+YOViKRGNx+6aWXdPPNN+v3v/+9pkyZove///3KZrP7nBMOhwf/4BcMBlUoFA75/gMmwsin30s63sxazSwi6QpJvA4HAIAasmzZMt144437rFuwcuVKPfjgg2pvb1d7e7tWr1495rpPAAAA40k8Htett96qW265ZZ8A5+Mf/7i+9KUvqbm5eVj7gw8+qHw+L0l65ZVX1NnZuc/LWl5NOBwePH+kdDqtRCKhxsZG7dixQz//+c8P8hsdunEfPjnnCpKukfSQpA2S7nHOPetvVQAAYKg5c+YM+4ufVF6H4OWXXx42FLy1tVWNjY166qmnql0iAADAEXfqqadq0aJFWrly5bD2BQsWaOnSpfv0f/jhh7Vw4UKdcsopuuCCC/SVr3xFM2fOlLR3zaeBn3vvvXef85cvX65Fixbpyiuv3OfYKaecolNPPVWvfe1r9Vd/9Vd6wxvecIS+5f6Zc65qN6sFixcvdgMruAMAMJFt2LBB8+fP97uMihvte5rZaufcYp9Kwih4BgMAVMNkef7x28E+f437kU8AAAAAAACoXYRPAAAAAAAAqBjCJwAAJrCJPr1+on8/AACAiYDwCQCACSoajaqzs3PCBjTOOXV2dioajfpdCgAAAF5FyO8CAABAZcyZM0dbtmxRR0eH36VUTDQa1Zw5c/wuAwAAAK+C8AkAalCp5FRyTiUn77O8XSw5uVG2S855+962G96v5MrHikOu67xzBreHXEtOcnIqlSQnDV5LKl/HSYPXHDjuvHPKNXhtw44NP0dD6igf2zuFauB7uCHbGnKt0pB7uRH9BraHHR9xjobVMlDHkO83ou/ANb0e3hfY+9337TO8XUPOHezzKvfQPu2vfg9peH0j77F3e/Dyo7YP/C737T+yfeQ99p6vEf1e7R5f+IuTteQ1UwUcjtzPbpB7ZZ2iy37idykAAGAMhE8Aas5AKJIvOvUXS8oXSyoUnfLFkvoPYDtfLKm/UFKh5FQoOZW8z2KpNGJ/5HGnQqmkYkkH3LdUkndOObwpFMvHRu7vEySVRu4PD5lwcMykgJnM2zZZ+dPbDphk3nEN9DXJtHdbGt6mwWsN3MOG3W9gd+Be3qUH+9ng/+zbPtq5A/fYe7+x76FRrzX2PYbXUd4wBfb28X5PQy49ej0j2vf2Hfk7GKXuIfcYKKm+jscQHL7VG1/SCemnxeRLAECtCAaDOvnkk1UoFNTa2qrvf//7ampqUnt7u1pbW3XDDTfo85//vCRp165dmjVrlj74wQ/qtttu08aNG/XBD35QXV1dyuVyOvvss3X77bfr8ccf16WXXqrW1tbB+9x8880677zzht37C1/4gj75yU8eUt3f+973dP7552v27NmH/uXHwFMfgH2USk59+aL6+gvK9BeVzZeUKxSVK5SU87aHt3mfBa8tX1LW+xxsG3LuQN/+QkmFYkn9XmhUKJYGA6dqMJNCAVMwYAoFAgqYFAoGvH1TwEyhoI2yH1DQpFCg3LcuHCofG7hW0IbsBxQMSMGAyawcggRtYNsUDAyEH96xEf0CgfI/3oNef/P6BLw+gSHbZuadI69t9H7DtvfpszdY2BvQDA9yBoMe2xt4DN0OjDxnSFAxEBDJhoZFe8+R7RskDQ+Chtc4cH0Ak1cxklS96/W7DAAABsViMa1Zs0aStHTpUq1YsUI33HCDJKm1tVU//elPB8OnH/7wh1qwYMHguddee62uu+46XXrppZKkZ555ZvDY2WefrQceeOBV73244dPChQsJnwDsK18sqSdbUE+uoHQ2r55sQd3efl9/OUAqfxaVGdjOF5Xp3xsuDR7PF9WbKyhXOPTwJxQw1YUCqgsHy5+hgOpCQUXD5c9EXUjNiYAioYAiwYDCwYBCwYAiQVM4GFA4FFA4sHc7FDBFQl6/EdvhYdewfbb3Bkd7A6XgwI8X7AAAxrdStElR9cvls7Iw458AALXlrLPO0tq1awf34/G45s+fr1WrVmnx4sW6++679a53vUvbtm2TJG3fvn3YepYnn3zyAd/r+uuvVyaTUVtbmxYsWKA777xTP/jBD3Trrbeqv79fZ555pr72ta9Jkq6++mqtWrVKZqZly5Zp7ty5WrVqla688krFYjH913/9l2Kx2BH6LRA+Ab4rFEtKZfLa05dXKtOvPb15dWXy6urrVzqTV3fOC5OyBXXn9oZL5fa8svkDC4oiwYBikaDikeDgZzwcUlM8otlNQ9oiIcXCA9tBxSIhRcMBRUNBRcNB1YX3BkrlkGlvuBTxgiQAAKrFoo2SpGz3HsWaZ/lcDQCgpvz8eumVZ/bf72DMPFl62xcPqGuxWNRjjz2mq6++elj7FVdcobvuukszZsxQMBjU7NmzB8On6667Tuecc45e//rX6/zzz9dVV12lpqYmSdITTzyhtra2wevcd999OvbYYwf3v/jFL+q2224bHHW1YcMG3X333fr1r3+tcDisj3zkI7rzzju1YMECbd26VevWrZMkdXV1qampSbfddptuvvlmLV68+NB/P2MgfAKOIOecevuL6uzJaVdPTh3d/erszWmX97mnrxwqdfXl1ZXpV1dvOVwai5lUHwmpIRpSfTSkhmhYTfGI5jbH1eDtN9TtPVZfF1LS61tfF1KiLqRYJKhYOKgwoRAAYAIKxMsP5L3pXYRPAICaMDD6aOvWrZo/f77e+ta3Djt+4YUX6h//8R81Y8YMvfvd7x527KqrrtIFF1ygBx98UD/5yU/0jW98Q08//bSkA5t2N9Rjjz2m1atX63Wve91gXdOnT9cll1yiF198UR/96Ef19re/Xeeff/5hfuP9I3wCDkCp5LSrN6cdqZy2pzJ6JZ3VjnRWnT395ZCpp38wcBprJFJjLKzmRERN8bCm1Ud03PR6NcXDaopFNCURVmOsHCxN8doa4+VgialhAACMLRyfIknKpDp9rgQAUHMOcITSkTaw5lNfX58uuOACrVixQtdee+3g8UgkotNPP1233HKL1q9fr/vvv3/Y+bNnz9ayZcu0bNkyLVy4cHCE0sFyzmnp0qX6p3/6p32OPf3003rooYf09a9/Xffcc4++853vHNI9DhThEyCpN1fQ5j192rw7oz/t7tP2roy2p7Pakcpqeyqrnd1Z5YvDX0EWDJiaExFNq6/TtPqIjp2W0NT6gf26we2Whjo1JyKMPAIAoALCDc2SpGzPbp8rAQBguHg8rltvvVWXXXaZPvKRjww79vGPf1xvfvOb1dzcPKz9wQcf1LnnnqtwOKxXXnlFnZ2dOuqoo/Tcc88d0D3D4bDy+bzC4bDOPfdcXXrppbruuus0ffp07d69W93d3UokEopEInrHO96hE088Ue9973slSQ0NDeru7j4yX34EwidMCs457Ujn9EJHj/60u0+bd/eVP/dktGV3nzp7+4f1j4YDmtUY04xknc5obdbMxqhmNUY1MxkttzfWaVqijlFJAAD4LFpfHvnU3034BACoPaeeeqoWLVqklStX6uyzzx5sX7BgwbC33A14+OGH9bGPfUzRaPklGl/5ylc0c+ZMPffcc/us+fSpT31Kl19++bDzly9frkWLFum0007TnXfeqc9//vM6//zzVSqVFA6HtWLFCsViMV111VUqlcqzdgZGRr3//e/Xhz70oYosOG7Ouf33mkAWL17sVq1a5XcZqJD+Qkkvd/bqhY4evdDRq007e8rbO3vU218c7BcKmI6aEtPcKXHNbY5pzpS45jbHdXRzXHOnxNSciPD6dgAYx8xstXPuyK+WiUNWqWew9pdf0rzvtunpRf+oU/7yE0f8+gCA8WXDhg2aP3++32VMeKP9nl/t+YuRTxiXnHPalspqw7a01m9Pa8P2tJ57pVt/2t2nYmlvoDq7Mapjp9frnYvn6tjp9Tp2WkJHT41rZjLKW9kAAJgAEsmpkqRS3x6fKwEAAGMhfELNc85pa1dGT29Oac3mPXpma0obtncrlclLKr8Rbt7UhF47s0EXL5ql46bX69iWerVOSyhRx//FAQCYyBrqE8q4iJRN+V0KAAAYA/8yR83J9Bf1xz/t0R/+tEdrNndpzeaUdvXkJEmRUEAnzUrq7Ytm6aRZSc2fldRrZzYQMgEAMElFw0HtUEKW6/K7FABAjXDOsYxKBR3K8k38ix2+680VtPrlPXrqpU499eJuPb2la/DNcse2JPTmE1rUNrdRbXOn6MSZDYqEmC4HAAD26rWEgv2VeTsPAGB8iUaj6uzs1NSpUwmgKsA5p87OzsEF0Q8U4ROqrlRyWr89rV88t1OP/3eHnt7cpULJKRgwnXxUo5a9sVVLWqfqtGOmqDEW9rtcAABQ43qDDYr0M+0OACDNmTNHW7ZsUUdHh9+lTFjRaFRz5sw5qHMIn1AVqUxeTz6/S7/YuFOPb+wYnEa3aE6jlr/pNTrzNVN1+jFTVM/0OQAAcJBywXrVF1hwHAAghcNhtba2+l0GRuBf+qiYdDavh5/doQfWbtOTz+9SoeSUjIb0phNa9GcnTtebTmhRS0Od32UCAIBxLhdKKpbd7HcZAABgDIRPOKJ6cwU9umGHHli7Xb/c2KH+YklHNcV09Rtb9daTZqhtbpNCQdZsAgAAR04hklS8r8fvMgAAwBgIn3DYnHNas7lLdz71Jz2wdpuy+ZJmJqN675JjdMkps9Q2t4mF3gAAQMUU6xpVr16pVJIC/JELAIBaQ/iEQ9aTK+gna7bqzt/+Seu3p5WIBPWXp83RZW1HafExUxQIEDgBAIAqiDYqICeXS8liU/yuBgAAjED4hIP2/I5ufe837fr3P25Vb39R82clddNfLNSlbUexYDgAAKi+aJMkqb+nS3WETwAA1BySAhywP/xpj/7l8Rf0yPodqgsFdPGi2bpyydE6lWl1AADAR4F4OXDqTe1SXQtvOAIAoNYQPmG/Vr+8R7c8vFG/eaFTTfGwPnbu8Vr6+nlqTkT8Lg0AAEDhRDl8ynR3+lwJAAAYDeETxrR+W1o3P7xR//ncTk2rj+hTb5+v95xxtBJMrQMAADUkUl8On7LpPT5XAgAARkOKgH10dOd0y8MbdfeqzUpGw/q7C0/U0rPmEToBAICaFEs2S5L6e3f7XAkAABgNaQIGFYolfe837fq/jz6vbL6oZW9o1bXnHK/GeNjv0gAAAMYUS06TJBV7GfkEAEAtInyCJOnZbSldf98zemZrSn92Yos+dfFJOral3u+yAABAlZlZVNKvJNWp/Kx4r3PuxhF93i/pK5K2ek23Oee+Vc06h2pINqroTKVsyq8SAADAqyB8muT6CyX982P/ra//8kVNiYe14q9O00Unz+TtdQAATF45Sec453rMLCzpSTP7uXPutyP63e2cu8aH+vaRjNUprYSU6fK7FAAAMArCp0msfVevrr3rj1q7JaXLT5+jT719vprivMEOAIDJzDnnJPV4u2Hvx/lX0f5Fw0HtVFzBHOETAAC1iPBpkvrxH7foUz9ep2DA9PX3nqYLF87yuyQAAFAjzCwoabWk4yStcM49NUq3d5jZmyT9t6TrnHObR7nOcknLJenoo4+uYMVSjzUo2N9d0XsAAIBDE/C7AFRXoVjSZ//jWV1399NaMLtRP/8fbyJ4AgAAwzjnis65NklzJJ1hZgtHdPkPSfOcc4skPSLpjjGuc7tzbrFzbnFLS0tFa84EE4rk0xW9BwAAODSMfJpEUn15XbPyD3ri+V1a9oZWffKi1yoUJH8EAACjc851mdkvJF0oad2Q9s4h3b4l6cvVrm2kbDCpusKf/C4DAACMgvBpkti8u09Lv/M7bd7Tpy+/Y5He9bq5fpcEAABqkJm1SMp7wVNM0lslfWlEn1nOue3e7p9L2lDlMvfRH04qlmHaHQAAtYjwaRLYtLNH7/3WU8rki/q3v1mi181r9rskAABQu2ZJusNb9ykg6R7n3ANm9jlJq5xz90u61sz+XFJB0m5J7/etWk8h0qBEb6/fZQAAgFEQPk1wz25L6X3f/p3MpLuWL9H8WUm/SwIAADXMObdW0qmjtH96yPY/SPqHata1P6W6JtWpX8pnpXDU73IAAMAQLPgzga3bmtJ7bv+t6kIB3fPBswieAADAxBVtLH9mU/7WAQAA9kH4NEG17+rV+7/7O9XXhXTPh87Sa1rq/S4JAACgYizWJEnK9XTupycAAKg2wqcJaGc6q/d953cqlpz+9eozNWdK3O+SAAAAKioYL4dPvWnCJwAAag1rPk0w3dm8ln7399rVk9O//c0SHTedEU8AAGDiC9eXX6iSTe/2uRIAADASI58mEOecPn7P03p+R7f+5b2nq21uk98lAQAAVEVd/RRJUq6H8AkAgFpD+DSBfP2XL+rh9Tv0DxfN15tPaPG7HAAAgKqJJadKkvI9e3yuBAAAjET4NEH8ZtMufeWh5/T2RbO07A3z/C4HAACgquLJ8rS7Yh/hEwAAtYbwaQJ4JZXVR1f+Ua3TEvrSOxbJzPwuCQAAoKqSiYT6XJ1cpsvvUgAAwAiET+Occ07X/2it+vqL+sZfn676OtaQBwAAk08yFlZKCVk25XcpAABgBMKnce7Hf9yqxzd26O8uPFHHTW/wuxwAAABf1IUC6lZcgRzhEwAAtYbwaRzr6M7pcw+s1+nHTNHSs+b5XQ4AAIBvzEy9Vq9QPu13KQAAYATCp3HsxvvXqa+/qC+9Y5ECAdZ5AgAAk1s22KBIvtvvMgAAwAiET+PUQ8++op8984o+du7xOm56vd/lAAAA+C4balC0QPgEAECtqVj4ZGafMbOtZrbG+7loyLF/MLNNZrbRzC4Y0n6h17bJzK4f0t5qZk957XebWcRrr/P2N3nH51Xq+9SSfLGkf/rZBp0wo17L3/Qav8sBAACoCflwg2KlHr/LAAAAI1R65NNXnXNt3s/PJMnMTpJ0haQFki6U9DUzC5pZUNIKSW+TdJKk93h9JelL3rWOk7RH0tVe+9WS9njtX/X6TXh3/X6z2jv7dP3bXqtwkMFrAAAAklSINCrueqVSye9SAADAEH4kF5dKuss5l3POvSRpk6QzvJ9NzrkXnXP9ku6SdKmZmaRzJN3rnX+HpMuGXOsOb/teSed6/Ses3lxB//zo8zqjtVl/duJ0v8sBAACoGaW6RgXkJN54BwBATal0+HSNma01s++Y2RSv7ShJm4f02eK1jdU+VVKXc64won3YtbzjKa//MGa23MxWmdmqjo6OI/PNfPKdJ1/Srp6crn/bazXBczYAAICDE2ssf2YJnwAAqCWHFT6Z2aNmtm6Un0sl/YukYyW1Sdou6ZYjUO8hcc7d7pxb7Jxb3NLS4lcZh213b7++8asXdcGCGTrt6Cn7PwEAAGASCcTKz0e57t0+V9GQZbQAACAASURBVAIAAIYKHc7JzrnzDqSfmX1T0gPe7lZJc4ccnuO1aYz2TklNZhbyRjcN7T9wrS1mFpLU6PWfkL7+yxfU11/Q/7rgRL9LAQAAqDnBRDl8ynTvVp3PtQAAgL0q+ba7WUN2/0LSOm/7fklXeG+qa5V0vKTfSfq9pOO9N9tFVF6U/H7nnJP0C0mXe+cvlfSTIdda6m1fLuk/vf4TTk+uoJVP/UkXL5qt46Y3+F0OAABAzYl44VO2e8L+LRIAgHHpsEY+7ceXzaxNkpPULumDkuSce9bM7pG0XlJB0t8654qSZGbXSHpIUlDSd5xzz3rX+ntJd5nZ5yX9UdK3vfZvS/q+mW2StFvlwGpCunfVZnXnClr2xla/SwEAAKhJdQ3NkqRczx6fKwEAAENVLHxyzv31qxy7SdJNo7T/TNLPRml/UeW34Y1sz0p65+FVWvtKJafv/aZdpx3dpLa5TX6XAwAAUJNiyfJ7Zwq9hE8AANSSSr/tDkfALzbuVHtnn656A6OeAAAAxtLQ0KiCC6jYR/gEAEAtIXwaB77763bNTEZ14cKZfpcCAABQs5KxiNKKS5kuv0sBAABDED7VuI2vdOvJTbv0vtcfo3CQ/1wAAABjScbCSruELJvyuxQAADAEaUaN+95v2hUNB/Se1x3tdykAAAA1rS4UUFoJBfoJnwAAqCWETzUsVyjqp2u36aKFszQlEfG7HAAAgJpmZuoL1Cvcn/a7FAAAMAThUw174r93KZ0t6JK22X6XAgAAMC5kQw2KFLr9LgMAAAxB+FTD7n96m6bEw3rjcdP8LgUAAGBcyIYaFCV8AgCgphA+1ai+/oIeWb9Dbzt5FguNAwAAHKBCOKl4qcfvMgAAwBCkGjXqP5/bqUy+qEsWMeUOAADgQBUijYooL+UzfpcCAAA8hE816j+e3qbpDXU6o7XZ71IAAADGDRdNljeyvPEOAIBaQfhUg9LZvH6xsUNvXzRLwYD5XQ4AAMC4YdGm8kamy99CAADAIMKnGvTwszvUXyjpklOYcgcAAHAwAvFy+JTr3e1zJQAAYADhUw16YO02zZkS06lzm/wuBQAAYFwJxadIkrJpwicAAGoF4VONyeaL+q8XOnXe/BkyY8odAADAwYg0lNfLzHZ3+lwJAAAYQPhUY/7w8h7lCiWdffw0v0sBAAAYd6L15fCpv2ePz5UAAIABhE815slNuxQMmM58zVS/SwEAABh3Yo3l8KnYS/gEAECtIHyqMb/etEunzm1SfV3I71IAAADGnWQ8rl5XpyJvuwMAoGYQPtWQVF9ea7em9Eam3AEAABySZCyslBIS4RMAADWD8KmG/OaFXXJOeuNxhE8AAACHIhkNK+0SslzK71IAAICH8KmGPLlplxKRoE6Z2+R3KQAAAONSNBxQtxIK9RM+AQBQKwifasivN+3SktdMVTjIfxYAAIBDYWbqC9QrlO/2uxQAAOAh5agRm3f3qb2zT29gyh0AAMBhyYYaVJdP+10GAADwED7ViN+8sEuSWGwcAADgMPWHGhQr9vhdBgAA8BA+1YgnN3WqpaFOx0+v97sUAACAcS0fTirueqVS0e9SAACACJ9qgnNO//VCp15/7FSZmd/lAAAAjGvFumR5I8ui4wAA1ALCpxqwI53Trp6c2njLHQAAwGFzUe+ZivAJAICaQPhUA57ZWn4wOvmoRp8rAQAAGP8sNhA+dflbCAAAkET4VBOe2ZpSwKSTZif9LgUAAGDcC3rhU3/vHp8rAQAAEuFTTVi3NaVjW+oVj4T8LgUAAGDcC9U3S5Ky6d0+VwIAACTCp5qwbmtKC5lyBwAAcETU1U+RJOV6CJ8AAKgFhE8+25nOamd3jvAJAADgCIk2lEc+9RM+AQBQEwiffMZi4wAAAEdWor5RBRdQsY8FxwEAqAWETz57ZmtKZtICFhsHAAA4IpLxiFJKqNTHguMAANQCwiefrdua1mumJZSoY7FxAACAIyEZDSvt4lI25XcpAABAhE++Y7FxAACAIysZCymlhAI5wicAAGoB4ZOPOrpzeiWdZb0nAACAIygWDqpHCYX6CZ8AAKgFhE8+Wret/EDEyCcAAIAjx8zUF2xQON/tdykAAECET75at6UcPrHYOAAAwJGVDTYoWiB8AgCgFhA++eiZrSm1TkuoIRr2uxQAAIAJpT/coGixR3LO71IAAJj0CJ98xGLjAAAAlVEINyqsvJTP+F0KAACTHuGTT7qzeW1LZTV/VoPfpQAAAEw4pai3rEGWRccBAPAb4ZNPXu7skyS1Tk34XAkAAMDE4+qayhvZLn8LAQAAhE9+ae/slSQdQ/gEAABwxAXiA+ETI58AAPAb4ZNPBkY+zZsW97kSAACAiScQK4dP+d7dPlcCAAAIn3zSvqtX0xvqFI+E/C4FAABgwgknpkiSsulOnysBAACETz5p7+zVPKbcAQAAVERdQ7Mkqb9nj8+VAAAAwieftHf26ZipTLkDAACohJgXPuV7CZ8AAPAb4ZMPenMFdXTnNG8aI58AAAAqoSERU4+LqtBH+AQAgN8In3ww8KY7pt0BAABURjIaVkoJub4uv0sBAGDSI3zywcCb7ph2BwAAUBnJWEhpF5dyKb9LAQBg0iN88sHgyCem3QEAAFREMhpWWgkFCZ8AAPAd4ZMP2nf1alp9nerrQn6XAgAAMCHFI0GllVCoP+13KQAATHqETz5o7+zTPKbcAQAAVIyZKROoVyRP+AQAgN8In3zwcmcvU+4AAAAqLBdqUF2xx+8yAACY9A4rfDKzd5rZs2ZWMrPFI479g5ltMrONZnbBkPYLvbZNZnb9kPZWM3vKa7/bzCJee523v8k7Pm9/96hlff0F7UjnGPkEAABQYf3hRsVKvVKp6HcpAABMaoc78mmdpL+U9KuhjWZ2kqQrJC2QdKGkr5lZ0MyCklZIepukkyS9x+srSV+S9FXn3HGS9ki62mu/WtIer/2rXr8x73GY36fi9r7pjpFPAAAAlVSMNJQ3siw6DgCAnw4rfHLObXDObRzl0KWS7nLO5ZxzL0naJOkM72eTc+5F51y/pLskXWpmJukcSfd6598h6bIh17rD275X0rle/7HuUdNe9t5018q0OwAAgIoq1TWVN7Jd/hYCAMAkV6k1n46StHnI/havbaz2qZK6nHOFEe3DruUdT3n9x7rWPsxsuZmtMrNVHR0dh/G1Dt9Lu8ojn45m2h0AAEBlRb3wKUP4BACAn0L762Bmj0qaOcqhG5xzPznyJR15zrnbJd0uSYsXL3Z+1vJyZ6+mJiJKRsN+lgEAADDhBeKN5Q2m3QEA4Kv9hk/OufMO4bpbJc0dsj/Ha9MY7Z2Smsws5I1uGtp/4FpbzCwkqdHr/2r3qFntvOkOAACgKoLxKZKkQt+e/T/0AgCAiqnUtLv7JV3hvamuVdLxkn4n6feSjvfebBdRecHw+51zTtIvJF3unb9U0k+GXGupt325pP/0+o91j5rWvqtPxzDlDgAA1Cgzi5rZ78zsae+txp8dpc+YbyOuJZH6ZklSrnuPz5UAADC5HVb4ZGZ/YWZbJJ0l6adm9pAkOeeelXSPpPWSHpT0t865ojeq6RpJD0naIOker68k/b2k/2lmm1Re0+nbXvu3JU312v+npOtf7R6H830qLZsv6pV0VvN40x0AAKhdOUnnOOdOkdQm6UIzWzKiz6hvI641dQ3lkU+5nk6fKwEAYHI7rBHIzrkfS/rxGMduknTTKO0/k/SzUdpf1Chvq3POZSW982DuUateSWUlSbObYj5XAgAAMDpvhHmPtxv2fkaumXmppM942/dKus3MzDu3ZsTjSeVdUIVeFhwHAMBPlZp2h1Hs7M5JkqY31PlcCQAAwNjMLGhmayTtlPSIc+6pEV3GehvxyOv4+sbhZDyilBIq9jHtDgAAPxE+VVGHFz61ED4BAIAa5i2X0KbyC13OMLOFh3id251zi51zi1taWo5skQcgGQsp7eJyGUY+AQDgJ8KnKuroLk+7I3wCAADjgXOuS+WXwlw44tDgW4dHvI24piSjYaWVkGVTfpcCAMCkRvhURR09OQUDpuZ4xO9SAAAARmVmLWbW5G3HJL1V0nMjuo31NuKakoyFlXZxBfsJnwAA8NNhLTiOg9PRndO0+ogCAfO7FAAAgLHMknSHmQVV/kPlPc65B8zsc5JWOefuV/ltxN/33ka8W9IV/pU7tkQkqJQSCvVv87sUAAAmNcKnKtrZnWPKHQAAqGnOubWSTh2l/dNDtsd8G3EtMTNlgg2KFNJ+lwIAwKTGtLsq6ujOqaWe8AkAAKBa+kNJRQs9Uu3NCgQAYNIgfKqiju6cpjdE/S4DAABg0shHGhRSQcr3+V0KAACTFuFTlRRLTp29/Uy7AwAAqKJipLG8wRvvAADwDeFTlezp61ex5AifAAAAqqhU54VPmS5/CwEAYBIjfKqSnemcJBE+AQAAVJHFGPkEAIDfCJ+qpKOH8AkAAKDaArEp5Y0sI58AAPAL4VOVdHSXw6fphE8AAABVE0qUw6di7x6fKwEAYPIifKqSgfBpWj3hEwAAQLVEvPAp17Pb50oAAJi8CJ+qpKM7p0QkqERdyO9SAAAAJo1oQzl86id8AgDAN4RPVbKzO8t6TwAAAFXWEI+p28VU6GPNJwAA/EL4VCUd3TlNb4j6XQYAAMCkkoyFlVJCxT7WfAIAwC+ET1XS0ZNj5BMAAECVJWMhdbu4lEn5XQoAAJMW4VOVdHQTPgEAAFRbMloe+WQ5pt0BAOAXwqcqyOaL6s4WCJ8AAACqrCEaUtrFFexP+10KAACTFuFTFXR05yRJLfWETwAAANWUiISUVkJhwicAAHxD+FQFOwfCpyThEwAAQDUFAqZMsEGRQrffpQAAMGkRPlUBI58AAAD80x9uULTUJxULfpcCAMCkRPhUBR095fBpOms+AQAAVF0+nCxvZHnjHQAAfiB8qoKOdFZmUnMi4ncpAAAAk04x0ljeyPLGOwAA/ED4VAUdPTlNTdQpFOTXDQAAUG0uSvgEAICfSEOqoKM7pxam3AEAAPjCok3lDabdAQDgC8KnKiB8AgAA8E8wMaW8kWHkEwAAfiB8qoKO7hxvugMAAPBJKF4On4qETwAA+ILwqcKcc+roYeQTAACAXyIN5fAp173b50oAAJicCJ8qrKsvr3zRaTrhEwAAgC8S8Qb1u6DyvXv8LgUAgEkp5HcBE11HT06SNI3wCQAAwBfJeEQpJRQgfAIAwBeMfKqwVCYvSZoSD/tcCQAAwOSUjIaUdgk51nwCAMAXhE8Vluorh0/JKOETAACAH5KxsNJKSFnCJwAA/ED4VGHpbDl8aowRPgEAAPghGQsr5RKybMrvUgAAmJQInypsYNpdkvAJAADAF8loSGnFFepP+10KAACTEuFThQ2GT1HWdgcAAPBDIhJSWgmF84RPAAD4gfCpwtKZgurrQgoF+VUDAAD4IRAwZYMNqit0S875XQ4AAJMOiUiFpTJ5Rj0BAAD4LBdqUFBFqb/X71IAAJh0CJ8qLJ3Ns94TAACAzwqRZHmDRccBAKg6wqcKS2UInwAAAPxWqmssb2S7/C0EAIBJiPCpwtKZvBoJnwAAAPw1GD4x8gkAgGojfKqwdCavZJTwCQAAwFfxKeXPDCOfAACoNsKnCktnC4x8AgAA8Fko3lTeYNodAABVR/hUQYViST25gpIx3nYHAADgp1CiPPKp2Ef4BABAtRE+VVB3tiBJjHwCAADwWbS+HD7le3b7XAkAAJMP4VMFpTJ5SWLNJwAAAJ81xKNKu5j6e/f4XQoAAJMO4VMFpbPl8ImRTwAAAP5KRkNKK6FiH+ETAADVRvhUQYMjnwifAAAAfJWMhZV2CblMyu9SAACYdAifKiidYc0nAACAWpCMhpVyCd52BwCADwifKmjvyCfedgcAAOCnZCyktOIK5Bj5BABAtRE+VRBrPgEAANSGZKw88inUn/a7FAAAJh3CpwpKZfIKBUyxcNDvUgAAACa1+kh55FMk3+13KQAATDqETxWUzuTVGAvLzPwuBQAAYFILBEzZUIMipT6pmPe7HAAAJpXDCp/M7J1m9qyZlcxs8ZD2eWaWMbM13s/Xhxw73cyeMbNNZnarecmMmTWb2SNm9rz3OcVrN6/fJjNba2anDbnWUq//82a29HC+SyWkMnnedAcAAFAj8uFkeSPLuk8AAFTT4Y58WifpLyX9apRjLzjn2ryfDw1p/xdJfyPpeO/nQq/9ekmPOeeOl/SYty9JbxvSd7l3vsysWdKNks6UdIakGwcCq1qRzhYInwAAAGpEIdxY3iB8AgCgqg4rfHLObXDObTzQ/mY2S1LSOfdb55yT9K+SLvMOXyrpDm/7jhHt/+rKfiupybvOBZIecc7tds7tkfSI9gZZNSGVySsZ5U13AAAAtaAU9UY+Zbr8LQQAgEmmkms+tZrZH83sl2Z2ttd2lKQtQ/ps8dokaYZzbru3/YqkGUPO2TzKOWO178PMlpvZKjNb1dHRcchf6GB1e2s+AQAAoAZEm8qfWcInAACqab/DcszsUUkzRzl0g3PuJ2Octl3S0c65TjM7XdK/m9mCAy3KOefMzB1o/wO43u2SbpekxYsXH7Hr7g9rPgEAANQOi3krNBA+AQBQVfsNn5xz5x3sRZ1zOUk5b3u1mb0g6QRJWyXNGdJ1jtcmSTvMbJZzbrs3rW6n175V0txRztkq6S0j2h8/2ForxTmndJaRTwAAALUiGGfNJwAA/FCRaXdm1mJmQW/7NSovFv6iN60ubWZLvLfcvU/SwOip+yUNvLFu6Yj293lvvVsiKeVd5yFJ55vZFG+h8fO9tpqQyReVLzolo4RPAAAAtSCcaJYklfoY+QQAQDUd1mrYZvYXkv6fpBZJPzWzNc65CyS9SdLnzCwvqSTpQ8653d5pH5H0PUkxST/3fiTpi5LuMbOrJb0s6V1e+88kXSRpk6Q+SVdJknNut5n9b0m/9/p9bsg9fJfOFCSJkU8AAAA1IpGoV86F5Hr3KOp3MQAATCKHFT45534s6cejtN8n6b4xzlklaeEo7Z2Szh2l3Un62zGu9R1J3zm4qqsjlclLkpIx3nYHAABQC5KxsNJKKNZbM3+vBABgUqjk2+4mtXS2HD4x8gkAAKA2JGNhpV1cRabdAQBQVYRPFZLq80Y+seYTAABATUhGw0opIZchfAIAoJoInyqEkU8AAAC1JRkLKeUSsizhEwAA1UT4VCF713wifAIAAKgFyWh5zadALu13KQAATCqETxUy8La7ZJQFxwEAAGpBMhZWyiUUzhM+AQBQTYRPFZLK5JWIBBUK8isGAACoBQ11IXUrrki+W3LO73IAAJg0SEYqJJ3Ns94TAABADQkETJlQgwIqSv09fpcDAMCkQfhUIalMnvWeAAAAakwhlCxvZFP+FgIAwCRC+FQhacInAACAmlOoayxvZHjjHQAA1UL4VCGpDNPuAAAAao2rY+QTAADVRvhUId3ZgpJRwicAAICaEmsqf2YZ+QQAQLUQPlUII58AAABqTyA2pbzBtDsAAKqG8KkCCsWSenIFJWMhv0sBAADAEKHEwMgnpt0BAFAthE8V0J0tSBIjnwAAAGpMJDFFJWdymT1+lwIAwKRB+FQBqUxekljzCQAAoMY0xCLqUUz9vUy7AwCgWgifKiCdLYdPjHwCAACoLclYWCmXUKF3t9+lAAAwaRA+VUA6U5521xBlzScAAIBakoyGlVZcxT5GPgEAUC2ETxWQL5UkSeEQv14AAIBakoyFlHIJKUv4BABAtZCOVECp5CRJQTOfKwEAAMBQ5ZFPCVk27XcpAABMGoRPFVAcCJ8ChE8AAAC1pNFb8ynYn/K7FAAAJg3Cpwrwsicx8AkAAKC2DKz5FM4z8gkAgGohfKqAkmPkEwAAQC2qj5bXfAoXM1Ix73c5AABMCoRPFVBkzScAAICaFAyY+kMN5Z0Mi44DAFANhE8VMDDyyQifAAAAak4+kixvZFn3CQCAaiB8qgCm3QEAANSuQqSxvJFl5BMAANVA+FQBxVL5k2l3AABgvDGzuWb2CzNbb2bPmtnHRunzFjNLmdka7+fTftR6qFyU8AkAgGoK+V3ARLR32p3PhQAAABy8gqSPO+f+YGYNklab2SPOufUj+j3hnLvYh/oOm0Wbyhus+QQAQFUw8qkCSiWm3QEAgPHJObfdOfcHb7tb0gZJR/lb1ZEViHvhE2s+AQBQFYRPFVBkzScAADABmNk8SadKemqUw2eZ2dNm9nMzWzDG+cvNbJWZrero6KhgpQcnnJhS3mDaHQAAVUH4VAHewCcFmHcHAADGKTOrl3SfpP/hnEuPOPwHScc4506R9P8k/fto13DO3e6cW+ycW9zS0lLZgg9CPFGvnAvLZRj5BABANRA+VcDAtDsGPgEAgPHIzMIqB093Oud+NPK4cy7tnOvxtn8mKWxm06pc5iFLRkNKKaF8726/SwEAYFIgfKqAIms+AQCAccrMTNK3JW1wzv2fMfrM9PrJzM5Q+Zmys3pVHp5kLKy0i6vQx7Q7AACqgbfdVcDA2+4ChE8AAGD8eYOkv5b0jJmt8do+KeloSXLOfV3S5ZI+bGYFSRlJVzjnPQCNA8loWCklNJPwCQCAqiB8qoDB8Ik1nwAAwDjjnHtS0qs+xDjnbpN0W3UqOvKSsZBSLiFl9/hdCgAAkwLT7iqgWCp/BgmfAAAAak4yGlZacQWyI9dRBwAAlUD4VAF7p935XAgAAAD2kYyGlXIJBft52x0AANVAPFIBe992x8gnAACAWpOMhZRWQuF8tzR+lqoCAGDcInyqgKL3EMO0OwAAgNpTX1de8ymgkpTr9rscAAAmPMKnCvAGPvG2OwAAgBoUCgbUH2oo72SZegcAQKURPlVAqeRE7gQAAFC78uFkeSPb5W8hAABMAoRPFVB0TkHSJwAAgJpVqhsInxj5BABApRE+VUDJORYbBwAAqGEu2lTeyDDyCQCASiN8qoDytDvCJwAAgFoViDWWN5h2BwBAxRE+VUCxJKbdAcD/Z+/eoyS963rff751faqq6+menunuuWeSzOROjEkMwe0FASGw0XAQPODWRI1msYWz3ce9tqLsJVsQxe1esg5G5aBkCR6OEUEhaHJCQEDYEkjEkPtlJreZyVx6unu6+n6p53f++D1VXT3dPTM9/VRXV837tdazqp5LVT3dySTdn/l+vz8A2MBSxU3+CW13AAA0HeFTE/i2u1bfBQAAAFaSL/UoktF2BwDAOiB8aoLIOaVInwAAADasciGnMVeQo+0OAICmI3xqgmrklGbmEwAAwIYVFrIadSXNTYy0+lYAAOh4hE9NQOUTAADAxhYGWVVUUnWSyicAAJqN8KkJokjMfAIAANjAwkJGo64kx8wnAACajvCpCaqOtjsAAICNzFc+FVntDgCAdUD41ARRRNsdAADARlab+ZSaofIJAIBmI3xqgsg5pah8AgAA2LBqM58ys2OtvhUAADoe4VMTVJ2UpvIJAABgw6rNfMpUp6T52VbfDgAAHY3wqQmiyDFwHAAAYAPrymc0qpLfmab1DgCAZlpT+GRmf2hmT5nZI2b292bW03DuN81sv5k9bWZvaDh+U3xsv5m9t+H4hWb27fj435hZLj6ej/f3x+f3nOkzWi1yjsonAACADSyTTmk23eV3GDoOAEBTrbXy6X5JVznnrpb0jKTflCQzu0LSOyRdKekmSX9qZmkzS0v6E0lvlHSFpHfG10rSH0j6iHNur6QRSbfFx2+TNBIf/0h83YqfscavJxHViJlPAAAAG91crts/maLyCQCAZlpT+OSc+5Jzbj7efUDSzvj5zZLucs7NOOeel7Rf0g3xtt8595xzblbSXZJuNjOT9BpJn41f/0lJb2l4r0/Gzz8r6bXx9St9RssxcBwAAGDjc/k4fKLyCQCApkpy5tMvSro3fr5D0sGGc4fiYysd3yzpZEOQVTu+6L3i86Px9Su91xJmdruZPWRmDw0ODp7TF7caEQPHAQAANjxXqIVPVD4BANBMmTNdYGZflrR1mVPvc859Ib7mfZLmJX062dtLhnPu45I+LknXX3+9a/bnVRk4DgAAsOGlgnhcKeETAABNdcbwyTn3utOdN7Ofl/RmSa91ztWCncOSdjVctjM+phWOD0nqMbNMXN3UeH3tvQ6ZWUZSd3z96T6jpSLnlCJ9AgAA2NAypTh8YuYTAABNtdbV7m6S9OuSftI5N9lw6m5J74hXqrtQ0j5J35H0oKR98cp2OfmB4XfHodVXJb0tfv2tkr7Q8F63xs/fJumf4utX+oyWi5xTmplPAAAAG1qxWNK0ssx8AgCgyc5Y+XQGd0jKS7rfzwDXA865dznnHjezz0h6Qr4d793Ouaokmdl7JN0nKS3pTufc4/F7/Yaku8zsdyX9m6RPxMc/IemvzGy/pGH5wEqn+4xWY7U7AACAjS8sZDXqSspPnRQ/uQEA0DxrCp+cc3tPc+5Dkj60zPF7JN2zzPHntMxqdc65aUlvX81ntFoUSakkR7kDAAAgcWGQVcWV1Dt5UtlW3wwAAB2MiKQJIudY7Q4AAGCDCwsZjaqk6uRIq28FAICORvjUBFVH2x0AAMBGFwa+7c4xcBwAgKYifGqCiJlPAAAAG15YyKqiooyB4wAANBXhUxNUabsDAADY8GqVT+lZwicAAJqJ8KkJokgiewIAANjYwkJGFRWVmR3zP8ABAICmIHxqgoiZTwAAABterfLJFEmzY62+HQAAOhbhUxNUI9ruAAAANrpykFFFJb/D3CcAAJqG8KkJqHwCAADY+DLplGbSZb/DincAADQN4VMTRE5KUfkEAACw4VXzoX9C5RMAAE1D+NQE1cgpTfYEAACw4UX5Hv9kmsonAACahfCpCWi7AwAAaBNBt3+k7Q4AgKYhfGqCKHK03QEAALSBVLFW+UTbHQAA11DusAAAIABJREFUzUL41ARV55Sm8gkAAGDDyxVCVZWi7Q4AgCYifGoCBo4DAAC0h3Ixr3EVqHwCAKCJCJ+aIIqcyJ4AAAA2vjDI6qQryTHzCQCApiF8aoKqc0qTPgEAAGx4YSGjiiuqOkn4BABAsxA+NYGvfCJ8AgAA2OjCIKtRV1I0OdLqWwEAoGMRPjVB5ET4BAAA0AbCQlYVleSY+QQAQNMQPjVBNXJK850FAADY8GqVT8ZqdwAANA0RSRNEzrHaHQAAQBsICxlVVFR6ttLqWwEAoGMRPjVB5Jj5BAAA0A5qlU/p6rQ0P9Pq2wEAoCMRPjVBNXJKEz4BAABseLWZT5KkKVrvAABoBsKnJoicaLsDAABoA+Ugo4qLwyeGjgMA0BSETwmLIidJInsCAADY+LLplKYzZb/D0HEAAJqC8ClhVefDJ9ruAAAA2kM1F/onVD4BANAUhE8Ji+LwibY7AACA9uCCbv+EmU8AADQF4VPCosg/stodAABAe7Cgxz+h7Q4AgKYgfEpYve2O7ywAAEBbSJcInwAAaCYikoRV6wPHqXwCAABoB8VCUdPKMfMJAIAmIXxKmHOETwAAAO0kLGRVUYmZTwAANAnhU8JqlU9pBo4DAAC0hXKQ0cmoKEflEwAATUH4lLAqq90BAAC0lTDIalQlRZMjrb4VAAA6EuFTwuLsSWRPAAAA7SEsZDXqSopouwMAoCkInxJWb7tj5hMAAEBbCAM/88lN0XYHAEAzED4lrL7aHaVPAAAAbSEsZDTqSkrNUPkEAEAzED4lbKHtjvAJAACgHfjKp6LSs2NSFLX6dgAA6DiETwmrDRxP850FAABoC2Ehq4oryeSkmUqrbwcAgI5DRJKwetsdlU8AAABtIQwyqqjod6aZ+wQAQNIInxLm6pVPhE8AAADtoBz41e4kSdPMfQIAIGmETwmrtd1R+QQAANAecpmUptNlv0PlEwAAiSN8ShhtdwAAAO0nynf7J1NUPgEAkDTCp4TVVruj7Q4AAKB9uCAOn2i7AwAgcYRPCVuofGrxjQAAAOCsWaHHP6HtDgCAxBE+Jaw+84n0CQAAoG1kC6GqStF2BwBAExA+Jay+2h0znwAAANpGWMxpXCUqnwAAaALCp4RVI//IwHEAAID2EQZZVVRi5hMAAE1A+JSw+swnvrMAAABtIyxkdDIqyFH5BABA4ohIEkbbHQAAQPsJg6xOupKiyZFW3woAAB2H8ClhDBwHAABoP2Ehq4qKclNUPgEAkDTCp4TV2+6ofAIAAGgb5SCjUcfMJwAAmoHwKWFRre2OyicAAIC2URs4npqh8gkAgKQRPiUsqq9219r7AAAAwNkLC1lVXEmp6ow0N93q2wEAoKMQPiWsPvOJtjsAAIC2EQYZjarkd2i9AwAgUWsKn8zsD83sKTN7xMz+3sx64uN7zGzKzB6Ot481vOY6M3vUzPab2UfNfEpjZr1mdr+ZPRs/boqPW3zd/vhzrm14r1vj6581s1vX8rUkJYpouwMAAO3LzHaZ2VfN7Akze9zMfnWZa1b8+axd+cqnot+ZpvUOAIAkrbXy6X5JVznnrpb0jKTfbDh3wDl3Tby9q+H4n0n6ZUn74u2m+Ph7JX3FObdP0lfifUl6Y8O1t8evl5n1Snq/pFdKukHS+2uBVSvF2ROVTwAAoF3NS/ovzrkrJN0o6d1mdsUp1yz781k7KzdWPk1R+QQAQJLWFD45577knJuPdx+QtPN015vZNkmhc+4B55yT9ClJb4lP3yzpk/HzT55y/FPOe0BST/w+b5B0v3Nu2Dk3Ih+E1YKslqnWB463+EYAAADOgXPuiHPuu/HzMUlPStpxymUr/XzWtvKZtKbTXX6HyicAABKVZETyi5Lubdi/0Mz+zcy+bmY/HB/bIelQwzWHtPDDzIBz7kj8/KikgYbXHFzmNSsdX8LMbjezh8zsocHBwVV+WatTa7uj8gkAALQ7M9sj6fslffuUU2f1c9h6/gyWhCjf458w8wkAgESdMXwysy+b2WPLbDc3XPM++RLtT8eHjkja7Zz7fkm/Jun/NbPwbG8qropyq/pKTv9+H3fOXe+cu76vry+pt11WxMBxAADQAcysS9LnJP1n51zlXN5jPX8GS0TQ7R+pfAIAIFGZM13gnHvd6c6b2c9LerOk18ahkZxzM5Jm4uf/amYHJF0i6bAWt+btjI9J0jEz2+acOxKXbR+Pjx+WtGuZ1xyW9OpTjn/tTF9Ps1UZOA4AANqcmWXlg6dPO+f+bplLVvr5rK2lCt3SuJj5BABAwta62t1Nkn5d0k865yYbjveZWTp+fpH8MMrn4ra6ipndGK9yd4ukL8Qvu1tSbcW6W085fku8qsqNkkbj97lP0uvNbFM8aPz18bGWqlc+ET4BAIA2FP+M9glJTzrn/miFy1b6+aytFYtFTStP2x0AAAk7Y+XTGdwhKS/pfv9zih6IV7b7EUkfMLM5SZGkdznnhuPX/Iqkv5RUkJ8RVZsT9WFJnzGz2yS9KOmn4+P3SHqTpP2SJiX9giQ554bN7IOSHoyv+0DDZ7RMbbW7NG13AACgPf07ST8n6VEzezg+9luSdkuSc+5jWuHns3YXBlmNWZcCwicAABK1pvDJObd3heOfky/VXu7cQ5KuWub4kKTXLnPcSXr3Cu91p6Q7V3HLTVetDxxv8Y0AAACcA+fcNyWd9ieZ0/181s7CQkajrqg+Zj4BAJCoJFe7g2i7AwAAaFdhkNVIVJRj5hMAAIkifEpYVBs4TtsdAABAWwkLWZ10JcInAAASRviUsGo88ylF+AQAANBWwiCrioqKCJ8AAEgU4VPCapVPKb6zAAAAbSUsZFRxJRkznwAASBQRScJqM5/SzHwCAABoK77yqaTU7JgURa2+HQAAOgbhU8KqtYHjtN0BAAC0lbCQ1agryeSkGaqfAABICuFTwuptd4RPAAAAbSUMMqqo6HdovQMAIDGETwmLsyfa7gAAANpMOfCVT5Ikho4DAJAYwqeEVeuVTy2+EQAAAKxKOfADxyVR+QQAQIIInxIWOSczyWi7AwAAaCtBNq2pdJffmabyCQCApBA+JSxyTmmCJwAAgLYUBd3+CZVPAAAkhvApYdWIYeMAAABtqxY+MfMJAIDEED4lLHJOKb6rAAAAbSlbCFVVirY7AAASREySsGpE2x0AAEC7Cgs5TVgXbXcAACSI8ClhkXO03QEAALSpsJBVRSXa7gAASBDhU8KiyCmVInwCAABoR2GQ0agrUvkEAECCCJ8SVnVOacInAACAthQWshqJinLMfAIAIDGETwmLnET2BAAA0J7CIKuTrihH2x0AAIkhfEpYFDHzCQAAoF2FhYwqrkT4BABAggifElaNaLsDAABoV2HgB47bTKXVtwIAQMcgfEqYb7sjfAIAAGhHYSGrUVdSqjojzU21+nYAAOgIhE8Ji5xTiu8qAABAWwqDjEZV8ju03gEAkAhikoRVI6c0lU8AAABtKSxkVXFFvzM92tqbAQCgQxA+JcxXPhE+AQAAtKMwyC5UPk1T+QQAQBIInxIWOVa7AwAAaFflIEPlEwAACSN8ShhtdwAAAO0ryKY1lS77HWY+AQCQCMKnhEVOtN0BAAC0sSjf7Z9Q+QQAQCIInxIWRU5kTwAAAO3LCrXwiconAACSQPiUsKpzSpM+AQAAtK1SoaBpC6h8AgAgIYRPCYucGDgOAADQxsJCVuPWxcwnAAASQviUMNruAAAA2lsYZFRRibY7AAASQviUsGpE2x0AAEA7CwtZnYyKtN0BAJAQwqeERc7RdgcAANDGwiCr4agoNzXS6lsBAKAjED4ljPAJAACgvYWFjEYdlU8AACSF8ClhtN0BAAC0tzDIatSV5Bg4DgBAIgifEhY5KUX4BAAA0LbCQlYVFZWaHZOiaqtvBwCAtkf4lDDfdtfquwAAAMC5CoOMRl3J79B6BwDAmhE+JawaOaWZ+QQAANC2wkJWFcInAAASQ/iUsGrkaLsDAABoY2GQ1ahq4RNznwAAWCvCp4Q5J9ruAAAA2lhYyKjiin6HyicAANaM8ClhVcdqdwAAAO1sUeUTK94BALBmhE8JiyKnFDOfAAAA2laQTWsyXfY7VD4BALBmhE8J86vdET4BAAC0tXy3f2TmEwAAa0b4lDDa7gAAANpfttClqtK03QEAkADCp4RFkah8AgAAaHNhIaeJVBdtdwAAJIDwKWGRc0rzXQUAAGhrYSGrMSvRdgcAQAKISRJWZeA4AABA2ysHGY25EpVPAAAkgPApYZFzSjHzCQAAoK2FQVYjrsjMJwAAEkD4lLDISWkqnwAAANpaWMhopFqg8gkAgAQQPiXMt921+i4AAACwFmGQ1UhUkmPmEwAAa0b4lLAoou0OAACg3YWFrCoq+son51p9OwAAtDXCp4RFztF2BwAA0ObCIKNRV5JVZ6W5qVbfDgAAbY3wKWFVBo4DAAC0vbCQ1ahKfofWOwAA1oTwKWFRJKWofAIAAGhrYZBVxdXCJ4aOAwCwFmsOn8zsg2b2iJk9bGZfMrPt8XEzs4+a2f74/LUNr7nVzJ6Nt1sbjl9nZo/Gr/momU9xzKzXzO6Pr7/fzDad6TNaJXJOaSI9AACAttZdyCxUPk1R+QQAwFokEZP8oXPuaufcNZL+QdJvx8ffKGlfvN0u6c8kHyRJer+kV0q6QdL7a2FSfM0vN7zupvj4eyV9xTm3T9JX4v0VP6OVqs5R+QQAANDmfOVT0e9Q+QQAwJqsOXxyzlUadkuSasuB3CzpU857QFKPmW2T9AZJ9zvnhp1zI5Lul3RTfC50zj3gnHOSPiXpLQ3v9cn4+SdPOb7cZ7SEc07O0XYHAADQ7pj5BABAcjJJvImZfUjSLZJGJf1YfHiHpIMNlx2Kj53u+KFljkvSgHPuSPz8qKSBM3zGkYZjMrPb5SujtHv37tV9casQxbFbmoHjAAAAbS2fSWk61eV3qHwCAGBNzqryycy+bGaPLbPdLEnOufc553ZJ+rSk9zTzhuOqKHfGCxe/5uPOueudc9f39fU16c6kapw+kT0BAAC0NzOTgm6/w8wnAADW5Kwqn5xzrzvL9/u0pHvkZzodlrSr4dzO+NhhSa8+5fjX4uM7l7leko6Z2Tbn3JG4re54fHylz2iJyMXhE+kTAABA2ysVAk1PFBRQ+QQAwJoksdrdvobdmyU9FT+/W9It8Yp0N0oajVvn7pP0ejPbFA8af72k++JzFTO7MV7l7hZJX2h4r9qqeLeecny5z2iJWuVTmplPAAAAba9cyGoi1cXMJwAA1iiJmU8fNrNLJUWSXpT0rvj4PZLeJGm/pElJvyBJzrlhM/ugpAfj6z7gnBuOn/+KpL+UVJB0b7xJ0oclfcbMbos/46dP9xmtUq98InwCAABoe2GQ0ZhK2kzlEwAAa7Lm8Mk591MrHHeS3r3CuTsl3bnM8YckXbXM8SFJr13NZ7RCFPlH2u4AAADaX33FO2Y+AQCwJmtuu8OCqqu13bX4RgAAALBmYZDVSFSk7Q4AgDUifEoQA8cBAAA6R1jIaLhakGi7AwBgTQifEhRFzHwCAADoFL7yqSQ3NdLqWwEAoK0RPiWo3nZH5RMAAEDbC4OMKq4omx2XqvOtvh0AANoW4VOC4sInpal8AgAAaHv1geOSNFNp7c0AANDGCJ8SVGu7I3sCAABof2GQVcUV/Q5DxwEAOGeETwmqRrTdAQAAdIqwkFmofJoifAIA4FwRPiUoYuYTAABAx/CVT3H4xIp3AACcM8KnBNXCJ6PvDgAAoO0tmvlE2x0AAOeM8ClB1cg/MnAcAACg/YVBVqNUPgEAsGaETwlaaLtr8Y0AAABgzYJsSlNpZj4BALBWxCQJqka03QEAAHQKM1M236V5y9B2BwDAGhA+Jahe+UT4BAAA0BHCYk5TqS7a7gAAWAPCpwTFhU+sdgcAANAhwiCjceui7Q4AgDUgfErQQttdi28EAAAAiQgLWY2pSOUTAABrQPiUoIWB46RPAAAAnSAMsjrpSsx8AgBgDQifEhRFzHwCAADoJGEho5MRlU8AAKwF4VOCqo7V7gAAADpJGGQ1VC0w8wkAgDUgfEpQFPlH2u4AAEC7MrM7zey4mT22wvlXm9momT0cb7+93ve4nsJCViNRUW56VIr/ohEAAKwO4VOCFmY+tfhGAAAAzt1fSrrpDNd8wzl3Tbx9YB3uqWXCIKNRV5JFc9LcZKtvBwCAtkRMkiDa7gAAQLtzzv2zpOFW38dGERayGlXJ79B6BwDAOSF8ShADxwEAwHniVWb2PTO718yuXOkiM7vdzB4ys4cGBwfX8/4SEwZZVVzR7zB0HACAc0L4lKBqLXxi5hMAAOhc35V0gXPu+yT9saTPr3Shc+7jzrnrnXPX9/X1rdsNJqkcZBYqn6apfAIA4FwQPiUozp5E4RMAAOhUzrmKc248fn6PpKyZbWnxbTVNWMiq4mrhE5VPAACcC8KnBC0MHCd9AgAAncnMtlo84NLMbpD/eXKotXfVPGHAzCcAANYq0+ob6CRVZj4BAIA2Z2Z/LenVkraY2SFJ75eUlSTn3MckvU3SfzSzeUlTkt7hXPw3cB0oLGSY+QQAwBoRPiUoYrU7AADQ5pxz7zzD+Tsk3bFOt9NyhWxaUylmPgEAsBa03SWItjsAAIDOYmYqFQJNp4pUPgEAcI4InxJUjfwjbXcAAACdIwwymkyVmfkEAMA5InxKUK3yKcV3FQAAoGOEhazGrUTlEwAA54iYJEFRPHA8ReUTAABAxwiDrCoqMfMJAIBzRPiUoCoznwAAADpOWMjopCvRdgcAwDkifEpQXPhE5RMAAEAHCYOsRqoF2u4AADhHhE8JWmi7a/GNAAAAIDFhIauhaoG2OwAAzhHhU4KqEW13AAAAnSYMMhquFqXZcak63+rbAQCg7RA+JWhhtTvCJwAAgE4RFrIaVcnv0HoHAMCqET4lqB4+MfMJAACgY4RBVhVX9Du03gEAsGqETwmqRv4xTfgEAADQMcJCpqHyifAJAIDVInxK0ELbXYtvBAAAAIlZXPlE2x0AAKtFTJKghdXuqHwCAADoFH7mU5ffmaLyCQCA1SJ8SlA1rnyi7Q4AAKBzhEFWo46B4wAAnCvCpwTFhU+sdgcAANBBwkJGFTFwHACAc0X4lKAociJ3AgAA6CyFbFpzqbzmLUvbHQAA54DwKUFV55QmfQIAAOgoZqYwyGo63UXbHQAA54DwKUGRcwwbBwAA6EBhIauJVBdtdwAAnAPCpwT5tjvCJwAAgE4TBlmNq0TlEwAA54DwKUHVSLTdAQAAdKCwkNGoupj5BADAOSB8SpBvu2v1XQAAACBpYZDVaFSg8gkAgHNA+JSgyDmlSJ8AAAA6ThhkNRQVmfkEAMA5IHxKUDVySjPzCQAAoOOEhYyG5uPKJ+dafTsAALQVwqcEUfkEAADQmcIgq6FqQYrmpdmJVt8OAABthfApQVEkKp8AAAA6UFjI+oHjEq13AACsEuFTgqoMHAcAAOhIYSGjiiv6HYaOAwCwKoRPCYoi2u4AAAA6URhkNaqS35mi8gkAgNVYU/hkZh80s0fM7GEz+5KZbY+Pv9rMRuPjD5vZbze85iYze9rM9pvZexuOX2hm346P/42Z5eLj+Xh/f3x+T8NrfjM+/rSZvWEtX0sSIueUJnwCAADoOGEhS+UTAADnaK2VT3/onLvaOXeNpH+Q9NsN577hnLsm3j4gSWaWlvQnkt4o6QpJ7zSzK+Lr/0DSR5xzeyWNSLotPn6bpJH4+Efi6xS/7h2SrpR0k6Q/jd+/ZapOSjHzCQAAoOMsqnxi5hMAAKuypvDJOVdp2C1JOtO6szdI2u+ce845NyvpLkk3m5lJeo2kz8bXfVLSW+LnN8f7is+/Nr7+Zkl3OedmnHPPS9ofv3/LRBEznwAAADqRn/lUC5+ofAIAYDXWPPPJzD5kZgcl/Qctrnx6lZl9z8zuNbMr42M7JB1suOZQfGyzpJPOuflTji96TXx+NL5+pfda7h5vN7OHzOyhwcHBc/xKz4y2OwAAgM4UBlmNKW67Y+YTAACrcsbwycy+bGaPLbPdLEnOufc553ZJ+rSk98Qv+66kC5xz3yfpjyV9vllfwNlwzn3cOXe9c+76vr6+pn1ONXK03QEAAHSgYi4tS6U1ky5R+QQAwCplznSBc+51Z/len5Z0j6T3N7bjOefuMbM/NbMtkg5L2tXwmp3xsSFJPWaWiaubasfV8JpDZpaR1B1fv9J7tUzkCJ8AAAA6kZkpDDKaTJeVZ+YTAACrstbV7vY17N4s6an4+NZ4LpPM7Ib4c4YkPShpX7yyXU5+YPjdzjkn6auS3ha/162SvhA/vzveV3z+n+Lr75b0jng1vAsl7ZP0nbV8PWsVOdF2BwAA0KHCQlaTRuUTAACrdcbKpzP4sJldKimS9KKkd8XH3ybpP5rZvKQpSe+IA6N5M3uPpPskpSXd6Zx7PH7Nb0i6y8x+V9K/SfpEfPwTkv7KzPZLGpYPrOSce9zMPiPpCUnzkt7tnKuu8etZkyoDxwEAADpWOcioMtGlHcx8AgBgVdYUPjnnfmqF43dIumOFc/fIt+edevw5LbNanXNuWtLbV3ivD0n60Cpuuaki55QifQIAAOhIYZBVZbwk0XYHAMCqrHm1OyyInFOamU8AAAAdKQyyGokKtN0BALBKhE8JYrU7AACAzhUWMhquFiXa7gAAWBXCpwRFkZTiOwoAANCRwiCrE/MFaW5Cqs61+nYAAGgbRCUJipxjtTsAAIAOFRayOlEt+B1a7wAAOGuETwmqOtruAAAAOlUYZFRxJb9D+AQAwFkjfEpQxMwnAACAjhUWshpVHD4x9wkAgLNG+JSgKm13AAAAHSsMsqq4ot+ZJnwCAOBsET4lKIoksicAAIDOtKjyifAJAICzRviUoIiZTwAAAB0rLGQ0yswnAABWjfApQdWItjsAAIBOFQZZVZj5BADAqmVafQOdhMonAACAzhUWsppRTvOpnDK03QE4F85Js+PS5FC8DUsTJ/zzmYpU2CR1DfitvNU/5rtafdfAmhE+JShyUorKJwAAgI5UyqWVMmkm3aUMbXcAJGluWppqCJBqgVL9+TJbdXZ1n5Hrkrr6pa6tUnnAP3b1L4RTtcdCr5SiuQkbE+FTgqqRU5rsCQAAoCOZmcJCVpOpskq03QGdaeqkNHZ0mdBoWJo8sfTY7PjK7xX0SMXNUmmL1LNb2n6N319265Xyof/88WPS+FFpLH4cP+7vafyYdOQRafzL0uzY0s9LZRaqproGFoKqcu1Y/LzUL2VyzfseAssgfEpQ5ByVTwAAAB0sDLKaiErqo/IJaG9RJA0/Jx17VDr6mHTsMf9YObT89bkuHxDVwqItl0jFLYuPNW6FTVL6HH7dLm3228AVp79udiIOpI4vH1SNHpQOPegDs+UUepdWTpW3SeE2qbzdHytvldLZ1X8NwDIInxIURcx8AgAA6GRhIaPKZJfEzCegfcyMSccel44+uhAyHX9Cmpv05y0tbdkn7b5R2nqV1L3LVyvVg6ReKRu09ms4Va4kbb7Yb6dTnZMmBhcqp8aPLQRVY/H+0H5/Ppo75cUmlfriQCrewlowtX3heGGTxO/BOAPCpwRVnVOaP3QAAAAdKwyyGp0oStMvtfpWAJzKOenkSwsB09FH/PORFxauCbqlgVdI194iDVzlw6a+yzdeuJSUdNYHRuH2018XRX52VeVlH0SNvSxVjvjHsaPS6OG4kmpo6WszwUIgVd4aB1TbTnm+rXO/xzgrhE8JYuA4AABAZwuDrEaiop/LAqB15qZ89VJjy9yxx6WZWkusSb0XSdu+T7rmZ33INHCV1L2TKp3lpFK+2qu0Rdp29crXzc/E4dQRvzUGVJUj0pGHpafvleanlr62sKkhoGpo7+veJfVe6OdiZfLN+xrRUoRPCfJtd62+CwAAADRLWMhouFqQ5kZ9lQW/xALN5ZwPOY4+tng+09B+yUX+mlyXNHCl9Iq3xSHTK6T+y6V8V2vvvRNl8tKmC/y2Euek6dE4nDq1kip+fuxxaeL4wj9DSZJJ4Q4fRG26QNp0Yfx8j39e7G32V4cmInxKUNU5pUmfAAAAOlY5yGpwriClqn6Vq3y5NTcSRSypjs4zOyENPiUdf8qHE7WwaWp44Zqe3T5cuvJ/W2ib69nDn4eNxEwq9Pit//KVr6vO+wDq5EFp5HnfHjn8vH/+zJf8uUZB90IQ1RhK9V7oQ6tUuolfFNaK8ClBDBwHAADobGGQ1eFqQUrJt96td/jknPSdP5e+/H6pq1/a/So/JHn3q6TN+/gFHO1hblo68Yx0/Elp8En/ePxJ6eSLC9dkAqn/CunyN/uwaetVvrop6G7dfSNZ6czCPKrdr1x6fnaiIZB6wYdSw8/7wfFP/ePiAemprA8mTw2lNu3xW660Ll8SVkb4lKDIifAJAACgg4WFjJ508S8x06OSdq3fh0+NSF94j/TUP0gX/qgPvp69X/reX/vzhU3SrhsXwqjt1zA/Ba01P+vb4xoDpsGnpOHnFtqtUlm/0tyO66Tv/1lfKdN3uQ8OqGQ5v+VKPnAcuHLpuagqjR5aWjE18oJ08MGG2V+xroEVqqYu8nOu+D2+6QifElSNnNL8ZRMAAEDHCoOsRlULn9Zx6PjBB6XP/qKflfL635VufLevcnJOGjogvfQt6aUHpIMPSM/c61+Tzvtf6He/0odRu27wARWQtOq8/8X/+BO+Ze74Ez5kGtovRfP+GktJvRf7aqarfkrqu8w/33yxX5ENWI1UeuXZU875sL5WKVUPqF6QXviG9MjfSHIL1+dD/+9h78WnPF7EnKkEET4lKHKO1e4AAAA6WFjIquKKfmd69PQXJyGKpG/dIX3ld3xryi/eJ+28fuG8mbRlr9+u/Tl/bHzQh1Avxdu//LH0zY/4c/1X+Mqd9Ph6AAAgAElEQVSoWoVUz+6N/zf+0xWpcthvc9M+QGvcWL59/USRdPKFxQHT8Sd9C111Nr7IfFVJ/+XSpW/y/871X+bbQvlnhfVg5kOjYq8P4E81Ny2dfMmHUkMHpOED/vHQd6THPqdFwVShd2kgtXmvf96qmX9tivApQZFj5hMAAEAnC4PMQuXTVJMrnyaGpM+/S3r2S9LlPyH95B1+gO+ZdPX56y//Cb8/Oykd/tc4jPqW9OhnpYfu9OfK2+M2vXgbuGp9W51mJ32oNHooDpheXng+GgdOM5XTv0emcEog1bM0oFpuy5U2fvCWJOd8FVJ1zs/Kqc7Hj/F+VF16bnp0IWCqhUxzkwvv2b3Lh0wXv8Y/9l8ubblUyhVb93UCZ5INpL5L/Haq+RlfJTW0f3Ew9cI3pEfuWnxtqb8hmIpDqd44oOLPwBKETwmqRk7p8+l/YAAAAOeZdat8euF/SZ+7TZockt70P6Uf+KVzD0pyRenCH/ab5EOG408sVEa99C3p8b+Lry1Lu34gbtN7pa+yOtdBvfMzi0Ok0UM+XKofO+RbY05V6vMrV22+WLrwR6TuHX6/e6cfQj190r9uyRYfH35emvqufz4/tfL9pbJnGVTF16QyPryJqvHjfMP+csdW2HfVM1+z6H1rAdH8CkHRKecWhUqNr5k/t3+OklTe5tvkrvv5hZlMfZdKQXju7wlsRJm8/3e779Kl52Yn/byyWiA1fEAaes7/BcHDp6zMV97u/xt2ajtf74Xn7Sw+wqcERU603QEAAHSwsJDVmIpyMlkzZj5FVekbfyR97ff8MNxf+htp2/cl+xmptLT1FX674Zf9sZMHF4Kog9+Wvvp7kpxkaf/59VX1bvSr7FXnpLEjpwRLp1QuTQwu/ezCJinc6QOlXTfEwdLOhYAp3J7sL2ZzUwuh1Jm2ysvSsSf889mx5O7hdCzlQ636lvaPll68n84uXJPO+uAsk5NSpYVzteNL9jMNx7P+PVc6t2g/I+W6fHUIs8IAH+Rvvcpvp5quNARTtcf90hN3S1PDDRearxjcfJF/7N7lg/XuHf55uF3KFtbtS1pPhE8JiSLfF0r2BAAA0LnCICOnlGYzJeWTrnwaOyb93S9Lz39desXbpTd/ZP1mivTs8tvVb/f7UyelQw/Gg8y/LT30CemBP/HnCr1xxZJb/B75bv+LU/cOH1h174wrluKAKdy+/q0o2YLfwm2re111bmlo5aqLA6HGbaUQacmxhuOW9kPjAbS/IPQrjG6/Zum5qZGGQCoOpYafk47dJ00cX3p9cUscSO1s+O/ozjio2uFX7mvDlSAJnxJSdf5/vrTdAQAAdK5SLqOUSdPpsvJJznw68E/S390uzYz72U7f/7OtnUdU6JH2/bjfJGl+VjryPR9GDR/wv/ycGix1UgtWOutnZ3X1tfpOALS7wiZp53V+O1Vje/LoId+OPBpvQwek576+tBIzlfH/zQ13NlRNxeFULagKujfcTDvCp4REcfhE2x0AAEDnSqVM5SCriVRZ3UlUPlXnfYvdN/7Izxi59Yt+ps5Gk8n5WVC7fqDVdwIAnSOT9wPKey9a+Zrp0YVAqrbV2p0PPiA9/vLSmW65rlMqp3Y1hFQ7pZ496155SfiUkCjyj6x2BwAA0NnCQkYTKvnB12sxekj63C/5aqJrb5Fu+gNWSAIALBZ0+23gyuXPR1Vp/HgcSB1cqKIaPeiPHfne0hl87zsqpdZ3thThU0LqbXe0bQMAAHS0MMiqMl2SpobO/U2e/v+kz7/LzxZ6618szFoC0FHmqpGGxmd1Ynwm3vzzoYbng2P++ejUrEr5jDYVc+ouZLWpmNWmYk49xZx6in6/p5iLj2XjYzkVc2kZRRDnr1Taz7ULt/kVSpczN71QLTV+vCVDzQmfElKtDxznDz0AAEAnC4OsRqeK0vRzq3/x/Kz0ld+RvnWHX23u7Z/0S3ADaBvTc9U4MFoaJg2Oz+jE2IyGJvzxk5Nzy75HkE1pS1deW7ry2rmpoGt29ai7mNXkTFUnp+Z0ctK/1zPHxjU6Nafxmfll30eScumUuhvCqZ5CHFqV4sdCLbRa/JjLUDlx3sgG/v81Lfz/DeFTQpwjfAIAADgfhIWMhk8UpflVtt2NvCD97S9IL39XuuF26cc/6H8hALAhTM7O65lj4zpycspXJC1TpXRibEYTs9VlX18OMnGglNO+/i696qLN2tyVq4dMfWX/fHNXXqVVVivNzkc6OTWrk5NzOjk5p5HJWZ2cnNVIvO+f+/0Xhyb18MGTOjk5p9lqtOJ7lnLpelVVbymnvq68+sqLt/5yoL5yXmGQoboKa0L4lJBa5VOageMAAAAdLQyyGqoWpOqkr2TK5M78osc/L939n/zzn/4r6YqfbO5NAlhRFDkdGpnSE0cqeupoRU8dGdNTRyt6cXhScU2BJL9Y2KZiTlviAOnqnT31531deW0p57S5lNeWcl6bSzkF2XTT7jmXSam/HKi/fPaBtXNOk7O+kmpkIg6upuLAaiJ+jAOtoYlZPTc4ocGxmWUDq1wmpb6uvPrDfD2kqgVTC0GVD9moqMJyCJ8SUmW1OwAAgPNCWMhqcK4gpeRXIerqW/niuWnpS++THvwLacd10tvulDbtWa9bBc57lek5PX10TE8dqejJ+PHpo2P16iUz6cLNJV2xPdRbr92pS7eWtWtTUVvKOfUWc8q08VBfM1Mpn1Epn9GOnrOb8eOcU2VqXoPj0zpemdFgPJPq+Jh/HByb0YtDk3rwhWGNrNBS2FPMnlVQ1V3IUk11HiF8SkgtIU/zhwcAAKCjhUFWz80HUk6nD59O7Jf+9uelY49KP/h/SK/57bOrkgKwavPVSC8MTS6qZHryyJgOn5yqX9NdyOrybWW9/fpdunxbWZdtDXXJQFmFXPMqltqNmam7mFV3Mau9/eXTXjs7H2loIg6nFgVV0/Wg6qEXRzQ4NqOZ+aXVVNm0qa8rr209BV2wuag9m0v1xz2bS+ouZpv1ZaIFCJ8SsjBwvMU3AgAAgKYKCxmNquR3pleY+/TIZ6Qv/mcpk5d+5jPSJW9YvxsEOtzwxOyiSqanjo7pmWNj9YAjnTJd3FfSdRds0n+4cbcu3xbq8q2hBsI8lTYJymVS2tZd0Lbu01dVOec0NjO/JKSqBVUvn5zStw4M6e++e3jR63qK2TiIKuqCzSXt2RI/bi5pU5GqqXZD+JSQevhE+gQAANDRykFWFbdC+DQ7Id3769K//T/S7h+UfuovpO4d63+TQAeYnY90YHC8Xs1UC5uOj83Ur9nSldfl28q65VUX6LKtoS7bVtbe/i7lM1QzbRRmpjDIKgyyuriva8Xrpmareml4Ui8MTejFoQm9MDSpF4cm9OALI/rC915eNI+rHGQWVUpdsLmoC7eUdMHmkrZ05QimNiDCp4TQdgcAAHB+CIOGyqephvDp+JO+zW7waelH/qv0o++V0vy4DZxJFDkdHJnUM8fG9cwxX8X09NEx7T8+rvn4L/lz6ZT2DXTph/f11VvmLt1aVl853+K7R1IKubQu3VrWpVuXtvvNzFd1cHhqUSj1wtCkHj08qnsfO1ovBpH8Kn6LK6UWKqb6y3kKRlqE/xsmZGHgeItvBAAAAE0VFhorn0b930L+219J9/y6lC9LP/f30sU/1tqbBDYg55xeHp32AdPRsXrYtP/4uKbmqvXrdvQUdMlAl15zWb8u2xbq8q1l7dlSUraNh39jbfKZtPb2d2lv/9LKqblqpMMjU3HF1GT98amjY7r/iWOaqy4EU0E2pQt644qpLSVdtKVUf9+eIjP5monwKSELM59IUQEAADpZGGRVUdHvjB6SPvdL0mOflS78Uemtfy6VB1p7g0CLOec0ODajp4/FAdPRMT1zfEzPHhvX+Mx8/br+cl6Xbi3rnTfs1qVbu7RvoKx9/V0qBwyaxtnLplPas6WkPVtKS85VI6eXT041hFK+YuqFoQl9/ZnBRYPQN5dyujgOovb2ddWfb+8OaONLAOFTQlxc+ZSmhA8AAKCjhYWMZpTTfCqvzDf/SLKU9Jr/Jv3Qr0kp5szg/DI0PqNnjo3r2eO+Ve7ZY+N6+tiYRqfm6tf0lnK6ZKBLb712hy4ZKMcblSZovnTKtKu3qF29Rf3Qvi2LzkWR0+GTU9p/fFwHBse1/7jf7nn0iE5OLvz7W8yldXFfly7uW6iS2tvfpd29JeUyVOOdLcKnhNTb7khEAQAAOlpY8FUZ48F29aRnpLd9QrrgB9ft8184MaFPfetFlYOM9vZ36eK+Ll3UV1KQJfhC84xOzenZY2N6+lgcMB0d07PHx3RifLZ+TRhkdMlAWf/+6m26pL9Ll2z1QdOWLuYyYeNJNQRTP3ZZf/24c05DE7M6cHxc+xtCqe88P6zPP/xy/bpMyrR7c1F7+7rq/y3e2+8rprryRC2n4juSENruAAAAzg9duYzMpLuu+FO968evlvIrr96UpImZed3x1f36xDeelyTNRVF90Rszaeemgm8VafgFaG9flzaVqC7BYtXIaWJ2XhMzfhufqWp8el7j8f7ErH8+ND6rZ+Kw6Whluv76Ui6tvQNlveay/oZKprIGwjztSWh7ZqYtXXlt6crrlRdtXnRuYmZezw1OaP/gWD2UOjA4oX966nh9OL4kbesOlvy3eG9/13m9Eh/hU0Lqq93RdgcAANDRUilTOZ/R0ah7XYIn55y+8PDL+v17n9Sxyozeeu0OvfemyxQWsnr+xES9XeTA4IT2Hx/XvxwYWjTHpLeUi+eXlHzrSPyL0I6ewoZZ9ck5p8rUvI5WpnWsMq2jlWkdjx+PVWZ0LD4+NVtVKZ9RIZdWKZdRMZdWKe8f/ZZRKR8/xvvF/MK1i/bzaRWzaWXaYIi1c04z81EcFlV9SBQHROPTtQDJn1v2+OzC68an5xcN9z6dfCalvf1d+sGLN2vfQNnPZeovb6h/d4D1VMpn9Iqd3XrFzu5Fx+eqkV4cmqy38NWqpv72oYOamF348xY2VKzu2VLSrt6idvcWtWtTQb2lzg6mCJ8SslD51OIbAQAAQNOFhawqDTNtmuXRQ6P67198XP/64oiu3tmtP/vZ63Tt7k3185dvC3X5tnDRa+pzTOJfgGrh1H2PH9PwxMH6dUE2pQu31NpFSvVfiC7ckmwL3/RcNQ6PZhZCpdFpHRub0bHRaR0b88HS9Fy05LU9xawGyoEGugNdOlBWMZfW5GxVk7M+ZJmcrer42LQmZxb2J2eri5ZdP5NcJlUPqkr5tAoNwVUtyCrm0kqZVI2kahSp6pyqkf9ez0dOkXOqRs4fr/rHqLYfLWyRi6+vn4vfL3KKnBZdW3uPuWqkqbnqohW7TifIptSVz6iUz6iUy6gryKi/HKi0JaOuOHgr5TMqB/E1+cXHu/L+NV35jPKZVEf/MgwkJZtOLbsan3NOR0anF82U2n98XF99elAn/vXQomtLuXQ9jNrdW9TuzcX6/o6eQtu3VhM+JaQ+84n0CQAAtDEzu1PSmyUdd85dtcx5k/R/SXqTpElJP++c++763mXrhUFWlenmhU9D4zP6n196Wnc9eFCbSzn9j5+6Wm+7budZ/ay5aI7Jpf2Lzg1PzC5USsV/M//wwRH9wyMvL2rh27WpuCSUOnUp8mrkdGLcVyUtCpPqlUs+bBpdJqQLsiltDQP1h4G+b2ePBsK8BsJAA2Ggrd2BBsqB+sP8Of2yVasS8kGUD6QmZhaCqcm4Cqh+bna+Hl5NzVY1MVvV5My8RianNDU7X9+PnJ/xkkqZ0ilTykyZ2vOUlDb/vHYunbKF680/ZtMpBdmF8+n4XDq+LlN/rRadL8ahUCmu9FocHDU85tqjkgs4X5iZtvcUtL2noB/e17fo3OTsvA6NTOmloUm9NOy3g8OTev7E0pX4JGlrGPgqqXo4VYirporqK2/8llfCp4TUV7vb4P/AAQAAzuAvJd0h6VMrnH+jpH3x9kpJfxY/nlfCQkaVqfkzX7hKc9VIf/WtF/WRLz+jqdmqbvt3F+o/vW6fwoSWnu8t5dRb6tUP7OlddHxqtqrnT0zUq6Vqj9/cf0KzpyxFPhAGGpqY0eDYjE4tMEqnTH1deQ10B7pgc1GvvKhXA2Gg/nLeh0pxwBQGmab9omRmCrJpBdm0epl3BWCDKuYy9Xlpp3LOaXB8RgfjUOqloal6OPW/9p/Q5xpmsEk+0N+1qbg4nKpVT20qqpBrfdUU4VNCqvH/kxk4DgAA2plz7p/NbM9pLrlZ0qec/5u3B8ysx8y2OeeOrMsNbhBhkNVLw5OJvuc3nz2h3/ni43r2+Lh+eN8Wvf8nrtDe/qW/lDRDIZfWFdtDXbF9cQtfNXI6PDKl/YNjOnDcz5c6Pjajq3aE9SBpIAy0NQw0EOa1uSvPDFQAWCMzU385UH850HUX9C45Pz1X1eGTC4FUY/XUA88NLZozJUlbuvLa3Vuoh1K/8mN7172Nj/ApIfWZT1S5AgCAzrZD0sGG/UPxsSXhk5ndLul2Sdq9e/e63Nx6SXLm08HhSf3uPz6h+x4/pt29Rf35LdfrdZf3b4gWinS8lPjuzUW95rJW3w0AQJKCbNovING3dNEL55xGJucWtfLVwqkHXxjRvY8d1a++7pJ1v2fCp4TQdgcAALCYc+7jkj4uSddff/3ZT4BuA37m09ra7iZn5/VnXzug//ufn1PaTP/1DZfqth+6sO2HygIAWsfM4hbrnK7Z1bPk/Hw1akmFKuFTQhg4DgAAzhOHJe1q2N8ZHzuvhIWMxmfmNV+NVj3g2TmnLz5yRL9/z5M6Mjqtm6/Zrve+8TJt6y406W4BAPBatSgB4VNC6m13VD4BAIDOdrek95jZXfKDxkfPt3lPkuoDwMdn5hetAHcmj788qt+5+wl954VhXbk91Eff+f1Lhn8DANBpEou8zOy/mJkzsy3xvpnZR81sv5k9YmbXNlx7q5k9G2+3Nhy/zswejV/z0XgpX5lZr5ndH19/v5ltOtNnrLeo1nZH5RMAAGhjZvbXkr4l6VIzO2Rmt5nZu8zsXfEl90h6TtJ+SX8u6VdadKstFRZ8+HS2K94NT8zqfX//qH7ij7+p/YPj+v23vkJ3v+eHCJ4AAOeFRCqfzGyXpNdLeqnh8LLL8JpZr6T3S7pekpP0r2Z2t3NuJL7mlyV9W/4Hm5sk3SvpvZK+4pz7sJm9N97/jZU+I4mvabWi+mp3rfh0AACAZDjn3nmG807Su9fpdjasMPA/RlemTz90fL4a6dPffkl/dP8zGp+Z1y2v2qP/83WXqLuYXY/bBABgQ0iq7e4jkn5d0hcaji27DK+kV0u63zk3LElmdr+km8zsa5JC59wD8fFPSXqLfPh0c/w6SfqkpK/Jh08bZqnf+swn2u4AAAA6Xr3y6TTh078cOKHfufsJPX1sTD948Wb995+8UpcMlNfrFgEA2DDWHD6Z2c2SDjvnvnfKcrArLcN7uuOHljkuSQMNgdJRSQNn+IxF4dN6LPMbRbTdAQAAnC9qM5+Wa7s7NDKp37vnSd3z6FHt6CnoYz97rd5w5VYZf0kJADhPnVX4ZGZflrR1mVPvk/Rb8i1368I558xsVUv1rscyv3H2RPgEAABwHigv03Y3NVvVx75+QB/7+gGZSb/245fo9h+5SEE23arbBABgQzir8Mk597rljpvZKyRdKKlW9bRT0nfN7AatvAzvYS200NWOfy0+vnOZ6yXpWK2dLm7dOx4f3zBL/S603bXi0wEAALCeFgaOz8k5p3sfO6oP/eOTOnxySv/+6m36rTddrh09hRbfJQAAG8OaVrtzzj3qnOt3zu1xzu2Rb3u71jl3VH4Z3lviFelu1MIyvPdJer2ZbYpXrXu9pPvicxUzuzFe5e4WLcyQultSbVW8W085vtxnrLta2x0znwAAADpfOZ+RmfS9Q6P6mT//tn7l099VOcjorttv1J/8zLUETwAANEhq4Phy7pH0JvlleCcl/YIkOeeGzeyDkh6Mr/tAbfi4/FK9fympID9o/N74+IclfcbMbpP0oqSfPt1ntELkmPkEAABwvkilTF35jL74vZfVU8zqg2+5Su/8gV3KpNf0d7sAAHSkRMOnuPqp9nzFZXidc3dKunOZ4w9JumqZ40OSXrvM8Q2z1G8pn9HFfSXlMvzAAQAAcD7436/fpfnI6Vdfu0+bSrlW3w4AABtWMyufzitvuHKr3nDlcjPZAQAA0In+25uvaPUtAADQFijTAQAAAAAAQNMQPgEAAAAAAKBpCJ8AAAAAAADQNIRPAAAAAAAAaBrCJwAAAAAAADQN4RMAAAAAAACahvAJAAAAAAAATUP4BAAAAAAAgKYhfALw/7d3fyG21WUYx78PmpUWHcuy8khKiWFSKRKW0IX90xJPF10YVkZCF1lZCaUFXYZRZEVlmNYxEk1ORRJligndpJWW/zNFK48d0ygrCjLr7WKtoWGcPTo6a1bn9/t+YDiz9p4zvO+8+6zz8O6190iSJEmSNBmXT5IkSZIkSZqMyydJkiRJkiRNxuWTJEmSJEmSJuPySZIkSZIkSZNx+SRJkiRJkqTJuHySJEmSJEnSZFw+SZIkSZIkaTIunyRJkiRJkjQZl0+SJEmSJEmajMsnSZIkSZIkTcblkyRJkiRJkibj8kmSJEmSJEmTcfkkSZIkSZKkybh8kiRJkiRJ0mRSVXPXsKmSPAD8dqJvvx/wx4m+9/8z++6LfffFvvvSUt8vqKpnz12E/scMNoke++6xZ+iz7x57hj777rFnaLPvhfmru+XTlJL8vKqOmruOzWbffbHvvth3X3rtW7u/Xh+7PfbdY8/QZ9899gx99t1jz9Bf377sTpIkSZIkSZNx+SRJkiRJkqTJuHzaWOfNXcBM7Lsv9t0X++5Lr31r99frY7fHvnvsGfrsu8eeoc++e+wZOuvb93ySJEmSJEnSZLzySZIkSZIkSZNx+SRJkiRJkqTJuHzaIEmOS3J7kjuTnDl3PVNJcmCSq5PcmuSWJKePtz8zyZVJ7hj/3HfuWjdakj2S/CLJ98bjg5NcO878m0n2mrvGKSTZkmRHkl8luS3JK1ufd5IPjo/vm5NcnOQprc47yVeT3J/k5mW3rTrfDD4//gxuTHLkfJU/fgt6/tT4GL8xyXeSbFl231ljz7cnecM8VT9xq/W97L4zklSS/cbjJmatPvSSwZYsymI9WJnFerBaDpu7ps2wWhabu6aNtp4M1pL15rAWrCeDtczl0wZIsgfwReB44DDgrUkOm7eqyTwMnFFVhwFHA6eNvZ4JXFVVhwBXjcetOR24bdnxJ4FzqupFwJ+BU2epanqfAy6vqhcDL2P4GTQ77yQHAO8Hjqqqw4E9gJNod97bgeNW3LZovscDh4wf7wbO3aQaN9p2HtnzlcDhVfVS4NfAWQDj+e0k4CXj3/nSeM7fHW3nkX2T5EDg9cDvlt3cyqzVuM4y2JJFWawHK7NYD1bLYU1bI4u1ZjuPPYO1ZDuPMYc1ZDuPPYM1y+XTxngFcGdV3VVVDwGXANtmrmkSVbWrqq4fP/8bw3+ABzD0e+H4ZRcCb56nwmkk2Qq8CTh/PA5wLLBj/JLmegZI8gzg1cAFAFX1UFU9SOPzBvYEnppkT2BvYBeNzruqfgz8acXNi+a7Dfh6Da4BtiR53uZUunFW67mqrqiqh8fDa4Ct4+fbgEuq6p9VdTdwJ8M5f7ezYNYA5wAfBpb/BpImZq0udJPBlqyRxZq2Mov1YI0c1oOVWez3M9ez4daZwZqxzhzWhHVmsGa5fNoYBwD3LDveSR8h4CDgCOBaYP+q2jXedR+w/0xlTeWzDCeG/4zHzwIeXHaSbHXmBwMPAF8bL3M/P8k+NDzvqroX+DTDMxC7gL8A19HHvJcsmm8v57p3AT8YP2+65yTbgHur6oYVdzXdt5rS9WN1RRZr3cos1oNFOaxpq2Wxqrpi3qo2TbMZex2W57BmrZHBmuXySY9LkqcB3wI+UFV/XX5fVRUNbW+TnADcX1XXzV3LDPYEjgTOraojgL+z4vLfBue9L8OzTgcDzwf2YZXLZHvR2nwfTZKPMbyk5aK5a5lakr2BjwIfn7sWSeu3VhZrTcdZ7FFzWItWy2JJ3jZvVZuvtwwG/eSwXjOYy6eNcS9w4LLjreNtTUryJIawc1FVfXu8+Q9LL8kY/7x/rvomcAxwYpLfMFzOfyzD6++3jJcCQ7sz3wnsrKqlZ1R3MISgluf9WuDuqnqgqv4FfJvhMdDDvJcsmm/T57ok7wROAE4eAx+03fMLGYL9DeP5bStwfZLn0nbfakuXj9UFWaxlj8hiSb4xb0mbYlEOa91qWexVM9e0WVrO2GtakMNatVYGa5bLp43xM+CQDL8Nay+GN8S7bOaaJjG+19EFwG1V9Zlld10GnDJ+fgrw3c2ubSpVdVZVba2qgxhm+6OqOhm4GnjL+GVN9bykqu4D7kly6HjTa4BbaXjeDJd4H51k7/HxvtRz8/NeZtF8LwPeMf4mtKMZLoPftdo32N0kOY7h5RwnVtU/lt11GXBSkicnOZjhDbh/OkeNG62qbqqq51TVQeP5bSdw5PjvvtlZqzndZLAla2SxZi3IYs1fCbNGDmvdalms+TdaH7WcsRdaI4c16VEyWLNcPm2A8X1g3gv8kOHEeGlV3TJvVZM5Bng7wzNOvxw/3gicDbwuyR0Mz1acPWeRm+QjwIeS3MnwHlAXzFzPVN4HXJTkRuDlwCdoeN7js4s7gOuBmxjOk+fR6LyTXAz8BDg0yc4kp7J4vt8H7mJ40+2vAO+ZoeQnbEHPXwCeDlw5nte+DDCeyy9lCPuXA6dV1b9nKv0JWdD3Ik3MWu3rLIMtWZTF1KbVcljT1shiTVlnBmvGenJYK9aZwZqV9q9okyRJkiRJ0ly88kmSJEmSJEmTcfkkSZIkSZKkybh8kiRJkiRJ0mRcPkmSJEmSJJAkMJYAAAAlSURBVGkyLp8kSZIkSZI0GZdPkiRJkiRJmozLJ0mSJEmSJE3mvxYjW2ck3p/3AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 1440x720 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-nhrZDHvPx1-",
        "colab_type": "text"
      },
      "source": [
        "Let's now evaluate our model on both the training and testing sets."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L9mKLM7BkuVz",
        "colab_type": "code",
        "outputId": "7884e7ea-0349-4824-cd80-7c4add36f76c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "print('RMSE of training set:', evaluate(train_set))\n",
        "print('RMSE of testing set:', evaluate(test_set))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "RMSE of training set: 1.0002025748958006\n",
            "RMSE of testing set: 1.108309473614627\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JEZtM1UiQUrA",
        "colab_type": "text"
      },
      "source": [
        "Now we will pick a user from the database and look at his/her preferences."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t-KBJYqbU--g",
        "colab_type": "code",
        "outputId": "5a80faa9-26a8-42d1-ed08-0c75c83635ec",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "user_id = 45\n",
        "df_join[df_join['userId'] == user_id].sort_values(by=['rating'], ascending=False).head(10)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>movieId</th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "      <th>userId</th>\n",
              "      <th>rating</th>\n",
              "      <th>timestamp</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>39202</th>\n",
              "      <td>2078</td>\n",
              "      <td>Jungle Book, The (1967)</td>\n",
              "      <td>Animation|Children|Comedy|Musical</td>\n",
              "      <td>45</td>\n",
              "      <td>5.0</td>\n",
              "      <td>950726248</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>46609</th>\n",
              "      <td>2694</td>\n",
              "      <td>Big Daddy (1999)</td>\n",
              "      <td>Comedy</td>\n",
              "      <td>45</td>\n",
              "      <td>5.0</td>\n",
              "      <td>951170525</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>46142</th>\n",
              "      <td>2657</td>\n",
              "      <td>Rocky Horror Picture Show, The (1975)</td>\n",
              "      <td>Comedy|Horror|Musical|Sci-Fi</td>\n",
              "      <td>45</td>\n",
              "      <td>5.0</td>\n",
              "      <td>950741248</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>55533</th>\n",
              "      <td>3578</td>\n",
              "      <td>Gladiator (2000)</td>\n",
              "      <td>Action|Adventure|Drama</td>\n",
              "      <td>45</td>\n",
              "      <td>5.0</td>\n",
              "      <td>1020803425</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>33101</th>\n",
              "      <td>1584</td>\n",
              "      <td>Contact (1997)</td>\n",
              "      <td>Drama|Sci-Fi</td>\n",
              "      <td>45</td>\n",
              "      <td>5.0</td>\n",
              "      <td>950740871</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>56625</th>\n",
              "      <td>3717</td>\n",
              "      <td>Gone in 60 Seconds (2000)</td>\n",
              "      <td>Action|Crime</td>\n",
              "      <td>45</td>\n",
              "      <td>5.0</td>\n",
              "      <td>1007994717</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45369</th>\n",
              "      <td>2580</td>\n",
              "      <td>Go (1999)</td>\n",
              "      <td>Comedy|Crime</td>\n",
              "      <td>45</td>\n",
              "      <td>5.0</td>\n",
              "      <td>951756858</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>56996</th>\n",
              "      <td>3753</td>\n",
              "      <td>Patriot, The (2000)</td>\n",
              "      <td>Action|Drama|War</td>\n",
              "      <td>45</td>\n",
              "      <td>5.0</td>\n",
              "      <td>1007995101</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>57287</th>\n",
              "      <td>3793</td>\n",
              "      <td>X-Men (2000)</td>\n",
              "      <td>Action|Adventure|Sci-Fi</td>\n",
              "      <td>45</td>\n",
              "      <td>5.0</td>\n",
              "      <td>1007995171</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>57418</th>\n",
              "      <td>3798</td>\n",
              "      <td>What Lies Beneath (2000)</td>\n",
              "      <td>Drama|Horror|Mystery</td>\n",
              "      <td>45</td>\n",
              "      <td>5.0</td>\n",
              "      <td>1007995171</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       movieId                                  title  ... rating   timestamp\n",
              "39202     2078                Jungle Book, The (1967)  ...    5.0   950726248\n",
              "46609     2694                       Big Daddy (1999)  ...    5.0   951170525\n",
              "46142     2657  Rocky Horror Picture Show, The (1975)  ...    5.0   950741248\n",
              "55533     3578                       Gladiator (2000)  ...    5.0  1020803425\n",
              "33101     1584                         Contact (1997)  ...    5.0   950740871\n",
              "56625     3717              Gone in 60 Seconds (2000)  ...    5.0  1007994717\n",
              "45369     2580                              Go (1999)  ...    5.0   951756858\n",
              "56996     3753                    Patriot, The (2000)  ...    5.0  1007995101\n",
              "57287     3793                           X-Men (2000)  ...    5.0  1007995171\n",
              "57418     3798               What Lies Beneath (2000)  ...    5.0  1007995171\n",
              "\n",
              "[10 rows x 6 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lfsqp9f3QcXi",
        "colab_type": "text"
      },
      "source": [
        "Let's look at the least preferred items."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bkHatTbO19dL",
        "colab_type": "code",
        "outputId": "09a3fda4-1456-460c-d827-074f81b8094e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "df_join[df_join['userId'] == user_id].sort_values(by=['rating']).head(10)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>movieId</th>\n",
              "      <th>title</th>\n",
              "      <th>genres</th>\n",
              "      <th>userId</th>\n",
              "      <th>rating</th>\n",
              "      <th>timestamp</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>53154</th>\n",
              "      <td>3270</td>\n",
              "      <td>Cutting Edge, The (1992)</td>\n",
              "      <td>Comedy|Drama|Romance</td>\n",
              "      <td>45</td>\n",
              "      <td>1.0</td>\n",
              "      <td>950718581</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5111</th>\n",
              "      <td>173</td>\n",
              "      <td>Judge Dredd (1995)</td>\n",
              "      <td>Action|Crime|Sci-Fi</td>\n",
              "      <td>45</td>\n",
              "      <td>1.0</td>\n",
              "      <td>950741335</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>44072</th>\n",
              "      <td>2470</td>\n",
              "      <td>Crocodile Dundee (1986)</td>\n",
              "      <td>Adventure|Comedy</td>\n",
              "      <td>45</td>\n",
              "      <td>1.0</td>\n",
              "      <td>950740565</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>67895</th>\n",
              "      <td>5585</td>\n",
              "      <td>Ernest Scared Stupid (1991)</td>\n",
              "      <td>Comedy</td>\n",
              "      <td>45</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1034344038</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>62486</th>\n",
              "      <td>4502</td>\n",
              "      <td>Ernest Saves Christmas (1988)</td>\n",
              "      <td>Children|Comedy</td>\n",
              "      <td>45</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1034344018</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>60222</th>\n",
              "      <td>4104</td>\n",
              "      <td>Ernest Goes to Camp (1987)</td>\n",
              "      <td>Comedy</td>\n",
              "      <td>45</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1034344018</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>53115</th>\n",
              "      <td>3268</td>\n",
              "      <td>Stop! Or My Mom Will Shoot (1992)</td>\n",
              "      <td>Action|Comedy</td>\n",
              "      <td>45</td>\n",
              "      <td>1.0</td>\n",
              "      <td>950740376</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>46306</th>\n",
              "      <td>2672</td>\n",
              "      <td>Thirteenth Floor, The (1999)</td>\n",
              "      <td>Drama|Sci-Fi|Thriller</td>\n",
              "      <td>45</td>\n",
              "      <td>1.0</td>\n",
              "      <td>950741061</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>54299</th>\n",
              "      <td>3439</td>\n",
              "      <td>Teenage Mutant Ninja Turtles II: The Secret of...</td>\n",
              "      <td>Action|Children|Fantasy</td>\n",
              "      <td>45</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1034344018</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15160</th>\n",
              "      <td>569</td>\n",
              "      <td>Little Big League (1994)</td>\n",
              "      <td>Comedy|Drama</td>\n",
              "      <td>45</td>\n",
              "      <td>2.0</td>\n",
              "      <td>950722240</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       movieId  ...   timestamp\n",
              "53154     3270  ...   950718581\n",
              "5111       173  ...   950741335\n",
              "44072     2470  ...   950740565\n",
              "67895     5585  ...  1034344038\n",
              "62486     4502  ...  1034344018\n",
              "60222     4104  ...  1034344018\n",
              "53115     3268  ...   950740376\n",
              "46306     2672  ...   950741061\n",
              "54299     3439  ...  1034344018\n",
              "15160      569  ...   950722240\n",
              "\n",
              "[10 rows x 6 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iGu6-AnIQgyy",
        "colab_type": "text"
      },
      "source": [
        "Let's now look at the most likely preferences of the selected user."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EX_poCW7WcZO",
        "colab_type": "code",
        "outputId": "d8e8cd67-3b00-4836-8a00-d87b349070a9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "predictions = np.zeros((n_movies, 1))\n",
        "movie_to_column_items = np.array(list(movie_to_column.items()))\n",
        "df_result = pd.DataFrame(columns=['UserID', 'MovieID', 'Movie', 'Genres', 'Prediction'])\n",
        "\n",
        "for i, movie in enumerate(movie_to_column_items):\n",
        "    predictions[i] = predict(user_id, movie[0])\n",
        "    \n",
        "indices = np.argsort(-predictions, axis=0)\n",
        "\n",
        "for j in range(10):\n",
        "    movie_id = int(movie_to_column_items[np.where(movie_to_column_items[:, 1] == indices[j])][0][0])\n",
        "    df_row = pd.DataFrame({\n",
        "        'UserID': user_id,\n",
        "        'MovieID': movie_id,\n",
        "        'Movie': df_movies[df_movies['movieId'] == movie_id].iloc[0]['title'],\n",
        "        'Genres': df_movies[df_movies['movieId'] == movie_id].iloc[0]['genres'],\n",
        "        'Prediction': predictions[indices[j]][0][0]\n",
        "    }, index=[j])\n",
        "    df_result = df_result.append(df_row, sort=False)\n",
        "    \n",
        "df_result"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>UserID</th>\n",
              "      <th>MovieID</th>\n",
              "      <th>Movie</th>\n",
              "      <th>Genres</th>\n",
              "      <th>Prediction</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>45</td>\n",
              "      <td>417</td>\n",
              "      <td>Barcelona (1994)</td>\n",
              "      <td>Comedy|Romance</td>\n",
              "      <td>3.562902</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>45</td>\n",
              "      <td>27611</td>\n",
              "      <td>Battlestar Galactica (2003)</td>\n",
              "      <td>Drama|Sci-Fi|War</td>\n",
              "      <td>3.555125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>45</td>\n",
              "      <td>3969</td>\n",
              "      <td>Pay It Forward (2000)</td>\n",
              "      <td>Drama</td>\n",
              "      <td>3.548344</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>45</td>\n",
              "      <td>104841</td>\n",
              "      <td>Gravity (2013)</td>\n",
              "      <td>Action|Sci-Fi|IMAX</td>\n",
              "      <td>3.546065</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>45</td>\n",
              "      <td>1754</td>\n",
              "      <td>Fallen (1998)</td>\n",
              "      <td>Crime|Drama|Fantasy|Thriller</td>\n",
              "      <td>3.542547</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>45</td>\n",
              "      <td>114180</td>\n",
              "      <td>Maze Runner, The (2014)</td>\n",
              "      <td>Action|Mystery|Sci-Fi</td>\n",
              "      <td>3.541998</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>45</td>\n",
              "      <td>5785</td>\n",
              "      <td>Jackass: The Movie (2002)</td>\n",
              "      <td>Action|Comedy|Documentary</td>\n",
              "      <td>3.526598</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>45</td>\n",
              "      <td>1755</td>\n",
              "      <td>Shooting Fish (1997)</td>\n",
              "      <td>Comedy|Romance</td>\n",
              "      <td>3.515884</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>45</td>\n",
              "      <td>1711</td>\n",
              "      <td>Midnight in the Garden of Good and Evil (1997)</td>\n",
              "      <td>Crime|Drama|Mystery</td>\n",
              "      <td>3.500507</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>45</td>\n",
              "      <td>177765</td>\n",
              "      <td>Coco (2017)</td>\n",
              "      <td>Adventure|Animation|Children</td>\n",
              "      <td>3.495129</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  UserID MovieID  ...                        Genres Prediction\n",
              "0     45     417  ...                Comedy|Romance   3.562902\n",
              "1     45   27611  ...              Drama|Sci-Fi|War   3.555125\n",
              "2     45    3969  ...                         Drama   3.548344\n",
              "3     45  104841  ...            Action|Sci-Fi|IMAX   3.546065\n",
              "4     45    1754  ...  Crime|Drama|Fantasy|Thriller   3.542547\n",
              "5     45  114180  ...         Action|Mystery|Sci-Fi   3.541998\n",
              "6     45    5785  ...     Action|Comedy|Documentary   3.526598\n",
              "7     45    1755  ...                Comedy|Romance   3.515884\n",
              "8     45    1711  ...           Crime|Drama|Mystery   3.500507\n",
              "9     45  177765  ...  Adventure|Animation|Children   3.495129\n",
              "\n",
              "[10 rows x 5 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S2xU-gjQQrwZ",
        "colab_type": "text"
      },
      "source": [
        "Now the predictions for least preferred items."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d2nL5rsz2dR8",
        "colab_type": "code",
        "outputId": "3924eb3e-d164-48c7-9738-f2cff38bd4c1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "df_result = pd.DataFrame(columns=['UserID', 'MovieID', 'Movie', 'Genres', 'Prediction'])\n",
        "indices = np.argsort(predictions, axis=0)\n",
        "\n",
        "for j in range(10):\n",
        "    movie_id = int(movie_to_column_items[np.where(movie_to_column_items[:, 1] == indices[j])][0][0])\n",
        "    df_row = pd.DataFrame({\n",
        "        'UserID': user_id,\n",
        "        'MovieID': movie_id,\n",
        "        'Movie': df_movies[df_movies['movieId'] == movie_id].iloc[0]['title'],\n",
        "        'Genres': df_movies[df_movies['movieId'] == movie_id].iloc[0]['genres'],\n",
        "        'Prediction': predictions[indices[j]][0][0]\n",
        "    }, index=[j])\n",
        "    df_result = df_result.append(df_row, sort=False)\n",
        "    \n",
        "df_result"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>UserID</th>\n",
              "      <th>MovieID</th>\n",
              "      <th>Movie</th>\n",
              "      <th>Genres</th>\n",
              "      <th>Prediction</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>45</td>\n",
              "      <td>2901</td>\n",
              "      <td>Phantasm (1979)</td>\n",
              "      <td>Horror|Sci-Fi</td>\n",
              "      <td>2.054488</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>45</td>\n",
              "      <td>3104</td>\n",
              "      <td>Midnight Run (1988)</td>\n",
              "      <td>Action|Comedy|Crime|Thriller</td>\n",
              "      <td>2.220623</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>45</td>\n",
              "      <td>26761</td>\n",
              "      <td>Prime Suspect 2 (1992)</td>\n",
              "      <td>Crime|Drama|Mystery|Romance|Thriller</td>\n",
              "      <td>2.227821</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>45</td>\n",
              "      <td>96518</td>\n",
              "      <td>Prime Suspect 6: The Last Witness (2003)</td>\n",
              "      <td>Drama|Thriller</td>\n",
              "      <td>2.227821</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>45</td>\n",
              "      <td>160422</td>\n",
              "      <td>The Wooden Horse (1950)</td>\n",
              "      <td>Drama|War</td>\n",
              "      <td>2.227821</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>45</td>\n",
              "      <td>121099</td>\n",
              "      <td>101 Dalmatians II: Patch's London Adventure (2...</td>\n",
              "      <td>Animation|Children</td>\n",
              "      <td>2.237293</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>45</td>\n",
              "      <td>165489</td>\n",
              "      <td>Ethel &amp; Ernest (2016)</td>\n",
              "      <td>(no genres listed)</td>\n",
              "      <td>2.237293</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>45</td>\n",
              "      <td>96520</td>\n",
              "      <td>Prime Suspect: The Lost Child (1995)</td>\n",
              "      <td>Drama|Mystery</td>\n",
              "      <td>2.246765</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>45</td>\n",
              "      <td>93740</td>\n",
              "      <td>Salmon Fishing in the Yemen (2011)</td>\n",
              "      <td>Comedy|Drama|Romance</td>\n",
              "      <td>2.256236</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>45</td>\n",
              "      <td>91414</td>\n",
              "      <td>Arthur Christmas (2011)</td>\n",
              "      <td>Animation|Children|Comedy|Drama</td>\n",
              "      <td>2.275180</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  UserID MovieID  ...                                Genres Prediction\n",
              "0     45    2901  ...                         Horror|Sci-Fi   2.054488\n",
              "1     45    3104  ...          Action|Comedy|Crime|Thriller   2.220623\n",
              "2     45   26761  ...  Crime|Drama|Mystery|Romance|Thriller   2.227821\n",
              "3     45   96518  ...                        Drama|Thriller   2.227821\n",
              "4     45  160422  ...                             Drama|War   2.227821\n",
              "5     45  121099  ...                    Animation|Children   2.237293\n",
              "6     45  165489  ...                    (no genres listed)   2.237293\n",
              "7     45   96520  ...                         Drama|Mystery   2.246765\n",
              "8     45   93740  ...                  Comedy|Drama|Romance   2.256236\n",
              "9     45   91414  ...       Animation|Children|Comedy|Drama   2.275180\n",
              "\n",
              "[10 rows x 5 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ORbaz1CIOWIJ",
        "colab_type": "text"
      },
      "source": [
        "I hope you enjoyed this exercise. Something you can also try on your own is to implement gradient descent instead of MAP-estimation like I did here. Let me know about any comments you may have."
      ]
    }
  ]
}