14-Boosting_Stacking.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/UDEA-Esp-Analitica-y-Ciencia-de-Datos/EACD-04-MACHINE-LEARNING-1/blob/master/14-Boosting_Stacking.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
    "\n",
    "\n",
    "**Recuerda que una vez abierto, Da clic en \"Copiar en Drive\", de lo contrario no podras alamancenar tu progreso**\n",
    "\n",
    "Nota: no olvide ir ejecutando las celdas de código de arriba hacia abajo para que no tenga errores de importación de librerías o por falta de definición de variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget --no-cache -O init.py -q https://raw.githubusercontent.com/UDEA-Esp-Analitica-y-Ciencia-de-Datos/EACD-04-MACHINE-LEARNING-1/master/init.py\n",
    "import init; init.init(force_download=False); "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Boosting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Como vimos en la clase anterior, en Machine Learning los métodos de ensamble sigen la idea de entrenar múltiples modelos simples y combinar sus respuestas para producir un mejor resultado. A diferencia del Bagging en Boosting la combinación se hace de manera secuencial y no paralela:\n",
    "\n",
    "![alt text](local/imgs/Boosting1.png \"Boosting1\")\n",
    "\n",
    "La idea central de los métodos de Boosting es entrenar modelos secuencialmente haciendo que los modelos subsiguientes se enfoquen en las muestras que no han podido ser modeladas correctamente por los modelos anteriores. Cada modelo tiene un peso diferente dentro de la decisión final. La forma de asignar el peso a las muestras y a los clasificadores dependen del método de Boosting particular.\n",
    "\n",
    "![alt text](local/imgs/Boosting2.png \"Boosting2\")\n",
    "\n",
    "Imágenes tomadas de: https://quantdare.com/what-is-the-difference-between-bagging-and-boosting/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Los dos métodos reducen el error por varianza; boosting es mejor para reducir el bias pero puede presentar problemas de sobreajuste, razón por la cual algunos métodos de boosting requieren un ajuste cuidados de parámetros de regularización."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Adapting Boosting (Adaboost)\n",
    "\n",
    "En Adaboost el modelo de ensamble se define como una suma sopesada de $M$ modelos débiles:\n",
    "\n",
    "$$F({\\bf{x}}) = \\sum_{m=1}^M \\alpha_m f_m({\\bf{x}})$$\n",
    "\n",
    "donde $\\alpha_m$ son los pesos y $f_m(\\cdot)$ son los modelos base.\n",
    "\n",
    "El algoritmo de entrenamiento consiste en adicionar clasificadores débiles uno a la vez, en un proceso de optimización iterativo, tratando de encontrar en cada iteración el mejor par $(\\alpha_m, f_m)$ para adicional al ensamble actual. \n",
    "\n",
    "Dado un conjunto de muestras de entrenamiento $\\mathcal{D} = \\{({\\bf{x}}_i,y_i)\\}_{i=1}^N$\n",
    "\n",
    "***\n",
    "\n",
    "**Algoritmo AdaBoost**\n",
    "\n",
    "* Inicialice los pesos de las muestras $w_m(i) = 1/N$\n",
    "\n",
    "* Para $m$ desde 1 hasta $M$\n",
    "    * Entrene un clasificador $f_m: \\mathbb{R}^d \\rightarrow \\{-1,+1\\}$ teniendo en cuenta los pesos $w_m$\n",
    "    * Estime el error $\\epsilon_m = \\sum_{i:f_m({\\bf{x}}_i) \\neq y_i} w_m(i)$\n",
    "    * Estime el peso del clasificador $\\alpha_m = \\log\\frac{1 - \\epsilon_m}{\\epsilon_m}$\n",
    "    * Actualice los pesos $w_{m+1}(i) = w_{m}(i)\\exp(-\\alpha_m y_i f_m({\\bf{x}}_i))$\n",
    "* El modelo obtenido será igual a $F({\\bf{x}}) = \\text{sign}\\left(\\sum_{m=1}^M \\alpha_m f_m({\\bf{x}})\\right)$\n",
    "\n",
    "\n",
    "![alt text](local/imgs/AdaBoostAlgorithm2.png \"AdaBoost\")\n",
    "\n",
    "Imagen tomada de: https://towardsdatascience.com/understanding-adaboost-2f94f22d5bfe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Los pesos de cada una de las muestras dentro del algoritmo AdaBoost pueden ser tenidos en cuenta de dos formas:\n",
    "\n",
    "* Los pesos de cada muestra se pueden usar para establecer una nueva distribución $D_{m+1}(i) = \\frac{D_{m}(i)\\exp(-\\alpha_m y_i f_m({\\bf{x}}_i))}{Z_m}$, donde $Z_m$ es un factor de normalización que garantiza que la distribución cumplea la restricción estocástica $\\sum_{\\forall i} D_{m}(i) = 1$. Se puede usar un proceso de muestreo sobre la distribución $D_{m+1}$ en la cual tendrán mayor probabilidad de aparición las muestras sobre las que el modelo ha cometido más errores. Dicho muestreo se puede realizar fácilmente usando una distribución multinomial (numpy.random.multinomial).\n",
    "\n",
    "* Los pesos de cada muestra se pueden usar con clasificadores base que puedan tratar cada muestra con una importancia diferente. En el caso de los árboles de decisión, se puede, por ejemplo, usar el peso de cada muestra para medir la ganancia de información. Cuando se estima la probabilidad de quedar a cada lado de una partición $a$, en lugar de contar 1 por cada muestra, se usa el peso de dicha muestra (sample_weight)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEICAYAAABcVE8dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOydeXxU5fX/3+dOdhK2gIBACCgqCogKUqW44dK6r9U2KFRtVKy19dvFNr+vWm36tVpbqYqWulFIq627ldaqdd/BugIKYhLCTlhDCElmnt8f987MncxMMpNMZiaT83698srMfe7c+9x7537umfOccx4xxqAoiqJkPlaqO6AoiqIkBxV8RVGUXoIKvqIoSi9BBV9RFKWXoIKvKIrSS1DBVxRF6SWo4CuISIOIjEnBfq8SkY3O/ouTvO8Oj1lEpovI58nqkxIbIvJvESlLdT96IqJx+KlDRKqBIYAXaAHeAq40xqzpxn2+AiwyxtzfXfuIsR/ZwE7ga8aYjyK0lwJfAbudRbuB94G5xpgXktTNhCMi04F/+t8CBQSPEeBgY0xtN+5/f2Cls08DbAHmGWNu7659KumDWvip5wxjTCEwDNgI3JXi/iSLIUAe8FkH6/V3zs+hwAvAkyIyu5v71m0YY143xhQ6x3SIs7i/f1lbsRcRS0QSfp86+yoCLgJ+KSLHJ3ofIpKV6G0qXUMFP00wxjQBjwEH+5eJSD8R+bOIbBaRGhH5f/6b3xGC/+cs3+Ss189pyxORRSJSLyLbReR9ERkiIpXAdOBux6Vxt7O+cSw/RORhEblHRJ4TkV0i8q6I7Ofq08ki8rmI7BCReSLyqohcHumYRCRXRO4UkXXO353OsgMAv6tku4j8J4bzs8EYMxe4CfiN6zzsKyKPO+foKxH5gWv/HhH5hYh86RzLUhEZGeGYTxWRZc46a0Xkx87y40SkzrW9cSLyinNOPxORM11t7Z63eBCRN0TkFhF5G9sSLxGR/iLykIisF5E6EbnZ/SAQkctFZIWIbBORf/qPM4bz+i6wApjk2tYIEXnSdU6vdrUVON+t7c45u975pepvrxORn4jIJ07fO9re10TkAxHZKbZ773bXfv7i+g6/JyKDXOdntvPaEpEbXPfBwyLS12nb37nOlzj92iwi18d9QTIJY4z+pegPqAZOdF4XAAuAP7va/ww8DRQBpcAXwGVO26XAKmAMUAg8ASx02q4AnnW26QGOAPo6ba8Al7fphwH2d14/DNQDRwJZQBXwiNM2CNsNc67Tdi22K+ryKMd3M/AOsA8wGNtldYvTVursNyvKZyO2O8drgHHYBstS4AYgx2lbDZzirPsT4BPgQGz3yaFAcYRjXg9Md14PAA53Xh8H1Dmvs53z/QtnXycAu4ADOzpv7Vz/aMf4hvPdGOfsN8u5nvOcazrEOW7/d+E87Afogc66NwGvR9nn/oBxXgswDdiD/UsT55x+6DrO/Z2+zHDafwv8B+gPjAQ+Bapd269z+jYCyI9he+8D33ZeFwFTnddXA0852/AAk4FC1/mZ7bwux74vRjuffxp4yH2swH3YvyYPB/YCY1N976dMc1Ldgd7853zxG4Dt2MK5DpjgtHmAZmyfrn/9K4BXnNcvAXNcbQc628jCfhi8BUyMsM9X6Fjw73e1nQqscF5fArztahNgTdvtudq/BE51vT/FLw50XvDznOXTgKlAbZv2n7tu+M+Bs6Js333Mtc657dtmneMICv50YANgudr/CtzU0Xlr5/pHO8Y3gBtc74dji3Kua9nFwAvO6xeAWa62LEfYhkfYp18EtzvbNMBvCI7nTQNWt/nM/wJ/cp2rGa62KwkX/Etc7zva3lvYD+ziNuuUO+dhQoRjcAv+q0C5q+0Q59gt17EOdbV/AJzf3fd2uv6pSyf1nG2M6Y8tZN8HXhWRodjWdDZQ41q3BvvmB9g3QlsWtvW3EHgeeMRxpdwm9iBprGxwvW7E/gXh32dgQNnYd1Ad0YnUx33j6Eck/Me/FRgF7Ov85N8uItuxLckhzjojsR86HXEetkDXOC6qoyKssy+wxhjjcy1zXw+Ift46g3vgfhSQC2x0Hec9BI9zFHCPq20L4MO2siPifOcKgZ9hP9j8/vZR2C4k9zn9KTDUaR/Wpm+RAgza9r297X0X2435ueO2OdVZ/jDwIvA3x812q0QeE4j0HcvB/kXpP9ZEXpcejQp+mmCM8RpjnsCO2Pk69k3bgn3D+CkB1jqv10VoawU2GmNajDG/NMYcDBwNnI5tnYNt8XSW9bhERESEdkQlSh/XdWH/AOcAm7Ct9zXAV8aY/q6/ImOMXzTWAB360Y0x7xtjzsJ2PT0F/C3CauuAkRI6gOq+HonGfZ3WYAvVQNdx9jXGTHS1X9bmPOQb2z8ffQf2d+42Z19XuLa1MsI5PcNp30DoNY80VtC271G3Z4z53BhzEfa5vwN4XETyjDHNxpibjDHjsO+Hc4BIoZiRvmPNwOb2jr23ooKfJojNWdg+5OXGGC+28FSKSJGIjAKuAxY5H/kr8CMRGS0ihcCvgUeNMa0icryITBARD7bPvQXb4gM7EqizMffPARNE5GzH2rqaoKUWib8C/09EBjsDbje4+h8XYg86fx+4Efi5Y2m/B+wSkZ+JSL7Yg7TjRWSK87H7gVtEZKxzfidKm3h/EckRkTIR6WeMacE+X24r3s+72KL7UxHJFpHjgDOARzpzPPFg7DDdV4HfikhfZ6ByfxE5xlnlPqBCRMY5x9RfRM6PYxe3Aj8TkRzgbaBZRP5H7MF/j/NdOsJZ92/AL5x9jMD+DrRHu9sTkYtFZJBzPXdgPyx8InKCcy0twr/Dbv4KXCcipSJSBFQCf23zS0xxUMFPPc+KSAP2l7oS2xfrD1W8BjvSYTW23/IvwINO24PYrpvXsOPVm5z1wRbhx5xtLscWi4VO21zgfLGjOf4QT0eNMVuAC4DbsAcoDwaWYPtMI/Erp/1j7MHTD5xl8bBdRHY7nz8VuMAY86DTHy/2r5dJ2OdgC7bI93M++ztsgfo39rl4AHsQsC0XA9UishPbJx1mSRpjmrEF/pvOfuZh+6pXxHk8nWUm0AdYBmwD/o7zsDXG/B37WP/uHMPH2OMlsfIM9sPsMmNMK/Z5PhJ7jGkL8Eegr7PujdhGQzX2ef0b0a8/MWzvVGC5iOzCHhC+0DnX+2IHIuzEDt19Efv735Y/AY8Cr2PfJ7uwgwmUCGjildJpHOurDigzxryc6v4oyUdErsEeh5qR6r4oHaMWvhIXInKK83M+F3uAVLBDL5VegIgMF5GjHbfSOOBHwJOp7pcSG5oJp8TLUdg/rXOw3QtnG2P2pLZLShLJxXajlGK7lv6K7aJRegDq0lEUReklqEtHURSll5DWLp1BgwaZ0tLSVHdDURSlx7B06dItxpjBkdrSWvBLS0tZsmRJqruhKIrSYxCRmmht6tJRFEXpJajgK4qi9BJU8BVFUXoJae3DVxQltbS0tFBXV0dTU1Oqu6K0IS8vjxEjRpCdHXshXBV8RVGiUldXR1FREaWlpdjFUZV0wBhDfX09dXV1jB49OubPJcSlIyIPOtOLfRql/Tixp8T70Pm7IRH77a1UVVVRWlqKZVmUlpZSVVWV6i4pGUpTUxPFxcUq9mmGiFBcXBz3L69EWfgPA3djT8kXjdeNMacnaH+9lqqqKsrLy2lsbASgpqaG8vJyAMrKIpULV5SuoWKfnnTmuiTEwjfGvIY9A5HSzVRUVATE3k9jYyMVFRUp6pGiKD2FZEbpHCUiH4nIP0XkkGgriUi5iCwRkSWbN+ukNW2pra2Na7mi9HQuvfRS9tlnH8aPHx+yfOvWrZx00kmMHTuWk046iW3btnV5X7/+9a9D3h999NFd3ibAK6+8wltvvZWQbXWFZAn+B8AoY8yhwF3Y08hFxBgz3xgz2RgzefDgiNnBvZqSkpK4litKT2f27Nn861//Clt+6623MmPGDFauXMmMGTO49dZbu7yvtoKfKJHujOC3trYmZN9ukiL4xpidxpgG5/ViINuZ8k6Jk8rKSgoKCkKWFRQUUFlZmaIeKUr3cswxxzBw4MCw5U8//TSzZs0CYNasWTz1VLgd6fV6+clPfsKUKVOYOHEif/yjXcl5/fr1HHPMMUyaNInx48fz+uuvc/3117Nnzx4mTZoUGA8rLLTnO3/llVc49thjOeussxgzZgzXX389VVVVHHnkkUyYMIEvv/wSgGeffZapU6dy2GGHceKJJ7Jx40aqq6u57777+P3vf8+kSZN4/fXXqa6u5oQTTmDixInMmDEj8At99uzZXHnllUydOpWf/vSnvPrqq0yaNIlJkyZx2GGHsWvXri6dy6SEZYrIUOzJtY2IHIn9oKlPxr4zDf8XsaKigtraWkpKSqisrNQBW6Xb+eWzn7Fs3c6EbvPgffty4xlRPbztsnHjRoYNGwbA0KFD2bhxY9g6DzzwAP369eP9999n7969TJs2jZNPPpknnniCU045hYqKCrxeL42NjUyfPp27776bDz/8MOL+PvroI5YvX87AgQMZM2YMl19+Oe+99x5z587lrrvu4s477+TrX/8677zzDiLC/fffz2233cYdd9zBlVdeSWFhIT/+8Y8BOOOMM5g1axazZs3iwQcf5Ac/+EHggVVXV8dbb72Fx+PhjDPO4J577mHatGk0NDSQl5fXqXPlJyGCLyJ/BY4DBolIHfa8l9kAxpj7gPOBq0SkFdgDXGS0EH+nKSsrU4FXFBciEjFq5d///jcff/wxjz32GAA7duxg5cqVTJkyhUsvvZSWlhbOPvtsJk2a1OE+pkyZEnjA7Lfffpx88skATJgwgZdftmf4rKur48ILL2T9+vU0NzdHjZF/++23eeKJJwC4+OKL+elPfxpou+CCC/B4PABMmzaN6667jrKyMs4991xGjBgR6ymJSEIE3xjz7Q7a78YO21QUpYfSWUu8uxgyZAjr169n2LBhrF+/nn322SdsHWMMd911F6ecEj6n+2uvvcZzzz3H7Nmzue6667jkkkva3V9ubm7gtWVZgfeWZQX87ddccw3XXXcdZ555Jq+88go33XRT3MfVp0+fwOvrr7+e0047jcWLFzNt2jSef/55DjrooLi3Geh3pz+pKEq3o0l20TnzzDNZsGABAAsWLOCss84KW+eUU07h3nvvpaWlBYAvvviC3bt3U1NTw5AhQ/je977H5ZdfzgcffABAdnZ2YN3OsGPHDoYPHx7ok5+ioqIQ//vRRx/NI488AtjXePr06RG39+WXXzJhwgR+9rOfMWXKFFasWNHpvoEKvqKkLf4ku5qaGowxgSS73ib63/72tznqqKP4/PPPGTFiBA888ABgW78vvPACY8eO5cUXX+T6668P++zll1/OwQcfzOGHH8748eO54ooraG1t5ZVXXuHQQw/lsMMO49FHH+Xaa68FoLy8nIkTJ3baZXrTTTdxwQUXcMQRRzBoUDAu5YwzzuDJJ58MDNreddddPPTQQ0ycOJGFCxcyd+7ciNu78847GT9+PBMnTiQ7O5tvfvObneqXn7Se03by5MlGJ0BReiulpaX08TRzwZH7hizPz8/jmGOPjfgZS4SDDxpH0ZGJGeNZvnw548aNS8i2lMQT6fqIyFJjzORI62vxNEVJU2pra1l01WF8Z/DK8MavPon+wdXAxNMhr1+39U3pmajgK0qaUlJSQo4HvmotZtztdYHlo0aO5KNPPg5bf2+rj7m3/pwbsheCt/N+aCVzUR++oqQplZWVZFngNRZ7m1vY29yCJyubG355M3n5fcL+8vP70ESO/WFf4rM0lZ6PCr6ipCllZWUUDxyAEQsRYdSoUcyfPz/qgKIl4PXf0j5vEnuq9BTUpaMoaUyf/HyyycXn83W4rscSl+Crha+Eoxa+oqQxYrz4YrxNRQSvUcFXoqOCryQUTRRKND6MxH6bGsvjvOj4F0FPYM2aNRx//PEcfPDBHHLIISHx6loeOX5U8JWEoYlCiUeMwRD7zEa+DHPpZGVlcccdd7Bs2TLeeecd7rnnHpYtWwZoeeTOoILf01ixGN77U/Bv2TOp7lEAnY0r8Yjx4hNPzOv7xBmWy5BB22HDhnH44YcDdnmCcePGsXbtWkDLI3cGHbTtSezZDo9EqFP345VQGF44KtnobFyJR4jXwnceDt1h4f/zetjQTsJXZxg6Ab4Zm2VeXV3Nf//7X6ZOnQpoeeTOoILfk2jaYf//xq0w4QJY9hQ89z+wd1daCH5JSQk1NTURlyudQ4wP4vDh4/fhZ4iF76ehoYHzzjuPO++8k759+4a1a3nk2FDB70k0N9j/i4ZCn0HQxxH5lsbon0kilZWVlJeXh7h1dDauriHGF5eF7/U/HEw3CH6MlniiaWlp4bzzzguInh8tjxw/6sPvSex1BD+nyP6f7Ux12LInNf1pQ1lZGfPnz2fUqFExJQopseCLy4ePdKNLJwUYY7jssssYN24c1113XUiblkeOHxX8FBJvCKO3yf7C/PDJlZz0u1e57snPAWhp2t3tfY2VsrIyqqur8fl8VFdXq9h3EdulE7uFbySzXDpvvvkmCxcu5D//+U9g8HLx4sWAlkfuDFoeOUX4Qxjbuj/as4h3Ln2Mvs9extV978IMOYS8TR/zux0/ZMsZCxh0xNnJ6rqSRJZVHo2xsjnk56/GtP7lt/yB+73/Cxc/Bfsd3+X9a3nk9Cbe8shq4aeIzoQwepvsCaS/ecT+zCs7gtOPGANAaxpZ+EpisUyciVcZZuEriUUFP0V0JoTRu8d26Xjy+jr/7cGd1r3pMWirJB7Bh4njNg0IfncM2io9HhX8FBEtVLG9EEaf48O38uxB2+w8OynE16wWfsYSZ1im6YZB23R2+/ZmOnNdVPBTRGVlJQUFBSHLOgphNHt30Ww85OTmA5Cda3/epxZ+xmLhw9eZWjoJcunk5eVRX1+vop9mGGOor6+POxErIXH4IvIgcDqwyRgzPkK7AHOBU4FGYLYx5oNE7Lun4h+YraiooLa2lpKSEiorK9uNDjB7G9hNPrnZtgDkOC4dX3PqBL+qqipwDAMHDgTguwfu4tYZeViWBCPI8/rB1e/aOQRKzIgxxGWXJdjCHzFiBHV1dWzevDkh21MSR15eXtyJWIlKvHoYuBv4c5T2bwJjnb+pwL3O/15NWVlZ7OFfPi/9177MFvLIzbJv6rzcHPaabEwS4/Crqqq49tprqa+vD2vzLztvXAGrt3l57HPh9NNPY0JJMXy4CDavgKKhYdsoLi5m7ty5GsIZAcGHSWFYZnZ2dtRsUaXnkRCXjjHmNWBrO6ucBfzZ2LwD9BeRYYnYd6/hq9fI272WvSabPMfCz8u22EMOJsEWfrT8gKqqKr773e+Gif3hwyxmHZrNrEOzmT0pm8OHeXhseSu/eKGBM373Dkx3EmZ2ro+4jfr6ei699FKtqhkBC19QxGPAZzk2nA7aKhFIVmmF4cAa1/s6Z9n6tiuKSDlQDlqDJYRd9qm6puUa/uC38LM97CE3oZm2bfMD/CWOwXY/ubMQsy147jsFnLRf+Nfo31/aLoXa2tqgG2fXeioq/hAxk7G5uZmKigq18tsQdy0dyazyyEpiSbtaOsaY+cB8sBOvUtyd9GH3FgBqzBBysxwLP8tDvclBWhNn4beXH9A2ZPSIfT2ctF8Wb69p5eIn99DqzLnR1Aobd9uXrqSkBHL6QG4/2LW+3bBTraoZjmDijMPPrPLISmJJluCvBUa63o9wlimx0liPV7LZTR552baFn5tt0UQufZu2QGN7HjUXnhzILYza3F5+QElJCbU1NQzIt33Kx5fa/Tjn0T0BgXcTEnVUNBS21zJx7AjWrKkLW3dPi2Gf4fqLri0WPoijeFp3hGUqmUOyBP8Z4Psi8gj2YO0OY0yYO0dph8YtNOUMgD0SiNLJzbL4j+8wvr/jKbgtxoE1Tw6c/yCMiZx2f9CYkSGi7zOwp5VAFFHrY+XMOjT4tVm9zRcQ++LiYsCeei4s6qjfCPjiX3z4bYCisP02NBueP/CnYcsjMWfOHObPn4/X68Xj8VBeXs68efNi+mxPw860jaN4mqUWvhKdRIVl/hU4DhgkInXAjUA2gDHmPmAxdkjmKuywzO8mYr+9isatNGb1B2xXDtg1wO+VCxkxZhJnH1jQ3qeDvP8APDozavOymQCh9cbLF3s59lpbvDfU3cnq9V/w+7ftcYMVuwpZtGhRx773U34NX9n1YN5f8j6PP/44u3fbCWPDB/bh+qk+zhtf4BIqx6qV0P9z5szh3nvvDWzW6/UG3mei6Aumcz58HbRVIpAQwTfGRJiGKaTdAFcnYl+9lt1baMzqjwhke4I/8XOys1ja9yTOnmqnP7jj4iPG9h90Gnz2JLSTSPPBfz/gn//8F9u3b+P2k/L4/swzmOhsY+g+g2HIEO668x/x9X+fg+w/YMrUK5hy9f3BNmPgzgnw7LX2XzvM2wfm3dgXn9N/Y+DeJS384L77WLx4ccw5DT0FiXsSc7+Fry4dJZy0G7RV2vDfKvj4Udj4GQ1FR5ObZYXM7JOX7WFPi23NtRdhE+JaOfqadnd5+DQ4/PvOm18OYOJ4Vy6d8QVnVUoUInDufKh+E3AeRMa0eQ1guPnmmwMfAbh4YjYTh1iBSdPBPu6ZM2dy+eWXc//99/do4beIs7RChs54pSQGFfx0Z+lDsGUlDJ3Ax1nHk7c9VGzzsj00tXipqqpi1qxZeL2hN7o/wqbToidWqHvACRNctWkXqzcnsobPGBg0JmJLlkc4er9B5GV7uPmkG0KO8egRHnKjfIubmpqYPXs2QI8VfStOl47ooK3SDir46c6ujXDAN+DcP/LhYx+Tm7UppHlPw06eeXcdf1//JcXn3hBxE3uASx58r1O7f8AnbN2xhyHOe5/Px+cbGjjt96/hS2LQ7M1nHcIlR5VSXl4e4sP3mfZjWFpbW3t0fL/EaeH7tDyy0g4q+OmMMdCwAYqGYIzhg9ptgbIKYLtwVr30D7JHT8bK6xN1M7k5uezc07lp27wGausbAoLf1NzCxl2G4QPyuec7h2PFkfbfWc6/7y3WbLXdVP6BWX+UjgGys9p3MfXk+P546+FblpZHVqKjgp/O7NkG3mYoHMq7X21l5aYGRg8KCntFRQX1NTXwxiNRNxGcRWtap7rQeKNlZ3s6tLS24iOLu799OBNH9O/UNuNlSN88NuzcG3g/b968YETOwnPZUvcl8HHUz/fkjO14XTqWZeHFwqMuHSUCWh45ndm1wf5fNJQNO5oA+L9zJwSaO7JcPR5PlycR92LZfnuH1lYvXiyG9Y+vLGtXGNI3j43O8YchFoOKBzJjxoyIzVlZWSElp+OdRzjVxBuWaVmCD0tdOkpE1MJPJc274fU7IFrxs13r7P9FQ9le1wzAAUOCSUslJSWByJS2dDQ/bqwYJMQ90OptRcRiUJ/cLm03Hob2zePDNdsjN4r9QHrxxZeZM2cO9913X6B2e2FhIffdd1/gHMQUxZRmeOL04VsCXjxkq4WvREAFP5XUvGULfk5hsI55G3ZlFTP95IuoHvx1+n+9jH88+XcumWmLU2VlZdhE6JDYcsM+JMzCz8nOwrK633fvZ0jfXDbsaOKel1eFtZ1a38gQaaEAx9VzzTdh9LGQ1zds3fbqBKWr4Mc7aOsRwSsetfCViKjgp5JGp0TwFa9B8X4hTW1rxg8YWYSvqYGrrijHI6G19OOZRCVeDFZIkpbP5+1wkDTRHDqyPy2+r7j9+c/D2g7IbkRy91AKsH2NnUW8/0kw87GwdTszj3CqiT8s03Hp6KCtEgEV/FTiF/yCgSGL27oeAKy8QrxNDWEWaVyTqHSCthZ+3Kn+CeD0iftyyiFDIyYHf3j73XYoEQT7tSHyAG40F1i6DuoaY+IWfI9lu3Q0Dl+JhA7appLGrfbNnNsvZHEk14OVV4hvjz2JeTItUiNtBN/4bKs/yWR7LHKywv+wxH4IQTD9dk9kf39n5hFOJT4T/wQolog90K4uHSUCKvipZM9WyB8IVuhliCToVn4Rvr0NQHItUh+hYZm2hZ88/33HSLB//v/evRHXLCsrY/78+YwaNQoRYdSoUQkZ2O4ufMZg4UOs+KJ0VPCVaKhLJ0XUbWuk8asaik0hF/7oTj744AP7Bhdh0PHfpbmlOWT97AH70lTzUdItUl+bsEzLeIPZnGmAESto4bv6GY3udoElEq/PkC1xhmWKqEtHiYoKfoq45+UvOWPTenaQw8rsMRRNDa0jkx/hM1k71ybdIjVp4MNvn/gEvydhfM7xxPGA9Yg/d0ItfCUcFfwk8cbKLbz8ebAOzhvvvsf/5S/j6RUt1D56Vtj6ImLPMOWOvrltUTK7DDguHdyCH+ccq92MEbdLJ7NmxPT63TJq4SsJIn3u3Aznzhe/4OG3qnn0/TUseP0LyuSfALy3LrIlZoyhuroan89HdXV1ytwQBgGfe9DW2MvSBkmKhZ+KDF2fX/Dj9OG3kAWfPQVVF3RTz5Seigp+ktjV1MqBhXtpWHAFX952Dvuv+ydL13n59evNEdf3eNLDT24X7kpnC9/t0nFZ+C/+MmH78IfJ1tTUBOrul5eXd7vo+1odKz2uKB24x3MxDB0P6z/qpp4pPRV16SSJjVt3sP2T1xiXvZbxY7M4fJiHx5dHr2DpT/lPNXaUTlBIBRNX9cbuR4IuJ5eF7135Ap4TbwyuVv9lsDZRNPoMhsEHhC1OVYauz/llJXHF4QuvyGEwYj0se7q7uqb0UFTwk8S+e77gL2Mf5+BJwTjwt9eEu3PSbVJugyCuAUALX1wC1O1I8IG0e28zfYBq3xBGbFyGd+9u8OTCrvVY9x6NtEYpwOZgxIPvqrdhoJ317HHKR9StqcUSwur/d3c+hF/wiaOMhYjY0z9aHh24VcJQwU8CxhjuKHiIcdYOrnu+iddqWmnxwScbQ33OV111VdoIvR87ycodlumzQzXTBCNBH35Tcwt9gFXZB1Dq3Qj/t29gvRbj4eqWH7GLyJO959LCvdl3kj/vyLC21v+1C9Z9tsnL+HuDs3x1dz6E8douHYkrSkfsB5N4QsZeFAVU8JPCnm3rGCvGOGwAACAASURBVGet4fZVo/n9O+F+1XSz6t34RMJcOvEMInY3xhWW6Q9jzBp3Km83TYGWoBumvs/+jC8+rt1tPb9tJEN2fQbAZ+t2UJibxUVTSvjoo4+o/+AZTij1kGVBqy85Gbpev4UexxzCltjx+2rhK5FQwe9ujCHv3ikAvLBpQEhTokoYdyemTeKV7S9PH8FHgmGjfhdIVm4+R51/Y3ufisLYwKuHFy6heksjFx17DIceC0vuvgy2PEbfXKFon9AidVVVVd1SwM50wodvWY5LRzTbVglHBb+7aazHatnNUt9YJp15JV988VW3VbbsDoy7dAHxV2/sdlwuHeO3aBOQCdw3L5udTcFB9clHHwvPPEb9umroH3TldGeNfZ/Xn3gVXxy+Ty18JQoJuXNF5Bsi8rmIrBKR6yO0zxaRzSLyofN3eSL2m25EjNXeVg3AvNYzOWXGcWkRWx8P4WGZ6VhLxxF8RyDjqT0Tjb752aHzAOcU2v+dekZ+2ovg6SrGSZ6SOFw6Hsvtw1fBV0LpsoUv9ojSPcBJQB3wvog8Y4xZ1mbVR40x3+/q/tKVaJbeqDvK+TqwxuxDYW7P+0HVtnialcZx+D6TQMHPy2Z3s5dWr48sjwW5zkxjzaGC35019r3e+DNtRcCrUTpKFBKhQEcCq4wxqwFE5BHgLKCt4Gc01157LY2NjQhw7uGD6FuQA8DWJY/DCKgzgynM63mCj7R16fjSKw5fLPshhNvn3fVfIP3y7Wu1s6mVgX1yXBb+rpD1urPGfuB44niAeUTsKR7918jnS6tBdiW1JOKbMBxY43pf5yxry3ki8rGIPCYiI6NtTETKRWSJiCzZvHlzArrXvVRVVTFo0KDAzFRHHTKcx85o5sEZDTw4o4EzR+zgK98QWjz5DOmbvIm/E4U7CgbSL0rHXVrB+B9MifDh52cDBN06uY7gt7Hwo9XYP/XUU7tciiFYWiHOevg+EzwHauUrLpJ15z4LlBpjJgIvAAuirWiMmW+MmWyMmTx48OAkda9z+N049fX1ZFkwbpDF6EH25N6XfXwoBz89lulvTGVL2Yu894sTGVSYvIm/E0VYeeS0jNIJDctMhIXfN88RfP/Ard+l08aHH6nG/qxZs1iwYEFIKYaZM2cyaNCguITfPwgdf5QOwYey+vEVF4nwMawF3Bb7CGdZAGNMvevt/cBtCdhvati6OlCj5M35P+G0/YWcwYdw9YH1HDW4EdgCwOqadSz/8EsWLVrElAPTcwq9WDBi2ZE5Dh5MQizohOEWfOfBZMVhEUfDb+E/8cFaPlqzndzmbXwLwix8CK+xX1paGjaQC1BfXx9XBE/nonScz+KxH8tq4SsuEiH47wNjRWQ0ttBfBHzHvYKIDDPGrHfengksT8B+U8MjM2GTnZwz73gAD7AGnxGWNI1kcp7t3Wr1GYqLi3tEJE57GCwsv2g40TCJGBRNGELggeTrxCBnNEYOzCfHY/HwW9UA5NDCt/Jg985t9Ongs+0N2MZTg8c41rnlif029Ti/bgLjLBk2R4DSNbos+MaYVhH5PvA8tvo9aIz5TERuBpYYY54BfiAiZwKtwFZgdlf3mxJ21MGmz3ijzzf58d+WsXdvM/2PuRjvzi1sbWji/LGtTHaGM7I8WcydOzfFHe46RgTalh9Oo7DMkEzbBEbpDOuXz39vOIk9LbboVm/ZTfNDHl7+ZDVvN3wStn5hXhZXHbsf/Qtyog7k+ok1gifgw4/jfFtWG8FXl47iIiFhI8aYxcDiNstucL3+OfDzROwrpXz1GgC3bD2WDTMuBmAbUP/GXVj5ffGNGxpYdfall/V46x4cQW07Z2yaRelImyid2267ncdPOLvLyW19crPo44TSFvfJocHTh727tvL8Z+FVN7fububd1VsZN6wvU+b8jj2vvUZra+RJSIoKi/j5E6EPjX752fzPyQeQ7Qme22CUTnyDtkBw4nO18BUXPTBOMHWYjctoNtl8tLya7R/cay/0tbJ33ef0OeQ4fASLdX19+rEp6mWCcYdldmIGpm7HNcbw/nvvMgao37otpG49dD3rVUQoGjmB81q3ct73TgprX/BWNfe8vIq12/cAfRk8aQY7d+4MVrx0baewXz9eXL4xsMzrM2zd3czR+xVzzAHBQAW/hR/XoG3Ah68WvhKOCn4c7N2wnK/MMPIa1rF3TaiF5m3YSitBS8xKkwlMuopxlR8mgWGPCcNVWuG5f/yDC48iRGQTWrd+xBR4+x5Y/o+wh96sgTDrPPcSDzCAV159lUWLFrJ582YGDx7MzJkXc9yxx/LKqy+waNFCtm7ZwkVHDKDP6EN56v2b2dsa7PvWum0cQvyZtuD24avgK0FU8ONg77plrDIlXHrB6fzm7SdCIjGyWxvtyaMd4hloS2eMe07bdHXpOA+k7Tu2A91Yt370dHjzTng09ofHccBxMwAKgN2w/j545D7X8nygCXiX+Z++y/c+CQa0HSJfcWEu9HFCRGPB79JRC1+JRGaoUhLYsbOBoqb1fGm+xtVlZ7NfYWtIhcQbbr6Bz1Z8HFg/kyx8yxF6Y7z2bLZpFaUjAZdO8cABQL1dPMxFwurW7zcDrn4PWvZ0aTOnnX4669evD7wfUmjxz7IChn70AJV/ej2wPG9zETwFo4qLYt52lscWfC+aeKWEo4IfIx+tWM4xYjjxa0dQXJgbFnsN8FzVFlhpv7as2K2ytMYV5+71+sgiPp9y92Nhid2/U75xCmxdZZcHdkho3XoRGHxglzfzzw/X2eUPHAYX2K9z6j9n/PB+wRWNk8Ebh0sny3kY+zRKR4lAOt25acl/VmzkoP/9J/c8ZUfoHDA2+g3fJy8n8NryZMapNa45Yzszx2q3E/BVGyZNPBSAAQMHBrJe03G+gba/OLY0Gpq9hnHD+4eu2Ikw2CzHh99qJHQbioIKfhhtSxwvXPwGTS0+yg62fwzlDIzuHhAr+IPJk5VBFr4jGj5nyr30Kp7mEjbHfXHrrb9J6xLUbevvGGBDAxwxdljId+/fz//LXiGO8+136agPX4mEunRcRCpxvPjFVxl42EmcWWpgFdB336ifdwt+xvjwXYlNgTDBtPLhBzNKjeO7t9KpfxHwP4TcY0At+R4O86zm/QsM0AeoJ3vVbyFPwIoj0zZg4WuUjhKOCr6LSJNZTBmwg9/Kj+HNvZDbN1hEKwLiisyx4rhJ05mQevO+xM0olSjEZeH7OjEHbKpoOwZ08dHD+dqAlrD1rLwirhp+RMzb9SduqYWvRCIzVClBRArfO3ngBvbzbIIxZ8HIqe1+3l20y5OVGafWjtKxRcMf/ZJOFr5x+fADxdPSyeUUI1XvrGeRMRFamrg6ryjmrGG18JX26Hl3RoJx++wjuQJG5zdQ29IPLngYvnZVu9sSjzvxKjME3x2lk8jiZAnD6Ysx3oA1m04PpFhpL3TUnTXcUXnl7EBYplr4Sjg9785IIH6fvb9ueWBKORejc7azI29ETNtzu3Q8GeLDxxWlY9JRUB3B9/mCPvx4MlPThUgTqbQllrlyPc61cTImNEpHCSFDzNDOEclnDzDrlIkM65tFQX4+oz1fsGKfo2PansdyW/iZEaVjrGCtGq8zqXY6Wfh+H77PF4zSSfdB20i0Hcg1Ed07HWcNZ7d16aiFr7jo1YIf6eYpGdKfh79WHbpwxJSYticukc8UHz64aun4Leg0GrQNsfATWB45FbgHcktLSzs1V67fh+8NxOGr4CtBeuad0QU68tkPGZAPwDuTfsPOH65m9//UcMTp5TFt2+2392RUlI4tpN5A9cb0qYcf8OH7fK4pDtPogdRJos2V21HWcJYTpRNIvFILX3HRqwQ/Fp/98IH2fEZZA0fSt38xfYr6h60TjZCwzAzJtLXLD4f68NMq7DFg4XsDbpCe6NJpS6S5cmPJGg4M2mqUjhKBzDBDY6CqqopZs2aFiLyVnUtpaQmWZeHzGXJyc7n4ghNh+wNkFcU/gbonnYQwUbjqzfs6MSFHdxPw4RsTHKDMAMGH8Fj9WPC7dFq0tIISgcy4Mzpgzpw5XHzxxWEW/QPfm8SX39nIyovW8+V3NvDKebs4uNSetSq3b/yCb2WM3z6I26UTnIEpHV063oC4WWnicmpbpqOjkMpE4E+8avVH6fhU8JUgmadQbXhoYRV/qnocq3AQOR7D8ELoO3w/bjguj7Py/ss/tpXw+toszv/6QUzZvpjdmz/BZ4S8ouK495UpkTkhhFj4fh9++lj4AcH3Bgdt0yEkNlKZjkTNvtUewUFbdeko4WS84P/fG9uZdNUdZONlQc5vGGutBZYD8NTmkVz71CZ+fdvv2F7cB95ZzMAtS9hBn5DKl7GSDkKTcEIs/Pin3Otugi4dX6fmgO0uIoX8JnT2rShkO+6sFp+GZSrhZLzgPzF4HuOz6gLvf/FJCet2Z7Hyiy94t3YFCxYsoKysjBc/WAFA/73r+NIMY2hu/KcmYy18E+rDT6daOsFM26APPx2iiKLFyyds9q0oeAKZthqWqYST8YI/1NrBmzsG8+fXa1lZ7+Pl6k8BWxQWLlwYsLYK+w/mXd9BHGZ9yRJzIN/KiV/UPFlpJIQJwhAepWOlkQ/fL+7G5y6tkPrrUFJS0qk4+q7iT7xq9mlYphJOQn6bi8g3RORzEVklItdHaM8VkUed9ndFpDQR+40FjxiW7xnI/KUtvFwdjCO/8sorQ35aD+yTw4XNNzA95xFuseZ0ykr0ZKCFL1Z4lE46hWUa59eGz7jDMlPfv87G0XcV9eEr7dFlwRd7BO8e4JvAwcC3ReTgNqtdBmwzxuwP/B74TVf3GyuWGEaMGBkSz7xw4ULmzZsXsl7/AlusN+7cS0EnrHvIoIJpLtxROqSRj9xP0MIPunTSIQeis3H0XcWfeNVsNEpHCScRCnUksMoYsxpARB4BzgKWudY5C7jJef0YcLeIiIlWMCSBiDEMGFhMdXV1u+v1zw8O0hZ2wn8PmVROwYU7SsekX/E0cWfaBmrppMcDqTNx9F0lmHilPnwlnETcucOBNa73dc6yiOsYY1qBHUD8cY+dwMLElIiTk2UFrPx+BZ1zzWSihS/iCQh+OkbpBDJtjc9V6yeN+pdkgolXGqWjhJN2CiUi5UA5JGaAS/DFPAfr/ZdMZtWmBiaVxF5OwU1WBgq+EQtLjD3BiDcNi5NZLgvfcT1lQmmFzuIPy2z2e3LUwldcJEKh1gIjXe9HOMsirVMnIllAP6A+0saMMfOB+QCTJ0/ussvHtk5jE4DJpQOZXDqw8/vKSJdOMArG5/eRp4nLBNw+fB+YnlsPP1FYliCi5ZGVyCTCFHofGCsio0UkB7gIeKbNOs8As5zX5wP/SYb/HhzBT1JcdlYmRuk4v468Xm+weFoauUzEZeEHBm3TKGw0FWRbVjAsUy18xUWXTVJjTKuIfB94HvAADxpjPhORm4ElxphngAeAhSKyCtiK/VBICoIvaQLlyc48wfe7THyu8sPpZeG7qmX60qe0QirxWEKrT6N0lHASooTGmMXGmAOMMfsZYyqdZTc4Yo8xpskYc4ExZn9jzJH+iJ5kYFv4sR1mV4tdZWWkS8ddfji9omAA17V1hWWmU/9SQJZH1MJXIpKBChXEGBOz4Cei2FVGWpb+xCafNzBnbHqVH3Zl2qZRaYVUkmWJ+vCViKTTnZtwjM/YESYxCH57xa5ixZOBQhPqMrHntE0nC9o/QOuvpeMz0qujdMBOvmr2j5Cpha+4yOg7wx9VEovgJ6LYVSYOFhqJ5MNPn+MUdxy+8eFD0qYefqrItoRmr9bSUcLJbMH3xS740WL+u7vYVboTyGT1el2ThKePhR8IG/W6BT/FfUoxHo8E57SNYuGnYnIWJfVkuOD7wwg7VoBUFbtKd/xhj8cddyy3/cYugfTGG2+mskshBMIyHQvfYPV6H362ZdFke99obW1lb6s35G/BwirKr5xDTd1ajOUJjFep6Gc+mT1o65/SMIb67f6B2YqKCmpraykpKaGysjLptVDSjZraNUwF1m9YzyFD7V878+69j+x++6bFuXHX0vFb+L2dnCyLf3y6kbvzYN5/VvC7f/+rzRr9GXz1XwLvdr7/FNv+cz+zZtmpMulwXXsCVVVVPU4vMlrw47HwITXFrtKdDz/6mG+NhTknjWX/AfZIYFNTU7fP3BQroRa+UcEHfnnmISyp2cbut4s5cajBc+CBIe0Vv/gF/jHdgrFT6XPwsWz7zwN4vd6kTMOYCSxauJA5c+bQ2NiIJULdmjVcUX4FkN7nrlcIfm8uptVVlq+ph7FwwwErAWgy2Wzc2sDmbbtS3DOHNqUVTGZ7KWNi6phipo4phtX7c7BnKwcfv39I++3r3wtMzuLduZlBZ/yYkdf9HW/DNtY//ANmzpxJRUVFj7BYU8Gnbz7LhauuZeb/WEBhYPnq1kGckCaGUDQy+u7wOdUbYi2e1lXcPtBMGQj7b+0e+v+ulX3ugn3uguLbGti0dWfaDGYHwzLVpRNG/1GwLXzWLfd41e7P32T7G1U0fv4W2QOGkTt8HID69duhcf1KssXLH9YcwK9WjuVXK8fy+u5RjMnawpraNR1vIIVktIUfnPKu+wXfn7hV9hP7lHYmcSsdqayspLy8nB27dgaWpdNgduDXm/E6g7Yq+AEGlMKnj8HSBSFuzbJxsN/ts3jgwQftSLbGx5C9WQycYNh17GgWtTbSsGYFjY2N6tePgD8f5TfPrmLd5u0A3PKdI5g+FkaXpochFI2MFvx4wjK7SjBxq29gmT9xqyffLOk+mO2PufcnXqnguxh2qJ19/OwPwpq+Bnzt9Nw2Sx+AUVD0ndn89ndfYVr2tuvX74mDlgnBEfys7OCkSV5nUOTGG29IRY9iJsMFP3nVHf0JWu/vGcEH2/KBpSHLezJpPZjdNkqnl4dkhjDudPjxSvA2R13lySef5LbbbqdubR14snj10n5cXPAK9ceN4cEXlgO24TJz5kzefPPNwNSgiShF0lPxV429+ZZfcePNldTW1pKdmw/Aueeek8qudUhm+/CTKPh+n/aRty3jyj8tDVuudBOuTGDBp4O2bSncB/qNiPp3zuxreHtZLbfOW8jWvdn8eWkDE6xqHjh6LaX9Qx+e9957LyeeeCKQmFIkPRVxdOWc886juroan8/HCc558XrTO7M5s++OJE55p4lbqSEwPmOMhmV2Af+k6796vYUfbbGt1ByP0Ha2z5deeomqqqqElCLpqfgtfI97hjsn18d4W1PRpZjJbJeOSZ6Fn+6+7kzFipBpq3QO/3f1tZeeBuDKydn86Gu5fLTBy6qtwbr6nsVXs+jCYnK9uwHY1Gi4ZnETXtNLftE6PnxPVvBp6I8W86rgpw5/sa9kxeGnta87Qwlm2mqUTiIoKytj65pPoAn2G2CLWAseDhgczFafMNgAzaxvyKbFByV94Q9LLWp30jt+0Uay8K1gGfF0JqMF3+f3p/XycrkZjcvCF43DTwiTxo2F/0L+oOHsMds4v//DIe13e+dyuuddLs/+NcWygz/zG8bN/hXfKd2XsrILU9PpZGIcC99dRNA/b0Sa+/AzWvADE3Zopm3GIv46ScYAmmmbCHKz7XNaUFiIj+3U//vekPbyLIujig/hwhtOo3jHp/AODJBdlB76tVR0N+mIz0uL8ZDtCX7XLMfa97ZGj4hKBzJb8E3yEq+U1OCvze+vpWM0LLPrOAbSwH5F+PYIDf99LqS5AVg6ahSPH1UKW40j+A1sbUhvsUsYPi++toaF36XjTe85hDNaCbWWTuYTuLY+26WjFn7X8Z9Ty3gBaT/6LH8AAEOy97B1995kdjN1mFZa23zPJODDT+9B24y+O0wg0zaNJuxQEovln/HKAOrDTwT+X8RibEt2/vz5jBo1ChFh1KhRzJ8/PxickNcPxMOw7Ea2NraksNfJw39eQrB6hg8/swXfP8Vhb58CKYOxArV0fM4k5nqtu4z4Bd+HEaGsrCyQYFRdXR0aiSZCk5UP22r4+9OLM6ZoYLv4vHgJNSIDRfzUwk8dwbBMtfAzFv/D3OcNCJTSNfwWvmW8Hf5iqqqqombTLopat2EV9OsVVTbFePFJFJdOmlv4XRq0FZGBwKNAKVANfMsYsy3Cel7gE+dtrTHmzK7sN1ZMwIevIpCpWP4MRydKJ+ynttIJ7PvFwtvhmEhFRQX3H+Nl6oi1VOS9QfPsyQDseH0eX03an9GHTO323iYbiWjh21LqS/PEq67eHdcDLxljxgIvOe8jsccYM8n5S4rYA/gCLh218DMVvzW6fPlyNm1Yz56m5t7hVuhG/L+ILePtMJGttraWn7zQhNfbyuW5LzCnZLX9N/RT6p+/PRndTT4mguB7eobgdzUs8yzgOOf1AuAV4Gdd3GbiSHKmrZJ8/L/eXv7PS5x3gA9D76rc2B2I4yazIg1OtqGkpIQPa2oYcVt9yPJ3r59A4d6N3dbHVBJp0La3ROkMMcasd15vAIZEWS9PRJaIyDsicnZ7GxSRcmfdJZs3b+5S5/yDtir4mYv/Rjv/QB8H9dmFz9hi1VsqN3YLzjm1XTrtW/jRigY25Q+hqKU+yqd6NuLz4m0zLmj5vQhp7sPvUAlF5EUR+TTC31nu9YzfiRqZUcaYycB3gDtFZL9o+zPGzDfGTDbGTB48eHA8xxKGfwIUTbzKXPoPHs56bz+OL1rDvp7tfLwzOMdob6jc2B34DSRPDIO2/iqbbcM2s4pLGeirp6qqitLSUizLyhhXW/sWfnoLfocuHWPMidHaRGSjiAwzxqwXkWHApijbWOv8Xy0irwCHAV92rsuxY5JYD19JDUX9BjJhoUVNzVZnydZAW6+o3NgN+N1kFl68MTgBIhUNfGPBCvpsbuK6a+awaZs9PWZNTQ0zZ87koYce4sUXX0x8x5OELfhRBm0z3KXzDDDLeT0LeLrtCiIyQERyndeDgGnAsi7uNzbUpdMr0LkIEosEXDq+sPDDWLH6DgNg/33y7O35/xBeeukl5syZk6juJh0x4S4d8fjr4ad3aYWuDtreCvxNRC4DaoBvAYjIZOBKY8zlwDjgjyLiw37A3GqMSYrgB+LwNUono9G5CBKL38L3GG+n8xo8Iw6j+UMPb17URJPpzzpTjEHY7C3ktH8NYtEbX3DiB3VMGtmfMYMLO95gGiEmPFzVXzwt3ROvuiT4xph6YEaE5UuAy53XbwETurKfzqJx+L0HnYsgcbgt/M7WJjrssKmcf8tUphRuYlBuC4Nz9jIibw9fG7CeiSdfyVdmGNf97SOOHD2Qv11xVCK73+1EsvCDgt/Dffg9GXscGY3DV5Q4CAza0vkJZXKyLC68cCYzZ84MLPvG/ln8s6yAlqd+wfpNOZTd8SRfbdmdkD4nE8t4MVEybdN9isOMdm4brZapKHHjj2rz0HGUTnuUlZUxY0bQAbB1j22A9W3dxmUXnsnw/vlsa+x5JZXtiXYih2Wme5RORithIA7fk9GHqSgJJWjhh1uy8fLiiy9y1VVX4fF4qG+078dvnTGDefPmMaAgmx17WvD6okVzdz+dCRu1aMUX5tJx3qe5Dz+jlTDZc9oqSibgz7T1xFBaIRbmzZtHa2srq+q2ADD7W3YKz4A+ORgDO/akpqxyVVUV5eXl1NTUYIyJufCbZbzhgm/1DB9+RithwMLXxCtFiR1HzDyS4Ckj8/oBAnvs+ooD++QApMytU1FRQWNjY8iyWDK0xfjCBT9LBT/1aHlkRYkby2UgJbTctOWB/P7QaCfH9S9wBH93agQ/WiZ2RxnannbDMlXwU4bOaaso8eN2gSbCpRNC/gDY8DF8/DfGrHuO0bKeF5ZtZPOu5E+PGC0Tu6MMbcGHsSJn2qa74PeKsEz14StK7IhrhriEzxE8YDR8+RKseZeRwP05+zLjtaE07G2l8pzkputUVlZSXl4e4taJJUPbilBaQQdt04BAWKZa+IoSM24XaMJnELuoCq75wP478Sb2k3U8WPQn8je8H3H17iy+Fq3wW0cJfBZeTBs3sUddOmmA0dIKihIv7vsl4RZ+dj4U72f/HTEbhkxgWuvbXLnploBv30+kKJqZM2cyaNCghAl/u/P1RsEyXowV6hwJnDMV/BSixdMUJW4st0unO8uS5A+Aq95g8YjrGGS2wsOnhzRHiqIBqK+vT+m8uZYJLzmhFn4a4I/Dt9SloyixI91o4Udg3ehzWew9EjZ9BtuDETLtRcukcoIbC2/YoK2VlW2/MOkt+Bk+aOuUKtWwTEWJGbeFT6KjdCIwtF8Bv239Fqd63qPu3rN5evAVrOhzJCVn/pCaZ+ZSWXYYBw2IUHZYYOkd54Qt3uUZQNPxv+Qbh47slv56Ivrwe4ZLJ6MF3+/SCf0CK4rSHu5B287Ww4+HyaUDyBt6EPfsvoyr9z7A9HUPsyh7fzhoBgfVvsEv9vucjb6+NHhz2naUnIYVIYtyaGaI2cIPn5/CNw69qlv6axlf1EFbFfwUovXwFSV+Ql2g3W8sjSruw+JrpwPT4d99mPjufTx5fn/KHlzKOWdNBvM5Zz2dw7sfrw58pqCgICSipqqqioqKCnZvXsPmnxRSuvM9tjdeFkjuSiSRLHxLLfw0QOe0VZS4cd8vCU+86ojRx8Bbf2Bo1Qm8lAsY2GkKmPM/N7LhhhsiTnDjj+bxD/B+WJ/ND4ufgNueiHv3SwaexuQf/KXddawIiVeBQVv14aeOQC0d9eErSsz4p+uz3yTZWNpvBnz7EWjezYoNu/hy8276jzyYS6afwCWXXBLxI22jec6p2s6sk8czoG8hR06dGvOu+697jQO3vQrGQDvRSR58GAmVTnXppAOBOHz14StKrAjuWjpJFnzLggO/CcBBE+CgGD7SNpqnepuXXz76ESKCb/4bMe/68T9VMm7tbVD9BoyeHr2L+Ox+uhDLwmsk7aN0MtvXoYlXihI3IXH4yXbp2sK7NwAACv5JREFUdILO1sRpy87Bk+0XC06PbKlvr4WVL5BFK0i4rezFSnsLP6MF3wSidFTwFSVWQnz4PSBpsbKykoKCgpBlsdTEaVu2YWnNTv7aerzduD1CDsBfLoSq88nCR1NW37BmH5Za+CnFn3jVA760ipIuSJITr9oSb/2cztTEiVS24aF75/KY9xh7hS1fhH6gaSdsWg5TLucC369ZMuTCsG36sBC18FOIP/FKo3QUJWZCwjK7s7RCBDo7C1W8NXEilW3w7t7OKjMcgKd/VcbRJVmcO2UEz99/C3z2BGBg7Cl87BuDZGeHbdOLFdScOOjOAnFtyWglDLp0MvowFSWhdGvxtA7o7CxU8RKpbEPrjk3soJDVvqGcNaaFty7twxOn7eKUut/Cs9fS6jMccebltHq9eCI8CH1iIe2UR44k7J19wHWWLkXpiMgFwE3AOOBIY8ySKOt9A5gLeID7jTG3dmW/MaM+fEWJG8stZkm28Ds7C1W8lJSUUFNTE7LMu2szW577PWecVsn4hnewWkIfPFtyhrH5tEOwjLB82WfwjdAYIh8etm/fSmlpaSBf4NRTT2Xx4sXU1NQgIoE5OvzCPnpQHvsXNUGR+8HaxMO3Xx9T5c546WpY5qfAucAfo60gtkPwHuAkoA54X0SeMcYs6+K+O0ajdBQlbsSTusSrSELsX55IKisrufjiiwMC7Gf3py+xbuAItgwZDRSFtPmaG2nd8Q/weXl5y6dw3Xkh7a1GGLr7c66YPggYZD8rG97jymMGIQwCQp+fBVlw5YgvyZLCsP5t8pqwZYmgS4JvjFkOIO1bAUcCq4wxq511HwHOApIo+OrSUZRYCflFnOSAh87OQhUvZWVlvPnmm9x3330hoi8ibH9tQYef3xFB875oGsi0/K84dL+O9+8z9uff3lvKvSv3AUIFPisnj4c73kzcJCPxajiwxvW+Doia/iYi5UA5dP2prj58RYkfK4VhmX43RkVFRcQyColk3rx5TJs2LWRfp556KgsWLIhYh99NJG2afvvHIbpt6NhKLy6uY8+elWEPuPnz58d+IHHQoeCLyIvA0AhNFcaYpxPdIWPMfGA+wOTJk7v2u0YFX1HixkrhoC3Yot8dAh/rvqZNm8a1115LfX19xM9E+8URzR0VjYKCAubOnQsk5wEHMUTpGGNONMaMj/AXq9ivBdyFqUc4y7ofHbRVlLgJKUWS5EHbdKCsrIwtW7awaNEiRo0aBQTr3bcX4x8pAawtfve3ezudmWaxsyTDpfM+MFZERmML/UXAd5KwX0SrZSpK3IQUG+zFSYvx/tKI5I7yR+kkw3qPha6GZZ4D3AUMBp4TkQ+NMaeIyL7Y4ZenGmNaReT7wPPYYZkPGmM+63LPY8BfqlQtfEWJHXdYZk+opZNOJNMd1Rm6GqXzJPBkhOXrgFNd7xcDi7uyr07hjL6r4CtK7FiWhc8IlpgeUUtHiZ2Mu5rubLZ333kL0EFbRYkHEfA5ln0HIddKDyOjlLBtmnLz3iYA/vLXv6a4Z4rScxCRgOCnIkpH6T4y6mr663DcMXsyf7j0CE4Y1ojXSMLrcChKphPw3atLJ6PIqBmv/PU2Zo1cR740A/DfvftSW7uivY8pitKGoOCrSyeTyCjB9yc+DLplnWvplkAsraIoseFzfvzroG1mkVFXs7Mz3yiKEoq6dDKTjLqanZn5RlGUcHyB+PuMkoheT0a5dCD9Ex8UpSegFn5moldTUZQwjOigbSaigq8oShjBQVsV/ExCBV9RlDDUpZOZ6NVUFCUMX0AaVCIyCb2aiqKEoRZ+ZqJXU1GUMFTwMxO9moqihKGCn5no1VQUJQwV/MxEr6aiKNFRwc8o9GoqihIdDcPPKFTwFUWJSsiE5kqPRwVfUZSoaKZtZqGCryhKVER9+BmFXk1FUaJjqURkEl26miJygYh8JiI+EZncznrVIvKJiHwoIku6sk9FUZKJCn4m0dV6+J8C5wJ/jGHd440xW7q4P0VRkoBgAJ3iMNPokuAbY5YDiA7sKEpGoj78zCJZV9MA/xaRpSJS3t6KIlIuIktEZMnmzZuT1D1FUSKigp9RdGjhi8iLwNAITRXGmKdj3M/XjTFrRWQf4AURWWGMeS3SisaY+cB8gMmTJ5sYt68oSjcglv56zyQ6FHxjzIld3YkxZq3zf5OIPAkcCUQUfEVRUo/W0slMuv1qikgfESnyvwZOxh7sVRQl3VHBzyi6GpZ5jojUAUcBz4nI887yfUVksbPaEOANEfkIeA94zhjzr67sV1GU7sUfpaODtplFV6N0ngSejLB8HXCq83o1cGhX9qMoSopQwc8o9GoqihJCVVUVXq8XgL///TGqqqpS3CMlUajgK4oSoKqqivLyYOT0jl27KC8vV9HPEFTwFUUJUFFRQWNjI83GLotsDDQ2NlJRUZHinimJoKulFRRFySBqa2sB+N/3+nDmgQN44cOakOVKz0YFX1GUACUlJdTU1LDohU9Z9ELocqXnoy4dRVECVFZWUlBQELKsoKCAysrKFPVISSQq+IqiBCgrK2P+/PmMGjUKEWHUqFHMnz+fsrKyVHdNSQBiTPqWq5k8ebJZskTL5yuKosSKiCw1xkScn0QtfEVRlF6CCr6iKEovQQVfURSll6CCryiK0ktQwVcUReklpHWUjohsBmo6+fFBQG+bNF2PuXegx9w76OwxjzLGDI7UkNaC3xVEZEm00KRMRY+5d6DH3DvojmNWl46iKEovQQVfURSll5DJgj8/1R1IAXrMvQM95t5Bwo85Y334iqIoSiiZbOEriqIoLlTwFUVRegkZJ/gi8g0R+VxEVonI9anuTzIQkQdFZJOIfJrqviQDERkpIi+LyDIR+UxErk11n7obEckTkfdE5CPnmH+Z6j4lCxHxiMh/ReQfqe5LMhCRahH5REQ+FJGElgvOKB++iHiAL4CTgDrgfeDbxphlKe1YNyMixwANwJ+NMeNT3Z/uRkSGAcOMMR+ISBGwFDg7k6+ziAjQxxjTICLZwBvAtcaYd1LctW5HRK4DJgN9jTGnp7o/3Y2IVAOTjTEJTzTLNAv/SGCVMWa1MaYZeAQ4K8V96naMMa8BW1Pdj2RhjFlvjPnAeb0LWA4MT22vuhdj0+C8zXb+Msdai4KIjABOA+5PdV8ygUwT/OHAGtf7OjJcCHo7IlIKHAa8m9qedD+Oa+NDYBPwgjEm448ZuBP4KeBLdUeSiAH+LSJLRaQ8kRvONMFXehEiUgg8DvzQGLMz1f3pbowxXmPMJGAEcKSIZLT7TkROBzYZ8//btX/VKKI4iuPf0wRkG5sgwgqmkPS2m8oiGIvUKUxlaV4gL5E3sEswCBoQBEMg2wQERTCCJA+QNFY+QOBYzC3SbDezF+aeDwzzpxhOM4fLnZ9/1s6yZBu2nwNbwNuyZduLsRX+LfDk3v20PIuRKfvYH4Ej259q51km2/+AOfCydpaBzYDtsqd9DLyQdFg30vBs35bzX+CEbqu6F2Mr/B/AM0lrklaAHeBz5UzRs/ID8x1wZfugdp5lkLQq6WG5fkA3mHBdN9WwbO/bntp+Svctn9t+XTnWoCRNyiACkibAJtDb9N2oCt/2HbAHnNL9yPtg+0/dVMOT9B74BqxLupH0pnamgc2AXboV369yvKodamCPgbmk33QLmzPbTYwpNuYRcCHpEvgOfLH9ta+Xj2osMyIiFhvVCj8iIhZL4UdENCKFHxHRiBR+REQjUvgREY1I4UdENCKFHxHRiP/DwefIw0wUOgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.ensemble import AdaBoostRegressor\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "# Create a random dataset\n",
    "rng = np.random.RandomState(1)\n",
    "X = np.sort(5 * rng.rand(80, 1), axis=0)\n",
    "y = np.sin(X).ravel()\n",
    "y[::5] += 3 * (0.5 - rng.rand(16))\n",
    "\n",
    "# Fit regression model\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "clf_b = DecisionTreeRegressor(max_depth=5)\n",
    "clf_1 = AdaBoostRegressor(base_estimator=clf_b, n_estimators=10, random_state=0).fit(X, y)\n",
    "clf_2 = AdaBoostRegressor(base_estimator=clf_b, n_estimators=20, random_state=0).fit(X, y)\n",
    "# Predict\n",
    "X_test = np.arange(0.0, 5.0, 0.01)[:, np.newaxis]\n",
    "y_1 = clf_1.predict(X_test)\n",
    "y_2 = clf_2.predict(X_test)\n",
    "\n",
    "plt.plot(X,y,'o',color='black')\n",
    "plt.plot(X_test,y_1,label='10 estimators')\n",
    "plt.plot(X_test,y_2,label='20 estimators')\n",
    "# Plot the resu  et\")\n",
    "plt.title(\"Boosting of Decision Tree Regression\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeRegressor(max_depth=5, random_state=1634131518)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf_1.estimators_[1]."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gradient Boosting\n",
    "\n",
    "Es un algoritmo de Boosting que aproxima el problema en poco diferente al caso anterior, en lugar de ajustar los pesos de las muestras, el Gradient Boosting se enfoca en la diferencia entre la predicción actual y el valor deseado (ground truth). El modelo global se define de la misma manera que en el caso anterior:\n",
    "\n",
    "$$F({\\bf{x}}) = \\sum_{m=1}^M \\gamma_m f_m({\\bf{x}})$$\n",
    "\n",
    "\n",
    "Sin embargo, en este caso el algoritmo minimiza el error a través de una forma de gradiente. En el paso inicial la función $F_0 = 0$.\n",
    "La función de salida en el paso $m = 1,...,M$ está dada por: \n",
    "\n",
    "$$F_m({\\bf{x}}) = F_{m-1}({\\bf{x}}) + \\gamma_m f_m({\\bf{x}})$$\n",
    "\n",
    "El nuevo modelo $f_m$ intenta minimizar la función de pérdida:\n",
    "\n",
    "$$f_m = \\underset{f}{\\mathrm{argmin}} \\sum_{i=1}^N L(y_i,F_{m-1}({\\bf{x}}_i) + f({\\bf{x}}_i))$$\n",
    "\n",
    "Es decir, debe entrenarse usando los valores objetivo dados por:\n",
    "\n",
    "$$r_m(i) = -\\frac{\\partial L(y_i,F_{m-1}({\\bf{x}}_i))}{\\partial F_{m-1}({\\bf{x}}_i)}$$\n",
    "\n",
    "Estos valores se conocen como los residuales del paso $m$. Por lo tanto el algoritmo de Gradient Boosting tiene forma de algoritmo de gradiente."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "La longitud del paso en el gradiente se estima como:\n",
    "\n",
    "\\begin{equation*}\n",
    "\\gamma_m = \\arg\\min_{\\gamma} \\sum_{i=1}^{N} L\\left(y_i, F_{m-1}({\\bf{x}}_i)) - \\gamma f_m({\\bf{x}}_i) \\right)\n",
    "\\end{equation*}\n",
    " \n",
    "\n",
    "Los algoritmos para clasificación y regresión sólo difieren en la función de opérdida $L$.\n",
    "\n",
    "![alt text](local/imgs/Gradient_Boosting3.png \"AdaBoost\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Aunque el Gradient Boosting puede ser implementado para cualquier clasificador base, el algoritmo más usado usa **árboles de decisión** como modelos base. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEICAYAAABcVE8dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxU1fn48c8zM1kIOyFAWDIRQWULQcKiuKHgVsGtbg0KdYlgq1grSr+oVfuLxfarwheLGlckadVat1bqLtVqXYKigmgFTMK+hJ0Qksyc3x/3zjBJZrJOMsnM83695pWZe8/cc+7M5Jkz555FjDEopZSKfo5IF0AppVTr0ICvlFIxQgO+UkrFCA34SikVIzTgK6VUjNCAr5RSMUIDfgSISJGITLLv/4+IPNFK+d4tIvmtkVcdZThZRL6PQL4iIk+LyG4R+ayV827QObfmZ0E1jIg4ReSAiKRFuizhoAG/BhG5XEQ+FZGDIrLdvn+DiEhL5GeMuc8Yc21zjyMi6SJiRMQVjnKFi12mQb7HxpgPjTHHRqAoJwGTgf7GmLE1d4rIDBHx2P/cB0TkR/sL4pjmZtzQcw7XZyGQ/SXiO6fyGue4Opx5hcj/2oA894nIlyJyTkvnGy7GGI8xppMxpiTSZQkHDfgBROTXwELgj0AfoDcwE5gAxId4jrPVCqiaww0UGWMO1pHmP8aYTkBXYBJwCFghIsNbo4Atwf4S6WSf10zsc7Rvw2qmb6EKw4d2/t2AJ4AXRKRzuDNpa5WdNskYozdrtHFX4CBwcT3pngEeAZbZ6ScBPwG+BPYBG4C7azznSqAYKAXmAUXAJHvf3UB+QNrxwMfAHuAr4LSAfcuB3wEfAfuBt4Ce9r4SwAAH7NsJQcp+N/Ai8Lz9/C+AkQH7h9h57AFWA1NrvD7PAjvsc7kDcNj7BgH/AvYCO4Hn7e0f2GU6aJfpMuA0YGPAcYuAW4Gv7ec/DyQG7L8N2AJsBq61jzcoxHvTF3gN2AWsBa6zt18DlAMeuxz3BHnuDODfQbb/A3ixge9PD+Bpu6y7gVfs7TXP+XZgk/0efA+cEeKzMNV+H/bY78uQhr5uIV6fWucIuOzX9Ab7NVtrbx8KvGO/lt8R8H8BJAIPYn3WtwGLQ+Vtv2fLAx53sfMbFbBtAvCJfZ4rgVMC9h0N/Jsjn/dHgGcCPncG+DnW5/+9BhzvGvu12w+sBy63tx+D9Xn1fYb/XOP1SbcfdwPysf4PioDfABJwrv8CHrLzXg+cGenYVu39iHQB2soNOBuoAlz1pHvG/lBMwPqFlGj/Q4+wH2fY/wQX2OmHYgWZU4AE+x+liiABH+iH9aVwrn2syfbjFHv/cmCd/eHsYD+eb+9Ltz+YIctv51UJ/BSIwwoYP9r347D+4f8H69fM6fY/xbH2c58FXgU623n9F7jG3vcXrC8y3+txUkCe1QI0wQP+Z1jBugewBpgZ8J5sBYYBSfY/Wl0B/wPs4ANk2v+Up9v7ZhAkoAc8N+h+4GpgWwPfn9exAm93+/U8teY5A8diBcq+Ae/b0UE+C8dgfVFOto91m/3+xNf3ujXmHDkS0N6wy90B6IT1hXSVvX+0fZ6+z8Ii4GU7fResys/vQuTpD/j2sWYDhzlSURlgH/ss+zU9GyvgJtv7Pwfux/pMnoL1mXzG3ucL+E/bn48OdR3PLuteYLD9/FRgqH3/r1hfxL7P8IQar0+6/fjPwEtY/wcD7fdkesC5VmJ9ZpzAjcCGSMe2au9HpAvQVm7ANGBrjW2+mtwh7FoCVsB/tp5jLQAesu/fBTwXsK8jUEHwgH87sLTGsd4M+EAtB+4I2HcD8IZ9P52GBfxPAh47sGrPJ9u3rdi1dnv/X+znOO0yDw3Yd33AP/KzQB5W+3jNPBsS8KcFPP4D8Kh9/yng9wH7BtU8XsC+AVg1+M4B237PkeAwg6YF/LOByvreH6zg4QW6BzmG/5ztc9iO9cswLsj74/ss3Am8UOO92oT9i6Ku160x58iRgBZYC84G3q+R7kmOfKmXA+6AfScDP4TI81qsCs4erGBYRvVfC/OAp2s85127DAOxvhw6BOx7jtoBP62Bx+til+NCavwiwQrkjwD9Qrw+6VhfvFXAMQH7fwG8E3Cu3wXs8/2a6VnX+9KaN23DP6IU6BnYDmiMOdEY083eF/habQh8ooiME5H3RWSHiOzFaivtae/uG5jeWG3IpSHK4AYuEZE9vhvWxcbUgDRbA+6XYdXGGiOwLF5go13Gvli1EW9A2mKsWm1PrA97cZB9YNU+BfhMRFaLyNWNLFOoc6r22tW4X1NfYJcxZn+IMjZVP6xmDaj7/Rlg57+7roMZY9YCN2MF9+0i8pyI9A2StC8Br7f9vmyocT7N/SwECnxt3cCEGud5GdZ59sH6pfpVwL5/AL3qOPa/7f+jHli/Bk6qkdcVNfIaz5HPZKkx5lCIcoYqe9DjGWP2AVdgBemtIvKPgIvyv8b6jBeKyDciMj1IPr2wKj+h/g+g9nsCzXtfwkoD/hH/wapNnN+AtKbG4z9jtR0PMMZ0BR7FCoBg1aAH+BKKSBLWz8tgNmDVILsF3DoaY+Y3oUyhBJbFAfTHanPeDAywt/mkYdUqd2LVztxB9mGM2WqMuc4Y0xer5r84sGdOM2yxy1er7EFsBnrUuBjoL2MzXAh8aN+v6/3ZYOffrb4DGmP+bIw5Cev1NFhNFjVtJuD1tnuJDaD55xOyWAH3NwDv1jjPTsaYX2I1V1ZgNe/49nW1P/d1Z2B9Gc8CrhGRjIC8ng7ymv4R6/1PFpHEgMPU+gwYuzrdgONhjPmnMWYS1pfXWuAxe/sWY8y1xphUrC+EPBE5qkZW27F+RQb9P2gPNODbjDF7gHuwgtVPRaSziDhEJBOrGaYunbFqd+UiMhb4WcC+F4HzROQkEYkH7iX0654PTBGRs+z+v4kicpqI9A+RPtAOrCaFgfWkGy0iF9m/ZG7G+pL7BPgUq0Zym4jEichpwBSs5igP8AKQa78ubuAWu7yIyCUBZdyNFTx8vxS2NaBMobwA/FxEhthflHeGSmiM2YDVBPd7+3XLwLpA1+hxB/Zrf5SILMJqjrnH3hXy/THGbAH+ifX56W6/hqcEOfaxInK6iCRgNY0c4shrVfPcfyIiZ4hIHFYN9LB9ji3tNWCYiPzMPo84ERkrIsfan4UngAUikiKW/iJyZkMObIzZgdVU53svlwIXisjkgNd0ooj0NcasA74Bfisi8SJyElYHibqEPJ6IpIrIFPuzVIF1jcQLICKXioivpr4H6zPsqVH2Sqz/5/tEpJP9hfArmvAZixQN+AGMMX/ACmS3YQWqbVg1gNup+x/tBuBeEdmP1Wb/QsAxV2PVGP6MVWPZjdWMEiz/DVi/MP4HK4BvAObQgPfJGFMG5AIf2T9lx4dI+irWz/PdWL2HLjLGVBpjKrAC/DlYNfrFwFXGmO/s592I9Q+yHqvXxJ+x/nEBxgCfisgBrGAx2xiz3t53N7DELtOl9Z1HjXP6J/B/wPtYtbFP7F2HQzzlCqy21s1YFxV/a4x5pxFZnmCfwz6s6yVdgDHGmG/s8tT3/lyJ9UvoO6za4M1B8kgA5mO9xluxmgl+UzORMeZ7rOtKi+y0U4Ap9vvUoowxe7Euek7D+sxuxboekmAn+TVWU8ZnWBdB3wIGNyKLh4CpIjLUGFOE9SvqTqzXtMQ+vu81vQLrYm0p8Fusi+Kh3n/qOZ4T6/3aYh/vRKz/TYBxwOcichDrouwvTPC+9zdgfVkUYfXIWYJ1Datd8HUnUqrNE5EhwCogwRhTFenyqNYnIn8DVhpjfhfpsrRHWsNXbZqIXCgiCSLSHaut++8a7GOH3ZR0lN28ei5wHvBKpMvVXmnAV23d9VjNI+uw2lRnRbY4qpX1xRpfsR+rKeg6XxObajxt0lFKqRihNXyllIoRbXqyoZ49e5r09PRIF0MppdqNFStW7DTGpATb16YDfnp6OoWFhZEuhlJKtRsiUhxqnzbpKKVUjNCAr5RSMUIDvlJKxYg23YavlIqsyspKNm7cSHl5eaSLompITEykf//+xMXFNfg5GvCVUiFt3LiRzp07k56ejrTMss6qCYwxlJaWsnHjRo46quaknqGFpUlHRJ4Sa8HvVSH2nyYie0VkpX27Kxz5xqqCggLS09NxOBykp6dTUFAQ6SKpKFVeXk5ycrIG+zZGREhOTm70L69w1fCfAR6m7lnjPjTGnBem/GJWQUEBOTk5lJVZaysUFxeTk5MDQHZ2diSLpqKUBvu2qSnvS1hq+MaYDziyKpBqQfPmzfMHe5+ysjLmzZsXoRIppdqL1uylc4KIfCUi/xSRYaESiUiOiBSKSOGOHTtasXjtQ0lJsCm6Q29Xqj3bsGEDEydOZOjQoQwbNoyFCxf69+3atYvJkyczePBgJk+ezO7dda4u2SD33Xdftccnnnhis48JsHz5cj7+uDXWrqlbawX8L7AWPR6JtaBDyOlNjTF5xpgsY0xWSkrQ0cExLS0trVHblWrPXC4XDzzwAN9++y2ffPIJf/rTn/j2228BmD9/PmeccQY//PADZ5xxBvPnN2Ql0LrVDPjhCtJNCfhVVeGfBbxVAr4xZp8x5oB9fxkQJyI963maCiI3N5ekpKRq25KSksjNzY1QiZRqOampqRx//PEAdO7cmSFDhrBpk7WE7Kuvvsr06dZa49OnT+eVV2rXIz0eD3PmzGHMmDFkZGTw2GOPAbBlyxZOOeUUMjMzGT58OB9++CFz587l0KFDZGZm+q+HdepkrT++fPlyTj31VM4//3wGDhzI3LlzKSgoYOzYsYwYMYJ169YB8Pe//51x48YxatQoJk2axLZt2ygqKuLRRx/loYceIjMzkw8//JCioiJOP/10MjIyOOOMM/y/0GfMmMHMmTMZN24ct912G//617/IzMwkMzOTUaNGsX///ma9nq3SLVNE+gDbjDHGXvPVgbXEmGok3wdx3rx5lJSUkJaWRm5url6wVS3unr+v5tvN+8J6zKF9u/DbKSFbeKspKiriyy+/ZNy4cQBs27aN1NRUAPr06cO2bdtqPefJJ5+ka9eufP755xw+fJgJEyZw5pln8tJLL3HWWWcxb948PB4PZWVlnHzyyTz88MOsXLkyaP5fffUVa9asoUePHgwcOJBrr72Wzz77jIULF7Jo0SIWLFjASSedxCeffIKI8MQTT/CHP/yBBx54gJkzZ9KpUyduvfVWAKZMmcL06dOZPn06Tz31FDfddJP/C2vjxo18/PHHOJ1OpkyZwp/+9CcmTJjAgQMHSExMDFq2hgpLwBeRv2At9txTRDZirT0ZB2CMeRT4KTBLRKqwFm2+3OhE/E2WnZ2tAV7FlAMHDnDxxRezYMECunTpUmu/iATttfLWW2/x9ddf8+KLLwKwd+9efvjhB8aMGcPVV19NZWUlF1xwAZmZmfWWYcyYMf4vmKOPPpozz7TWbR8xYgTvv/8+YAXryy67jC1btlBRURGyj/x//vMfXnrpJQCuvPJKbrvtNv++Sy65BKfTCcCECRO45ZZbyM7O5qKLLqJ///71lrMuYQn4xpgr6tn/MFa3TaVUO9XQmni4VVZWcvHFF/uDnk/v3r3ZsmULqampbNmyhV69etV6rjGGRYsWcdZZZ9Xa98EHH/D6668zY8YMbrnlFq666qo6y5GQkOC/73A4/I8dDoe/vf3GG2/klltuYerUqSxfvpy777670efbsWNH//25c+fyk5/8hGXLljFhwgTefPNNjjvuuEYf01/uJj9TKdXiYn2QnTGGa665hiFDhnDLLbdU2zd16lSWLFkCwJIlSzj//PNrPf+ss87ikUceobKyEoD//ve/HDx4kOLiYnr37s11113HtddeyxdffAFAXFycP21T7N27l379+vnL5NO5c+dq7e8nnngizz33HGC9xyeffHLQ461bt44RI0Zw++23M2bMGL777rsmlw004CvVZvkG2RUXF2OM8Q+yi6Wg/9FHH7F06VLee+89/8XLZcuWAVbt9+2332bw4MG88847zJ07t9bzr732WoYOHcrxxx/P8OHDuf7666mqqmL58uWMHDmSUaNG8fzzzzN79mwAcnJyyMjIaHKT6d13380ll1zC6NGj6dnzSL+UKVOm8PLLL/sv2i5atIinn36ajIwMli5dWq27aaAFCxYwfPhwMjIyiIuL45xzzmlSuXza9Jq2WVlZRhdAUbEqPT2d4uLaa1m43W6KiopapQxr1qxhyJAhrZKXarxg74+IrDDGZAVLrzV8pdooHWSnwk0DvlJtlA6yU+GmAV+pNkoH2alw04CvVBuVnZ1NXl4ebrcbEcHtdpOXl6djMFST6QIoSrVhOshOhZPW8JVSKkZowFdhFesDhVR4lZeXM3bsWEaOHMmwYcP47W9/69/3448/Mm7cOAYNGsRll11GRUVFs/Las2cPixcv9j/evHkzP/3pT5t1TJ9XXnnFP8tnJGnAV2GjA4VUuCUkJPDee+/x1VdfsXLlSt544w0++eQTAG6//XZ+9atfsXbtWrp3786TTz7ZrLxqBvy+ffv65+BprqYE/HY7PbKKDboalwo3EfFPUVxZWUllZSUigjGG9957z18DDzU98sGDB7n66qsZO3Yso0aN4tVXXwVg9erVjB07lszMTDIyMvjhhx+YO3cu69atIzMzkzlz5lBUVMTw4cMBeOaZZ7jggguYPHky6enpPPzwwzz44IOMGjWK8ePHs2uXteDf448/zpgxYxg5ciQXX3wxZWVlfPzxx7z22mvMmTOHzMxM1q1bx8qVKxk/fjwZGRlceOGF/sVbTjvtNG6++WaysrJYuHAhf/3rXxk+fDgjR47klFNOafbrqRdtVdjoQKEo98+5sPWb8B6zzwg4p+6FSzweD6NHj2bt2rX84he/YNy4cezcuZNu3brhclkhrH///v558gPl5uZy+umn89RTT7Fnzx7Gjh3LpEmTePTRR5k9ezbZ2dlUVFTg8XiYP38+q1at8k+PXHM086pVq/jyyy8pLy9n0KBB3H///Xz55Zf86le/4tlnn+Xmm2/moosu4rrrrgPgjjvu4Mknn+TGG29k6tSpnHfeef4vqIyMDBYtWsSpp57KXXfdxT333MOCBQsAqKiowDfDwIgRI3jzzTfp168fe/bsafrrbNMavgobHSikWoLT6WTlypVs3LiRzz77jFWrVjX4uW+99Rbz588nMzOT0047jfLyckpKSjjhhBO47777uP/++ykuLqZDhw71HmvixIl07tyZlJQUunbtypQpUwArKPu+HFatWsXJJ5/MiBEjKCgoYPXq1bWOs3fvXvbs2cOpp54KWL9OPvjgA//+yy67zH9/woQJzJgxg8cffxyPx9Pg8w5Fa/gqbHJzc8nJyanWrKMDhaJIPTXxltatWzcmTpzIG2+8wa9//Wv27NlDVVUVLpeLjRs3+mepDGSM4W9/+xvHHntste1Dhgxh3LhxvP7665x77rk89thjDBw4sM78GzI98owZM3jllVcYOXIkzzzzDMuXL2/0eQZOj/zoo4/y6aef8vrrrzN69GhWrFhBcnJyo4/pL3eTn6lUDTpQSIXbjh07/E0Zhw4d4u233+a4445DRJg4caL/ompd0yMvWrQI3ySRX375JQDr169n4MCB3HTTTZx//vl8/fXXtaYwbor9+/eTmppKZWVltc4Kgcfu2rUr3bt358MPPwRg6dKl/tp+TevWrWPcuHHce++9pKSksGHDhmaVTwN+BEVjF8bs7GyKiorwer0UFRVpsFfNsmXLFiZOnEhGRgZjxoxh8uTJnHfeeQDcf//9PPjggwwaNIjS0lKuueaaWs+/8847qaysJCMjg2HDhnHnnXcC8MILLzB8+HAyMzNZtWoVV111FcnJyUyYMIHhw4czZ86cJpX3d7/7HePGjWPChAnVFiq5/PLL+eMf/8ioUaNYt24dS5YsYc6cOWRkZLBy5UruuuuuoMebM2cOI0aMYPjw4Zx44omMHDmySeXy0emRI8TXhbFm84fWiFVbotMjt206PXI7oV0YlVKtTQN+hGgXRqVUa9OAHyHahVG1F2252TeWNeV90YAfITrXuWoPEhMTKS0t1aDfxhhjKC0tJTExsVHPC0s/fBF5CjgP2G6MGR5kvwALgXOBMmCGMeaLcOTdXvkuzM6bN4+SkhLS0tLIzc1tdxdsCwoK/OfQo0cPAHbt2tVuz0dV179/fzZu3MiOHTsiXRRVQ2JiIv3792/ck4wxzb4BpwDHA6tC7D8X+CcgwHjg04Ycd/To0Ua1Lfn5+SY5OdkA9d6SkpJMfn5+vcdITk4Omk4p1XhAoQkRU8PSpGOM+QDYVUeS84Fn7fJ8AnQTkdRw5K3CL9T4gIKCAn7+859TWlraoOME63UU7BilpaVcffXVUTEOQak2LdQ3QWNvQDqha/j/AE4KePwukBUibQ5QCBSmpaW11JegCiE/P98kJSUFram73e4G1ewDbyJS7fh1HcPtdkfmpJWKIrR0DT+cjDF5xpgsY0xWSkpKpIsTc+oaH9CULqM1ex3VdQztkqpUy2qtgL8JGBDwuL+9TbUxdY0PaGyX0WC9juo6hnZJVapltVbAfw24Sizjgb3GmC2tlLdqhLrGB+Tm5tIhIY5+nSXobXhaD4an9aB/Fwfjhgxg6Z/uJ/u802DvJv/twbtvJb1H7WMclRzPg3ffWi0tFWVBy3LDDTfgcrkQEVwuFzfccEMLviJKRY+wzKUjIn8BTgN6AtuA3wJxAMaYR+1umQ8DZ2N1y/y5MabeSXKieS6dtqq+OX5K/ngKaQe/ap3CJPWES56BhE7+Tb+fP7/WsnP/LfVy5TWzqi1Pp1SsqmsuHZ08LcoE9otval/4kMfwemC+GwaMgaEXtNAZ2LyV8NZdUHmw3qR//76S858vJy0trV2PaVAqHDTgx4hGzcC56QtY/37jMji0Gz5eBBc9DhmXhqHE9dj1I+z4rtqmKVOnVnt8x8kJxDvh+LzaXwyJiYk88cQTGvhVTKkr4OuKV1GioKCA6dOn4/F4uHSYi+NTnfYeD/tf+hX0+v5IYmOg8GmoaMJiD/GdIf3ksJS5Xj2Osm4B/rnOVFvq7bJhLk4cEPxjXF5ezowZMwA06CuF1vDbvYKCAmbPnk2vbvHMPy+VLq5KTutUzGGvEy/iT+d0Vr8+v9vZk//t+f/Y6erVqPy8OPGKs/6ELWTNmjXVVv35fe93uKDTKpJ/H3rcn9vtrrUgtVLRSmv4UaKqsoJvHphCl4ptAHg8HjIrK/lXTk/6uPYTLxtZd7g7f90/gl9vP48K++1NTEjg+OOPr31AL1DR2FIYoKo5p9EsaQMHU4WLbdu2YTCUJaXSOe7rOp+j/fuVsmjAb0e2b/qRUeWfsNY1iIMJvdm2bRtVHqu2/aNJ5IEP9rH8y++B74GPAKsN/4G8PLKzJ0Su4GF35FxeXnQrcaVeOrjgUIjvIe3fr5RFA347sm/HBvoCByfMZeTES3A4HHVOW+t0OqN+yUST0AWA8yadzF/f+LDWfpfLVW3wVzh6MSnVXrW5qRVUaGWl1uDkTsn9gLprrklJSSxZsiTqg5nYAf+FZ59g1qxZWEM+LJ06deKZZ57xvwa+XkzFxcUYYyguLiYnJ0cnbVMxQwN+Gxc4c+UrBU8A0K23NUtFsEVUAJKTk6O+Zu/j6NAVgKpDe1i8eDFer9c/UdT+/furvQa6jrCKdRrw26iCggJ69uzJtGnT/DXSrs5DVBkHy95aDlhdDfPy8nC73YgIbreb/Px8du7cGRPBHsCVZAX88v27602r6wirWKfdMtug/AfmkFDyL7xeb7XtWV33kCQVnLDUod0MbW++9y5nfXARFb0yiO8asMTCmGvhmDOrpU1PT6e4uLjWMbTbpoom2i2znTlh98v07raPUk/HWvte2ZJCScmXEShV2yTJA1nuGck4rxcOWN1V2b4G4jvWCvi5ublBRyLrOsIqVmjAb2MOHz6E21nKQ8VHc9uSFUFSFOF2u1u9XG1Vx46dya68nefOHc/4gcnWxsdOgYraUy1EyzrCSjWVtuG3AYHT/Y7PPBaXePl2a/ARUVojra5zolVnWblhD4VFuygs2sV+byKe8uDTRmRnZ1NUVITX66WoqEiDvYopWsOPsBtuuIFHHnmEx68fw6SepXRwVALw3ebaASs5OZmFCxdqkAqQ0jkBgPn/PDLJ2pNxFRzX6SD9IlUopdooDfgRcsMNN5CX9ziTxw5ixjkjubbP9xQe6sePZV3YXpZC4bdf4Xa7temhHqldO/CPG09id9mRX0SVz3XEWbk9gqVSqm3SgB8Bvlr9JROH88IpVpfACuPkkvztFG0u9afTniMNM7xf12qP30voRHz5obDmoSN0VTTQgN+KfEGjuLiYn005iV8M3cdub0fO+ZuLHXsPVgv2TmfkZqRs70xcRxLLwhfwa64z4BuhCzrtsmpf9KJtKwkc1j9q2EAKjv+aExOL+OvGZD79dgPrN1Wf3tcXUFTjmfhOJFIONcYxNJWO0FXRQmv4reCdV56h+MPnuPXSsQCMTz6I15Qy/Ikqvtu8qlpap9NJTk6Ors/aDI7ETjgwUFlWbT3cptIRuipaaMBvYVWVlZzy5S1M6uOptv2d3X1Zs7n68n2zZulC3OHgSOwMQHnZPhLDEPDT0tKCjtDVaZdVe6MBv4Xt3rWNFPGQ++NQ7vvLJ/7thyr3+e9rrT68XHbAP7hvD4nd+zb7eDpCV0ULDfgtbN/OLaQAR408CV78umELjKtmietgBfxDP35CBU1Yt7eGS04aRLcFv+Hxxx9n67at9Ondh+uuu47JJw2iovhT3n7r7Wr7rrl+FlN+fhs49BKZalt08rQW9s2Hf2fEu9NYc2YBX/y4V7v2tYKVH79J5luXRrQML3a7np/e/IeIlkHFphafPE1EzgYWAk7gCWPM/Br7ZwB/BDbZmx42xjwRjrzbkmB9tY/panUP7NgjlewTz9MA3wqGjpnEsl1P4Sk/0OJ5/e3FF9l/8Eg+Ajx7/Cr2//eDFs9bqcZqdg1fRJzAf4HJwEbgc+AKY8y3AWlmAFnGmF825kg0Wd8AAB1mSURBVNjtqYZfs682WE02T991FZce+jN7f7mGrj2b356s2pZgy0x+cvtwOssh+l6+EEdcAl2OOQVc8REqoYo1LV3DHwusNcastzN7Djgf+LbOZ0WZ2bNnU1ZWhjMunmvPH0/HeKv9NnnXCjyJQpfuvSJcQtUSgvXg+XhnJ37VvwRe/hkA34y5nxE/mRmJ4ilVTTiuKvUDNgQ83mhvq+liEflaRF4UkQGhDiYiOSJSKCKFO3bsCEPxWpZvZarSUmuU7AUnD+HRYSt5YPAXPDD4C87o8D2b492IU6+PR6Ngy0z+7u9F/GrfdC5fdRIAf39xqa6bq9qE1upG8Hcg3RiTAbwNLAmV0BiTZ4zJMsZkpaSktFLxmsbXjFO6Zx/iikdc8Uw+piPlJo7+Dx2k9//uJ+u5rgy4/bNIF1W1kGDLTF5+8YXkPfYkz7/0Bl4jxHkOM23aNHr27KmBX0VUOKqdm4DAGnt/jlycBcAYUxrw8AkgKrovzJs3j3NPHkHu2H3WyE5gkGMVHxwYwKZ91ik/eOfvwZUQyWKqFpadnV3tYnx6err/Ws5+Euliv/2lpaU6B4+KqHDU8D8HBovIUSISD1wOvBaYQEQCFhtlKrAmDPlGVEFBAcXFxUwfWkUvdvP1Ng9fb/PwTHEfZv51C2DNX6//2LEncMqFA95EOscfmdOnSXPweL2NuykVQrNr+MaYKhH5JfAmVrfMp4wxq0XkXqDQGPMacJOITAWqgF3AjObm29rWl2xk1advA/Dj+vV8+umnXDL1dMZ3+p43dvbmiryvqqVPSkpi4cKFkSiqirDAC7n7vfF0iasehBszB8/qv8xj2PcPNzi9Bwc/TlzMoFOvaPBzVOwIy5VEY8wyYFmNbXcF3P8N8Jtw5BUpW1+ex9TdrxzZkHHk7lvfd6iWVlemipy2MG994FQM+z1xdHZVD/iNmYOnakMh2+jOp92n1ptWjIcpe/LZWbSaQac2utgqBmjXkQZKqChli/Sm19V/4YQTTsDYbfYVVYZvtq31p8vPz9dAHyFtZd56X16zZ89mX5WL7q4jq3E1dg6eDhW72ZYwkKmz/6/etJWVFZCbD97KxhdaxQSd7KOB4qoOss/VHeeA0Wx39aVwUxWFm6r4epsH37Abt9utwT6C2tK89dnZ2ezcuZO4Lr3o4qry9+DxzZ1UUFBAeno6DoeD9PT0kL13Ont2U57Qo0F5Oh12/c3rqTuhillaw2+gBM8BKlzWpFw6e2Lb1BbnrU/o0ouue9biDbiY2phfIl3NXn5MTG5QXg6ngyrjAFMVptKraKM1/AZK9JRR6bLmVg/W91pnvYy8UG3jkZy33hvfiU7mABXFn/pvf134PwxPrmLsgHj/bUDS4Vq/RA4f2k8ShzFJPRucnwcnxqM1fBWc1vAbqIMpwxPX2f+4Zt9rFXlt8ZdXVVIKHaQCnj7Tv+2VcwESa6Ud/5fqzVF7d26lF+Ds1PBpOapwINqGr0LQgN9AHc1BvPHNXz1JtRzfF3Cke+kEOmbKLfzjg+Oq1bprzrDpEuGJUat5+ZJ4Ntw/3r/d6bFmWo3v2vCA7xEnok06KgQN+DUE69Z36aWX0lEOQ0Ln+g+gIqqt/fLqndyD8y6cVm3bvsOuWr9EUjufwYUDy/Hu82KMQURwObvy3/gJHDPs5Abn58GpF21VSBrwA4S6mFZ1+BDTARK6RLR8KjoE+yVS0fUYzn1wSdAV0U7r27/Bx/bgRLxaw1fB6UXbAKG69eU9YvWBdnTQgK/CIzs7m6KiIrxeL0VFRSxbtiwsXUqtgK81fBWc1vAD+LrvXfiTifTpcuSl6dfRCxTjSuoaoZKpaBeq62hxcTEOh6PB1yM84tRumSqkmA/4gW32DoeDAX1TeClrRdC0vfoPbuXSqVgRbCEVH2NMg0cNe3Hi0ICvQojpJh1fm31xcTHGGDweD8cNsEY1Tnu3O0ct3M/QvCpeSb8Xbv2BfsNPinCJVbQKtpBKTQ1p4vGItuGr0GI64Adrsz+mtzUR2r/X7sF0TWPeH/O4YMZsaERfaKUaq+ZgvlDqGzXswYUYbcNXwcV0k07Jps3EpRwFAf9fg3t1oNy4+HHTdsThjFzhVMwJ7FKanp4etImnvlHDXnFqwFchxVwNP3DSqjt+dgKHZu2ifOaR2y/7fE1JVQ8N9iqigjXxNGTUsFecOLRJR4UQUzX8mv3sp/YtpbiqO89/d+R7z+lyMfjUyzkmUoVUiqaPGvaiNXwVWswE/IKCAqZPn05yn76cOXEcIjAi/iseLU7jzle/wev12v9U93JhGxqpqWJXU0YNe0V76ajQojLgf160C6/X+B8/9NBDvPrqq7hSj+Ppn3Xh3A5Hul2+t2Y3Xq+32vS1SoVDJFbfMhrwVR2iMuAvf+oOjOfIjIHHd4HjrxyPAGe4XqKgqAd/Wr6ZQ5WGlZv34Xa7I1dYFZUitfqWx+HCVXW4xY6v2reoDPi3xL2E03Eo6L7DHvjDGyV8vc2q0Ud6+lwVnepafaslA74RJw60DV8FF5UB3/mbI32VExITMeZI847XgMd+6HQ6deES1SIitfqW1YavAV8FF53dMl3x/ltq/zQqvfhvvmAvIixZskSDvWoRkVp9y4hLA74KKSwBX0TOFpHvRWStiMwNsj9BRJ63938qIunhyLchgvVnFhFmzpypwV61mKb2o28uI06c6EVbFVyzA76IOIE/AecAQ4ErRGRojWTXALuNMYOAh4D7m5tvQwVbf3bp0qUsXry4tYqgYlCk1j02DhdOreGrECSwfbtJBxA5AbjbGHOW/fg3AMaY3wekedNO8x8RcQFbgRRTT+ZZWVmmsLCwWeVTKpZ89tBlDNi3gtTfro10UVSEiMgKY0xWsH3haNLpB2wIeLzR3hY0jTGmCtgLJIchb6VUIIdTa/gqpDZ30VZEckSkUEQKd+zYEeniKNWuGHHhRAcRquDCEfA3AQMCHve3twVNYzfpdAVKgx3MGJNnjMkyxmSlpKSEoXhKxQ7j0H74KrRwBPzPgcEicpSIxAOXA6/VSPMaWOuAAz8F3quv/V4p1QTiwqkBX4XQ7IFXxpgqEfkl8CbgBJ4yxqwWkXuBQmPMa8CTwFIRWQvswvpSUEqFm8OJS9vwVQhhacM3xiwzxhxjjDnaGJNrb7vLDvYYY8qNMZcYYwYZY8YaY9aHI99wC5wrPz09nYKCgkgXSalGMY44XFrDVyFE5dQKTRGpya6UCiuHNumo0NpcL51IqWuyK6XaDYcLpxiMV4O+qk0Dvi1Sk10pFVb20pxVVTq9gqpNA74tUpNdKRVWTquV1lNVWWcyvV4Vm7QN35abm1utDR90rnyfSKzcpJpGHNa/dMlLd+GISwiapqS4hE0rCrk+ywVZA/nrip16vSpGaMC3NXXR6GinF7Pbl6oex3DYuBj0w1Mh0wwETjvOuu8QQ9aANM5csIrp062hMvq+Nkx7rAg1e/K0lqSTp0Veeno6xcXFtba73W6Kiopav0CqTsYY1u04QKUn9P91Zmamf1Gg57J7kdVxG+5c6z1OSkrSRYEaoGZFCNrOa1fX5Gka8FWdHA4HwT4jIqILv7dTgV/iv7tyLHcM/I5O95dxsPzIhV63290uaqyR0pYrQi09W6ayReOFML2YHX0CF2dZtaUcgK9u7su7Nw4iwWWFBF/TXTR8hltCe+3VpwE/THw/8YqLizHGRM0/TKRWblItx7c4i9Pp5M0vNvDCgVFsl56c3mM744Yemdm8rKyM6dOnt/vPcEtotxUhY0ybvY0ePdq0F2632wC1bm63O9JFa7b8/HzjdruNiBi3223y8/MjXSQVBvn5+SYpKckApm9qL2N+28XMvTij1mc4KSkp6Hseq5+LssNV5tZFz5seo39iOo6Y7L91HTS6TbwGWHOYBY2p2oYfJtrWrdojX0+T4uJiSu5Ip49jDxXGya6qREY9vJPS/Yf9aWfNmuVfGrQtX7Rsae+99xb9l99MXI21g9eafky69z1EJEIls2gbfitotz/xVEzLzs6mqKiI/Px85nzl5vFdo1l2cAgD4vdz8ogB1dI+8sgjTJo0CYjtqUg6lH7DMY5N9Dgqk95DJtB7yAQ6duzEZOcXVHna9pQWGvDDRNu6VXuWnZ3NlMuu46ZHlnPdk18DcHxap1rp3n33XQoKCtrtRctwMB67Zn/egyRd8TRJVzzNj33OAqCqsu4RzpGmA6/CRAduqfbO91nNycnhx6pUrh+0nUk3Hw34GvStpgr54l4+uuVYqjxVGCMY4M+rPeS9sz4mftH6Ar7TFe/fJuKbw6gS6BCJYjWIBvwwys7O1gCv2jXf53fh649z8VFlSBJYIR2EI/edDgcOe0bO41yb6XB8D/I/jo1ftMZrBXyXK86/Tew5jNr6pHUa8JVS1WRnZ/PRRx9xyn2PhEzjG5g1b948Hr4wlWM77o+JC7ZwpIYfF3ck4ONo2KR1kaZt+EqpWhYvXswZZ5wRcn9JSYn/gm+K+zi6uSpjItgD4LGCujNIDd9TVRGRIjWUBnylVFDvvPMOycnJQfcFttV7ErrS2RxorWJFnK9Jx1erBxD/OgTaS0cp1U4tXLiw/t5nid1IlEoqD5cRE3wBX46ET4evhu/RJh2lVDvlm4bB7XYjIrjd7lpt9Y4O3QDYt2dnpIrZurxVVOKEgAFW4oyzd2nAV0q1Y762eq/XS1FRUa22+nWbtgMw8aTxUTNpYF2M14MHZ7VtvoVnqjTgK6WiVUFBAS+//g4A3TslRM2kgXXyVtUK+A77Aq63jY+0bVa3TBHpATwPpANFwKXGmN1B0nmAb+yHJcaYqc3JVynVNsybN48B3a3gd9WoJLL6HAXA28/cF7W9diRIwPddtI32bplzgXeNMYOBd+3HwRwyxmTaNw32KuyicS2C9qCkpIR1W/dRbuK4Jm0jD4wr5YFxpTxx4maqDh+KdPFahrcKj9So4dsXbb1RftH2fGCJfX8JcEEzj6dUo0XrWgTtQVpaGlu27iTl0URSl6aSujSVuT9k4BIve3ZujnTxWobXg7dmDd/fD79tj7RtbsDvbYzZYt/fCvQOkS5RRApF5BMRqfNLQURy7LSFO3bsaGbxVCyI5ZkbI803aeCB7ZvYuv57tq7/nuIdVp/8vTs2Rbh0LUNM7Rq+bxCWx9POA76IvCMiq4Lczg9MZ0+8H2pyfbc9P/PPgAUicnSo/IwxecaYLGNMVkpKSmPORcWoWJ65MdKCdds8a+qlAJTt2hyVTW0SpIbvsNvwve094BtjJhljhge5vQpsE5FUAPvv9hDH2GT/XQ8sB0aF7QxUzNO1CCKrZrfNyeddDMCaLz6q1dQ2bdo0/5z67ZWYKry12vB9vXTaecCvx2vAdPv+dODVmglEpLuIJNj3ewITgG+bma9SfroWQdvSvZe1Lm7xmi9qNbWBNaf+DTfc0NrFChuHt3bAd7qsNnzTxi/aNne2zPnACyJyDVAMXAogIlnATGPMtcAQ4DER8WJ9wcw3xmjAV2GjaxG0LUlJHdlrOjK9bxGTbx0YItWblMwfg9cRT8eLHybl6Pbzo1+MBxOiht/W2/CbFfCNMaVArSn1jDGFwLX2/Y+BEc3JR6n66FoEbcvHfaYRX/Ihxhl6zezKinjGeL/i0y/ebWcBvwqvVA+dvou2JpoDvlJKBXPOrD9QUFDAtGnTgu53Op0c2LMT/tdN1aF9rVy65nEYD15HzRp+lFy0VUqppsjOzg45p35OTg6JSV0A8Ja3r6mVrSad9lnD14CvlGox77zzDrNmzcJp14CdTiezZs1i8eLF4HBwkETM4f0RK19Tuo06jAfjCN4P3+tt2wFfm3SUUi1q8eLFVoAPolySkIrIBHzfCG1fTyLfCG2gzutBDuPBSHy1bU77oi1aw1dKqeDKHUk4Kw9GJG/fCG1X977+bQ0Zoe0I2qRj1fi1SUcppUKodHXEVRWZgF9SUkLCgBH0y8mj4/DTq22vi4PaTToul1Xjb+sXbbVJRykVMR5XR+IryiivbP155NOOOpq9A0YDkNB/GAdXvWdtr2eEttN4qq1nCwEDr7QNXymlgvPGdyRh/06Ou/ON1s/8kgV0te/G9z6aDoPGkpCQwLRf/IK3v90W8mlpQS7auny9dLxRvACKUko1R//evak8tJ7bTz8u6P6VK7/kzTfeZPeePXTv1o2zzj6LzMzwDdL6+stC0oueZ0mfbBIuvguA/GLIf7Yw5HPeja8CV/WLtr41bdEavlJKBdehU1c6OMqZdVrtCXQLCgrIn3ekF80+IO/NJ0hOTmbhwoXhGVndvRBefperMwazbdz/NOgpA56Px9Wra/WNYl0O1SYdpZQKJbELlJXCvcm1dl1aVcWltziBzgC8sLqKaS8forS0tEHdJxvkoDXBb0oHSOnXtZ7ENjHgq9H7+Nr0o3lNW6WUapbRM6xgaby1dv3xvvv8C2wM7ekgOyOOUakOjL0x7rNfwlBgVDOCfoU9m2dcUt3pAnmroEYbvj/gaw1fKaVC6J4Op98RdFfeNU9RXFwMQAcXLDjb0D1R/PsnpFXBF0uaF/DLSq2/jQnU3qpavXR8jw8dPsyaLc2fG8jlEAb37tzs49Q6btiPqJRSYZCbm+sfCXuoCq7/R3m1/Y9dnEJOt5Ww68faAbihdq2z/h7a3fDnBA34Vo1/7da93LTww6aVJUDPTgkU3hH+hWI04Cul2iRf+/zs2bMpLS2tti8pKYljJ0+HLXnwf5nNz6x8T8PTemv3w/ddtJ2a0Yuhw45vdnHiXS0zJlYDvlKqzfKtc1BQUFBrgZtTL78Mvh3Hfz58n2eXLOFwRYX/eQnx8Vw1fTonjB9ffybv58KhxgT8IG34IuBwMahnBwYNT234sVqZBnylVJsXcoGbET/liim3Ulxcc3qGSv657XWKiv5U/8G/XwZ7GrHgfbAmHbC26UVbpZRqOaHmvqlvThy/Dt1h3fvw58sblt5zuI6Ar90ylVKqxaSlpfl789Tc3iDHngPbVsG+TQ1L33cUHHVK7e3i1ICvlFItKbA3j09SUhK5ubl1Pi/YdYFmDeRyOLVJRymlGqOxgdi3rzHPCbb4yZVXXslHH30UcrGWejlcULoWvn21ac8P5EyAY89u/nFqEGNCryofaVlZWaawMPQkRkqp6FIzEINVW8/LywvP3Dm29PT0oM1AIsLMmTNZtmxZ42v+i0+A7d+Gp4Ade8GcH5r0VBFZYYzJCrpPA75Sqq0IFYjdbjdFRUVhy8fhcBAq9olIrX0OhwOv14vb7Q79BXBoN+zbHDLP119/nf/7v0Vs2bqF1D6p3HTTjQDcc889HCo/MqisQ2Iid959D1N+fmsTzqzugI8xpsk34BJgNeAFsupIdzbwPbAWmNvQ448ePdoopWKHiBig1k1EwpqP2+0Omk9DbklJSSY/Pz/ocfPz843b7TYiYtxut5k1a5Y/r5rnlpSUZJKTk4Pm4Xa7m3xuQKEJFYtD7WjIDRgCHAssDxXwASewDhgIxANfAUMbcnwN+ErFllCBuDkBMJj8/PyQXy4NuQUrT35+vklKSmryMcP1BVdXwG/W+F1jzBpjzPf1JBsLrDXGrDfGVADPAec3J1+lVHTKzc0lKan6zJUN6XHTWNnZ2cycORMRqba95uNQgvXx9y2KHg4N7lLaSK2xiHk/YEPA4432tqBEJEdECkWkcMeOHS1eOKVU25GdnU1eXh5utxsRwe12h/2Crc/ixYtZunRptbxmzpxZ6wsnmGABucEDvQIkJye3yhecX6iqvznSJPMOsCrI7fyANMsJ3aTzU+CJgMdXAg/Xl6/RJh2lVATk5+eHbFunjjb8xl4X8B2nZrt/qOsDDUVLteH7D1J3wD8BeDPg8W+A3zTkuBrwlVKR4gvEgHE6nf62+7ou2NbXhu+7bhCOwB5KXQG/NQZefQ4MFpGjgE3A5cDPWiFfpZRqspATttWRHqoPADv33HOb1qe/hTSrH76IXAgsAlKAPcBKY8xZItIXqxnnXDvducACrB47TxljGtRApf3wlVKqcerqh9+sGr4x5mXg5SDbNwPnBjxeBixrTl5KKaWapzV66bSqgoIC0tPTcTgcpKenU1BQEOkiKaVUmxBVk6cFmxApJycHIKLtZkop1RZEVQ0/2MCHsrIy5s2bF6ESKaVU2xFVAb/ZK98opVQUi6qAH2o4cksNU1ZKqfYkqgJ+a83DoZRS7VFUBfzWnIdDKaXaG10ARSmlokhdA6+iqoavlFIqNA34SikVIzTgK6VUjNCAr5RSMUIDvlJKxQgN+EopFSM04CulVIzQgK+UUjFCA75SSsUIDfhKKRUjNOArpVSM0ICvlFIxQgO+UkrFCA34SikVI5oV8EXkEhFZLSJeEQk6HaedrkhEvhGRlSKi8x0rpVQEuJr5/FXARcBjDUg70Rizs5n5KaWUaqJmBXxjzBoAEQlPaZRSSrWY1mrDN8BbIrJCRHLqSigiOSJSKCKFO3bsaKXiKaVU9Ku3hi8i7wB9guyaZ4x5tYH5nGSM2SQivYC3ReQ7Y8wHwRIaY/KAPLCWOGzg8ZVSStWj3oBvjJnU3EyMMZvsv9tF5GVgLBA04CullGoZLd6kIyIdRaSz7z5wJtbFXqWUUq2oud0yLxSRjcAJwOsi8qa9va+ILLOT9Qb+LSJfAZ8Brxtj3mhOvkoppRqvub10XgZeDrJ9M3CufX89MLI5+SillGo+HWmrlKqmoKCA9PR0HA4H6enpFBQURLpIKkyaO/BKKRVFCgoKyMnJoaysDIDi4mJycqye1NnZ2ZEsmgoDreErpfzmzZvnD/Y+ZWVlzJs3L0IlUuGkAV8p5VdSUtKo7ap90YCvlPJLS0tr1HbVvmjAV0r55ebmkpSUVG1bUlISubm5ESqRCicN+Eopv+zsbPLy8nC73YgIbrebvLw8vWAbJcSYtjtdTVZWliks1OnzlVKqoURkhTEm6PokWsNXSqkYoQFfKaVihAZ8pZSKERrwlVIqRmjAV0qpGNGme+mIyA6guIlP7wnE2qLpes6xQc85NjT1nN3GmJRgO9p0wG8OESkM1TUpWuk5xwY959jQEuesTTpKKRUjNOArpVSMiOaAnxfpAkSAnnNs0HOODWE/56htw1dKKVVdNNfwlVJKBdCAr5RSMSLqAr6InC0i34vIWhGZG+nytAYReUpEtovIqkiXpTWIyAAReV9EvhWR1SIyO9Jlamkikigin4nIV/Y53xPpMrUWEXGKyJci8o9Il6U1iEiRiHwjIitFJKzTBUdVG76IOIH/ApOBjcDnwBXGmG8jWrAWJiKnAAeAZ40xwyNdnpYmIqlAqjHmCxHpDKwALojm91lEBOhojDkgInHAv4HZxphPIly0FicitwBZQBdjzHmRLk9LE5EiIMsYE/aBZtFWwx8LrDXGrDfGVADPAedHuEwtzhjzAbAr0uVoLcaYLcaYL+z7+4E1QL/IlqplGcsB+2GcfYue2loIItIf+AnwRKTLEg2iLeD3AzYEPN5IlAeCWCci6cAo4NPIlqTl2U0bK4HtwNvGmKg/Z2ABcBvgjXRBWpEB3hKRFSKSE84DR1vAVzFERDoBfwNuNsbsi3R5WpoxxmOMyQT6A2NFJKqb70TkPGC7MWZFpMvSyk4yxhwPnAP8wm6yDYtoC/ibgAEBj/vb21SUsdux/wYUGGNeinR5WpMxZg/wPnB2pMvSwiYAU+027eeA00UkP7JFannGmE323+3Ay1hN1WERbQH/c2CwiBwlIvHA5cBrES6TCjP7AuaTwBpjzIORLk9rEJEUEelm3++A1THhu8iWqmUZY35jjOlvjEnH+l9+zxgzLcLFalEi0tHuiICIdATOBMLW+y6qAr4xpgr4JfAm1oW8F4wxqyNbqpYnIn8B/gMcKyIbReSaSJephU0ArsSq8a20b+dGulAtLBV4X0S+xqrYvG2MiYluijGmN/BvEfkK+Ax43RjzRrgOHlXdMpVSSoUWVTV8pZRSoWnAV0qpGKEBXymlYoQGfKWUihEa8JVSKkZowFdKqRihAV8ppWLE/wdudzdfHQj4eAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "from sklearn.ensemble import GradientBoostingRegressor\n",
    "clf_1 = GradientBoostingRegressor(loss = 'lad', n_estimators=20, random_state=0).fit(X, y)\n",
    "clf_2 = GradientBoostingRegressor(loss = 'lad', n_estimators=30, random_state=0).fit(X, y)\n",
    "# Predict\n",
    "X_test = np.arange(0.0, 5.0, 0.01)[:, np.newaxis]\n",
    "y_1 = clf_1.predict(X_test)\n",
    "y_2 = clf_2.predict(X_test)\n",
    "\n",
    "plt.plot(X,y,'o',color='black')\n",
    "plt.plot(X_test,y_1,label='20 estimators')\n",
    "plt.plot(X_test,y_2,label='30 estimators')\n",
    "# Plot the resu  et\")\n",
    "plt.title(\"Gradient boosting of Decision Tree Regression\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_digits\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "digits = load_digits()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1oAAAFwCAYAAAChCgrnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAIABJREFUeJzt3X/QXXdeF/DPlyzZaWdDEppusUtJytYtLtakThF1kCSKArLOk3WUQUZMgs76Y9Ak8mtQNHlQfsiwJhEc6TCQBHTB/cNt6u9VN4myiE47TVgEu3ZpspXWbtNtQtnW1g3HP55beDabfD/nPuf73Ofc7Os1w1D6ufecc7/3e77nfLhPz7t0XRcAAAC083lrfQAAAAC3Go0WAABAYxotAACAxjRaAAAAjWm0AAAAGtNoAQAANKbRAhiRUkpXSrlvFbb7m6WUL12F7d5fSjlfSnm5lPI3Wm9/JVqNYSnlSybjtq7FcbW2WnMFgDY0WgCNlVK+qpTyC6WUq6WUT5ZSPlxK+YoZ7v9sKeUvLf93Xde9peu6X1uF3X1XRJzpum5D13X/aBW2X3Wjz9pK13Ufn4zbtdXY/nKllIullK9ZpW1vmzRlb1qN7QNwYxotgIZKKV8QEf8qIn40Ir4wIt4WEYsR8dpaHtcq2hoR/+NmxbH+GgQAq02jBdDWOyIiuq772a7rrnVd92rXdR/suu6X3nhBKeVbSym/Wkp5qZTy70spW2+0oVLKm0spP1JK+Xgp5flSyo+XUm5bVl+Y/Nneb5RSPlZK+bpSyvdHxB+JiB+b/Nnbj01e+9t/ZlZK2VhK+elSygullEullO8tpXzepLavlPLzk/2+VEp5upTy9Tc5vg9FxO5l+3pHKeVkKeWflFL+TSnlUxGxu8f+PlxKOVpKuVJK+bVSyh+e/PtnSimfKKXsvcn+b/hZJ76mlPK/Jtv8x6WUsoLx/4xfgia/nv29yfG+XEr5YClly3WvfU8p5dlSynOllO9Ytq2TpZS/v+z/3lVK+d+Tf/6ZiPiSiPiXk8/xXTc5nu+cbPfZUsq3Xlf7hlLKE5O58Ewp5ciy8n+e/O8rk+3/oVLK20spHyqlvFhKuVxK+WellE032i8AK6PRAmjroxFxrZRyqpTy9aWUzcuLpZSFiPhbEfGnI+LOiPgvEfGzN9nWD8VS47YjIu6LpV/H/u5kO38gIn46Ir4zIjZFxFdHxMWu6/72ZJvfNvmzt2+7wXZ/NCI2RsSXRsTOiPgLEbF/Wf0rI+LJiNgSET8cET+5vFF5Q9d1f/S6fX10UvrmiPj+iNgQET/fc3+/FBF3RMT7IuLnIuIrJp/5z8dSI/WWG+y/9lnfNdnG74uIb4yIr52M2zTjfyPfPDn2t0bE+oj4juvquyPid0fEn4iI7+7z54Bd131LRHw8Iv7U5HP88PWvKaV83WRff3yy/eu3+6lYGtdNEfENEfFXSyl7JrWvnvzvTZPt/9eIKBHxgxFxd0T8noi4JyKOZMcKQH8aLYCGuq77jYj4qojoIuInIuKFUsqjpZS7Ji/5KxHxg13X/WrXdZ+OiB+IiB3X/6oyaWzeExGHuq77ZNd1L09e+02Tl/zFiPipruv+Q9d1v9V13a93Xfc/s+MrS3/K900R8T1d173cdd3FiHhvRHzLspdd6rruJyb/bdKpiPhdEXHXZ23s5k53Xffhrut+KyL+X4/9Pd113YnJ/v55LN30f1/Xda91XffBiHg9lpquafxQ13VXuq77eESciaVmNaLn+Fec6Lruo13XvRoR71+23Tcsdl33qa7rPhIRJyLiz0153DfzjZN9/3LXdZ+K65qiruvOdl33kclc+KVYah533mxjXdc9NZk7r3Vd90JE/MPa6wGYnkYLoLHJTfy+ruu+OCJ+byz9anBsUt4aEccnf9J2JSI+GUu/Lrztus3cGRG3R8Tjy1777yb/PmKpGfnYCg5vS0R8fkRcWvbvLl23//+z7LO8MvnHz/pFqeKZKff3/LJ/fnWy3+v/3TT7j1j2GSLilWXv7zv+0273Dcs/+6VY+u5buPsG2/5tpZSvLKWcmfx55tVYaii33GxjpZS7Sik/V0r59VLKb0TEP629HoDpabQAVtHkV6aTsdRwRSzdLP/lrus2Lfuf27qu+4Xr3no5lhqML1/2uo1d171l2XbefrPdVg7pciz9yrT8F5wviYhf7/+pUsv3v9r7q33WG+k7/it1z7J//pKIeHbyz5+Kpcb5DV903fuyz/HcDba93Psi4tGIuKfruo0R8eOx1EDebNs/MPn3D3Rd9wWx9Cean/XnoQCsnEYLoKFSypeVUr69lPLFk//7nlj687FfnLzkxyPie0opXz6pbyyl/NnrtzP5s7ufiIijpZS3Tl77tlLK105e8pMRsb+U8sdKKZ83qX3ZpPZ8LP33UJ9l8ud574+I7y+lbJj8ydzfjKVfNJqbwf5u+llvotf4D/B3Sim3T7a/P5b+FDIi4nxE/MlSyheWUr4oIg5e977sc7w/IvaVUt5ZSrk9Ig5fV98QEZ/suu7/Tv77vW9eVnshIn7ruu1viIjfjIirpZS3xdJ/6wdAQxotgLZejqWHO/y3svTUvV+MiF+OiG+PiOi67gMR8Q8i4ucmf7L1yxFxw6f6RcR3R8RTEfGLk9f+x4i4f7Kd/x5LN/JHI+JqRJyL3/nV6HhE/JnJU/VulG3112PpF5Zfi6WHVbwvIn5qwGfOrOb+ss/6GaYc/5U4F0vf2X+KiB+Z/DdmERE/ExEXIuJiRHwwfqcBe8MPRsT3Tv6k8foHbETXdf82lv789EOT7X/oupf8tYj4vlLKy7H0wJT3L3vvK7H0cJIPT7b/B2MpcuD3x9Lc+dcR8S9W+oEBuLHSddP+1QUAsFwpZVtEPB0Rnz95yAYAn+P8ogUAANCYRgsAAKAxfzoIAADQmF+0AAAAGtNoAQAANPamaV68ZcuWbtu2bSve2ZUrV6r1Z599tlrfsGFDuo+77767Wl+3bl26jZu5ePFiXL58eapAx6FjlnnyySer9WvXrqXbyMZs06ZNUx3TcmMcs5dffrla/9jHPpZu47bbbqvW77///qmO6XqPP/745a7r7uz7+qFj9vzzz1frzz33XLW+fv36dB/ZmAw5NyOmH7OI1Z9r2fn39NNPp9u47777Wh3OZ1mL8zNbs9785jdX66v5ffUxxjWtxXXgne98Z6vDuaG1OD+zdS0bl+yeJSLi1Vdfrdazde2BBx64ae3jH/94vPjiizOda88880y1no3JHXfcke7jrrvuqtbn7T7tqaeeqtazeTb0fmGotRizbEyyfuDFF19M9/GWt7ylWh96be27pk3VaG3bti0ee+yxFR/UI488Uq0fOXKkWt+1a1e6j2wbQ5qGhx56aOr3DB2zTDYmfS4Ui4uL1frCwsI0h/QZxjhmZ8+erdb37NmTbmPHjh2D9pEppVya5vVDx+zYsWPVenZe9Vlwz5w5U60POTcjph+ziNWfa9n5t2/fvnQb2bo5xFqcn9malc2lkydPrnjfLYxxTWtxHVjN44tYm/MzW9eycelz7l24cKFaz272auvi7t270/1fb+iYHTx4fY72Z8rGpM+alu1j3u7TsnuGbJ4NvV8Yai3GLBuT7J6jz3UgWxeHXlv7rmn+dBAAAKAxjRYAAEBjGi0AAIDGNFoAAACNabQAAAAam+qpg0NlTxG5ePFitd7nyUlDn1jV54lzY5I9nefcuXPpNrKnwQ156uBaOH/+fLWePclp48aN6T6yuTo22bmXPX0ne3pX9hSpiPx76fNU0XmTrTfZ0ytvRdm5k61Zp06dSvexdevWQccwNqdPn67WszE7fPhwy8O5ZWTXz2zd6/Oa7L6ldgxDIy9WIlunM32eBpc9ZW+tn8J3vWy9yM7PTCn5k9W3b99erQ/93mYtezplNqZ91rRsLmb1Pk/Q7MMvWgAAAI1ptAAAABrTaAEAADSm0QIAAGhMowUAANCYRgsAAKAxjRYAAEBjTXO0suf4Z1kEFy5cqNazbJSIPAcrO8ax5Whlx9sib+JWy/LJMqGyPIo+c2BxcXGqY1pr+/fvr9azHKws4yrLr+uzjXmUZeRkOR198seGZj71+W5mKcsuunTpUrXeJ+cum2tDso3WwtAcrLFd12alz/lVk+UPRuTn59gyoTLZ/cDQrNKI/PzKxmzW15I+Ga41O3furNb7rNHzNo+GZo/t3bu3Wu9zbmbf26yyx/yiBQAA0JhGCwAAoDGNFgAAQGMaLQAAgMY0WgAAAI1ptAAAABrTaAEAADTWNEcre2Z9ls/QJycrM2+ZUMeOHavWs6yAq1evDj6GWy3fKMtOyTIr+mSvLCwsTHNIay47t7LsoiwTo88cmrfsoj6yzJhs3Pbt25fuI5uP2bj1yRuZpez8y/IU+6x52XVg3uZadu5k2YDzdl3sK8sWGpo9lF2f+8hyHfusAbOUHc+DDz5YrffJ/cvOv7Fl/w09nmwO9Mm5G5rlNWtD19gW58VY1nm/aAEAADSm0QIAAGhMowUAANCYRgsAAKAxjRYAAEBjGi0AAIDGNFoAAACNzTRHaxZ5TfOW1ZNl5GRZAps3bx58DPOWz5Adb5Z9kmVa9JHlJ82bLGcry0bpkwOSvSb7Xtbi3D19+nS1fujQoWp97969g4/h+PHj1fqJEycG72OWsu85yz46f/58uo/se8n0ydKbpWzNy3J++uRBZefn2LKNIvJjyubK0JytiHw+z1tO5dD7gXPnzqWvefrpp6v1sc217NqT5dhl92kHDhxIjyGby9k1etZj2med/lzhFy0AAIDGNFoAAACNabQAAAAa02gBAAA0ptECAABoTKMFAADQmEYLAACgMY0WAABAY00Di7NQt6EBZn2C9LJ99AlW/VyTjdmOHTtmdCT9HDlypFrPAl4zfQKNxxZ8vdqyz9tnzLIQ2CxUNfveV8PGjRsH1U+dOlWttwh1vNXWtFkEvGbhnmOThY1mIbF9rp1ZyPMTTzxRra/FdSIbl2xdKqUMen/E/AUSZ2vO7t27q/XDhw9X633OraHh9WMLNM7GdBb3WNn1tc9cbmnoZ7p69Wq13qIfmNU9hV+0AAAAGtNoAQAANKbRAgAAaEyjBQAA0JhGCwAAoDGNFgAAQGMaLQAAgMaa5mjde++91Xr2TPvsOf8tcgCyrAHGb9++fdX62bNnq/ULFy5U631yiRYWFqr1/fv3D3r/rGV5EllWTJ9Mi+x7GWMe1NDPna15fTJ49u7dW63PW6bb6dOnq/Usm6xF9skY51pNtuZlGVh9coey/KPs+ju2vMWI/HqfzbWdO3e2PJxRyOZCNibZmPbJ0XrwwQer9ZMnT1bra5GpOER2bvS5L83GZNY5WZnsupSdW0ePHq3WP/CBDww+hlmtWX7RAgAAaEyjBQAA0JhGCwAAoDGNFgAAQGMaLQAAgMY0WgAAAI1ptAAAABprmqO1devWaj3LPsjqfZ55n2X1zJssByDLY8oyayLyMcsyXGYtmwdZdlFW75PRkY1rllUythytbJ61yJ/LsouOHTs2eB9jk43r1atX022M7fwb6syZM9X68ePHB+8jyx7rk182JtkcyLKLsgyeiHxM5i17LCK/tp06dapan7eMuj6yz5TNg82bN1frWQ5XRH79m7e80+x4s3uOFjmUY8yxq8lyv4aOaUS/dW8W/KIFAADQmEYLAACgMY0WAABAYxotAACAxjRaAAAAjWm0AAAAGtNoAQAANFa6ruv/4lJeiIhLq3c4o7e167o7p3mDMTNmKzTVuBmziDDXVsKYTc+YrYxxm54xm54xm54xW5le4zZVowUAAEDOnw4CAAA0ptECAABoTKMFAADQmEYLAACgMY0WAABAYxotAACAxjRaAAAAjWm0AAAAGtNoAQAANKbRAgAAaEyjBQAA0JhGCwAAoDGNFgAAQGMaLQAAgMY0WgAAAI1ptAAAABrTaAEAADSm0QIAAGhMowUAANCYRgsAAKAxjRYAAEBjGi0AAIDGNFoAAACNabQAAAAae9M0L96yZUu3bdu2VTqU3Ec+8pH0NevWravW77///hW//+LFi3H58uWSHsQyQ8fsypUr1frzzz9frd93333pPrIxG2I1xuz111+vvj8bkxdffLFa7zMemzZtqtbvuOOOav3222+v1h9//PHLXdfdmR7IxGqfm88++2y1/olPfCLdxgMPPFCtD52H045ZRD5uQ8+/a9euVeuvvvpqtd5HNq7r16+/aW0t1rTM2OfaWoxZNo+yedhnzLI1beh3vhrnZ+bixYvV+m233VatZ9eKiIgNGzZU6/fcc0+6jZtZi7mWfebs/Oyz72zMhliLMXvllVfSY6qprdFvyMbsrrvuSrdxM2O8DmT3eX36gcyQa2dE/zVtqkZr27Zt8dhjj03zlqb6fKnZxeLMmTMrfv9DDz2U7v96Q8fs9OnT1frRo0er9UceeSTdRzZmQ6zGmGWL1rFjx6r1kydPVut9xmPPnj3V+r59+6r1HTt2VOullEvpQSyz2ufmkSNHqvVszCOGnXt9TDtmEfm4DT3/skbtwoUL1Xofjz76aLVeWzfXYk3LjH2urcWYZfMoG5M+Y/aud72rWs/WzcxqnJ+Zoetwn8+8a9euar3P2N/MWsy17DNn5+fDDz+c7iMbsyHWYszOnz9frWfzsM+9bTZmBw8eTLdxM2O8DmT3effee+/gfQy5dkb0X9P86SAAAEBjGi0AAIDGNFoAAACNabQAAAAa02gBAAA0NtVTB1db9oSvS5fyB3xkr8me3rSaT+Bbib1791br2fH2eWrSkKfVrIXsaTRnz56t1rPPm82RiIjjx49X69n3kj3tatayz5zNoxaPeR3juXnixIlq/dy5c9X6xo0bq/XDhw+nx5A9bWotIzdWQ3b+9pkHY1vHM0OfWpatiX3GIxv3eZR97mzc+4xbtjZm15uxnb/Zk4qze6wWT2qcN9mTGLOny/Z5+mx2f5w9CXls8yyTrWnzxC9aAAAAjWm0AAAAGtNoAQAANKbRAgAAaEyjBQAA0JhGCwAAoDGNFgAAQGOjytE6cODA4G3s3LmzWp+3LIHseLPskyxbIWL+crSyDI4sGyXL+cgyMSLyfKQ+4z4mQ7PFsuyViHwuZ99rn320luWdZXMte3+fc2/eMqEy2Zhl2WRHjx5teTijkGUTDZ2HWQ5XxK2VW/OGbB0+duxYtd7nfiE7P+ftnmPoXDt16lS6j+waO7Yxy+6zsoyr7N62zz3H2LI3M9k9QzaP+oxJZiz9gF+0AAAAGtNoAQAANKbRAgAAaEyjBQAA0JhGCwAAoDGNFgAAQGMaLQAAgMaa5mhlz83PMmOyLJFbUZZdkmUnZBket2I2ylAt8piyDIix5YBkeTFZ9kmWXdTn8169erVan7eckIh8zRqajxRx653D2bmTmbeMuj4WFhaq9a1bt1brWY5PnzUvG9dsHo5tzYvIz69s3Pbu3ZvuI8tlnDfZfVqWKdVnHmT7WIvMxNWUXX/7mLf74+y8OHTo0GwOZAT8ogUAANCYRgsAAKAxjRYAAEBjGi0AAIDGNFoAAACNabQAAAAa02gBAAA01jRHK8vZyOpZVkifHIF5y+LJMieOHDkyaPt9xizLP8uyuuZNlmnRZw7NWw7I0CymLBOjRU7Igw8+OHgbrWXf89DsoP379w96/zzK1pvMvffem75m+/bt1fri4mK1nuVazdoszo0sSy9bQ7J8pbWQZYNl82Tfvn3pPm6162P2eVp8z9n3kmXtzfo+b9euXYPe3+Iea+fOndV6do0eei85rezamY3p0OzPiPFkUPpFCwAAoDGNFgAAQGMaLQAAgMY0WgAAAI1ptAAAABrTaAEAADSm0QIAAGhMowUAANBY08DiLEQuC7o7ffp0tZ6F3EXMJlh1lrJA1Ozzbty4Md3HrRa4mMnGNAtLjBg+14cGIE4rCyvMAhWzAOarV6+mx5AFko8tJDYiPzeyQNNsTetjbOGdQw0Nzjxw4MDgY8i2Meu5mJ1/hw8frtaz9aZPcGc2l/tcf+dNNm59PvMYg5rHLgtqz9aI7Ho0a9l9VvZ5+tyXZmtEdl8zNtl1q8XnGcuY+EULAACgMY0WAABAYxotAACAxjRaAAAAjWm0AAAAGtNoAQAANKbRAgAAaKxpjtZQfTKfMrdaJtTBgwer9ePHj1frfcY028eQLKHXX3893f+0sjyJc+fOVesvvfRStd4n0yLLjeqTWzNL2XeY5bFlY7558+b0GGadHdbC0Lm2d+/ean379u3pMcxbTlamRXZRZui6WTt/V2NNy87PLIcny8DK5nGffYzR0Gyh7P1jW8dnIRuTPjmTmaeffrpaz/IHZ31+ZrLzr0XuV/a93Go5dy0ysLLrc3Z+t8rh8osWAABAYxotAACAxjRaAAAAjWm0AAAAGtNoAQAANKbRAgAAaEyjBQAA0NiocrSyvJg+mTMXLlyo1rMsgrHlcGX5DFkOQJ8MnizjIRuTWj7Sa6+9lu5/Wtl3ePTo0eb7vN7CwkK1nn1v8ybLJeqT1zaPY5JlxmQ5WVneWot8lXmTrUnZXOuT95TlZGXnby0/Zf369en+Zy1bE+cxw66P7No0dFyyeXQryjIVDx06NHgf2b1cdn7Wvvd169at6JiGyNas7DqSZQtG5N/L2O5dh8rOzZ07d6bbyMZdjhYAAMCc0mgBAAA0ptECAABoTKMFAADQmEYLAACgMY0WAABAYxotAACAxkrXdf1fXMoLEXFp9Q5n9LZ2XXfnNG8wZsZshaYaN2MWEebaShiz6RmzlTFu0zNm0zNm0zNmK9Nr3KZqtAAAAMj500EAAIDGNFoAAACNabQAAAAa02gBAAA0ptECAABoTKMFAADQmEYLAACgMY0WAABAYxotAACAxjRaAAAAjWm0AAAAGtNoAQAANKbRAgAAaEyjBQAA0JhGCwAAoDGNFgAAQGMaLQAAgMY0WgAAAI1ptAAAABrTaAEAADSm0QIAAGhMowUAANCYRgsAAKAxjRYAAEBjb5rmxVu2bOm2bdu24p09++yz1fpzzz1Xrb/97W9P97Fp06apjmkaFy9ejMuXL5dp3pON2bVr16rvf/7556v1l19+uVp/9dVXq/WIiHXr1lXr2Xe+YcOGm9ZWY8xW25NPPpm+5t57763W169fP+gYHn/88ctd193Z9/VDxyz7zLXvuK9snt11112Dtj/tmEUMH7fs/M3GNXt/RL7u3X777ek2bmaM52d2nXjxxRcH7+P++++v1mvn71qM2cWLF6v1T3/609V6n3OrxTlesxbn5zPPPFOtZ9fPO+64I93H0HWrZoxzLRuz2267Ld3H3XffXa3P25r2+uuvV+tPPfVUtd7nvjUbsyFWY8yGjkmfe9ehNm7cWK3fd9991XrfNW2qRmvbtm3x2GOPTfOWz3DkyJFqfXFxsVp/73vfm+5jYWFhmkOaykMPPTT1e7Ixu3LlSvX9x44dq9bPnj1brZ8/f75aj8hP8ocffrha37Vr101rqzFmq632ed5w8uTJan3ojWgp5dI0rx86Ztln7jMmmWyeHTx4cND2px2ziOHjlp2/2bhl74+IeN/73let79ixI93GzYzx/MyuE9m518ejjz5ardfO37UYs3379lXr2Tzqc261OMdr1uL8zD53dv3Mxr3PPoYY41zLxqzPepSd4/O2pmXN6Z49ewbVI/IxG2I1xmzomFy4cGHqY5pWtuY98sgj1XrfNc2fDgIAADSm0QIAAGhMowUAANCYRgsAAKAxjRYAAEBjUz11MJM9+Sh7gkf2xMA+T2bpui59zZhkT2bJnhrY4mlx2VOEsqfdZO8fm+ypZdl3ErG6MQKrIZtH586dG1Tv87TP1X6q2VrIngqaPTlp+/bt6T7mba4NlT2xs888yq412ZPisvePTbYG91nThu5jjPM0W/ey8/PQoUPpPrL7krWMKrmR7Hs8depUtZ6tWX3u07LXZN/b2OZaNqbZPOvzhL1sHvV5QuYsDf3Me/furdbf/e53p8eQPb59yNMtp+EXLQAAgMY0WgAAAI1ptAAAABrTaAEAADSm0QIAAGhMowUAANCYRgsAAKCxpjlaWbZB9sz6LN+ozzPvs/yFWT03v6/seIbmuVy6dCl9TbaPseUzZLJMi/3791frR48eTfeR5Sdl2WOzlp2bW7durdbnLdeklSx/aHFxcdD2szUvYnw5PKstW2/6rEfZmM3bfB2aLdbnOpLtY2im4xhlmVB98o2y6828XT8z2Vzqs15l28jWxSwHb9Y2b95crWd5Ti3GbGzz7KWXXhr0/uzeOLtn6bONWfGLFgAAQGMaLQAAgMY0WgAAAI1ptAAAABrTaAEAADSm0QIAAGhMowUAANBY0xytTJZ9cOXKlWq9T6bF51rmTJaTleWERORZA2PLZ8hk8+zAgQOD3h8RUUqp1rN5OOsx7XPu1Jw7d65aX1hYGLT9scrWpMzOnTur9Vtxvcqyx7KMnCyvqc93kq2L8zbuWS5ftob3yQ0bmtU1Rtk6m2Uq9pF9N2O7fma5X5kW5042H++9997B+5il7PqXzZFDhw6l+8jW1bHJ1vFMnzHJnDhxolqf1bnpFy0AAIDGNFoAAACNabQAAAAa02gBAAA0ptECAABoTKMFAADQmEYLAACgsZnmaGVZH1l+UZZJE9EvL+RWsnXr1mq9T+ZFNu5jG9MshyfLjMrm4Z49e6Y9pM8ytuyULE8tyybKxiTLJouIOHbsWPqascnymDJZ9kmWTxYxfxll2fm5uLg4mwOpGNualp1/2XoyNCcvYnjuzRhl45bV+2QXZZlP2bhm16NbUZblNbbr51DZPVaf7M7sXm5s8yz7DrM1OMvt63M/kY2rHC0AAIA5pdECAABoTKMFAADQmEYLAACgMY0WAABAYxotAACAxjRaAAAAjWm0AAAAGptpYHEWqJYFXZ46darh0XxuyELfIiKOHDlSrbcI8G0pC5nbvHlztf6BD3ygWu8TUjlvsrDDLDA1G5MstDMiDw/sE649a1kgeCYLPG5xbp04caJan3X4Z7aeZPVMnzG71QJRs8/zxBNPVOvZtTUiH5M+27jV9FmTsrD2LFg8DaGcAAALtklEQVT1VhvX7FoSka+Ln4shzpns+pmtq4888kjDo8ll32FWz+ZRn3k2Fn7RAgAAaEyjBQAA0JhGCwAAoDGNFgAAQGMaLQAAgMY0WgAAAI1ptAAAABqbaY7W0MyYLCukz2uyYxhbfkOWwZFlCfTJTpinPII+FhYWBtX75Jrs379/mkMavSynJ6v3kWVxjTFHKzumLGcry4tpIZuv85YZleXFnD59Ot3G0aNHq/VNmzZNdUyrLTuerD40ozJifHmJLWTjkq1rfa6f2bqWrQG1/KPXX3893f+0smzNxcXFaj27X+iz3mzcuLFaH9v5OVQ2D/vcg2X3pocOHarWa/N0NeZZJjv3suvAhQsX0n1kGZOz4hctAACAxjRaAAAAjWm0AAAAGtNoAQAANKbRAgAAaEyjBQAA0JhGCwAAoLGZ5mhlz8XPMitaZPkM2ce1a9cG739aQ3O0ssyMiIjjx49Pc0i3vD6ZM4cPH179A5mhbJ5lOSAHDhxI99FnLo5NlueSrSfZmpeNa59juNXyj7Ix2b59e7qNecsOy2Sfp5bF1KfeZx/zqEVOVibL2svyj2rvX79+/QqOqC5bh7Ocyc2bN1frO3fuTI+hxb3cmGRZatmY98mQzO71+qyLs5SNye7du6v1LGutzz3YWNY0v2gBAAA0ptECAABoTKMFAADQmEYLAACgMY0WAABAYxotAACAxjRaAAAAjZWu6/q/uJQXIuLS6h3O6G3tuu7Oad5gzIzZCk01bsYsIsy1lTBm0zNmK2PcpmfMpmfMpmfMVqbXuE3VaAEAAJDzp4MAAACNabQAAAAa02gBAAA0ptECAABoTKMFAADQmEYLAACgMY0WAABAYxotAACAxjRaAAAAjWm0AAAAGtNoAQAANKbRAgAAaEyjBQAA0JhGCwAAoDGNFgAAQGMaLQAAgMY0WgAAAI1ptAAAABrTaAEAADSm0QIAAGhMowUAANCYRgsAAKAxjRYAAEBjGi0AAIDG3jTNi7ds2dJt27btpvUXX3yx+v7nn3++Wr/77rur9VdeeaVa7+Ouu+6q1tetW3fT2sWLF+Py5ctlmv1lYzbUlStXqvVnnnkm3cb9999fra9fv36qY1puNcYsmwcf/ehHq/W3vvWt0xzODb35zW+u1u+4445B23/88ccvd113Z9/XD51n2bn77LPPVusbNmxI95Gd30PmWcT0YxYxfNx+5Vd+pVq//fbbq/V77rkn3UdtTRpqLda07DqQyeZqRMSrr75arb/jHe+o1mvzeTXG7Nq1a9X3Z+ffJz7xiWr9tttuq9Yj8jUru3Zm1uL8fOqpp6r1N72pfgv02muvpfvI1rU+a+PNjPGeI7un6HN+PvDAA9X6kDVvjGOWefLJJ9PXZOffpk2bVrz/MY5Zi3mWrXvZvW+m75o2VaO1bdu2eOyxx25aP3nyZPX9x44dq9YXFxer9SeeeKJa7+PgwYPVem2yPvTQQ1PvLxuzoU6fPl2tHzhwIN3Go48+Wq0POZlWY8zOnz9fff+uXbuq9fe85z1TH9P1sjHZt2/foO2XUi5N8/qh8yw7d48cOVKtZ2PeZxtDF+1px+yNfQ4Ztx07dgyqZ2tixLALaGYt1rQ+n7kmm6sRERcuXKjWH3744Wq9Np9XY8yy/4dZdu4cP368Ws8ay4h8zcqunZm1OD/37NlTrWfn1sWLF9N9tFgbb2aM9xzZPOhzfp45c6ZaH7LmjXHMMn3myKFDh6r1hYWFFe9/jGPWYp5l19+zZ89OcUSfre+a5k8HAQAAGtNoAQAANKbRAgAAaEyjBQAA0JhGCwAAoLGpnjqYyZ7Qkz0JKntCUAur/bS41rKnUR0+fLhaz566EjH8aW+zlo3J1atXq/Xs6ZZ9bN++vVrPniI0tjEf+kSx7EmQEcOfANRnLreWPdUzW9OyudrnyUlDv5ux6fOZa/qMR7aPoU8ubS073kceeaRaz57i1mfMs33M4zzMzr/snqXPPNi9e3e1/tJLL1Xrq/lU0ZXI1uHsCZc7d+5M9zG2z7zasnl27ty5wfsY8tTBtZDNs+weKXvaZ8TwJ9y24hctAACAxjRaAAAAjWm0AAAAGtNoAQAANKbRAgAAaEyjBQAA0JhGCwAAoLGmOVpDs4EOHDhQrbfIhJp1PspQWb5CluOTZaPMo6EZHNk865MXM7YcrEyW45Flj+3du7da75PTk52/2TbWIhNj48aNg96f5fL1+UzzmF9Uk82DbMz6zLVsjRhbXmJ2vFkeVLbOnzp1Kj2Gecvh6SPL5szmUrZu9jFvmVHZmG3durVa73N+ZvvIMpLWIlOxJjs/W6w38zaPMtm9eIt79aF5iq3mmV+0AAAAGtNoAQAANKbRAgAAaEyjBQAA0JhGCwAAoDGNFgAAQGMaLQAAgMaa5mgNdfz48Wq9T6ZNn7yQefLSSy9V69mYZDlbEfOXCTU0TyKbZ31yQM6ePVutjy3nY+iYvfvd7171Y9i9e/fgfbSWnRtZpszi4uLgY8jm49gyoTLZmGbzIMt0i5i//MDsO8zyXrK8p6NHj6bHsBY5dastG9dDhw5V61k+UkTEmTNnpjmkNZddu7JMxWxM+2SPnT59ulrPrhV9rtEtZZ8pG5MsD7WPecvRyuZZtkZnn7dFxt2s7n39ogUAANCYRgsAAKAxjRYAAEBjGi0AAIDGNFoAAACNabQAAAAa02gBAAA01jRHK8sSyJ5Zn+U39MlPyfJCFhYW0m3MUpbTcfDgwWq9xZjt2rWrWs8yK2ad75DNo2wOtMjoyF4ztkyaod9RlhfVRzbXs8y4tZDNtSzfKKv3yXvK5tKePXuq9Vmfn9lnHpqBM+sMnTGYxXpy5MiRaj3LrRljHmN2/cz0OXey6+fYDM2Yy+bB0DGPyOfirGWZUNk8yca8xX3avBmam5vd+/aRfS+tMir9ogUAANCYRgsAAKAxjRYAAEBjGi0AAIDGNFoAAACNabQAAAAa02gBAAA0ptECAABorGlgcSYLXMuCLvsElO3YsWOaQ1pzWYhrFgKZvT8b04g88DQLDxxbOG8WmJiNWZ+wxDGGc9ZknzkLB7x06VK13meeZbJ5OEZZUGW25vUJoczWtFmFLvY19HiyuXT69On0GMYWTD8PsrmYhbbOep71kR1zFm7fIlC8RYBvS0Ov11nQbB9bt26t1sd2fc3mdlbPrr997m1bXGNnKVtPuq4btP0+cyQ7f2fVL/hFCwAAoDGNFgAAQGMaLQAAgMY0WgAAAI1ptAAAABrTaAEAADSm0QIAAGisaY5WlhWQ5Tdkz7zPsn4ixpnlUZNlAWRjtnv37mr95MmT6TFk39u8yXJ2Dhw4UK33GY8+WVtjkuU9ZTkeJ06cqNb7ZHxkuRrZMY5RNteeeOKJav3BBx9M95GNWzYXZ70mZsebrUnZmpiNacT85Whl50/2HWYZdH3mWXYM+/fvT7cxa1lOVmZoRl1EPp/HlqOVye45snV6cXEx3ce8XT+HanGPdfHixeEHMkey86ZPjtZYcnX9ogUAANCYRgsAAKAxjRYAAEBjGi0AAIDGNFoAAACNabQAAAAa02gBAAA01jRHK8vhyPIVspysPpkZY3lufitZJk2W85Hlq0Tk4z62bLIsk2Lv3r3VepYD0ievYh4zn2qOHj1arR86dKha75NblOWzzKPDhw8Pen+fMckyzrZv3z7oGFrL1uDs/D116lS1nmW6zaNszIZmk/XJg8r2sXPnznQbs5aNW1bPrm19snpuxXWtpkUmVDbXbjUtxuxWu7fNZPdhfe5tx8IvWgAAAI1ptAAAABrTaAEAADSm0QIAAGhMowUAANCYRgsAAKAxjRYAAEBjpeu6/i8u5YWIuLR6hzN6W7uuu3OaNxgzY7ZCU42bMYsIc20ljNn0jNnKGLfpGbPpGbPpGbOV6TVuUzVaAAAA5PzpIAAAQGMaLQAAgMY0WgAAAI1ptAAAABrTaAEAADSm0QIAAGhMowUAANCYRgsAAKAxjRYAAEBj/x8s2bt4D/LE/wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1080x576 with 55 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "real_data = digits.data[:44].reshape((4, 11, -1))\n",
    "fig, ax = plt.subplots(5, 11, subplot_kw=dict(xticks=[], yticks=[]),figsize=(15,8))\n",
    "for j in range(11):\n",
    "    ax[4, j].set_visible(False)\n",
    "    for i in range(4):\n",
    "        im = ax[i, j].imshow(real_data[i, j].reshape((8, 8)),\n",
    "                             cmap=plt.cm.binary, interpolation='nearest')\n",
    "        im.set_clim(0, 16)\n",
    "ax[0, 5].set_title('Selection from the input data')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'Accuracy')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAIABJREFUeJzt3Xt8HVW5//HPN+mN9Aq9cGtLy52KXCMFBFFQLMjlUPVwFSpwkCOgIuiBoyKnHg9e0KMeUH8Fy10R8FYVLHdQQGgLtKVAS1tKSUtp0tLSJG3SJM/vj5mUTZp0h5LJ3km+79crr86smdn76WRnnr3WmllLEYGZmdmWlBQ6ADMzK35OFmZmlpeThZmZ5eVkYWZmeTlZmJlZXk4WZmaWl5OFmZnl5WRhZmZ5OVmYmVlevQodQEcZNmxYjBkzptBhmJl1KbNmzaqKiOH59us2yWLMmDHMnDmz0GGYmXUpkl5rz35uhjIzs7ycLMzMLC8nCzMzy8vJwszM8nKyMDOzvJwszMwsLycLMzPLq9s8Z2Gte3JRFf9ctKrVbXvuMJAT9tvpXWUVb9Xy5MJVfPrgkZSWaFP5mtp6fv3MUjbUNwKwTZ9enDF+NIO36f2+4nvm1dWUCMrHbPeu8rkVa1m2Zj0T9t3hfb2+mXUMJ4tu7PfPVnD53bNpCpDeva156vUFK9Zx6Sf2RBILV1Zz5o3/5M2363hyURXXfnZ/epWWUFVdx1k3Ps3LK9Ztep0I+Muc5dx23ni2699nq+JbU1vPeTfPoHZjIz859QBO3D9JXI8vqOSC22ayYWMTX/vkXlz0sd239hSYWQfJNFlImgD8FCgFboyI77XYvgswFRgOrAbOioiKdNto4EZgFBDA8RGxJMt4u5NfP72Ub/xxLofvNpQbzi6nrM+7f9WNTcF//n4uP3t4Ies3NnLKgSP53K+eRhLnfngsU594lbqGJv7z+H2YdNMzLFuzntvPG88RewwD4NH5K/nCbbM4bcpT3H7eeEYM6veeY/zlY4uprm9g350G8+U7n2PDxkaGlPXhojueZbcRA9h9xAB+OH0+6+sbuezYJKGZWWEomr9idvQLS6XAAuATQAUwAzg9Il7M2edu4C8RcYuko4HPR8Tn0m2PAt+NiAckDQCaIqK2rfcrLy8PD/eRuO2pJXzrT/M4eu8R/PzMg+jXu7TV/Zqagsl/eZGbn1xC71IxbEBf7jh/PLsOH8DUf7zK5L+8SO9S0bdXKVMnfYhDxr67qejJRVWcf8tMth/UjzvOH89OQ7bZtK2xKbj1qSVUvLUeAAEn7r8T+48aAsDKdRv4yA8e4ZMf2IHvTdyPC26byd9fqaK0ROy782Bu/fwhDOjXi2/8YS53znid844Yyzc/tY8ThlkHkzQrIsrz7pdhsjgMuDoiPpmuXwkQEdfk7DMPmBARryu5CqyNiEGSxgFTIuKI9r6fk0XijbXrOeqHj3LE7sP45VkH06fXlu9hiAh+/MACHp1fyc/PPIhR25Vt2nbnM0u58R+vcu1n9+eA9CLf0qzXVjNp6gwGbdOb3/zboYweWkZDYxOX3T2bPz2/nP59SpFEfUMTJSVww9nlHLnHcL79pxe4/emlPPTVoxgzrD91DY187e45vL1hI/93+oEM7Nd7U3yT//IiNz2xhDPHj+Y7J+9LSYkThllHKYZk8RmSRHB+uv45YHxEXJyzz6+BpyPip5ImAr8DhgFHAucD9cBY4EHgiohobOv9nCwSV/5+LvfMep1HLv8oI7cty39AB5hbsZbPTX2avr1KuOXcQ/jJA6/wt3kr3tXf0Nzvsbiyhm+dsA+T//Iinzl4JNdM3C/v60cEP5w+n58/uoiJB+3MDz69H71KfSOfWUdob7IodAf35cB1kiYBjwPLgEaSuI4EDgSWAr8FJgG/yj1Y0gXABQCjR4/urJiL1pKqGu6e+Tpnjh/daYkC4IMjB3PnBYdy1o3PcPxP/05TwFUnjOPcI8Zu2mfYgL7cecGhnDP1Gb71p3n0KS3hkqP3aNfrS+LrE/amrE8p196/gIq31rPrsP6b7Td4m9588WO7v687tCKCW55cwssr1m31a5h1tlHblWV+I0iWyWIZSed0s5Fp2SYRsRyYCJD2S3w6ItZIqgCej4jF6bY/AofSIllExBRgCiQ1i4z+H13GTx5cQK9ScdHRnX/30N47DOKuLxzK5XfP5tQPjeLUD22evIeU9eH288fztbvncODoIe/q42iPi4/egwF9ezHl8cW8tqpms+1V1fX8Y2HVVt+h1dQUXDXtBW7/51KGDejzrluHzYrZvjsNzvw9smyG6kXSwX0MSZKYAZwREfNy9hkGrI6IJknfBRoj4qq0c/xZ4OMRUSnpJmBmRFzf1vv19Gao+SvWMeGnj3PBR3blyuP2KXQ4BfHIyyu58PZZ7DK0jNvPH8+Ige2/Q6uxKfiP383hnlkVfOGoXbliwt7uTLceoeDNUBHRIOliYDrJrbNTI2KepMkkF/5pwEeBayQFSTPURemxjZIuBx5KO75nATdkFWtX19QUfO++lxjQpxcXfmS3QodTMB/bewQ3TfoQ5986k4k/f7LNTvnWLF+znmeXruHSj+/Jl47Z3YnCrIXMahadrafWLBoam/jaPXP4w3PL+Oan9uH8I3ctdEgFN+u11Vw97UVq6hvafUyJxJnjR/P5D4/Nv7NZN1LwmoVlr76hiS/f+Rz3vbCCy4/d04kidfAu2/HnS9p917WZtYOTRRe1YWMjX7zjWR5+eSXfOmEc5x3hb8Rmlh0niy6otr6Bf7t1Jk8uWsV3T9mXM8fvUuiQzKybc7LoYt7esJFzb5rBs0vf4kef3Z+JB40sdEhm1gM4WRS5DRsbufS3z7Oqph5I7tpZsXYD/3f6QXxqvx0LHJ2Z9RQeM6HIPfDim9z3wgo2NjZRIhgztD83nFPuRGFmnco1iyI3bfZyRgzsyz0XHu4nis2sYFyzKGJr12/ksfmVnLDfTk4UZlZQThZFbPq8FdQ3NnHSATvl39nMLENOFkXsz7OXs8vQMvYfmf0gYWZmW+JkUaQq19XxxMIqTtxvJ49TZGYF52RRpO6d+wZNgZugzKwoOFkUqWmzl7PX9gPZc/uBhQ7FzMzJohgtqaph1mtvuVZhZkXDyaII/ezhV+jbq4TPHOyhPMysODhZFJlX3lzHH59bxjmHj2H7Qe2f6c3MLEtOFkXmxw8soKxPLy48qufOeGdmxcfJoojMrVjLfS+s4NwjxrJd/z6FDsfMbBOPDVVgudPaXnv/fIaU9eb8Iz2RkZkVFyeLApqxZDVn3vA09Y1Nm8quOG5vBvXrXcCozMw252RRQI8vqKQxgi8fswcSlPUp5ezDxhQ6LDOzzThZFNDzr69hz+0Hcukn9ix0KGZmW+QO7gKJCOYuW+tBAs2sS8g0WUiaIGm+pIWSrmhl+y6SHpI0R9Kjkka22D5IUoWk67KMsxCWrq5lTe1G9h81pNChmJnllVmykFQKXA8cB4wDTpc0rsVu1wK3RsR+wGTgmhbbvwM8nlWMhfT862sA2M81CzPrArKsWRwCLIyIxRFRD9wJnNxin3HAw+nyI7nbJR0MbA/cn2GMBTOnYi19e5V4oEAz6xKyTBY7A6/nrFekZblmAxPT5VOAgZKGSioBfgRcnmF8BTWnYg377jyY3qXuNjKz4lfoK9XlwFGSngOOApYBjcAXgXsjomJLB0u6QNJMSTMrKyuzj7aDNDQ2MXfZWjdBmVmXkeWts8uAUTnrI9OyTSJiOWnNQtIA4NMRsUbSYcCRkr4IDAD6SKqOiCtaHD8FmAJQXl4edBGvrKxmw8YmDnDntpl1EVkmixnAHpLGkiSJ04AzcneQNAxYHRFNwJXAVICIODNnn0lAectE0ZXNqWju3HayMLOuIbNmqIhoAC4GpgMvAXdFxDxJkyWdlO72UWC+pAUkndnfzSqeYvL862sZ1K8XY4aWFToUM7N2yfQJ7oi4F7i3RdlVOcv3APfkeY2bgZszCK9g5lSsYf9RQ5BU6FDMzNql0B3cPc6GjY28vGKdO7fNrEtxsuhk85a/TWNTsL/7K8ysC3Gy6ETr6xv5yYMLKBEcMNrJwsy6Do8620mq6xo4/5YZPP3qar438YOMGOj5tc2s63Cy6ARr129k0k3PMKdiLT859QBOPqDlg+xmZsXNySJjq2vqOXvq08xfsY7rzziICfvuUOiQzMzeMyeLDK1ct4Gzbnya11bVMuXscj6214hCh2RmtlWcLDLyxtr1nHHD07z59gZumvQhDt99WKFDMjPbak4WGfn+fS/z5tsbuPXcQygfs12hwzEze19862wG1tc3cv+Lb3LyATs5UZhZt+BkkYEHX3qT2vpGTtx/p0KHYmbWIZwsMjBt9nJGDOzL+LFDCx2KmVmHcLLoYGvXb+Sx+ZWcsN9OlJZ4oEAz6x6cLDrY9HkrqG9s4qQD3ARlZt2Hk0UH+/Ps5Yzeroz9PaqsmXUjThYdqHJdHU8srOLE/Xf0XBVm1q04WXSge+e+QVPASft77Ccz616cLDrQtNnL2Wv7gey1w8BCh2Jm1qGcLDpIxVu1zHrtLXdsm1m35GTRQf48+w0ATtzPycLMuh8niw4ybfZyDhg1hNFDywodiplZh3Oy6AALV67jpTfe5iQP72Fm3ZSTRQeY9vxySgQn7LdjoUMxM8tEpslC0gRJ8yUtlHRFK9t3kfSQpDmSHpU0Mi0/QNJTkual207NMs73IyKYNns5h+46lBGDPK+2mXVPmSULSaXA9cBxwDjgdEnjWux2LXBrROwHTAauSctrgbMj4gPABOAnkoZkFev7MXfZWpasqnUTlJl1a1nWLA4BFkbE4oioB+4ETm6xzzjg4XT5kebtEbEgIl5Jl5cDK4HhGca61aY9v5zepeK4fd0EZWbdV5bJYmfg9Zz1irQs12xgYrp8CjBQ0rvG9ZZ0CNAHWJRRnO/Lw/NXcvhuwxhc1rvQoZiZZabQHdyXA0dJeg44ClgGNDZvlLQjcBvw+YhoanmwpAskzZQ0s7KysrNi3mTF2g0srqzhCM+vbWbdXJbJYhkwKmd9ZFq2SUQsj4iJEXEg8I20bA2ApEHAX4FvRMQ/W3uDiJgSEeURUT58eOe3Uj25qAqAw3bzJEdm1r1lmSxmAHtIGiupD3AaMC13B0nDJDXHcCUwNS3vA/yBpPP7ngxjfF+eWLiKbct6M27HQYUOxcwsU5kli4hoAC4GpgMvAXdFxDxJkyWdlO72UWC+pAXA9sB30/J/BT4CTJL0fPpzQFaxbo2I4MlFVRy221BKPCOemXVzvbJ88Yi4F7i3RdlVOcv3AJvVHCLiduD2LGN7v5asquWNtRu4aDf3V5hZ91foDu4u64mFSX/F4e6vMLMewMliKz21aBU7Du7H2GH9Cx2KmVnmnCy2QlNT0l9x+G7DPH2qmfUIThZb4aUVb/NW7UY3QZlZj5E3WUi6RNK2nRFMV/HUolUAfNgP45lZD9GemsX2wAxJd6WjyPb4dpfHFlSy6/D+7DDYo8yaWc+QN1lExDeBPYBfAZOAVyT9j6TdMo6tKM2tWMvfX6nyKLNm1qO0q88iIgJYkf40ANsC90j6QYaxFaVr75/PtmW9Oe+IsYUOxcys07Snz+LLkmYBPwCeAD4YEf8OHAx8OuP4isozr67msQWVXHjUbgzs51FmzaznaM8T3NsBEyPitdzCiGiSdEI2YRWfiODa6fMZMbAvZx82ptDhmJl1qvY0Q90HrG5ekTRI0niAiHgpq8CKzeOvVPHMktVccvTubNOntNDhmJl1qvYki18A1Tnr1WlZj3L9wwsZue02nPqh0YUOxcys07UnWSjt4AaS5icyHoCwGC1YuY6P7jWcPr38HKOZ9TztufItlvQlSb3Tny8Di7MOrJhsbGxiTe1Ghg3oW+hQzMwKoj3J4kLgcJJZ7iqA8cAFWQZVbN6qqQdwsjCzHitvc1JErCSZ5a7HqqpuThZ9ChyJmVlh5E0WkvoB5wEfADaNbxER52YYV1Gpqq4DYKhrFmbWQ7WnGeo2YAfgk8BjwEhgXZZBFZtVNWmy6O+ahZn1TO1JFrtHxLeAmoi4BfgUSb9Fj7GquRlqoGsWZtYztSdZbEz/XSNpX2AwMCK7kIpPVXU9fUpLGNi3x90xbGYGtO95iSnpfBbfBKYBA4BvZRpVkVlVXcfQAX08K56Z9VhbTBaSSoC3I+It4HFg106JqshUpcnCzKyn2mIzVPq09tc7KZaitaqm3s9YmFmP1p4+iwclXS5plKTtmn/a8+LpzHrzJS2UdEUr23eR9JCkOZIelTQyZ9s5kl5Jf855D/+nDrequp6h/Z0szKznak+fxanpvxfllAV5mqQklQLXA58gefJ7hqRpEfFizm7XArdGxC2SjgauAT6XJqNvA+Xpe81Kj32rPf+pjhQRVFXX+YE8M+vR2vME99ZOCXcIsDAiFgNIuhM4GchNFuOAr6bLjwB/TJc/CTwQEavTYx8AJgC/2cpYtlp1XQN1DU3uszCzHq09T3Cf3Vp5RNya59Cdgddz1pvHlco1G5gI/BQ4BRgoaWgbx+6cL9YsND9j4WYoM+vJ2tMM9aGc5X7AMcCzQL5k0R6XA9dJmkRyt9UyoLG9B0u6gHRQw9Gjs5lnovnpbT+QZ2Y9WXuaoS7JXZc0BLizHa+9DBiVsz4yLct97eUkNQskDQA+HRFrJC0DPtri2EdbiW0KMAWgvLw8Wm7vCFWbahZuhjKznmtrZvKpAdrTjzED2EPSWEl9SEaunZa7g6Rh6bMcAFcCU9Pl6cCxkrZNHwg8Ni3rdM2DCPrWWTPrydrTZ/FnkjuSIEku44C78h0XEQ2SLia5yJcCUyNinqTJwMyImEZSe7hGUpA0Q12UHrta0ndIEg7A5ObO7s7W3GexnWsWZtaDtafP4tqc5QbgtYioaM+LR8S9wL0tyq7KWb4HuKeNY6fyTk2jYFZV1zF4m96eTtXMerT2JIulwBsRsQFA0jaSxkTEkkwjKxJVNfW+bdbMerz2fF2+G2jKWW9My3qEqnV1DPNts2bWw7UnWfSKiPrmlXS5x3zVXuWahZlZu5JFpaSTmlcknQxUZRdScVlVXec7ocysx2tPn8WFwB2SrkvXK4BWn+rubhoam3irdqNrFmbW47XnobxFwKHpQ3NERHXmURWJ1TXpA3muWZhZD5e3GUrS/0gaEhHVEVGdPij3350RXKE1P709zM9YmFkP154+i+MiYk3zSjpM+PHZhVQ8PC6UmVmiPcmiVNKmq6WkbYAecfVc5XGhzMyA9nVw3wE8JOkmQMAk4JYsgyoWzeNCuc/CzHq69nRwf1/SbODjJGNETQd2yTqwYlBVXU/vUjGoX3tyqplZ99XeAY/eJEkUnwWOBl7KLKIisqq6jqH9+yKp0KGYmRVUm1+ZJe0JnJ7+VAG/BRQRH+uk2ApuVU09wwa6v8LMbEvtKy8DfwdOiIiFAJIu7ZSoikRVWrMwM+vpttQMNRF4A3hE0g2SjiHp4O4xVlV7XCgzM9hCsoiIP0bEacDewCPAV4ARkn4h6djOCrBQ1m3YyIq3NzByyDaFDsXMrODydnBHRE1E/DoiTiSZC/s54D8yj6zAnl68msam4NDdhhY6FDOzgntP079FxFsRMSUijskqoGLx5KJV9O1VwkGjty10KGZmBee5Qtvw5KIqysdsS7/epYUOxcys4JwsWlFVXcfLK9Zx+G7DCh2KmVlRcLJoxVOLVgFwuPsrzMwAJ4tWPbmoioF9e/HBnQcXOhQzs6LgZNGKJxauYvyuQ+lV6tNjZgYZJwtJEyTNl7RQ0hWtbB8t6RFJz0maI+n4tLy3pFskzZX0kqQrs4wz1+ura1m6upYP7+4mKDOzZpklC0mlwPXAccA44HRJ41rs9k3grog4EDgN+Hla/lmgb0R8EDgY+IKkMVnFmuud/gp3bpuZNcuyZnEIsDAiFkdEPXAncHKLfQIYlC4PBpbnlPeX1AvYBqgH3s4w1k2eWFTFsAF92XP7AZ3xdmZmXUKWyWJn4PWc9Yq0LNfVwFmSKoB7gUvS8nuAGpKxqZYC10bE6pZvIOkCSTMlzaysrHzfAUcETy5axeG7DfWw5GZmOQrdg3s6cHNEjCSZ1/s2SSUktZJGYCdgLHCZpF1bHpw+TV4eEeXDhw9/38HUNTRRua6OvXYY+L5fy8ysO8kyWSwDRuWsj0zLcp0H3AUQEU8B/YBhwBnA3yJiY0SsBJ4AyjOMFYCaugYABvT1zHhmZrmyTBYzgD0kjZXUh6QDe1qLfZYCxwBI2ockWVSm5Uen5f2BQ0nm18hUbX0jAGV9PMSHmVmuzJJFRDQAF5PM2f0SyV1P8yRNlnRSuttlwL+lc3z/BpgUEUFyF9UASfNIks5NETEnq1ib1dQnNYv+rlmYmb1LplfFiLiXpOM6t+yqnOUXgQ+3clw1ye2znaqmLqlZOFmYmb1boTu4i0pzn0V/N0OZmb2Lk0WO2rQZqqyPaxZmZrmcLHK80wzlmoWZWS4nixyuWZiZtc7JIkdNvWsWZmatcbLIUVvXgAT9ejlZmJnlcrLIUVPfSFnvUkpKPC6UmVkuJ4sctfUNfsbCzKwVThY5qusanSzMzFrhZJGjtq7B40KZmbXCySJHTX0D/X3brJnZZpwsctTWN1Lm22bNzDbjZJGjps41CzOz1jhZ5Kitb3SfhZlZK5wsctTU+dZZM7PWOFmkIsI1CzOzNjhZpOoammhoCtcszMxa4WSRap5/2xMfmZltzski1TxLXplrFmZmm3GySL1Ts3CyMDNryckiVdM88ZEfyjMz24yTRaq2zjULM7O2OFmkNtUs3MFtZraZTJOFpAmS5ktaKOmKVraPlvSIpOckzZF0fM62/SQ9JWmepLmS+mUZa/P827511sxsc5ldGSWVAtcDnwAqgBmSpkXEizm7fRO4KyJ+IWkccC8wRlIv4HbgcxExW9JQYGNWsUIylwV4/m0zs9ZkWbM4BFgYEYsjoh64Ezi5xT4BDEqXBwPL0+VjgTkRMRsgIlZFRGOGsVKb3jrrPgszs81lmSx2Bl7PWa9Iy3JdDZwlqYKkVnFJWr4nEJKmS3pW0tdbewNJF0iaKWlmZWXl+wq2Jr11dpverlmYmbVU6A7u04GbI2IkcDxwm6QSkuaxI4Az039PkXRMy4MjYkpElEdE+fDhw99XIM2z5JWU6H29jplZd5RlslgGjMpZH5mW5ToPuAsgIp4C+gHDSGohj0dEVUTUktQ6DsowVmrqGylzE5SZWauyTBYzgD0kjZXUBzgNmNZin6XAMQCS9iFJFpXAdOCDksrSzu6jgBfJUG19gzu3zczakNlX6YhokHQxyYW/FJgaEfMkTQZmRsQ04DLgBkmXknR2T4qIAN6S9GOShBPAvRHx16xiBaipc83CzKwtmV4dI+Jekiak3LKrcpZfBD7cxrG3k9w+2ylq6xs84qyZWRsK3cFdNDxLnplZ25wsUjX1je6zMDNrg5NFKrl11jULM7PWOFmkauob3WdhZtYGJ4tUbX2DZ8kzM2uDkwVQ39DExsZwzcLMrA1OFrwzPLn7LMzMWudkwTuDCPpuKDOz1jlZkDxjAZ74yMysLU4W5CQLN0OZmbXKyQKoTZuhPP+2mVnrnCxwM5SZWT5OFrhmYWaWj5MFUFPvmoWZ2ZY4WQC1da5ZmJltiZMF79Qs/FCemVnrnCxIOri36V1KaYkKHYqZWVFyssBzWZiZ5eNkgeeyMDPLx8mCpGbhzm0zs7Y5WZCMOuvbZs3M2uZkAdTUuWZhZrYlmSYLSRMkzZe0UNIVrWwfLekRSc9JmiPp+Fa2V0u6PMs4a+sbPIigmdkWZJYsJJUC1wPHAeOA0yWNa7HbN4G7IuJA4DTg5y22/xi4L6sYm9XUNVLmu6HMzNqUZc3iEGBhRCyOiHrgTuDkFvsEMChdHgwsb94g6V+AV4F5GcYIJA/lDXCfhZlZm7JMFjsDr+esV6Rlua4GzpJUAdwLXAIgaQDwH8B/ZRjfJrV1jb511sxsCwrdwX06cHNEjASOB26TVEKSRP43Iqq3dLCkCyTNlDSzsrJyqwKob2iivrGJ/u7gNjNrU5Zfp5cBo3LWR6Zluc4DJgBExFOS+gHDgPHAZyT9ABgCNEnaEBHX5R4cEVOAKQDl5eWxNUGubx6e3M1QZmZtyvIKOQPYQ9JYkiRxGnBGi32WAscAN0vaB+gHVEbEkc07SLoaqG6ZKDrSCfvtyO4jBmT18mZmXV5mySIiGiRdDEwHSoGpETFP0mRgZkRMAy4DbpB0KUln96SI2KoawtYaXNab6844qDPf0sysy1EnX5szU15eHjNnzix0GGZmXYqkWRFRnm+/Qndwm5lZF+BkYWZmeTlZmJlZXk4WZmaWl5OFmZnl5WRhZmZ5OVmYmVle3eY5C0mVwGtbcegwoKqDw+koxRqb43rvijW2Yo0Lije2Yo0Lti62XSJieL6duk2y2FqSZrbngZRCKNbYHNd7V6yxFWtcULyxFWtckG1sboYyM7O8nCzMzCwvJ4t0iPMiVayxOa73rlhjK9a4oHhjK9a4IMPYenyfhZmZ5eeahZmZ5dWjk4WkCZLmS1oo6YoCxjFK0iOSXpQ0T9KX0/KrJS2T9Hz6c3wBYlsiaW76/jPTsu0kPSDplfTfbQsQ11455+V5SW9L+kqhzpmkqZJWSnohp6zV86TEz9LP3RxJmU2o0kZcP5T0cvref5A0JC0fI2l9zrn7ZSfH1ebvTtKV6fmaL+mTWcW1hdh+mxPXEknPp+Wdec7auk50zucsInrkD8mETIuAXYE+wGxgXIFi2RE4KF0eCCwAxpHMRX55gc/TEmBYi7IfAFeky1cA3y+C3+UKYJdCnTPgI8BBwAv5zhPJfPP3AQIOBZ7u5LiOBXqly9/PiWtM7n4FOF+t/u7Sv4XZQF9gbPp3W9qZsbXY/iPgqgKcs7auE53yOevJNYtDgIURsTgi6oE7gZMLEUhEvBERz6bL64CXgJ0LEUs7nQzcki7fAvxLAWOBZGreRRGxNQ9ldoiIeBxY3aK4rfN0MnBrJP4JDJG0Y2fFFRH3R0RDuvpPYGQW7/1lhWs9AAAGvElEQVRe49qCk4E7I6IuIl4FFpL8/XZ6bJIE/Cvwm6zevy1buE50yuesJyeLnYHXc9YrKIILtKQxwIHA02nRxWkVcmohmntIpru9X9IsSRekZdtHxBvp8gpg+wLEles03v3HW+hz1qyt81RMn71zSb59Nhsr6TlJj0k6sgDxtPa7K6bzdSTwZkS8klPW6eesxXWiUz5nPTlZFB1JA4DfAV+JiLeBXwC7AQcAb5BUfzvbERFxEHAccJGkj+RujKS+W7Bb6iT1AU4C7k6LiuGcbabQ56k1kr4BNAB3pEVvAKMj4kDgq8CvJQ3qxJCK8nfXwum8+4tJp5+zVq4Tm2T5OevJyWIZMCpnfWRaVhCSepN8AO6IiN8DRMSbEdEYEU3ADWRY9W5LRCxL/10J/CGN4c3m6mz678rOjivHccCzEfEmFMc5y9HWeSr4Z0/SJOAE4Mz0AkPazLMqXZ5F0jewZ2fFtIXfXcHPF4CkXsBE4LfNZZ19zlq7TtBJn7OenCxmAHtIGpt+Oz0NmFaIQNJ20F8BL0XEj3PKc9sXTwFeaHlsxnH1lzSweZmkY/QFkvN0TrrbOcCfOjOuFt71Ta/Q56yFts7TNODs9G6VQ4G1Oc0ImZM0Afg6cFJE1OaUD5dUmi7vCuwBLO7EuNr63U0DTpPUV9LYNK5nOiuuHB8HXo6IiuaCzjxnbV0n6KzPWWf04hfrD8ndAgtIvg18o4BxHEFSdZwDPJ/+HA/cBsxNy6cBO3ZyXLuS3IUyG5jXfI6AocBDwCvAg8B2BTpv/YFVwOCcsoKcM5KE9QawkaRt+Ly2zhPJ3SnXp5+7uUB5J8e1kKQtu/mz9st030+nv+fngWeBEzs5rjZ/d8A30vM1Hzius3+XafnNwIUt9u3Mc9bWdaJTPmd+gtvMzPLqyc1QZmbWTk4WZmaWl5OFmZnl5WRhZmZ5OVlYtyPpovTBpS5N0gBJFxU6DjNwsrAuRFJI+lHO+uWSrm6xz1nA0Iio7uz42iLpZkmf2YpDv0tyO2RbrztJ0nXvMZaiOS/WtThZWFdSB0yUNGwL+5QC38nizdMneDuFpDLgHxFxf6FjMQMnC+taGkimjby05Ybmb+8RcUtERPM3aEkfTQd4+5OkxZK+J+lMSc8omadjt3S/4ZJ+J2lG+vPhtPxqSbdJegK4TVI/STelxz4n6WOtxCJJ1ymZe+FBYETOtoPTeGZJmq5WRgGVdCLwCPANSQ9K2r61WNLdR0l6VMlcBt/OeY2vSnoh/flKaydT0tfS/+scSf+VlvWX9FdJs9NjT23PL8a6P387sa7memCOpB+8h2P2B/YhGXZ6MXBjRByiZPKYS4CvAD8F/jci/iFpNDA9PQaSOQOOiIj1ki4jGa/tg5L2JhmRd8+I2JDzfqcAe6XHbQ+8CExNx/X5P+DkiKhML8TfJRn5Ndc/gEPTpPcFkqE5Lmsllkkk4yftC9QCMyT9leQp388D40me4n1a0mMR8VzzG0g6lmRoikPSfaYpGSRyOLA8Ij6V7jf4PZxn68acLKxLiYi3Jd0KfAlY387DZkQ6Jo6kRUBz085coLlm8HFgXDL8DgCDcjrJp0VE83sdQXLBJyJelvQaycBxc3Le7yPAbyKiEVgu6eG0fC+SC/sD6fuUkgwr0dJOwK1KZrArAypztuXGAvBApAPZSfo97wwJ8YeIqMkpPxJ4Lue4Y9Of5rIBJMnj78CPJH0f+EtE/L2V+KwHcrKwrugnJOPw3JRT1kDarCqphGT2w2Z1OctNOetNvPM3UELybT63hkB6Ua/poLgFzIuIw/Lsdx3JbGd/S5u5vp2zrWUsLcfrae/4PQKuiYj/t9mGZPrN44H/lvRQRExu52taN+Y+C+tyImI1cBfJ4HPNlgAHp8snAb3f48veT9IkBYCkA9rY7+/Amek+ewKjSQa3y/U4cKqk0rRPorn2Mh8YLumw9Pjekj7Qyntsyzu1iXNa2Z7rE0rmYN6GZIa0J9IY/0VSmZLRgk9Jy3JNB85trj1J2lnSCEk7AbURcTvwQ5LpRc1cs7Au60fAxTnrNwB/kjQb+BvvvTbwJeB6SXNI/i4eBy5sZb+fA7+QNJekNjMpIupa7PMH4GiSvoqlwFMAEVGf3kL7s7QvoBdJLWlei+MnA/dIWg08TDLPc1ueIZnfYCRwe0TMhKTDn3eG8b4xt78ijeV+SfsAT6W1p2rgLGB34IeSmkhGXf33Lby39SAeddbMzPJyM5SZmeXlZGFmZnk5WZiZWV5OFmZmlpeThZmZ5eVkYWZmeTlZmJlZXk4WZmaW1/8HCwQqE+mGatYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "Performance = []\n",
    "N = digits.data.shape[0]\n",
    "ind = np.random.permutation(N)\n",
    "for i in range(1,200,2):\n",
    "    clf = GradientBoostingClassifier(n_estimators=i, max_depth=4, random_state=0)\n",
    "    clf = clf.fit(digits.data[ind[:int(N*0.7)],:], digits.target[ind[:int(N*0.7)]])\n",
    "    Performance.append(clf.score(digits.data[ind[int(N*0.7):],:], digits.target[ind[int(N*0.7):]]))\n",
    "Perform = np.array(Performance)\n",
    "plt.plot(np.arange(1,200,2),Perform)\n",
    "plt.xlabel(u'Número de árboles')\n",
    "plt.ylabel('Accuracy')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stacking"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Staking es similar al primer algoritmo de ensamble descrito en la clase anterior, en el que se usan modelos base diferentes entrenados sobre el mismo conjunto de datos. Sin embargo, a diferencia del ensamble de modelos en el que la decisión final se toma en consenso entre los clasificadores base (**voting**), en Stacking la combinación de los modelos se realiza usando un modelo adicional (meta-modelo), en un areglo de dos capas. En la primera capa se entrenan los modelos base y en la segunda se entrena un nuevo modelo que toma como entradas las salidas de los modelos en la primera capa y produce la decisión final.\n",
    "\n",
    "<img src=\"local/imgs/Stacking.png\"\n",
    "     alt=\"Árbol de decisión\"\n",
    "     width=\"500px\" />"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEICAYAAABcVE8dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOydeXhTZfbHP2+S7hulZS9tyg5lqVAWURRkKyrgzmhRGHUQFAcdndEZ3Mf+xm0ccZRxGHepKC4oCpQdxQUBEdl32lJka4G20DXJ+/vjJm3aJm3aJk2bvp/n6UOae3Pfcy/Nueee95zvK6SUKBQKhcL30XnbAIVCoVA0DsrhKxQKRQtBOXyFQqFoISiHr1AoFC0E5fAVCoWihaAcvkKhULQQlMNXNGuEEO8KIZ51su0NIcTjXrCppxBiuxCiQAjxx0Ye+ykhxEIX990ghLjb0zYpmg7K4Ss8ghDiciHED0KIPCHEWSHE90KIwdZt04UQ33naBinlTCnl3z09jgP+AqyXUoZJKV/1wvgKhUOUw1e4HSFEOPA18G+gNdAJeBoo8aZdjUgcsNvbRigUVVEOX+EJegBIKRdJKc1SyiIp5Sop5Q4hRG/gDeBSIcQFIcR5ACHENUKIX4QQ+UKIY0KIp+wPaPfEcN66fXrVQYUQYUKI9UKIV4VGebpHCDFSCJEthHhICHFaCHFCCPF7u89GCSG+so6/RQjxbE1PIUKISUKI3VZ7NljPCyHEOmAU8Jr1/Ho4+OwG6/F/sO7zlXX8NLvxjXb7D7e+l2f9d7jdtnghxDfW9NFqILrKWMPsrtuvQoiRTs6nm/U4eUKIHCHEx87OXdF8UQ5f4QkOAGYhxHtCiAlCiEjbBinlXmAm8KOUMlRK2cq66SJwB9AKuAaYJYS4DkAIEQesQHtiaAMkAtvtBxRCRAFrge+llH+UjjVD2gMRaE8cdwGv29n2utWG9sA0649DrE58EfCA1Z7lwFdCCH8p5VXARmC29fwOODnM74DbrbZ0BX4E3kF7ItoLPGkdqzWwDHgViAJeBpZZzxfgQ+BnNEf/d3u7hRCdrJ991nrch4HPhBBtHNjzd2AVEAnEoF1rhY+hHL7C7Ugp84HLAQn8DzgjhFgqhGhXw2c2SCl3SiktUsodaA71Suvm24A11ieGMillrpTS3uF3BL4BPpFSPlaDaWXAM9ZjLAcuAD2FEHrgRuBJKWWhlHIP8F4Nx5kCLJNSrpZSlgEvAUHA8Bo+U5V3pJSHpZR5aDezw1LKNVJKE/AJcIl1v2uAg1LKD6SUJinlImAfMFEIEQsMBh6XUpZIKb8FvrIbYyqwXEq53HpdVwNbgaudXJs4oKOUslhK6fE5FkXjoxy+wiNIKfdKKadLKWOAvmhO+RVn+wshhlrTMWeEEHloTwG29ERn4HANw12D5nDfqMWsXKtDtVEIhKJF6QbgmN02+9dV6Qhk2n6RUlqs+3eqZXx7Ttm9LnLwe6ijsaxkWsfqCJyTUl6sss1GHHCzNZ1z3po+uxzo4MCevwAC2GxNVd1Zh3NRNBOUw1d4HCnlPuBdNMcPWuRflQ+BpUBnKWUEmvMW1m3H0NIezvgfkA4sF0KE1MPEM4AJLZVho3MN+/+G5kwBEEII6/7H6zF2bVQay0qsdawTQGSVc461e30M+EBK2cruJ0RK+VzVQaSUJ6WUf5BSdgTuAeYLIbq591QU3kY5fIXbEUL0sk6Oxlh/7wzcCmyy7nIKiBFC+Nt9LAw4K6UsFkIMQUvj2EgDxgghbhFCGKwTnIlVhp0N7EfLpQfVxV4ppRn4HHhKCBEshOiFNp/gjMXANUKI0UIIP+AhtAqkH+oyrossB3oIIW6znvsUoA/wtZQyEy1F87QQwl8IcTkw0e6zC9FSP+OFEHohRKB18jqm6iBCiJvt3j+HdlO2eOB8FF5EOXyFJygAhgI/CSEuojn6XWiOEWAdWtniSSFEjvW9e4FnhBAFwBNoThUAKWUWWt75IeAs2oTtAPsBrZO0M4Bs4EshRGAdbZ6NNqF7EvgAbQ7BYRmplHI/Wn7830AOmpOdKKUsreOYtSKlzAWuRTv3XLTUy7VSStt1uw3tWp9Fm+h93+6zx4DJwN/QnmKOAX/G8fd+MNr/1wW0J605Usoj7j4fhXcRagEUhaI6QojngfZSSqfVOgpFc0NF+AoF5Wmo/tb6/SFoZZtLvG2XQuFODN42QKFoIoShpXE6os0x/BP40qsWKRRuRqV0FAqFooWgUjoKhULRQmjSKZ3o6GhpNBq9bYZCoVA0G37++eccKaUj+Yym7fCNRiNbt271thkKhULRbBBCVO3MLkeldBQKhaKFoBy+QqFQtBCUw1coFIoWQpPO4SsUCt+nrKyM7OxsiouLvW1KsyIwMJCYmBj8/Pxc/oxy+AqFwqtkZ2cTFhaG0WhEEx5V1IaUktzcXLKzs4mPj3f5c25J6Qgh3rYuG7fLyfaR1qXTtlt/nnDHuC2VtLQ0jEYjOp0Oo9FIWlqat01SKOpNcXExUVFRytnXASEEUVFRdX4qcleE/y7wGnZKfQ7YKKW81k3jtVjS0tKYMWMGhYWFAGRmZjJjxgwAUlJSvGmaQlFvlLOvO/W5Zm6J8K1Lq511x7EUNTN37txyZ2+jsLCQuXPneskihULRXGjMKp1LhRC/CiFWCCESnO0khJghhNgqhNh65syZRjSveZCVlVWn9xUKhWukpqaSkJBA//79SUxM5KeffuKVV16pFmC5yrvvvsvs2bOrvf/GG2/w/vs1JUM8R2NN2m4D4qSUF4QQVwNfAN0d7SilXAAsAEhKSlLKblWIjY0lM7N6I11sbKyDvRUKhSv8+OOPfP3112zbto2AgABycnIoLS1lypQpTJ06leDgYLeNNXPmTLcdq640SoQvpcyXUl6wvl4O+Akhomv5mMIBqamp1f74goODSU1N9ZJFCkXz58SJE0RHRxMQEABAdHQ0n376Kb/99hujRo1i1KhRAMyaNYukpCQSEhJ48sknyz+/ZcsWhg8fzoABAxgyZAgFBQWVjr9s2TIuvfRScnJyeOqpp3jppZcAGDlyJI888ghDhgyhR48ebNy4EdDStLfccgt9+vTh+uuvZ+jQoW6RmWmUCF8I0R44JaWU1sUldGjLtSnqiG1idu7cuWRlZREbG0tqaqqasFX4BE9/tZs9v+W79Zh9Oobz5ESnWWQAxo0bxzPPPEOPHj0YM2YMU6ZM4Y9//CMvv/wy69evJzpai09TU1Np3bo1ZrOZ0aNHs2PHDnr16sWUKVP4+OOPGTx4MPn5+QQFVSyrvGTJEl5++WWWL19OZGRktbFNJhObN29m+fLlPP3006xZs4b58+cTGRnJnj172LVrF4mJVZdwrh9ucfhCiEXASCBaCJGNtramH4CU8g3gJmCWEMIEFAG/k0qIv96kpKQoB69QuJHQ0FB+/vlnNm7cyPr165kyZQrPPfdctf0WL17MggULMJlMnDhxgj179iCEoEOHDgwePBiA8PDw8v3XrVvH1q1bWbVqVaX37bnhhhsAGDRoEBkZGQB89913zJkzB4C+ffvSv39/t5ynWxy+lPLWWra/hla2qVAoFE6pLRL3JHq9npEjRzJy5Ej69evHe++9V2n70aNHeemll9iyZQuRkZFMnz691jr4rl27cuTIEQ4cOEBSUpLDfWxpJL1ej8lkcs/JOEFp6SgUTRjVZNc47N+/n4MHD5b/vn37duLi4ggLCyvPx+fn5xMSEkJERASnTp1ixYoVAPTs2ZMTJ06wZcsWAAoKCsodd1xcHJ999hl33HEHu3fvdtmeyy67jMWLFwOwZ88edu7c6ZbzVNIKCkUTRTXZNR4XLlzg/vvv5/z58xgMBrp168aCBQtYtGgRycnJdOzYkfXr13PJJZfQq1cvOnfuzGWXXQaAv78/H3/8Mffffz9FRUUEBQWxZs2a8mP36tWLtLQ0br75Zr766iuX7Ln33nuZNm0affr0oVevXiQkJBAREdHg82zSa9omJSVJtQCKoqViNBodluDGxcWV53p9gb1799K7d29vm9GkMJvNlJWVERgYyOHDhxkzZgz79+/H39+/0n6Orp0Q4mcppcP8kYrwFYomimqya7kUFhYyatQoysrKkFIyf/78as6+PiiHr1A0UVSTXcslLCzMI8u7qklbhaKJoprsFO5GOXyFoomSkpLCggULiIuLQwhBXFwcCxYsUBO2inqjUjoKRRNGNdkp3ImK8BUKhaKFoBy+wq2oRiFFc0Sv15OYmEhCQgIDBgzgn//8JxaLhZUrV5KYmEhiYiKhoaH07NmTxMRE7rjjDm+bXC9USkfhNlSjkKK5EhQUxPbt2wE4ffo0t912G/n5+Tz99NOMHz8e0JQtX3rpJacSCc0BFeEr3IZajUvhC7Rt25YFCxbw2muv0ZQbU+uDivAVbkM1CikazIpH4aR7dGPKad8PJlRXvqyJLl26YDabOX36NO3atXOvPV5ERfgKt+GsIUg1CikUTQMV4SvqTanJwuvrD3Eyr5gbBnYiNTW1Ug4fVKOQoo7UMRL3FEeOHEGv19O2bVtvm+JWlMNX1Jtnvt7Nwk1ZGHSCvKIy3rhdrcalaP6cOXOGmTNnMnv2bIQQ3jbHraiUjhdpriWM3xw4wwvp+1i4KYt7ruzCwNhIzhWWAlo1TkZGBhaLhYyMDOXsFc2CoqKi8rLMMWPGMG7cuEpr1voKKsL3Es21hPH4+SKmvb0ZgNG92vKX8b24N+1njuZc9LJlCkX9MZvNte6zYcMGzxviYVSE7yWaawnjyTxtSbcXb+rP/+5IQq8TtA7x51xhmZctUygUtaEcvpdoriWMuRdKAOjVPhydTstvtgr259zFUp+rWVYofA3l8L1Ecy1hzL2o5eqjQisWY2gd7I/JIiko8ewCzAqFomEoh+8lmqvWuS3Cbx1S4fBbBfsBcP5i46d1TGYLFot6slAoXMEtDl8I8bYQ4rQQYpeT7UII8aoQ4pAQYocQYqA7xm3ONFet89yLpYQFGAj00wPa5POf/zgLgKTLRxIdHd2oVUfD/rGWae9s9vg4CoUv4K4I/10guYbtE4Du1p8ZwH/cNG6zpjmWMP686wB5p48jhEAIwdSpUzmReQiAglILubm5SCnLq44cOf20tDSio6PLjxEdHV3nm4OUkvsX/ULOhVI2Hsxxy7kpFL6OWxy+lPJb4GwNu0wG3pcam4BWQogO7hhb4X6c9QekpaWx+dc9lF08V2l/S1EBAPqg8ErvO6o6SktL4/e//z25ubnl7+Xm5nLnnXe67PSllPxr9QG++vU3AHq1D6vbCSoUVQgNDa323lNPPUWnTp1ITEykT58+LFq0yAuWuZfGyuF3Ao7Z/Z5tfa8aQogZQoitQoitZ86caRTjFBXY+gMyMzOrRepz584FIZDmypOzlsI8AHRB1R1v1aqjuXPnUlZWPddfWlrqcknqmr2neXXdISKC/BhibE2wv97V01Mo6sSDDz7I9u3b+fLLL7nnnnsc/u02J5rcpK2UcoGUMklKmdSmTRtvm9PiqKk/ICsrC4QOqpRfWkoKkRZztQgfqlcd1VR2WltJqsUimTBvI7MW/kxEkB8bHh5JoL8eNWer8DTdu3cnODiYc+fO1b5zE6axOm2PA53tfo+xvqdoYtTUHxAbG0uxEGCp2pUosRQVoKvi8B1VHcX26k/xwBRKju3m/LfvVd5WS0nqucJS9p7IZ0T3aO4b1Y3IEH8EqPp/H+L5zc+z7+w+tx6zV+tePDLkkQYdY9u2bXTv3r3Zi6k1VoS/FLjDWq0zDMiTUp5opLEVdaCq0xUG//L3U1NTETq9QwdrLsonKCKaqKioGquOJs2cS2BMH8KH3lDpfX9//1pLUm09AAW/ruLyHu0QQpCevoKMzMw6n2dzIa+wjB8O5WAyW7xtSovkX//6FwkJCQwdOrTJd8G7glsifCHEImAkEC2EyAaeBPwApJRvAMuBq4FDQCHwe3eMq3A/9hLHgXEDaPe7VArW/4/UP95MSkoK/z38JUcOVERgUVFRzJs3jy8K4tHrEvlo8dOYLRKL9aZQVsVRGdp1h99sD3cCkOXHqK1KKcfaA5D+5Sfl2ifSYuH0mRzuvfde5s+fz6HTBZwp0G4MEuuNye7+1DcmgvBAv3pencZBSsk/Vx3g5qQYXl17iM+2ZXPToBheunmAt03zOA2NxN3Ngw8+yMMPP8zSpUu56667OHz4MIGBgd42q964xeFLKW+tZbsE7nPHWIqasU2u1lee2Lbv3LlzKRk5HYCrb76dlJQbAWgdFU3cqJF8sLBylP/N+1vZdCSX37+zme8P51JqqjkiFTo9uReKKzVw1UbuBc2R2yaJAZAWhBC88cYbLF+1DnnDCwid80ncrm1CeHpSX5KMkeW9BE2NnAulvLb+EB9tOUa0taP505+zeeya3rQKdv16KdzHpEmTeOutt3jvvfe45557vG1OvfFJtczPD35Or9a96N26d6PrWR8/X8SN83/gohdkBkrLyigq8ofrXyAGsAB/3Sp4ZvfX+PvVJaqNJmzaf8F6Dpf071u+RUqJzsE1HZfQnszcQn47X8w1/TrQtU2I06ObLJJX1hzkTEFJHR2+FuGbL56veFNKEDqklBw7eYrOOj35m5dQeOgn/P39+cuf/8zYsWMBOHzmInO/2MnUt35iaHxrbh2ipa+Gd42ibXjTidpsKbOcCyXkF5XRNiyA0wUlnCssUw7fQxQWFhITE1P++5/+9Kdq+zzxxBPcdttt/OEPf0Cna3L1Li7hcw6/sKyQf/z0D4rNxcSGxZIcn0yyMZnukd0bZfxjZws5mV/MhL7t6RAR1ChjAuzbt5eVK1chZfXIWhceTsqdd9X5mIF+OuZvOFypCkYCOgf30JsGxXDToJjqGxyw6UgucJCcCyX0xPUa+tyLpUiLGUvxhQp7pAWsNyAhtC9h2dlsSo7togR4evbtxL/7LikpKQztEsXwrlEs23mCF1fu56ejWuvIiO7RfHDXUJft8DQmuwtearYwKC6SFbtOcqFYaRV5Coul9jmSQYMGsX///kawxnP4nMMP9gtm9U2rWZO1hvSMdN7c+SYLdiyga0RXxsePJ9mYTHxEvMfGt+Wupw03MqxLlMfGsZGWlsacOXMqNTJV5bwQPLHkn/U6/v82Hik/J9DOz1GEXxfahAUAsG7faZ5aupu8osq1za2C/fhk5nAigvwwWyQPLd7O0dxCjp0tJAATVL2p2eyx/Wu33WQyMXfu3PJUlTE6hPtGdePGgTEUlZlZvPUY/9lwmF6Pr9AOgUAIaBceyIo5I7yS9jGZK653j3ahjOndjhW7TlJQ3LxrwBXex+ccPkCrwFbc1OMmbupxEzlFOazOXE360XTmb5/P/O3z6dW6F+ONmvOPCXMtKnUVKaETZwg5fwBORbj12FX5etkyXnn6KdrrimnfxvkjZseOHeHUnnqN0YNsWl+UcEpzop3LMmhTFginnKdsaqN9qYleumNs/P4Y0SH+XNOldfm2U3nF/Jx1ntwjQUREh5JbUMyeXzcTHx1CYodALhnSip1Ft/Lpp59gNltoHZjDb3pb07Z2DapWETkqNW0foaVw7hvVDT+doMRkKZ/b3XeygG8PnCH3YimdWjXeU5oNkzXanPe7RCYndmLXcW3OQqmRKhqKTzp8e6KDorm1163c2utWTl48yaqMVazMWMm8bfOYt20e/aL7Md44nvHG8bQPad/g8QLP7ef7wDlaIaqHuRa49vcGoHpbeGXy4T+X1muMZX7ATusPVhGkfBqkhhQCpNtS0SbgQJUdAoBPtJdtgVUBQIH1Jxuui4LH77EpjX7MBnmQUeAwwoea6/tDAwz8aVzPSu999nM23x4447VSSLM1paO35s7CArWvqS+ndKSUPrd+rKepT/+Jzzt8e9qHtOeOhDu4I+EOsguyWZmxkpUZK3lp60u8tPUlBrYdyHjjeMYZxxEdFF2vMQzFWmrlt0sepGO3RHeaX42bb7mlxv90nU7H7NmzuWLEiHqPMeej7VzZsw03XKIpYfx92V7ahQUw44ou9T5mTew9mc+raw/x0LgedGsTyp4T+fx73SEeHteDrm2q39iylz5D27ILjB49mm+2WsVa7a6JwWCoVN/vShWTn8E6F2D2TkOXbVxDucPXJtx9NaUTGBhIbm5ueQ+HonaklOTm5ta5RLRFOXx7YsJiuKvfXdzV7y4y8jJIz0hnZcZK/rH5Hzy/5XkGtxvM+PjxjIkdQ2RgpOsHturMXOh4GSSM9ZD1GlsudiTTSdNRcHAwCxYs4IoGKnCuEYG0aRXLDQl9APh2ZSTdwkIhYVCDjuuM/KBcVlg2MTVmKN26RXOg9DgrLFH8OeFKcODwz61YQGDZcdasWcO0+x7mGyh3+KGhobzxxhvlDt3VdYT9rI62ag9BY2GL8A3WSpCQAG0e4YKPpnRiYmLIzs5GaWfVjcDAwEqVRa7QYh2+PcYIIzMHzGTmgJkcOneI9Ix00jPSeebHZ0jdlMqwjsNINiZzVexVhPtX14uphFV2oKZa8LpQU0Rq3yRlj6uNTK6gE6JSlY47Jm1rwt8aXZdanW35ClshAQ73Nwl//KUW+T751FNc9c9vSEtbyOTE6tp8NekEVXL4es0Gk5cifFsOX6/XrnOAQY+/QeezOXw/Pz/i4z1XSKGooHkWk3qQbpHdmH3JbL667isWX7uYaQnTyMjL4PHvH2fkxyO5f939LDuyjItlFx1+Xkqbw2/YvdSmGT916lSHypXgeBGVhQsXkpOT4zZtfSGoVKUjZUWq3BPYnK2tcevsxRL0OlGex66KSeePv7Q2ZFnNdHZDcnUdYYPV0dpuOsfPF5F9rrDa5zyFyVI5pQMQHmjw6Ry+onFQEb4ThBD0jupN76jePDDwAXbm7CxP+2w4toEAfQBXxFxBsjGZETEjCDJYqzks1i+lvv6XtmrqwZ6qEWlKSopHF07R6USleQKtDt9zHj/AUNXhlxIZ7F++YHpVTDp//NEifJudzuyLjY11mAKrOqlbEeFrNjy2ZCdlZsnCuxunVt9krpzSAW1yee+JfD7fls2onm2JrEPDmkJhQzl8FxBC0L9Nf/q36c/DSQ/zy+lfSD+azqrMVazOXE2QIYiRnUeSbEwmzFysfagBKR1HqQd7apMRdicCHKR0PDeefxWHf+5iGa1DnHcJm3X++FM1wne8r6MUmCNFT5vDt02e5hWVUVhaVSHUc5Tn8PUVJ9K5dTAbD+awLes8s0d14+HxPZ19XKFwis87/IWbMtl+TGvFt1gkORdLefmWAUSHOs4J14ZO6BjUbhCD2g3ikSGPsPXUVtKPprMmaw0rjq4gWPgzJro1iQV7iLMMxE9Xd6Gu2hx6bTLC7kTL4bu38aomqubwC8vMBPs7/zM1iQD8MJXbBjit9LDXCaqpSsfmaMusufRSs4WCRkynlOfw7e5c/7sjiTMFJUx+/fvyeQ3QbowSSYChaeoCKZoWPu/wX1lzgMJSM5HB/hw/XwTA1oxzJPdteM29QWdgWIdhDOswjLnD5vLTiZ9I2ziPdSHFLD3wMq9mvs3o2NEkxyczuN1g9E6i/nvvvZcFCxZgNpvR6/WEhIRw4cIFh/s6ikg9iRDCXmwSi8W5Q3UHFdG15vSKS80E1dDtatYFEEgpyAqFzpqeQFxJgflXmbQtNVkatULGNq6fXUon0E9P59bBRAT5VSrPnPrWT+QUlLDu4ZGNZp+i+eLzDt9skdw0KIZnJvcl90IJg55dw4m8IreP46fz4/JOlxMasoM++1bx+YRX2Vaym+VHl/PZwc+ICoxibNxYkuOTuaTtJeisui/33nsv//lPRReT2WzmwoULGAwGTKbKTsad1TeuohOVGzxkI6d0CstMtAtzXmtstj1BmUvLy+8b+gRSHuFbbzo2h99YzUGmKo1X9oQGGCgoNrFwUyY5F0rYbNUDenPjEUICDOh1gvEJ7YkIatoS0Arv0CIcvs0BtA7xJ8Cg40ResecGtJjwB4ZFD+N3XadRZCpiY/ZG0jPSWXJoCR/t/4i2wW3xO+zHd299x8VDjqt9zGYzcXFx9ZY5dhc6IbDXlbJ4uErHFl2XWB1+UamZwBrWrDXrrKk5U3FFhN/A2jPbZKm9wzdbJCUmS6No6zjK4dsICzTwS9Y5vjlQuWb92WV7y19fKDZx5+WqzFFRHZ93+BZZEfEJIegQEchv590f4ZdjK8vUa44hyBDEOOM4xhnHcbHsIhuObeCFpS9wss1J4h+Lp/RMKXmb88j7KY/irIobkZSSjIwMz9npIroqZZkez+FXKcssLrPUmNIx6azVKqYSLFJ73dAo3L/KpK1tPqFX30SyDu72+A3YlsM3OIjwwwIN5FvnE9LuHsqlXaK4WGriYomZ4jIzI1/aUG6vQlGVFuDwJXq7iK9DRBB7TuTzydZjHhnPcCqPREA4mKz9YvEXzJ07l8zMTHTBOsIHhhMxJILo5GjaXNOGkpMl5P2kOX/TqaZRcy2qNF5JPJvD1+kEBp0oj66LymrO4VsqRfjaNW+odbbI2laWeaGwBBAcP3O2Uj8E4BGn76gs00aY3Wpd7cID0ekEYYF+hAX6UVymBRuWemisKFoGPu/w7VM6AD3bh/HjkVz+/OkOj4w3XZ/L9X6gqzJBW7W23lJo4fx35zn/3Xn0oXrCB4UTMTSCNhPb0HZyW4ILg3nj1zdINiZjjDB6xFZX0OkaN4cPWh6/PIdfaiK4hpSOSV8R4UupSS809AmkfOLYeqcrLi0Dgz86/wrlTEcduu6iXDzNSUrHhk1m2ka5dpzy9won+LzDt0hZqWnn8Wv7cJeH8psfbcmi+FtrvXaVxquaauvNF8yc++Yc5745h3+kPxPmTCBiSASvb3+d17e/Tu/WvTU55/hkOoVWlwzwJIKqZZmebbwCq8M3W7BYJMVlNefNK0f42suGO3zrpK3JgpQSqTcgABEQXGk/T/VD2MpB/RymdLQI36AThFfpPhbWZ5v6qCgqWgYtwOGD3s4B6HWCzq2Da/hE/YkM9uc02pdVVHH4rjiHWbNmMX/+/PLfT148Wa7o+cq2V3hl2yv0j+5fLufcLtIlhfQAACAASURBVKSde0/AATpB5bLMxojw9VqEb5u4Dapx0tYuh2+pvSzTFQy2skyLBZNFlq+kpfOv/HfjqX6IqvLI9ticfJCfvlpqzba7Rfl7hRN8XktHS+k0zlg6ITDYHH6VlE5NzkGv11dz9qDJOU9LmMaH13zI8huWM2fgHEotpby49UXGfjqWaSumsWjfInKKctx/MlaqiadZPF+a6Gd1+EXWnHSNOXx99Qi/ofaVR/hmWWkxdp1dhO/Jfoiacvjh1nLLcAdll7YnGxXgK5zh0w6/XFulkTy+ToAezUnp9JW/kKmpqQQHV44Qg4ODWbhwISaTqZqzr0rnsM7c3e9uPpn4CUuvW8qsxFnkleTxfz/9H6M/Gc3dq+7m0wOfcr74fI3HqSuNLZ4Gmp5OqdlCYak2cV1b4xVgzeG7J8L3syvLtHf40e1jykXqFixYUEl22Wg0otPpMBqN5eJ29aW8SsdBDn9s73ZMH27kkQm9qm0T5RG+8vgKx/h0Ssdc/ojfOA5frxPohfZlrTpp62pbvyvER8Qza8AsZg2YxcFzBzU556PpPP3j09XknMP8XV8k3BE6UVk8zdNlmVAxaWurOqkxpWM3aWux7tbQG7xOJ9DrBCazrFTi+Oe5T3Dfmvcr7euqxn5dqKnxKjLEn6cmJTj8nO3JRrl7hTPc4vCFEMnAPEAPvCmlfK7K9unAi8Bx61uvSSnfdMfYNWGWzr84nmDLli10x4JJ6hgyZAipTzxa6UvvCWXL7pHd6R7ZndmJs9l7dq+m6Hl0JY99/xh+P2rdv8nGZEZ2HkmwX93nLhw1XjVKlY7ZQlGpNYdf46St1eHv/pyO+m3M1B+jw85fIbtha9HO0u9n0LFIAkQkM/WHAUg4EgV+bSrtd2zR88y+xATYq1eaOLboIYg7Xa+xB2TkMFOfg/+mA3WuMf2D4SCBJbfXa1yF79Nghy+E0AOvA2OBbGCLEGKplLLqqtkfSylnN3S8uuCuVntXSEtLI23hxzw7LhwzOo4dy/JorXZVhBD0iepDn6g+PDjwQXbk7NAUPTNWsf7YegL1gYyIGcGE+AmM6DSCQINrS6NVS+nQCBF+1Rx+DRF+YUAb8mUw4Ts/oQvwqB/wc8NteFiP9tecbT0mwDHrjx2PJgE4upZFsOapeo19GXCZH7C27p+da4D1Z6KB+q1hrPBt3BHhDwEOSSmPAAghPgImA1UdfqNjdlPVhivMmTOH0k4D0ROKGT3IMgqLPFerXRNCCAa0GcCANgP48+A/s+3UNtIz0lmduZrVmasJNgQzsvNIJsRPYHjH4fjrnWurV2280qQVPJ/S+eFwLpuO/AhQYx1+qV8El5T8l8N/H8s3+88wY+HPLL7nUgbERDTIhmH/t4YJ/TpyS1IM183/AYDJAzrywk39K+3Xs1cvhxVYsbGx7N+3r15j/2v1ft78LoPdT4+v2wdNxfC8EZ2ltPZ9FS0Sdzj8TlSOe7IBRytF3CiEuAI4ADwopXTY6iqEmAHMgIaXvVkaIaWTlpbGnDlzyM3NJaSjRI8FEzqQWjqiMbXrHaETOpLaJ5HUPolHhzzKlpNbWJmxkjVZa1h+dDlhfmFcFXsVE+InMKTDkGpyztqla9zGq+nDjbQPD6R9RCARQX706+TceesEmNEjDYGY9AGU4I/wCwS/hqV0zIYgivGjRGjHBDhbqq903Oxzhcz823M8+eSTFBdXyGIEBgYyZMJ1GAeN4rcTJ+jYOozUJ/7q8o2/WARg1gfU/Rysq6wJqaQVFI5prEnbr4BFUsoSIcQ9wHvAVY52lFIuABYAJCUlNWj+yZZ79lREWm1lKosFPWYs6MonOhtTu742DDoDl3a8lEs7XsrcoXP58cSPrMxYydqstXx5+EtaBbRiTNwYko3JJLVLQq/TO1jT1vMpsnEJ7RmX4Jp8tc0Wi6x9icO64K/XVSvLvFBSIUu8MzuPia99B4TTeuq/qn1+I8DVV9IRsBRfYOodKcyZM8cltVOTWTosyawVoT0JCdl4i7UomhfucPjHgc52v8dQMTkLgJQy1+7XN4EX3DBurZRP2nrIP1XrnpUWDJgxoccWFTemdn1d8NP7cUXMFVwRcwUl5hK+O/4dK4+uZNmRZXx64FOig6IZGzeWUkNHzLKiBNAipcfLMuuCzRRpp4fvDvsMelGpLDMs0FBJE//g6QIAnp6UQFRo5ZTY7PtmcyZHU7MM6NSH8KRJGCLakZv7m0vzOmaLrN9TqfUmoRy+whnucPhbgO5CiHg0R/874Db7HYQQHaSUJ6y/TgL20gh4MqWTlpZWbX1UKSU6LJjRgZRERUV5RdK4rgToAxgdO5rRsaMpMhXxbfa3pB9N5/ODn1MSXMIZGcmLWyYy3ji+0TThXcVWgmmRta9pWxf89DprWabmPKNC/CstIm5TXJ0yuHM16YdJ339Rbou5IEdz+JEdMZ37zSUNHpPFUt78VVfKpF45fIVTGuzwpZQmIcRsYCVaWebbUsrdQohngK1SyqXAH4UQkwATcBaY3tBxXcHWau9uB2VL5VRDWjBYHX5QUCDz5s1z67iNQZAhqFy64WLZRSa//V+K/Lfx4b4PeX/P+4R0bc3P+Vey7+xt9Izs6XXnb99s5M6UjkEnyC8u41R+CaCtpZB1tuJp7vj5YqJC/B3q/Ngvll529jcA/CI7Ysvy1zavYzLXM8IHLOhUDl/hFLd02kopl0spe0gpu0opU63vPWF19kgp/yqlTJBSDpBSjpJS1q98oY54qg7fqRCatKAXZsxSz+uvvdYsovuaCPELoZVlGEbzbDbcsoGnLn0GS2k0Oy98yc1f3cykLybx+vbXOXz+sNdstJcTsD3RjR83tsFdryEBBjYezOGvn+8EoEOrIHIulNJj7gp6zF3Bos1ZBAc4rh6y76q2FOVjKS2i9ZgZ6MOigdrndUyWeubwATM6hEVF+ArH+HSnrS3i07s5CnUWoUlrhG9Cx2233urWMb2FENrkd0RABJO6Tuaht/25c3Q74mKPsDJjJf/99b+88esbdGvVjWRjMsnxycSFxzWafTq7CH/jxu+AEI4fP95g3fr/u74f24+dAzRRvF7twzFGBWORmormm98dpaTMcSRtG8tWvXXh15WED74O/zZGhLmw1nkds0U6lFVwBbPQqZSOwik+raVTkdJx73GdRWhto6PRY8GMvtEE2zyNtoi5dh1tEXSwoRW39LyFt8a/xdqb1/LokEcJ8w/jte2vce2Sa7nlq1t4e9fb/HbhN4/bV1GlI/l48WLtTbuUhi1nXld6tg9jyuBYpgyOZVxCe2Kjgvnz+F48ktyLx67tw6u3XsKCO5Kcfj4lJYWcnBwWLlxI+AmtE6xtXLdyDZ6a9HfKzJaGpXRQKR2FY3zb4XsopeNMCG3q1BT0mDGjazT9Hk+jLXGovXbUudwmuA0pvVN4f8L7rL5pNQ8nPYxe6PnXz/9i/GfjSVmewgd7PuDUxVMesc9ePyYn92xlQ614ohdi0oCOJHZuVet+KSkp7Pt1CwAPP/ZMubOfMWMGmZmZlZ5E0tLS+GTrMXZk5zlc3tAVzKgIX+Ecn3b4nhJPS0lJYcGCBcTFxVVST7xq1EjrpK2+SZUuNgR78bTyRcKdnJtNznnRtYvK5ZxLTCW8sOUFxn46lunp0/lo30fkFuU6PkA9KC/LtEB0dHTFL3Z4uxci0E9PeKCBMwXaBLCjOSDtSeQxHv9yF6cLihlsbF2vsSyoKh2Fc3w8h+85eWRHQmjr959GjxkTOq9Xr7gL+8arCr352j9nk3O+u9/dHMk7wsqjK0nPSCf1p1T+sfkfDGk/hGRjMmPixhARUH8ZBPsc/o033cxn2ZVXfPKkbn1daBMWwJkLmsN39sRxPK+YTmUW/nFDP24dUr+blIrwFTXh4w5f+9fdk7bO0AuBxILFhx6c7MXTLPWsc+8S0YVZibOYOWAmB88fJP1oOukZ6Tz141M8u+lZLu14KcnxyYzqPKrOcs4VdfiSoUOH8Vn2DmI6dSSr4EyDJKjdTZuwAPadKGDR5ixiR/6O3LNnq+0T1Wc4AN3bhtZ7HFWWqagJn3b49RFPS0tLq7dmveYIbZ22voF9hC8bKFUhhKBHZA96RPbg/kvuZ8/ZPeWR/9zv5uKv89fknOOTuTLmSpfknEX5pG3FDenHH3+gQ0TDtHTcTc92YWw6clYr8xySQlQN+w7rYySmbet63aws6NCpCF/hBJ92+HVN6TR0MQudAJ2wdtr6CEJUpEhs1TruyJAJIUiISiAhKoEHBj3AjjM7ytfvXXdsHYH6QK6IuYIJ8RO4vNPlTuWcbbZINzdeuZsnJyYwa2S38t8/X/I5Lzz/PMePH6dTp05cddVVLPlmG0TFYy4qIDOzoF4lpWaVw1fUgG87fGtE6qoDcD6Z5prEsU4n0GHBJH0rwpdVcvjudqg6oSOxbSKJbRN5OOlhtp3exsqMlazOXM2qzFUEG4IZFTuKCUZNztnPbvlInV2Vjju1dNyNTidoH1Fx07p3+m3cO71CgcRoNHI+MxP4tvy9uvzt2VApHUVN+LTDr+i0dW1/Z5Nprpb1aYuYmynGub58c0PnMIfvufH0Oj2D2w9mcPvBPDrkUTaf3KzJOWeuYdmRZYT5hzE6djQTjJqcs/2krW2qtilG+LXh7G8sMzMTnU7ncnrRohqvFDXg0w7flUlG+5y9TqfDbK7+ZXG1rE+vw9p45UspHfsqHc9oEznDoDMwvONwhncczmNDH+PHEz+SflRbyOWLQ18QGRBJfPAw9MExlFmudKt4WmNjr79Tlbp0DZvRqxy+wim+7fBrqcOvmrN35OzrUtYnrBG+b03a2uXw61CW6W6qyTlnf0d6RjprM9cRHFfC1JWfYwy8FH1QDDTDTtPU1NTKays4wJUUj+q0VdSE74SiDrBV6TjrtHUmgqbX6ys1VNWlSkfnY2WZWpVOw8oy3U2APoDRcaN58coX+XPvjyjKvo1ekf3Ykb+SYOMb3Lx8Ii9teYldObsq1eQ3Zao28zmjtvSiRagIX+Ec347wJfhhwr/kLFysvr0w5xjRwdW/XEJITp86XfHGxRyXxvMvzscfk7bEoY/gqPGqKekEBRoCMRX052+D/siy3Uf51/df0C3pOGn70nhvz3vEhMaQHJ9MsjGZHpE9mnRDnH0zn9FodJjiqS29aEGHXk3aKpzg4w5f8pn/k/T/5KjD7acfrqHB5cWudR6vD4AOtpvr/tkmi92krWzkHL4r2Iun+YtgTPmX8PKVj2LiIuuy1pGekc47u97hzZ1vYgw3MiF+AsnGZLq06uJly2vGUYrHlfSiBR1+qAhf4Rifd/gdRS4F7QYTNmgKW7Zs4cNFH1JaWur0M/7+/tx2620MHjy4zuOdyCti/obDbLAM4MaGGN6EsC/LdCSe5m2Eg7JMnRBE+Edwfffrub779ZwtPsuazDWkZ6Tzxq9v8J9f/0P3yO5MMGrOv3N45xpG8A62SL+uTYCqDl9REz7t8M0WbcnBosjeLD0YzLQ5bzucmNXr9VgslvIv1eB6tuLnnyzgg7Xf1r5jM8J+0rYxyjLrSuU1ba3vVbGvdWBrbul5C7f0vIXThadZnbma9KPpvPrLq7z6y6v0ierDBOMExhvH0yG0Q6PaXxOO9JpqwyJ06FRKR+EE30k2O8AiJTok2375hdtvv92hswewWCxYLBYyMjIapLvSlByhu3Ccw286J1qR0nGt8aptcFtSeqfwwdUfsOrGVTw06CEEgn/+/E/GfTaOqcunsnDPQk4XnnZ+EBepSfPeU6gqHUVN+HSEb7GADsnBQ4dqrNZwl3yuJ1Q5vY0j8bQm5O8rN17VsYqoQ2gHpvedzvS+08nKz2Jlhqbr8/yW53lhywsMajeoXNEzKqgm9ZvqNFSmo76oKh1FTfh0hG+WEoEsL890hDvlc5tS5OsuKufwm96kbbl4mqVhTyCx4bH8of8f+GzSZ3w5+UtmDphJbnEuz/70LKM/Gc2MVTP4/ODn5JXkuXS8mmQ6PIkST1PUhE87fItFc/jO/L1er69TnX1t+GCAX0VaoeK9poJ9hO+uOYYurbpwb+K9fDn5Sz6d+Cl39r2TYwXHePKHJxm5eCT3rb2Prw5/xYXSC06P0VCZjvqiUjqKmvDtlI7UUjqO5rCEELz33ntufbz21Qi/qTVe2WNvS8WkrXvsE0LQs3VPerbuqck55+4hPUPT8v82+1v8df6MiBlBsjGZK2KuqCTn7EwqwdOrb6mUjqIm3OLwhRDJwDxAD7wppXyuyvYA4H1gEJALTJFSZrhj7JowS61KR9ipK1rtYebMmW7PpfpuDl97LZtihG99RrXl8D1lmxCChOgEEqITeHDQg+w4s4P0jHRWZqxkbdZaggxBmpyzcQKXx1xe7zr6hmIRenQWFeErHNPglI4QQg+8DkxA6z26VQjRp8pudwHnpJTdgH8Bzzd0XFeQ1hz+iBEjKq0/+8EHHzB//ny3j9dYK2s1JqKSPHITzOFTuUqnMZ4+bHLOjw55lDU3reHt8W8zsctENp/YzAMbHuDKj69kV+wu/vLGX4jrElcvmY76YkGHTqV0FE5wR4Q/BDgkpTwCIIT4CJgM7LHbZzLwlPX1p8BrQgghPSx0otXhS7p370lGxmJPDgU0rcjXXTgST2tKKR1RKYff+LbZyzn/dehf2XxiM+kZ6azJWkOBpYBOz3bijtg7SI5PZkj7IR63RwodQnXaKpzgDoffCThm93s2MNTZPlJKkxAiD4gCXBOpqSe2xiuha5y56aYU+boLxzl8b1pUmfIFUKyTtt78LzDoDAzvNJzhnYbz+LDH+eG3H0jPSGdV5iqWHFpC68DWjIkdQ3J8MgPbDkSvc7+qqgU9epXDVzihyU3aCiFmADOg4RNcUoJeyEZz+M5UOZszjhqvmtJ9zb7xSnohwneGn96PKztfyZWdr6TYVMx3xzU556WHl7L4wGLaBLVhnHEcycZkBrQZ4LZgwSJUlY7COe5w+McBezGSGOt7jvbJFkIYgAi0ydtqSCkXAAsAkpKSGpTyMVsnr4RoHIfvg/7eSeNV0znRijVttTLcpvh/EGgIZEzcGMbEjaGwrJBvs79lxdEVfLL/E9L2ptEhpAPjjeNJNibTJ6pPg66vRK+kFRROcYfD3wJ0F0LEozn23wG3VdlnKTAN+BG4CVjn6fw9gMX6hy888OjsCF+s0nHUeNVUomiwa7zyUg6/rgT7BWtyzfHJXCi9wPpj61lxdAUL9yzk3d3v0jmsM8nGZMYbx9dLztkidOhVDl/hhAY7fGtOfjawEq0s820p5W4hxDPAVinlUuAt4AMhxCHgLNpNweNIq3ZO40X4TdvZ1AdB0268qjxp690cfl0J9Q9lYteJTOw6kbySPNZmrWXF0RW8test/rfzf8RHxGuibvHj6RLhmpyzVoevInyFY9ziCaWUy6WUPaSUXaWUqdb3nrA6e6SUxVLKm6WU3aSUQ2wVPZ7GYkvpuOihGip21ZQcobvQ6Zq2PHLFpK32BNJcn7IiAiK4ofsN/G/c/1h38zoeG/oYUYFR/OfX/zD5i8ncuPRG3tz5JscKjtV4HIle5fAVTmlyk7buRFodvk7UntJxh9hVU3KE7qI5iac1h5SOK0QFRTGl1xSm9JrC6cLTrMpYRXpGOvO2zWPetnkkRCUwIV6Tc24f0r7SZy1Ch06ldBRO8G0tHVt5mgtRnzvErnzB2VTFPoffFKUVhF2VjsWDnbbeom1wW6b2mcrCqxey8saV/GnQn5BIXtr6EmM/Hcvty28nbW8aZwrPAFodvlriUOEMn47wLWbXJ23dIXblm2WZ9kscau81pbOsqNLRIvymVEHkbjqGduT3fX/P7/v+nqz8rHJdn+c2P8fzm58nqX0S7QNzubRQ0sbbxiqaJD4d4UtrpKNzoQ7fWc1/XXoBfNDfO268akInWi2H33RM8yix4bHM6D+Dzyd9zheTv+CeAfdwpvAMX4UdYWxsB+5ZfQ9LDi5xKufsjcVZFN6nRTh8V6p0UlNTCQ4OrvReXcWufDG6FNbGK6PRyPjxyQCsXrXKy1ZVoKtUltk4WjpNja6tunJf4n0svW4ps84PYnpePpn5mTzxwxOMXDyS2WtnV5Jzts1XZWZmIqUsn69STt/38W2HX4eUTkpKCgsWLKgkstYYYldNnV07dwLaJLa0OtPnn3uuyTiHirJMfGbStr4IIWhrDueBc3msmPQli65ZREqvFPad3cffvvsbV358JQ+uf5AnP3ySIlNRpc8WFhYybdq0JvP/2hxojk9Jvp3Dl3Wrw6/PotG+zrq1a6DftWiZe82ZFhcXMXfu3CZxrZpzHb4nsAjtKy2w0De6L32j+/KnpD/x65lfST+qyTkH3RJE78m9yf8ln7zNeVzYeQFZJjGbzY2yDKMv4K0lLBuKT0b4F0tMXCwxUWrSHL4rOXyFY86fP6e9EKIiZSUtHl+5yVXsxdOakpaOt5C24MZSUZqpEzouaXsJfx36V9bevJbCdwo5/8N5QhNCiftjHL3m9aLT3Z0I7R9KUUkRU6dObTYRq7fw1hKWDcUnI/ykZ9dQVGYmmjz+GEijFY5rX5BWgJbzTk1NbdJ3e1doFRGhvRC6iusopcdXbnIVe/E0XyzLrCvS1nOy+HbQ+VXbrge+vbo9O3buoPTHcxxuHcSO9qHsHhpO5OWRBJWZSTh1EWNGPvfd2/QjVm/hrSUsG4pPOvyHx/fEbLEQVHIGvkdzVh7G9ojX5n5Nd7+5POLVxrixY1l1Cs3ZW69jQIA/qalPe9cwK5XE01SEz6GgvuzN70nvwrNO9+kUYSCwj5GjR4/SqaSEKzJLKMvKZXerIH6KDmV7h1C2xoQTk2jikdWPkCWzeCTlEXSNJFHSHPDWEpYNxScd/l2Xx2sv8oMazeHX9IjXnB3+wIGXsGrFPuLi4jhjvY5PPP44KSm3etkyDVGlSqeF+3syAnrzp/B/suKeETXuFwWkV8lDw0Ugh4kJftx/R2seOuVPxPAIPjR/yNL3l3Jd7+tIjk+mf3R/hBCkpaUxd+5csrKyiI2N9YknWlfx1hKWDcUnHX45to7DRnD4zfURrzZsEfTuPXv55sAZ7k3bxrXXXuNdo+ywn7SVLbQs0x77Fcpqw+ac586dWylaLS22MLawiOCPc9iSIwlLDCNiaARpljQW7l1Ix5COdC7szGf/9xnnM88DvvNE6yr216453fB8+xmtER2+Oxq3miIVa8bKJimtYLPlne8z+PFwbpOyzRsIUdER7QopKSlkZGSwcOHC8j6UYpO2LdAAlhILeT/lkfVqFrvv20349+F0adWFTaZNxPw1hu7PdaftDW0JiAloFpOW7sR27SwWCxkZGU3e2YPPO3ybFoDnT9MdjVtNEZv//GBTJum7TgJNq6O4Q0QgQ+Jbs+9kPucKy1p8SkcnBJK6LzVh60PR6/WVHL49liILP/zvBy4/dTn75+zn+NvHKcsto821bej+bHe6pXajKLGIo3lH3XAmCk/QMlI6jaD+Uv6Ip/UpERcX1ywe8WrDFjG/kL4fgLBAA1EhAd40qRKBfnoW33MpD3z0C19s/80nu53rgqZuWr/P2v5WX3tMS80EGhxfyzlz5tApqhOZ32Zy7ttz6MP1RCRFEDEkgjaT2zDpi0n0jOypLfRiTCYmLKZ+Bincjo9H+I2X0oHKucvm8ohXG/bR/NLZl7H9iXFEhvh7zyAnJHbWymEvlJR52RLvIoRwOYfviJSUFP72xDNA9QjfRm5uLldffXX5E60538zZdWc59eopZvvN5i+D/0KAIYB52+Yx4fMJ3Pr1rby3+z1OXjxZb7sU7sHHHX7jpXR8FXuhtLjWIU1WEXR83/aM69OOGSNcWxnKVxHULYfviInX3Qg4j/ABli9f7lCKZObUmdze53bSrk4j/cZ0Hhz0IGZpLpdzvmPFHaTtTSOnKKdhRirqRctI6bTwx/yGYJ8iCQ9qun8uHSKCWHBHkrfN8Dr26qb1xhAIQGLfXrz3606Hu2RlZdUqRdIptBN39r2TO/veSWZ+JulHK+ScX9jyAkntkhhvHM/YuLFEBkY2zGaFS/h26NvIKR1fxBbQRwT5tfj8eHNACOoxZVsFgzZH88DsWURFRTncpa7VZ3Hhcdwz4B6WTF7CkklL+EO/P3C68DR/3/R3Ri0exczVM1lycAn5pfkNtV5RA77tCZXDbzDhgVp7fv+YCC9bonAFd0b4mIqZN2+e26vPukV2Y/Yls1l63VI+mfgJ0xKmkZGfock5fzyS+9fez9dHvuZi2cWGnIXCAU33Gd0deMHhPzWxD7FRwbXv2Ey4pl8H4qKCMUaHeNsUhQvUtQ7fIXprFZapxKMNRkIIerXuRa/WvXhg4APsytlVvorXhuwNBOgDuCLmCsYbx3NFzBUEGYIaPGZLx7cdvu3hthFTEdMvi2+0sRoDnU7QP6aVt81QuIhANNzh63Sg9wdTMVC7bLg7JBaEEPRr049+bfrxUNJDbD+9nfSMdFZlrGJ15mqCDEGM7DySZGMyl3e6HH9906sUaw74tsNXKR1FC6Mu0go1YggEU0mtu3lCF14ndAxsN5CB7QbyyOBH2HpqK+kZ6azOXM2KoysI9QvlqtirSDYmM6zjMPwcqIIqHNMghy+EaA18DBiBDOAWKeU5B/uZAdt0f5aUclJDxnUZ5fAVLYyGNF5VwhBQHuHXhKdFA/U6PUM7DGVoh6H8bejf+OnET6QfTWdd1jqWHl5KREAEY2LHkByfTFK7JAw6345hG0pDPeGjwFopZXdgrfV3RxRJKROtP43j7EE5/BZEc1xuzhPUV1qhGi5G+I0pGuin8+PyTpfz7OXPsmHKBl4d9SqXdbyM5UeX84dVf2D0J6N5dtOzbD25FUt5l73CnobeDicDI62v3wM2AI808JjuQzVetQia63JzQ6fIRgAAFNpJREFUnsBtEb5dDr8mvKUL76/3Z1TsKEbFjqLIVMTG7I2kZ6TzxaEv+Hj/x7QNbsu4uHFMiJ9Av+h+qqTYSkM9YTsp5Qnr65NAOyf7BQohtgohNgkhrqvpgEKIGdZ9t545c6Zh1qnGqxZBc11uzhNo0gpuOJCLEX5TEA0MMgQxzjiOl0e+zDdTvuG5Ec/RJ6oPH+//mJTlKUz4fAIv//wye3L3uGd+oxlTq8MXQqwRQuxy8DPZfj+pXUlnVzNOSpkE3Aa8IoTo6mw8KeUCKWWSlDKpTZs2dTkXBwdTKZ2WgK+uRVAfNGkFd6R0XMvh21Q2q0os2J6sGjvVFuIXwjVdruHfV/2bDVM28Oxlz2KMMPLB7g+Y8vUUJn4xkX//8m8OnjvoUTuaKqIhfxxCiP3ASCnlCSFEB2CDlLJnLZ95F/haSvlpbcdPSkqSW7durbd9ZP4A70yA27+ArqPqfxxFk8ZoNDpMK8TFxZGRkdH4BnmRx7/YxbKdJ9j2+NiGHeidayDzuwYFSxYpkRYHuXQh0Ousx9UHwO2fQ9zweo/jCueLz7Mmaw3pR9PZcmoLFmmha0TXckVPY4TRo+M3JkKIn60BdjUamsNfCkwDnrP++6WDwSOBQilliRAiGrgMeKGB47qGyuG3CJrrcnOeQMvhuyHCH/04HFrToEO89u9/k5eX53DboEGDuHr0FbBpPueydnEhLLFBY9WOP0Ojr2Zo9NWcLc7l+xPr+Pa31by+/XVe3/46XcJ7cGWnsYzoOIb2wR1rPJJOJ+gYEVhtXqCo1EzOhdrTYK6g1wk6tnJ/o1lDHf5zwGIhxF1AJnALgBAiCZgppbwb6A38VwhhQUshPSel3NPAcV1DpXRaBM11uTlPoBMCiztmbWOHaT8N4IExTzhNL+m//QnT01/BpvnMS9/Ju8vWN2isuhMN3IowXI0hfAcHi3ZwJP913tn7OuaizpTl98eU3x9pciwp8sS1fbjz8spNltfP/559JwvcY11oAFsfG+OWY9nTIIcvpcwFRjt4fytwt/X1D0C/hoxTb5TDbzHU1g3aknDkYvOKynhr4xGiwwIY26cdHSI8L1PgrIIHwGw2l2v2hOjKePH6/h63xznagu/nSk+yJ28ju/O/5WTQMmi3jM7BCSSEX0Gf8MsI9WsNwNNf7eFoTmWdHyklR3IuMqpnG67u16HBFgX66Rt8DEf4dpeCcviKFobOSZXOxoNneHXdIQCe+HI3fx7fk/tGdfOoLampqUydOtXhNr1eX+Hw9WXcnNTZo7a4RmdgMPAnMvIySM9IZ2XGStJP/odVp/7L4HaDGR8/nujwYM4Wllb6ZGGpmVKThaFdoprIuTjGtz2hcviKFoYzaYWiUjMA70wfDOC21ENNpKSkMHp0tQQAgNYnodNRJvwJFiaP21JXjBFGZg6YyZLJS/h80ufc3e9uThae5JkfnyE36m9sK3mRLw59US7nfPaidgNo3QRXg7PHtz2hmrRVtDCcNV6VmLTgJ6FTOP1jIsgvapylINesWcOsWbO0iB4tsp81axbz588HoEz4EyS8tyylK2Wj3SO7c/8l9/PVdV+x+NrFdBDjKZK/8fj3j2tyzuvu56vDy0CU0Dq4aTv8FpLSUY1XipaBM2kFm8MPMOgJD/Qjv7jxnOz8+fPLHXxVTMKfIEodbvM0de3QFkLQO6o3A0OnsmbfWN6f0Y4VR1ewKmMVG45tILSHgQ8zLscSPJERMSOapJyzj4e+jS+PrFB4FacRvpbSCTDoCA8yUFDcNNIopboAr0X49e3Qjgzx5/zFMvpF9+ORIY+w+ubVTI9/kbLzgzmY9ysPffMQV358JY98+wjrs9ZTavbODc0RLSTC9/H7mkJhRedkjcOSMluEryMswK/RUjq1UYo/gV5y+PXt0I4K8cdkkfxjxT789FowuSM7kpJTk/nynnkcyPuV9KPprMlaw/KjywnzC9PknOOTGdphqFflnJXDVyh8CIHjxqsSkwV/gw4hRNOK8EUAgdI7EXB9hd/6doogyE/P298drfR+z3ZhRAYFMix4GMM6DGPusLls+m0T6RnprM1ay5eHv6RVQCtGx45mQvwEktolodd5pvzSGcrhKxQ+hJbDr06JyUyAQfsehAf6UVSmlRH6G7z73SjBjwDhHYdf3w7tYV2i2Pv35FqP76fzY0TMCEbEjKDEXML3x78nPSOd5UeX89nBz4gKjGJs3FiS45O5pO0l6BrBT/m2J1QOX9HCcCatUGKyEGDQosmwQC3OK2jEiVtnlOBfY4TvSfG12oTf3EmAPoCrYq/ihSte4Jsp3/DSlS8xsN1AlhxawvT06Yz9dCwvbHmBnWd2elTRU0X4CoUP4UweuaTMUhHhB2k55PxiE1GhAY1pXjVK8Cccxz0Bjqpopk6dypw5c5g37//bu/vYuur7juPvr2/s2E6cJuSJNE6ugbINUnW0OJk2ULXRQmiaBtpBH2KyP1bVEwGtU9GmTpa6VZ2lSasQFZuUuoxp1E7Qqi4CStRA1bSMdCNxaDLIQxkNdp6gScizHT/E97s/7r1+vNeO773nnvScz0u6iu+5597zPYr98c+/8zu/33dKEsxh3KFdM6OG1Q2rWd2wmp7BHnYc3cH2d7az5dAWvn/g+yydvZTVDat59LZHqUyUtr8/2kk4vOqNRulIPGS/08e3EvuvDDGzMv3jPrc2HSJ/ueWX5Swtpz6qqMozLDPXKBqA999/n+bm5kisajarchZrb1zLk594kp9/4ed8645v0TCngZ3Hd5Y87CHyga8bryReKjJDkMcPzRwY1aXzRzctoK56BkfPTgzTrPfO9/HKW6cYHAp2qcA+r2Sm555hcrLRMlFc4GZO1Rzu/9D9bLp7E1vWbgnkGNFOQgW+xExFpok/sYU/0qVTXZng4T++iXO9g/T05x6t88jm1/mzp3exff97gdZ72SupzNOHP9VomSgvcBPU0M2Y9OGrS0fiIfutPr6FP3qUDkD9vPSyhMfPXeZ3FtdN+Jx3z10GRuaICcplr6LK+6Hn9ITXvv3Nr/PYY4/Re/lyzvcuq6/P+b7QVM2Gyuqwq5hUTAJfLXyJh+yiHOOnV+i/kqKueqTVuDSzuMbRM705Az/rUp6/AErlks+kOtUL/zRx1dMHgAcenQHkq+98zveFpm4JfO3gNd3AVOCLRIgNd+mM3d4/mGLB7JGfg2XXpQP/y//eya/+4d7h/v2swcyfCPm6fEplM59iYX0D6z5y/aT77d69m+dfeJ6zZ84y77p5rPvMOlauXDn82uYtmxkYGPlrpKqqivVfWj+8T+AO/wwO/QgGemDm7PIcswAKfJEIyV60nRD447p0FtVVc9fvLeKnh05y4lwfNyyYNWb/bND39A8FWu/JoTr2Lv5T1q26ddL9Vq76Cisfacv52oOfb6W7u2fc1kG2HnuJrq7c7ym5ikQ68PsvXNOBH+0kVB++xEz2O338zVejb7zK+ouP3wjAsXGjdVIppzczf37QXToDQykqZxT381nonDglNXNO+t++C+U7ZgFiEvjRPk2RrOEW/rjt/TmmUVg6L92tc+zs2IuiPQMjIR90l87gUIqqRHE/n/lG80w1yqekqjNr3/Yr8MOjwJeYGRmlMzby+wbHdukAXD+nmkSFcXx84I/qxgmyhT+UclIOlUUGfmtrK7W1tWO2Xc2cOCWdtkEt/GuBxuFLvGRH6VwZGgn8vsEhLvZdYcHssasxzUhUsOQD1RO6dEaHfJAt/OxNXeMDf7pBXMicONlpG7q7u3F3uru72bBhAxs3bizsZKozgd9/vrD3l0m0k1A3XknM3LQwffH1wImRlma2yyY79n60pXNrJnbpZEK+KlER6EXbgeHAH+nDzxXEVzONQlNTE11dXaRSKbq6uqacHyfXtA3uzqZNm9i4ceP0W/7ZFn7/9NcKDnKCuPGinYTq0pGYaWy4jkSFsfPXIzckZVvw9fMmLrlXP682b+AvmjMz0C6dwcyyi6OvLRS6CtV05bugmw390b9wHnroIRKJBGaWP5Crp+7SyRXshf6CK1RRwzLN7EHg74FbgFXu3plnv3uB7wAJ4Cl3/8dijnvVNEpHYmb2zBnc+aEFfO+Vwyz5QDWfuGUx75xOD1nM1cKvn1fDby72jZkb/0JmcZTFc6p5672L/OLtYO5mPdObHjc/ukunXCNu8i1+AhOnpQBIpdJZknfd26rZgPHGnl/wmabHOXLkCMuXL2fNmjVs27aN7u7uzEymPuZzampq8v6CC2IWz2LH4b8JfA74br4dzCwB/AtwN3AM2G1mz7v7gSKPPTW18CWGnlz/UTa2v843ntvPN57bD0CiwlhUN3Eq5Pp5NbjDC/tOMDCUYv+J87yw713M4JYldezpPsv6p14LtN65NSN3ABe6CtV0tba2smHDhoLmns8ZyGYMVFTDoW389YpBWDET+A10/xsrVgArck25kAJ6gImvXRoIZg6jogLf3Q/CyIWiPFYBb7v74cy+zwL3AeULfE2PLDEyp7qSZ/58Fbu7zvDO6R5OXezn5sWzqaiY+HNwY6bP/7Ef7APS/fYrb5jHF1Yu555bF3PfbUtJ5VoVvUSqZlTwkfq5w88LXYVqupqamti5cyebNm0aE/qjW+GTyfUXx0u/vsIfLHI+f2vx97OeHQjonlh3L/oB/AxozPPaA6S7cbLPNwD/PMlnNQOdQOfy5cu9KK8+4f53c9z7Lhb3OSIRlUqlfOf/nfLXDr/vR8/0+OCVobBL8vb2dk8mk25mnkwmvb29vWzHevjhh722ttZJD/HL+0gmkxM+y8ymfN/4x/z58yccr7a2tqhzBjo9X77me8FHAvgnpLtuxj/u8wACf/Tj9ttvL/ik3d39vx5PB35/T3GfIyKx0d7e7vPnz88b0vkCOZlMTivss59T6l9wkwX+lJ3b7v5Jd/9wjsdzU7034ziwbNTz+sy24GlYpohMU1NTE6dPn6a9vZ1kMglAIpGelmKyMf65bgAbL9v9PfpzpjuktBjlmDxtN3Czmd1AOui/CKwvw3F10VZECjbd9W6z+7a0tEwYpZN93traWvY1dEcrdljmZ4EngYXAi2a2191Xm9kHSXfjrHH3K2b2KLCd9LDMp919f9GVXw218EWkjMJYFH06ih2lsxXYmmP7CWDNqOfbgG3FHKsgauGLiAyLXBKOvpvtiSceT2/UjVciItEK/PG3KV84n57IqGPz5pArExEJn3kBd5qVS2Njo3d25pytIaeGhga6u7tpvWsmlQm4c1mCVUsT3PTMdXR1dQVXqIjINcLM9rh7Y67XIrXEYfbut698rJLaynQ3zq7jQ+Vd+UZE5BoVqcDPzsOx6NuXxmzPjqUVEYmzSPXhF7ryjYhIHEQq8AtZ+UZEJC4iddFWRCTuJrtoG6kWvoiI5KfAFxGJCQW+iEhMKPBFRGJCgS8iEhMKfBGRmFDgi4jEhAJfRCQmFPgiIjGhwBcRiQkFvohITCjwRURiQoEvIhITCnwRkZgoKvDN7EEz229mKTPLOR1nZr8uM3vDzPaameY7FhEJQbFLHL4JfA747lXs+yfufrrI44mISIGKCnx3PwhgZqWpRkREAlOuPnwHXjKzPWbWPNmOZtZsZp1m1nnq1KkylSciEn1TtvDN7CfA9TleanH3567yOHe6+3EzWwS8bGaH3P2VXDu6exvQBuklDq/y80VEZApTBr67f7LYg7j78cy/J81sK7AKyBn4IiISjMC7dMxslpnVZb8G7iF9sVdERMqo2GGZnzWzY8AfAi+a2fbM9g+a2bbMbouBV81sH7ALeNHdf1zMcUVEZPqKHaWzFdiaY/sJYE3m68PA7xdzHBERKZ7utBWRMTo6OmhoaKCiooKGhgY6OjrCLklKpNgbr0QkQjo6Omhubqa3txeA7u5umpvTI6mbmprCLE1KQC18ERnW0tIyHPZZvb29tLS0hFSRlJICX0SGHTlyZFrb5beLAl9Ehi1fvnxa2+W3iwJfRIa1trZSW1s7ZlttbS2tra0hVSSlpMAXkWFNTU20tbWRTCYxM5LJJG1tbbpgGxHmfu1OV9PY2OidnZo+X0TkapnZHnfPuT6JWvgiIjGhwBcRiQkFvohITCjwRURiQoEvIhIT1/QoHTM7BXQX+PYFQNwWTdc5x4POOR4KPeekuy/M9cI1HfjFMLPOfEOTokrnHA8653gI4pzVpSMiEhMKfBGRmIhy4LeFXUAIdM7xoHOOh5Kfc2T78EVEZKwot/BFRGQUBb6ISExELvDN7F4z+5WZvW1mXw+7nnIws6fN7KSZvRl2LeVgZsvMbIeZHTCz/Wb21bBrCpqZVZvZLjPblznnb4ZdU7mYWcLMfmlmPwq7lnIwsy4ze8PM9ppZSacLjlQfvpklgLeAu4FjwG7gS+5+INTCAmZmHwcuAc+4+4fDridoZrYEWOLur5tZHbAHuD/K/89mZsAsd79kZpXAq8BX3f1/Qi4tcGb2NaARmOPua8OuJ2hm1gU0unvJbzSLWgt/FfC2ux929wHgWeC+kGsKnLu/ApwJu45ycfd33f31zNcXgYPA0nCrCpanXco8rcw8otNay8PM6oFPA0+FXUsURC3wlwJHRz0/RsSDIO7MrAH4KPBauJUEL9O1sRc4Cbzs7pE/Z+AJ4G+AVNiFlJEDL5nZHjNrLuUHRy3wJUbMbDbwQ+Cv3P1C2PUEzd2H3P02oB5YZWaR7r4zs7XASXffE3YtZXanu38M+BTwSKbLtiSiFvjHgWWjntdntknEZPqxfwh0uPt/hl1PObn7OWAHcG/YtQTsDmBdpk/7WeAuM2sPt6TgufvxzL8nga2ku6pLImqBvxu42cxuMLMq4IvA8yHXJCWWuYD5r8BBd3887HrKwcwWmtnczNc1pAcmHAq3qmC5+9+6e727N5D+Wf6puz8UclmBMrNZmYEImNks4B6gZKPvIhX47n4FeBTYTvpC3n+4+/5wqwqemW0B/hv4XTM7ZmZfDrumgN0BbCDd4tubeawJu6iALQF2mNn/km7YvOzusRimGDOLgVfNbB+wC3jR3X9cqg+P1LBMERHJL1ItfBERyU+BLyISEwp8EZGYUOCLiMSEAl9EJCYU+CIiMaHAFxGJif8HR9cck/GZq0UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.ensemble import StackingClassifier\n",
    "from sklearn.ensemble import StackingRegressor\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "\n",
    "clf_a = DecisionTreeRegressor(max_depth=3)\n",
    "clf_b = DecisionTreeRegressor(max_depth=5)\n",
    "clf_c = LinearRegression()\n",
    "\n",
    "estimators = [('DT1', clf_a), ('DT2', clf_b), ('lr', clf_c)]\n",
    "\n",
    "clf = StackingRegressor(estimators=estimators, \n",
    "                        final_estimator=RandomForestRegressor(n_estimators=10, random_state=42)).fit(X, y)\n",
    "\n",
    "# Predict\n",
    "X_test = np.arange(0.0, 5.0, 0.01)[:, np.newaxis]\n",
    "y_1 = clf.predict(X_test)\n",
    "y_2 = clf.estimators_[0].predict(X_test)\n",
    "y_3 = clf.estimators_[2].predict(X_test)\n",
    "\n",
    "plt.plot(X,y,'o',color='black')\n",
    "plt.plot(X_test,y_1,label='Stacking')\n",
    "plt.plot(X_test,y_2,label='DT')\n",
    "plt.plot(X_test,y_3,label='LR')\n",
    "# Plot the resu  et\")\n",
    "plt.title(\"Stacking of models\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "StackingRegressor(cv=None,\n",
       "                  estimators=[('DT1',\n",
       "                               DecisionTreeRegressor(ccp_alpha=0.0,\n",
       "                                                     criterion='mse',\n",
       "                                                     max_depth=3,\n",
       "                                                     max_features=None,\n",
       "                                                     max_leaf_nodes=None,\n",
       "                                                     min_impurity_decrease=0.0,\n",
       "                                                     min_impurity_split=None,\n",
       "                                                     min_samples_leaf=1,\n",
       "                                                     min_samples_split=2,\n",
       "                                                     min_weight_fraction_leaf=0.0,\n",
       "                                                     presort='deprecated',\n",
       "                                                     random_state=None,\n",
       "                                                     splitter='best')),\n",
       "                              ('DT2',\n",
       "                               DecisionTreeRegressor...\n",
       "                  final_estimator=RandomForestRegressor(bootstrap=True,\n",
       "                                                        ccp_alpha=0.0,\n",
       "                                                        criterion='mse',\n",
       "                                                        max_depth=None,\n",
       "                                                        max_features='auto',\n",
       "                                                        max_leaf_nodes=None,\n",
       "                                                        max_samples=None,\n",
       "                                                        min_impurity_decrease=0.0,\n",
       "                                                        min_impurity_split=None,\n",
       "                                                        min_samples_leaf=1,\n",
       "                                                        min_samples_split=2,\n",
       "                                                        min_weight_fraction_leaf=0.0,\n",
       "                                                        n_estimators=10,\n",
       "                                                        n_jobs=None,\n",
       "                                                        oob_score=False,\n",
       "                                                        random_state=42,\n",
       "                                                        verbose=0,\n",
       "                                                        warm_start=False),\n",
       "                  n_jobs=None, passthrough=False, verbose=0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[DecisionTreeRegressor(ccp_alpha=0.0, criterion='mse', max_depth=3,\n",
       "                       max_features=None, max_leaf_nodes=None,\n",
       "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                       min_samples_leaf=1, min_samples_split=2,\n",
       "                       min_weight_fraction_leaf=0.0, presort='deprecated',\n",
       "                       random_state=None, splitter='best'),\n",
       " DecisionTreeRegressor(ccp_alpha=0.0, criterion='mse', max_depth=5,\n",
       "                       max_features=None, max_leaf_nodes=None,\n",
       "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                       min_samples_leaf=1, min_samples_split=2,\n",
       "                       min_weight_fraction_leaf=0.0, presort='deprecated',\n",
       "                       random_state=None, splitter='best'),\n",
       " LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.estimators_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Consultar\n",
    "\n",
    "* Stochastic Gradient Boosting Tree\n",
    "\n",
    "Modificaciones recientes de gran interés:\n",
    "\n",
    "* XGBoost [Paper original](https://arxiv.org/pdf/1603.02754.pdf)\n",
    "* LightGBM\n",
    "* CatBoost\n",
    "\n",
    "[CatBoost vs. Light GBM vs. XGBoost](https://towardsdatascience.com/catboost-vs-light-gbm-vs-xgboost-5f93620723db)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}