0b - unit 05.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6aee698d",
   "metadata": {},
   "source": [
    "# Exercise: Supervised learning using different cost functions\n",
    "\n",
    "In this exercise, we will have a deeper look at how cost functions can change:\n",
    "\n",
    "1. how well models appear to have fit data\n",
    "2. the kinds of relationships a model represents\n",
    "\n",
    "## Loading the data\n",
    "\n",
    "Let's start by having a look at the data. To make this exercise simpler, we will only use a few datapoints this time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "            date  amount_of_precipitation  max_temperature  min_temperature  \\\n",
       "762   1950-02-01                     0.00               27                1   \n",
       "4414  1960-02-01                     0.15               52               44   \n",
       "8067  1970-02-01                     0.00               50               42   \n",
       "11719 1980-02-01                     0.37               54               36   \n",
       "15372 1990-02-01                     0.08               45               37   \n",
       "19024 2000-02-01                     1.34               49               41   \n",
       "22677 2010-02-01                     0.08               49               40   \n",
       "25234 2017-02-01                     0.00               43               29   \n",
       "\n",
       "        rain         year  \n",
       "762    False  1950.087611  \n",
       "4414    True  1960.087611  \n",
       "8067   False  1970.087611  \n",
       "11719   True  1980.087611  \n",
       "15372   True  1990.087611  \n",
       "19024   True  2000.087611  \n",
       "22677   True  2010.087611  \n",
       "25234  False  2017.087611  "
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>date</th>\n      <th>amount_of_precipitation</th>\n      <th>max_temperature</th>\n      <th>min_temperature</th>\n      <th>rain</th>\n      <th>year</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>762</th>\n      <td>1950-02-01</td>\n      <td>0.00</td>\n      <td>27</td>\n      <td>1</td>\n      <td>False</td>\n      <td>1950.087611</td>\n    </tr>\n    <tr>\n      <th>4414</th>\n      <td>1960-02-01</td>\n      <td>0.15</td>\n      <td>52</td>\n      <td>44</td>\n      <td>True</td>\n      <td>1960.087611</td>\n    </tr>\n    <tr>\n      <th>8067</th>\n      <td>1970-02-01</td>\n      <td>0.00</td>\n      <td>50</td>\n      <td>42</td>\n      <td>False</td>\n      <td>1970.087611</td>\n    </tr>\n    <tr>\n      <th>11719</th>\n      <td>1980-02-01</td>\n      <td>0.37</td>\n      <td>54</td>\n      <td>36</td>\n      <td>True</td>\n      <td>1980.087611</td>\n    </tr>\n    <tr>\n      <th>15372</th>\n      <td>1990-02-01</td>\n      <td>0.08</td>\n      <td>45</td>\n      <td>37</td>\n      <td>True</td>\n      <td>1990.087611</td>\n    </tr>\n    <tr>\n      <th>19024</th>\n      <td>2000-02-01</td>\n      <td>1.34</td>\n      <td>49</td>\n      <td>41</td>\n      <td>True</td>\n      <td>2000.087611</td>\n    </tr>\n    <tr>\n      <th>22677</th>\n      <td>2010-02-01</td>\n      <td>0.08</td>\n      <td>49</td>\n      <td>40</td>\n      <td>True</td>\n      <td>2010.087611</td>\n    </tr>\n    <tr>\n      <th>25234</th>\n      <td>2017-02-01</td>\n      <td>0.00</td>\n      <td>43</td>\n      <td>29</td>\n      <td>False</td>\n      <td>2017.087611</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 1
    }
   ],
   "source": [
    "import pandas\n",
    "from datetime import datetime\n",
    "\n",
    "# Load a file containing our weather data\n",
    "dataset = pandas.read_csv('Data/seattleWeather_1948-2017.csv', parse_dates=['date'])\n",
    "\n",
    "# Convert the dates into numbers so we can use it in our models\n",
    "# We make a year column which can contain fractions. For example\n",
    "# 1948.5 is half way through the year 1948\n",
    "dataset[\"year\"] = [(d.year + d.timetuple().tm_yday / 365.25) for d in dataset.date]\n",
    "\n",
    "\n",
    "# For the sake of this exercise, let's look at Feb 1st for the following years:\n",
    "desired_dates = [\n",
    "    datetime(1950,2,1),\n",
    "    datetime(1960,2,1),\n",
    "    datetime(1970,2,1),\n",
    "    datetime(1980,2,1),\n",
    "    datetime(1990,2,1),\n",
    "    datetime(2000,2,1),\n",
    "    datetime(2010,2,1),\n",
    "    datetime(2017,2,1),\n",
    "]\n",
    "\n",
    "dataset = dataset[dataset.date.isin(desired_dates)].copy()\n",
    "\n",
    "# Print the dataset\n",
    "dataset\n"
   ]
  },
  {
   "source": [
    "## Comparing two cost functions\n",
    "\n",
    "Let's compare two common cost functions, the _sum of squared differences_ (SSD), and the _sum of absolute differences_ (SAD). These both calculate the difference between each predicted value with the expected value. The difference is simply: \n",
    "\n",
    "* SSD squares that difference, and sums the result;\n",
    "* SAD converts differences into absolute differences, then sums them.\n",
    "\n",
    "To see these in action, we need to first implement these functions:"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy\n",
    "\n",
    "def sum_of_square_differences(estimate, actual):\n",
    "    # Note that with numpy to square each value we use **\n",
    "    return numpy.sum((estimate - actual)**2)\n",
    "\n",
    "def sum_of_absolute_differences(estimate, actual):\n",
    "    return numpy.sum(numpy.abs(estimate - actual))"
   ]
  },
  {
   "source": [
    "They're very similar. How do they behave? Let's test with some fake model estimates.\n",
    "\n",
    "Let's say that the correct answers are `1` and `3`, but the model estimates `2` and `2`:"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "SSD: 2\nSAD: 2\n"
     ]
    }
   ],
   "source": [
    "actual_label = numpy.array([1, 3])\n",
    "model_estimate = numpy.array([2, 2])\n",
    "\n",
    "print(\"SSD:\", sum_of_square_differences(model_estimate, actual_label))\n",
    "print(\"SAD:\", sum_of_absolute_differences(model_estimate, actual_label))"
   ]
  },
  {
   "source": [
    "We have an error of `1` for each estimate, and both methods have returned the same error. \n",
    "\n",
    "What happens if we distribute these errors differently? Let's pretend we estimated the first value perfectly, but were off by `2` for the second value: "
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "SSD: 4\nSAD: 2\n"
     ]
    }
   ],
   "source": [
    "actual_label = numpy.array([1, 3])\n",
    "model_estimate = numpy.array([1, 1])\n",
    "\n",
    "print(\"SSD:\", sum_of_square_differences(model_estimate, actual_label))\n",
    "print(\"SAD:\", sum_of_absolute_differences(model_estimate, actual_label))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ddecb338",
   "metadata": {},
   "source": [
    "SAD has calculated the same cost as before, because the average error is still the same (`1 + 1 = 0 + 2`). According to SAD, the first and second set of estimates were equally good. \n",
    "\n",
    "By contrast SSD has given a higher (worse) cost for the second set of estimates (<code>1<sup>2</sup> + 1<sup>2</sup> < 0<sup>2</sup> + 2<sup>2</sup></code>). When we use SSD, we not only encourage models to be accurate, but to be consistent in their accuracy.\n",
    "\n",
    "\n",
    "## Differences in action\n",
    "\n",
    "Let's compare how our two cost functions affect model fitting.\n",
    "\n",
    "First, fit a model using the SSD cost function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "58a56759",
   "metadata": {},
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "hovertemplate": "year=%{x}<br>min temperature=%{y}<extra></extra>",
         "legendgroup": "",
         "marker": {
          "color": "#636efa",
          "size": 20,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          1950.0876112251883,
          1960.0876112251883,
          1970.0876112251883,
          1980.0876112251883,
          1990.0876112251883,
          2000.0876112251883,
          2010.0876112251883,
          2017.0876112251883
         ],
         "xaxis": "x",
         "y": [
          1,
          44,
          42,
          36,
          37,
          41,
          40,
          29
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "legend": {
         "tracegroupgap": 0
        },
        "shapes": [
         {
          "line": {
           "color": "Crimson"
          },
          "path": "M1950.0876112251883 26.093609277984513 L 1950.4242946422737 26.168057787177304 L 1950.760978059359 26.24250629637004 L 1951.0976614764445 26.31695480556283 L 1951.4343448935301 26.39140331475562 L 1951.7710283106155 26.465851823948412 L 1952.107711727701 26.540300333141147 L 1952.4443951447863 26.61474884233394 L 1952.7810785618717 26.689197351526673 L 1953.117761978957 26.763645860719464 L 1953.4544453960425 26.8380943699122 L 1953.791128813128 26.91254287910499 L 1954.1278122302135 26.98699138829778 L 1954.464495647299 27.061439897490516 L 1954.8011790643843 27.135888406683307 L 1955.1378624814697 27.21033691587604 L 1955.474545898555 27.284785425068833 L 1955.8112293156405 27.359233934261567 L 1956.147912732726 27.43368244345436 L 1956.4845961498115 27.50813095264715 L 1956.821279566897 27.58257946183994 L 1957.1579629839823 27.657027971032676 L 1957.4946464010677 27.731476480225467 L 1957.831329818153 27.8059249894182 L 1958.1680132352385 27.880373498610993 L 1958.504696652324 27.954822007803728 L 1958.8413800694095 28.02927051699652 L 1959.178063486495 28.10371902618931 L 1959.5147469035803 28.178167535382045 L 1959.8514303206657 28.252616044574836 L 1960.188113737751 28.32706455376757 L 1960.5247971548365 28.401513062960362 L 1960.861480571922 28.475961572153096 L 1961.1981639890073 28.550410081345888 L 1961.534847406093 28.62485859053868 L 1961.8715308231783 28.69930709973147 L 1962.2082142402637 28.773755608924205 L 1962.544897657349 28.848204118116996 L 1962.8815810744345 28.92265262730973 L 1963.21826449152 28.997101136502522 L 1963.5549479086053 29.071549645695256 L 1963.891631325691 29.145998154888048 L 1964.2283147427763 29.22044666408084 L 1964.5649981598617 29.294895173273574 L 1964.901681576947 29.369343682466365 L 1965.2383649940325 29.4437921916591 L 1965.575048411118 29.51824070085189 L 1965.9117318282033 29.592689210044625 L 1966.248415245289 29.667137719237473 L 1966.5850986623743 29.741586228430208 L 1966.9217820794597 29.816034737623 L 1967.258465496545 29.890483246815734 L 1967.5951489136305 29.964931756008525 L 1967.931832330716 30.03938026520126 L 1968.2685157478013 30.11382877439405 L 1968.6051991648867 30.188277283586785 L 1968.9418825819723 30.262725792779634 L 1969.2785659990577 30.337174301972368 L 1969.6152494161431 30.41162281116516 L 1969.9519328332285 30.486071320357894 L 1970.288616250314 30.56051982955063 L 1970.6252996673993 30.63496833874342 L 1970.9619830844847 30.709416847936154 L 1971.2986665015703 30.783865357129002 L 1971.6353499186557 30.858313866321737 L 1971.9720333357411 30.93276237551453 L 1972.3087167528265 31.007210884707263 L 1972.645400169912 31.081659393900054 L 1972.9820835869973 31.15610790309279 L 1973.3187670040827 31.23055641228558 L 1973.655450421168 31.305004921478314 L 1973.9921338382537 31.379453430671163 L 1974.3288172553391 31.453901939863897 L 1974.6655006724245 31.52835044905669 L 1975.00218408951 31.602798958249423 L 1975.3388675065953 31.677247467442214 L 1975.6755509236807 31.75169597663495 L 1976.012234340766 31.82614448582774 L 1976.3489177578517 31.90059299502053 L 1976.6856011749371 31.975041504213266 L 1977.0222845920225 32.04949001340606 L 1977.358968009108 32.12393852259879 L 1977.6956514261933 32.19838703179158 L 1978.0323348432787 32.27283554098432 L 1978.369018260364 32.34728405017711 L 1978.7057016774497 32.4217325593699 L 1979.0423850945351 32.49618106856269 L 1979.3790685116205 32.570629577755426 L 1979.715751928706 32.64507808694822 L 1980.0524353457913 32.71952659614095 L 1980.3891187628767 32.79397510533374 L 1980.725802179962 32.86842361452648 L 1981.0624855970475 32.94287212371927 L 1981.3991690141331 33.01732063291206 L 1981.7358524312185 33.091769142104795 L 1982.072535848304 33.166217651297586 L 1982.4092192653893 33.24066616049032 L 1982.7459026824747 33.31511466968311 L 1983.08258609956 33.38956317887585 L 1983.4192695166455 33.46401168806864 L 1983.7559529337311 33.53846019726143 L 1984.0926363508165 33.61290870645422 L 1984.429319767902 33.687357215646955 L 1984.7660031849873 33.761805724839746 L 1985.1026866020727 33.83625423403248 L 1985.439370019158 33.91070274322527 L 1985.7760534362435 33.98515125241801 L 1986.1127368533291 34.0595997616108 L 1986.4494202704145 34.13404827080359 L 1986.7861036875 34.208496779996324 L 1987.1227871045853 34.282945289189115 L 1987.4594705216707 34.35739379838185 L 1987.796153938756 34.43184230757464 L 1988.1328373558415 34.506290816767375 L 1988.469520772927 34.58073932596017 L 1988.8062041900125 34.65518783515296 L 1989.142887607098 34.72963634434575 L 1989.4795710241833 34.804084853538484 L 1989.8162544412687 34.878533362731275 L 1990.152937858354 34.95298187192401 L 1990.4896212754395 35.0274303811168 L 1990.826304692525 35.101878890309536 L 1991.1629881096105 35.176327399502384 L 1991.499671526696 35.25077590869512 L 1991.8363549437813 35.32522441788791 L 1992.1730383608667 35.399672927080644 L 1992.509721777952 35.47412143627338 L 1992.8464051950375 35.54856994546617 L 1993.183088612123 35.623018454658904 L 1993.5197720292085 35.69746696385175 L 1993.856455446294 35.77191547304449 L 1994.1931388633793 35.84636398223728 L 1994.5298222804647 35.92081249143001 L 1994.86650569755 35.995261000622804 L 1995.2031891146355 36.06970950981554 L 1995.539872531721 36.14415801900833 L 1995.8765559488063 36.218606528201065 L 1996.213239365892 36.29305503739391 L 1996.5499227829773 36.36750354658665 L 1996.8866062000627 36.44195205577944 L 1997.223289617148 36.51640056497217 L 1997.5599730342335 36.590849074164964 L 1997.896656451319 36.6652975833577 L 1998.2333398684043 36.73974609255049 L 1998.57002328549 36.81419460174328 L 1998.9067067025753 36.888643110936016 L 1999.2433901196607 36.96309162012881 L 1999.580073536746 37.03754012932154 L 1999.9167569538315 37.11198863851433 L 2000.253440370917 37.18643714770707 L 2000.5901237880023 37.26088565689986 L 2000.9268072050877 37.33533416609259 L 2001.2634906221733 37.40978267528544 L 2001.6001740392587 37.484231184478176 L 2001.9368574563441 37.55867969367097 L 2002.2735408734295 37.6331282028637 L 2002.610224290515 37.70757671205649 L 2002.9469077076003 37.78202522124923 L 2003.2835911246857 37.85647373044202 L 2003.6202745417713 37.93092223963481 L 2003.9569579588567 38.005370748827545 L 2004.2936413759421 38.079819258020336 L 2004.6303247930275 38.15426776721307 L 2004.967008210113 38.22871627640586 L 2005.3036916271983 38.3031647855986 L 2005.6403750442837 38.37761329479139 L 2005.9770584613693 38.45206180398418 L 2006.3137418784547 38.52651031317697 L 2006.6504252955401 38.600958822369705 L 2006.9871087126255 38.675407331562496 L 2007.323792129711 38.74985584075523 L 2007.6604755467963 38.82430434994802 L 2007.9971589638817 38.89875285914076 L 2008.333842380967 38.97320136833355 L 2008.6705257980527 39.04764987752634 L 2009.0072092151381 39.122098386719074 L 2009.3438926322235 39.196546895911865 L 2009.680576049309 39.2709954051046 L 2010.0172594663943 39.34544391429739 L 2010.3539428834797 39.419892423490126 L 2010.690626300565 39.49434093268292 L 2011.0273097176507 39.56878944187571 L 2011.3639931347361 39.6432379510685 L 2011.7006765518215 39.717686460261234 L 2012.037359968907 39.792134969454025 L 2012.3740433859923 39.86658347864676 L 2012.7107268030777 39.94103198783955 L 2013.047410220163 40.015480497032286 L 2013.3840936372487 40.089929006225134 L 2013.7207770543341 40.16437751541787 L 2014.0574604714195 40.23882602461066 L 2014.394143888505 40.313274533803394 L 2014.7308273055903 40.387723042996186 L 2015.0675107226757 40.46217155218892 L 2015.404194139761 40.536620061381655 L 2015.7408775568465 40.611068570574446 L 2016.0775609739321 40.68551707976724 L 2016.4142443910175 40.75996558896003 L 2016.750927808103 40.83441409815276 L 2017.0876112251883 40.908862607345554",
          "type": "path"
         }
        ],
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "box": [
           {
            "boxpoints": "outliers",
            "notched": false,
            "type": "box"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "line": {
              "width": 1
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "#C8D4E3"
          },
          "height": 400,
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "margin": {
           "b": 2,
           "l": 2,
           "r": 2,
           "t": 30
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           }
          },
          "scene": {
           "camera": {
            "eye": {
             "x": 1.5,
             "y": 1.5,
             "z": 0.1
            }
           },
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.5
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          }
         }
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "year"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "min temperature"
         }
        }
       }
      }
     },
     "metadata": {}
    }
   ],
   "source": [
    "from microsoft_custom_linear_regressor import MicrosoftCustomLinearRegressor\n",
    "import graphing\n",
    "\n",
    "# Create and fit the model\n",
    "# We use a custom object that we have hidden from this notebook as\n",
    "# you do not need to understand its details. This fits a linear model\n",
    "# using a provided cost function\n",
    "\n",
    "# Fit a model using sum of square differences\n",
    "model = MicrosoftCustomLinearRegressor().fit(X = dataset.year, \n",
    "                                             y = dataset.min_temperature, \n",
    "                                             cost_function = sum_of_square_differences)\n",
    "\n",
    "# Graph the model\n",
    "graphing.scatter_2D(dataset, \n",
    "                    label_x=\"year\", \n",
    "                    label_y=\"min_temperature\", \n",
    "                    trendline=model.predict)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8694563",
   "metadata": {},
   "source": [
    "Our SSD method normally does well, but here it did a poor job - the line is a far distance from the values for many years. Why? Notice the datapoint at the bottom left doesn't seem to follow the trend of the other datapoints. 1950 was a very cold winter in Seattle, and this datapoint is strongly influencing our final model (the blue line). What happens if we change the cost function?\n",
    "\n",
    "### Sum of absolute differences\n",
    "\n",
    "Let's repeat what we've just done, but using SAD."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "71d741ff",
   "metadata": {},
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "hovertemplate": "year=%{x}<br>min temperature=%{y}<extra></extra>",
         "legendgroup": "",
         "marker": {
          "color": "#636efa",
          "size": 20,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          1950.0876112251883,
          1960.0876112251883,
          1970.0876112251883,
          1980.0876112251883,
          1990.0876112251883,
          2000.0876112251883,
          2010.0876112251883,
          2017.0876112251883
         ],
         "xaxis": "x",
         "y": [
          1,
          44,
          42,
          36,
          37,
          41,
          40,
          29
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "legend": {
         "tracegroupgap": 0
        },
        "shapes": [
         {
          "line": {
           "color": "Crimson"
          },
          "path": "M1950.0876112251883 38.425387160211244 L 1950.4242946422737 38.43229097881747 L 1950.760978059359 38.4391947974237 L 1951.0976614764445 38.44609861602993 L 1951.4343448935301 38.45300243463616 L 1951.7710283106155 38.45990625324239 L 1952.107711727701 38.46681007184861 L 1952.4443951447863 38.47371389045484 L 1952.7810785618717 38.48061770906107 L 1953.117761978957 38.4875215276673 L 1953.4544453960425 38.49442534627352 L 1953.791128813128 38.50132916487975 L 1954.1278122302135 38.50823298348598 L 1954.464495647299 38.51513680209221 L 1954.8011790643843 38.522040620698434 L 1955.1378624814697 38.52894443930467 L 1955.474545898555 38.53584825791089 L 1955.8112293156405 38.54275207651712 L 1956.147912732726 38.549655895123344 L 1956.4845961498115 38.55655971372958 L 1956.821279566897 38.5634635323358 L 1957.1579629839823 38.570367350942036 L 1957.4946464010677 38.57727116954826 L 1957.831329818153 38.58417498815449 L 1958.1680132352385 38.59107880676071 L 1958.504696652324 38.59798262536694 L 1958.8413800694095 38.60488644397317 L 1959.178063486495 38.611790262579404 L 1959.5147469035803 38.61869408118563 L 1959.8514303206657 38.625597899791856 L 1960.188113737751 38.63250171839808 L 1960.5247971548365 38.63940553700431 L 1960.861480571922 38.64630935561054 L 1961.1981639890073 38.653213174216766 L 1961.534847406093 38.660116992823 L 1961.8715308231783 38.667020811429225 L 1962.2082142402637 38.67392463003545 L 1962.544897657349 38.68082844864168 L 1962.8815810744345 38.6877322672479 L 1963.21826449152 38.694636085854135 L 1963.5549479086053 38.70153990446036 L 1963.891631325691 38.708443723066594 L 1964.2283147427763 38.71534754167282 L 1964.5649981598617 38.722251360279046 L 1964.901681576947 38.72915517888527 L 1965.2383649940325 38.736058997491504 L 1965.575048411118 38.74296281609773 L 1965.9117318282033 38.749866634703956 L 1966.248415245289 38.75677045331019 L 1966.5850986623743 38.763674271916415 L 1966.9217820794597 38.77057809052264 L 1967.258465496545 38.77748190912887 L 1967.5951489136305 38.7843857277351 L 1967.931832330716 38.791289546341325 L 1968.2685157478013 38.79819336494755 L 1968.6051991648867 38.80509718355378 L 1968.9418825819723 38.81200100216001 L 1969.2785659990577 38.818904820766235 L 1969.6152494161431 38.82580863937247 L 1969.9519328332285 38.832712457978694 L 1970.288616250314 38.83961627658492 L 1970.6252996673993 38.846520095191146 L 1970.9619830844847 38.85342391379738 L 1971.2986665015703 38.860327732403604 L 1971.6353499186557 38.86723155100984 L 1971.9720333357411 38.87413536961606 L 1972.3087167528265 38.88103918822229 L 1972.645400169912 38.887943006828515 L 1972.9820835869973 38.89484682543474 L 1973.3187670040827 38.90175064404097 L 1973.655450421168 38.9086544626472 L 1973.9921338382537 38.91555828125343 L 1974.3288172553391 38.92246209985966 L 1974.6655006724245 38.929365918465884 L 1975.00218408951 38.93626973707211 L 1975.3388675065953 38.94317355567834 L 1975.6755509236807 38.95007737428457 L 1976.012234340766 38.956981192890794 L 1976.3489177578517 38.96388501149703 L 1976.6856011749371 38.97078883010325 L 1977.0222845920225 38.97769264870948 L 1977.358968009108 38.98459646731571 L 1977.6956514261933 38.99150028592194 L 1978.0323348432787 38.99840410452816 L 1978.369018260364 39.00530792313439 L 1978.7057016774497 39.01221174174062 L 1979.0423850945351 39.01911556034685 L 1979.3790685116205 39.02601937895307 L 1979.715751928706 39.032923197559306 L 1980.0524353457913 39.03982701616553 L 1980.3891187628767 39.04673083477176 L 1980.725802179962 39.05363465337798 L 1981.0624855970475 39.06053847198421 L 1981.3991690141331 39.06744229059044 L 1981.7358524312185 39.074346109196675 L 1982.072535848304 39.0812499278029 L 1982.4092192653893 39.08815374640913 L 1982.7459026824747 39.09505756501535 L 1983.08258609956 39.10196138362158 L 1983.4192695166455 39.10886520222781 L 1983.7559529337311 39.11576902083404 L 1984.0926363508165 39.12267283944027 L 1984.429319767902 39.129576658046496 L 1984.7660031849873 39.13648047665272 L 1985.1026866020727 39.14338429525895 L 1985.439370019158 39.15028811386518 L 1985.7760534362435 39.157191932471406 L 1986.1127368533291 39.16409575107764 L 1986.4494202704145 39.170999569683865 L 1986.7861036875 39.17790338829009 L 1987.1227871045853 39.184807206896316 L 1987.4594705216707 39.19171102550254 L 1987.796153938756 39.198614844108775 L 1988.1328373558415 39.205518662715 L 1988.469520772927 39.21242248132123 L 1988.8062041900125 39.21932629992746 L 1989.142887607098 39.226230118533685 L 1989.4795710241833 39.23313393713991 L 1989.8162544412687 39.240037755746144 L 1990.152937858354 39.24694157435237 L 1990.4896212754395 39.253845392958596 L 1990.826304692525 39.26074921156482 L 1991.1629881096105 39.267653030171054 L 1991.499671526696 39.27455684877728 L 1991.8363549437813 39.28146066738351 L 1992.1730383608667 39.28836448598974 L 1992.509721777952 39.295268304595965 L 1992.8464051950375 39.30217212320219 L 1993.183088612123 39.309075941808416 L 1993.5197720292085 39.31597976041465 L 1993.856455446294 39.322883579020875 L 1994.1931388633793 39.32978739762711 L 1994.5298222804647 39.33669121623333 L 1994.86650569755 39.34359503483956 L 1995.2031891146355 39.350498853445785 L 1995.539872531721 39.35740267205201 L 1995.8765559488063 39.364306490658244 L 1996.213239365892 39.37121030926448 L 1996.5499227829773 39.3781141278707 L 1996.8866062000627 39.38501794647693 L 1997.223289617148 39.391921765083154 L 1997.5599730342335 39.39882558368938 L 1997.896656451319 39.40572940229561 L 1998.2333398684043 39.41263322090184 L 1998.57002328549 39.41953703950807 L 1998.9067067025753 39.4264408581143 L 1999.2433901196607 39.43334467672052 L 1999.580073536746 39.44024849532675 L 1999.9167569538315 39.44715231393298 L 2000.253440370917 39.45405613253921 L 2000.5901237880023 39.46095995114543 L 2000.9268072050877 39.46786376975166 L 2001.2634906221733 39.47476758835789 L 2001.6001740392587 39.48167140696412 L 2001.9368574563441 39.488575225570344 L 2002.2735408734295 39.49547904417658 L 2002.610224290515 39.5023828627828 L 2002.9469077076003 39.50928668138903 L 2003.2835911246857 39.516190499995254 L 2003.6202745417713 39.52309431860149 L 2003.9569579588567 39.52999813720771 L 2004.2936413759421 39.536901955813946 L 2004.6303247930275 39.54380577442017 L 2004.967008210113 39.5507095930264 L 2005.3036916271983 39.55761341163262 L 2005.6403750442837 39.56451723023885 L 2005.9770584613693 39.57142104884508 L 2006.3137418784547 39.578324867451315 L 2006.6504252955401 39.58522868605754 L 2006.9871087126255 39.592132504663766 L 2007.323792129711 39.59903632326999 L 2007.6604755467963 39.60594014187622 L 2007.9971589638817 39.61284396048245 L 2008.333842380967 39.61974777908868 L 2008.6705257980527 39.62665159769491 L 2009.0072092151381 39.633555416301135 L 2009.3438926322235 39.64045923490736 L 2009.680576049309 39.64736305351359 L 2010.0172594663943 39.65426687211981 L 2010.3539428834797 39.661170690726046 L 2010.690626300565 39.66807450933227 L 2011.0273097176507 39.674978327938504 L 2011.3639931347361 39.68188214654473 L 2011.7006765518215 39.688785965150956 L 2012.037359968907 39.69568978375718 L 2012.3740433859923 39.702593602363415 L 2012.7107268030777 39.70949742096964 L 2013.047410220163 39.716401239575866 L 2013.3840936372487 39.7233050581821 L 2013.7207770543341 39.730208876788325 L 2014.0574604714195 39.73711269539455 L 2014.394143888505 39.74401651400078 L 2014.7308273055903 39.75092033260701 L 2015.0675107226757 39.757824151213235 L 2015.404194139761 39.76472796981946 L 2015.7408775568465 39.77163178842569 L 2016.0775609739321 39.77853560703192 L 2016.4142443910175 39.785439425638145 L 2016.750927808103 39.79234324424438 L 2017.0876112251883 39.799247062850604",
          "type": "path"
         }
        ],
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "box": [
           {
            "boxpoints": "outliers",
            "notched": false,
            "type": "box"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "#C8D4E3",
             "linecolor": "#C8D4E3",
             "minorgridcolor": "#C8D4E3",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "line": {
              "width": 1
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             },
             "opacity": 0.8
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "#C8D4E3"
          },
          "height": 400,
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "margin": {
           "b": 2,
           "l": 2,
           "r": 2,
           "t": 30
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "#EBF0F8",
            "linecolor": "#EBF0F8",
            "ticks": ""
           }
          },
          "scene": {
           "camera": {
            "eye": {
             "x": 1.5,
             "y": 1.5,
             "z": 0.1
            }
           },
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "#DFE8F3",
            "gridwidth": 2,
            "linecolor": "#EBF0F8",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "#EBF0F8"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "#DFE8F3",
            "linecolor": "#A2B1C6",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.5
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "#EBF0F8",
           "linecolor": "#EBF0F8",
           "linewidth": 2,
           "showline": true,
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "#EBF0F8",
           "zerolinewidth": 2
          }
         }
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "year"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "min temperature"
         }
        }
       }
      }
     },
     "metadata": {}
    }
   ],
   "source": [
    "# Fit a model with SSD\n",
    "# Fit a model using sum of square differences\n",
    "model = MicrosoftCustomLinearRegressor().fit(X = dataset.year, \n",
    "                                             y = dataset.min_temperature, \n",
    "                                             cost_function = sum_of_absolute_differences)\n",
    "\n",
    "# Graph the model\n",
    "graphing.scatter_2D(dataset, \n",
    "                    label_x=\"year\", \n",
    "                    label_y=\"min_temperature\", \n",
    "                    trendline=model.predict)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5145f572",
   "metadata": {},
   "source": [
    "It's clear that this line passes through the majority of points much better than before, at the expense of almost ignoring the measurement taken in 1950. \n",
    "\n",
    "In our farming scenario, we're interested in how average temperatures are changing over time. We don't have much interest in 1950 specifically, so for us this is a better result. In other situations, of course, we might consider this result worse.\n",
    "\n",
    "\n",
    "## Summary\n",
    "\n",
    "In this exercise, you learned about how changing the cost function used during fitting can result in different final results. \n",
    "\n",
    "We also learned how this behaviour is because these cost functions describe the 'best' way to fit a model, although from a data analyst's point of view there can be drawbacks no matter which cost function is chosen."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "name": "python3710jvsc74a57bd0a970c7e51787c6ef7f3c805442fbb5a30ae2745635e1d2514e62d936ba85c55c",
   "display_name": "Python 3.7.10 64-bit ('ml_crash_course': conda)"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}