Exercise_08_Forecasting.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
      "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n"
     ]
    },
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'statsmodels'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[1], line 10\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01msklearn\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mlinear_model\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m LinearRegression\n\u001b[1;32m      9\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01msklearn\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mmetrics\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m mean_squared_error\n\u001b[0;32m---> 10\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mstatsmodels\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mtsa\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mar_model\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m AutoReg\n\u001b[1;32m     11\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01mstatsmodels\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mtsa\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01marima\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mmodel\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m ARIMA\n\u001b[1;32m     12\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mwarnings\u001b[39;00m\n",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'statsmodels'"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from pandas.plotting import lag_plot, autocorrelation_plot\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from statsmodels.tsa.ar_model import AutoReg\n",
    "from statsmodels.tsa.arima.model import ARIMA\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Before this exercise, please install the library:**\n",
    "\n",
    "- statsmodels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 1: Forecasting\n",
    "\n",
    "In this exercise you need to forecast daily minimum temperatures using autoregression, ARIMA, and linear regression model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**a)** Import temperature time-series dataset **daily-minimum-temperatures.csv** in the data folder. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.read_csv('./data/daily-minimum-temperatures.csv',header=0, index_col=0)\n",
    "data.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualize the data\n",
    "####################\n",
    "# Your Code Here   #\n",
    "####################"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Quick check for autocorrelation using **[lag_plot()](https://pandas.pydata.org/docs/reference/api/pandas.plotting.lag_plot.html)**, **[corr()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.corr.html)**, and **[autocorrelation_plot()](https://pandas.pydata.org/docs/reference/api/pandas.plotting.autocorrelation_plot.html)**.\n",
    "\n",
    "Hint: Refer to **[Pearson correlation coefficient](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient)**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# plot the figures and print the correlation matrix.\n",
    "####################\n",
    "# Your Code Here   #\n",
    "####################"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a regression model, it would look as follows: $X(t+1) = b_0 + b_1 \\cdot X(t)+ b_2 \\cdot X(t-1)$. \n",
    "\n",
    "Predict the value at time step $t$ given the observation at the last two time steps (t and t-1)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**b)** Implement a 7-lag forecast AR model using [**AutoReg**](https://www.statsmodels.org/stable/generated/statsmodels.tsa.ar_model.AutoReg.html) and forecast the newest 14-day temperature  based on the historic dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = data.values\n",
    "train, test = X[0:len(X)-14], X[len(X)-14:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "lags = 7\n",
    "\n",
    "# train autoregression\n",
    "####################\n",
    "# Your Code Here   #\n",
    "####################\n",
    "\n",
    "# make predictions\n",
    "####################\n",
    "# Your Code Here   #\n",
    "####################\n",
    "\n",
    "# plot results\n",
    "####################\n",
    "# Your Code Here   #\n",
    "####################"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**c)** Implement **[ARIMA](https://www.statsmodels.org/stable/generated/statsmodels.tsa.arima.model.ARIMA.html)** model and grid search the optimal hyperparameters for the model.\n",
    "\n",
    "Hints:\n",
    "\n",
    "- Fit an ARIMA(p,d,q) model, which sets the lag value to p for autoregression, uses a difference order of d to make the time series stationary, and uses a moving average model of q.\n",
    "\n",
    "- A rolling forecast is required given the dependence on observations in prior time steps. A crude way to perform this rolling forecast is to re-create the ARIMA model after each new observation is received. We manually keep track of all observations in a list called history that is seeded with the training data and to which new observations are appended each iteration.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# It will takes a long time for training. You can adjust the grid search range according to your PC condition.\n",
    "ar_values = [6, 8]\n",
    "diff_order_values = [1, 2]\n",
    "ma_values = [3, 4]\n",
    "\n",
    "# return the lowest rmse, the corresponding parameters and predictions to\n",
    "# the best_score, best_params, best_predictions respectively.\n",
    "best_score, best_params, best_predictions = float(\"inf\"), None, None \n",
    "for p in ar_values: # p\n",
    "    for d in diff_order_values: # d\n",
    "        for q in ma_values: # q\n",
    "            order = (p,d,q)\n",
    "            history = train\n",
    "            predictions = []\n",
    "            for observation in test:\n",
    "                model = ARIMA(history,order=order).fit()\n",
    "                pred = model.forecast()[0]\n",
    "                predictions.append(pred)\n",
    "                history = np.append(history, observation)\n",
    "            rmse = np.sqrt(mean_squared_error(test, predictions))\n",
    "            if rmse < best_score:\n",
    "                best_score, best_cfg, best_predictions = rmse, order, predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# print the error\n",
    "print(f\"The best error: {best_score}\")\n",
    "\n",
    "# Plot the predictions of the best model\n",
    "plt.figure()\n",
    "plt.plot(best_predictions, label='predictions')\n",
    "plt.plot(test, label='test')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**d)** Generate a **7**-lag input dataset manually, i.e., use the previous 7 days data \\[x(t-7), x(t-6)...x(t-1)\\] as the input to predict the following day X(t) using a **[linear regression model](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html)**.\n",
    "\n",
    "Hint: Refer to the [pd.Dataframe.shift()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.shift.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lags = 7\n",
    "train, test = data[0:len(X)-21], data[len(X)-21:]\n",
    "\n",
    "# Generate the shifted training/test dataset\n",
    "\n",
    "####################\n",
    "# Your Code Here   #\n",
    "####################\n",
    "\n",
    "print(\"Training dataset:\")\n",
    "print(train_shifted.head(10))\n",
    "print(\"Test dataset:\")\n",
    "print(test_shifted.head(10))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Generate features (X) and targets (y) based on the shifted dataset.\n",
    "####################\n",
    "# Your Code Here   #\n",
    "####################\n",
    "\n",
    "print(train_X[:1])\n",
    "print(train_y[:1])\n",
    "print(test_X[:1])\n",
    "print(test_y[:1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Implement a **[linear regression model](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html)** and calculate the root-mean-square-error of the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "####################\n",
    "# Your Code Here   #\n",
    "####################\n",
    "print(f'Test RMSE: {error}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot the prediction in the test phase compared with the true cases.\n",
    "####################\n",
    "# Your Code Here   #\n",
    "####################\n",
    "\n",
    "# Print the coefficient\n",
    "####################\n",
    "# Your Code Here   #\n",
    "####################\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}