3.2 Regresi - Multiple Linear Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2145170b-ec69-4207-872f-bf7fd4059b6c",
   "metadata": {},
   "source": [
    "# Multiple Linear Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05efb0d8-b134-4aa7-9bfc-93b85436b57d",
   "metadata": {
    "tags": []
   },
   "source": [
    "- Jika SLR hanya menggunakan satu prediktor maka MLR menggunakan lebih dari satu prediktor \n",
    "- rumus SLR ...    **Y=βo+β1X1+....+βnXn**\n",
    "    - Y  = variable target\n",
    "    - X  = variable prediktor\n",
    "    - βo = konstanta, perpotongan garis regresi dengan sumbu Y (nilai estimasi jika x = 0)\n",
    "    - β1 = koefisien regresi (slope)\n",
    "    - n  = banyaknya hitungan "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56c19a63-f911-4e33-a270-ac1377c0d4ad",
   "metadata": {},
   "source": [
    "#### 1. Membaca Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "58813f58-c01f-4c31-999b-7d16bab29b54",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>R&amp;D Spend</th>\n",
       "      <th>Administration</th>\n",
       "      <th>Marketing Spend</th>\n",
       "      <th>State</th>\n",
       "      <th>Profit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>165349.20</td>\n",
       "      <td>136897.80</td>\n",
       "      <td>471784.10</td>\n",
       "      <td>New York</td>\n",
       "      <td>192261.83</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>162597.70</td>\n",
       "      <td>151377.59</td>\n",
       "      <td>443898.53</td>\n",
       "      <td>California</td>\n",
       "      <td>191792.06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>153441.51</td>\n",
       "      <td>101145.55</td>\n",
       "      <td>407934.54</td>\n",
       "      <td>Florida</td>\n",
       "      <td>191050.39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>144372.41</td>\n",
       "      <td>118671.85</td>\n",
       "      <td>383199.62</td>\n",
       "      <td>New York</td>\n",
       "      <td>182901.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>142107.34</td>\n",
       "      <td>91391.77</td>\n",
       "      <td>366168.42</td>\n",
       "      <td>Florida</td>\n",
       "      <td>166187.94</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   R&D Spend  Administration  Marketing Spend       State     Profit\n",
       "0  165349.20       136897.80        471784.10    New York  192261.83\n",
       "1  162597.70       151377.59        443898.53  California  191792.06\n",
       "2  153441.51       101145.55        407934.54     Florida  191050.39\n",
       "3  144372.41       118671.85        383199.62    New York  182901.99\n",
       "4  142107.34        91391.77        366168.42     Florida  166187.94"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('dataset/50_Startups.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebea9fb6-c4b8-4ab5-9a2e-5c5564f467a3",
   "metadata": {},
   "source": [
    "#### 2. Data Pre-processing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91629cf1-727b-4984-804a-58cc9a51594f",
   "metadata": {},
   "source": [
    "##### 2.1 mengetahui informasi data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "b3573992-0783-4965-988b-9b970f0ac882",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 50 entries, 0 to 49\n",
      "Data columns (total 5 columns):\n",
      " #   Column           Non-Null Count  Dtype  \n",
      "---  ------           --------------  -----  \n",
      " 0   R&D Spend        50 non-null     float64\n",
      " 1   Administration   50 non-null     float64\n",
      " 2   Marketing Spend  50 non-null     float64\n",
      " 3   State            50 non-null     object \n",
      " 4   Profit           50 non-null     float64\n",
      "dtypes: float64(4), object(1)\n",
      "memory usage: 2.1+ KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "488bb710-f134-4039-964a-99251e97bb67",
   "metadata": {},
   "source": [
    "**insight :**\n",
    "- terdapat 4 kolom numerik dan satu kolom kategorik\n",
    "- tidak ada null values\n",
    "- kolom kategorik harus diubah menjadi numerik melalui one-hot encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1085ed9-c8fe-4cff-b440-6daecae5f7d7",
   "metadata": {},
   "source": [
    "##### 2.2 Mentukan Variable Prediktor dan Variable Target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "138ded29-42dd-486a-bf83-38ec0b1daea6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "x = df.iloc[:,:-1] # prediktor\n",
    "y = df.iloc[:,4] #target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "bc6a3356-67d9-4db3-9e06-6a1669e99d55",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   R&D Spend  Administration  Marketing Spend       State\n",
      "0  165349.20       136897.80        471784.10    New York\n",
      "1  162597.70       151377.59        443898.53  California\n",
      "2  153441.51       101145.55        407934.54     Florida\n",
      "3  144372.41       118671.85        383199.62    New York\n",
      "4  142107.34        91391.77        366168.42     Florida\n"
     ]
    }
   ],
   "source": [
    "print(x.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "e9b89413-b31b-4e41-abff-030d966bfccc",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    192261.83\n",
      "1    191792.06\n",
      "2    191050.39\n",
      "3    182901.99\n",
      "4    166187.94\n",
      "Name: Profit, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "print(y.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e687633b-455b-4fa9-9362-349ac9b52cb6",
   "metadata": {},
   "source": [
    "setelah menentukan variable target dan prediktor, kolom kategorik harus diubah menjadi kolom numerik. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a21407b4-a074-4313-9711-b3c2334f1fed",
   "metadata": {},
   "source": [
    "##### 2.3 One-Hot Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc41bba7-0322-457c-9c8c-fef333924867",
   "metadata": {},
   "source": [
    "tahapan One-hot encoding adalah mengubah data kategorik menjadi numerik."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "825e6964-6822-4719-b502-a5f8a369d59f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Florida</th>\n",
       "      <th>New York</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Florida  New York\n",
       "0        0         1\n",
       "1        0         0\n",
       "2        1         0\n",
       "3        0         1\n",
       "4        1         0"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# pertama membuat dummy variable (variable contoh), berfungsi sebagai variable yang menyimpan data diskrit menjadi numerik\n",
    "states = pd.get_dummies(x['State'], drop_first=True) # drop_first akan menghapus kolom pertama dari variable dummy supaya terhinda dari variable dummy trap\n",
    "states = states.astype(int) \n",
    "states.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04c99c77-e44b-4d70-a48f-acd75d253268",
   "metadata": {},
   "source": [
    "setelah proses encoding selesai, maka `states` yang merupakan variable hasil encoding digabungkan dengan variable prediktor x "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "56a45bcd-5f40-4dca-b37a-883623e66700",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "x=pd.concat([x,states],axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "4248f314-f8f5-4c78-a8e2-970dfa8b7171",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   R&D Spend  Administration  Marketing Spend       State  Florida  New York\n",
      "0  165349.20       136897.80        471784.10    New York        0         1\n",
      "1  162597.70       151377.59        443898.53  California        0         0\n",
      "2  153441.51       101145.55        407934.54     Florida        1         0\n",
      "3  144372.41       118671.85        383199.62    New York        0         1\n",
      "4  142107.34        91391.77        366168.42     Florida        1         0\n"
     ]
    }
   ],
   "source": [
    "print(x.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c7ff31d-5278-41b1-af10-e4de76327312",
   "metadata": {},
   "source": [
    "Selain itu kita juga harus menghapus kolom `State` pada variable yang ada pada variable x karena tidak digunakan "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "be7575b7-b735-4ef1-a179-5ce58ffe13aa",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "x=x.drop('State',axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "e7fac393-cbca-488b-b225-63f8d0f3382a",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   R&D Spend  Administration  Marketing Spend  Florida  New York\n",
      "0  165349.20       136897.80        471784.10        0         1\n",
      "1  162597.70       151377.59        443898.53        0         0\n",
      "2  153441.51       101145.55        407934.54        1         0\n",
      "3  144372.41       118671.85        383199.62        0         1\n",
      "4  142107.34        91391.77        366168.42        1         0\n"
     ]
    }
   ],
   "source": [
    "print(x.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b818d2a-a58b-4684-8d5c-54decf01199f",
   "metadata": {},
   "source": [
    "#### 3. Splitting Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "e1f526c6-da58-4d48-aab2-732f1fff6957",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0.2, random_state = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42ae1d03-3517-41e8-9c77-3e405a04e31d",
   "metadata": {},
   "source": [
    "#### 4. Pemodelan Linear Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "3f9a4f65-35c1-4af6-a01c-e73b9cb4bf2b",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-2 {color: black;background-color: white;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LinearRegression()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LinearRegression</label><div class=\"sk-toggleable__content\"><pre>LinearRegression()</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "LinearRegression()"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regressor = LinearRegression()\n",
    "regressor.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2875894-640d-4cb9-b0e3-83e0450e30c4",
   "metadata": {},
   "source": [
    "#### 5. Melakukan prediksi data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "fec3499d-1479-4f19-81ca-f8e5b6fe665b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "y_pred = regressor.predict(x_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "c96b6369-3455-4438-a1af-2242b9552219",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Actual</th>\n",
       "      <th>Predicted</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>103282.38</td>\n",
       "      <td>103015.201598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>144259.40</td>\n",
       "      <td>132582.277608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>146121.95</td>\n",
       "      <td>132447.738452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>77798.83</td>\n",
       "      <td>71976.098513</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>191050.39</td>\n",
       "      <td>178537.482211</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>105008.31</td>\n",
       "      <td>116161.242302</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>81229.06</td>\n",
       "      <td>67851.692097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>97483.56</td>\n",
       "      <td>98791.733747</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>110352.25</td>\n",
       "      <td>113969.435330</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>166187.94</td>\n",
       "      <td>167921.065696</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Actual      Predicted\n",
       "28  103282.38  103015.201598\n",
       "11  144259.40  132582.277608\n",
       "10  146121.95  132447.738452\n",
       "41   77798.83   71976.098513\n",
       "2   191050.39  178537.482211\n",
       "27  105008.31  116161.242302\n",
       "38   81229.06   67851.692097\n",
       "31   97483.56   98791.733747\n",
       "22  110352.25  113969.435330\n",
       "4   166187.94  167921.065696"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame({'Actual': y_test, 'Predicted': y_pred})\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8b81ec1-88b4-41b1-b498-44757096f30b",
   "metadata": {},
   "source": [
    "#### 6. Melakukan Uji performa Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "fc4a6539-9f00-45a6-90aa-9fef15fd9926",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model R^2 Square value 0.9347068473282423\n"
     ]
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "print('Model R^2 Square value', metrics.r2_score(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3993e4e9-8eeb-439f-9b14-f46fb628c2d3",
   "metadata": {},
   "source": [
    "**insight :**\n",
    "- nilai R-square sebesar 93% menunjukan bahwa model regresi dapat dengan baik memprediksi data "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e4cf2ca-8e2e-45b0-b576-a7fb431debd4",
   "metadata": {},
   "source": [
    "#### 7. Analysis Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6647f85d-04fb-4452-8839-e08d27ae5d52",
   "metadata": {},
   "source": [
    "##### 7.1 Menggunakan 5 prediktor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "6e9ce4a7-ccb4-4e3e-8c95-fa6c7686f620",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>         <td>Profit</td>      <th>  R-squared:         </th> <td>   0.951</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.945</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   169.9</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Mon, 27 Nov 2023</td> <th>  Prob (F-statistic):</th> <td>1.34e-27</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>08:41:08</td>     <th>  Log-Likelihood:    </th> <td> -525.38</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>    50</td>      <th>  AIC:               </th> <td>   1063.</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>    44</td>      <th>  BIC:               </th> <td>   1074.</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>     5</td>      <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "         <td></td>            <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>const</th>           <td> 5.013e+04</td> <td> 6884.820</td> <td>    7.281</td> <td> 0.000</td> <td> 3.62e+04</td> <td>  6.4e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>R&D Spend</th>       <td>    0.8060</td> <td>    0.046</td> <td>   17.369</td> <td> 0.000</td> <td>    0.712</td> <td>    0.900</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Administration</th>  <td>   -0.0270</td> <td>    0.052</td> <td>   -0.517</td> <td> 0.608</td> <td>   -0.132</td> <td>    0.078</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Marketing Spend</th> <td>    0.0270</td> <td>    0.017</td> <td>    1.574</td> <td> 0.123</td> <td>   -0.008</td> <td>    0.062</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Florida</th>         <td>  198.7888</td> <td> 3371.007</td> <td>    0.059</td> <td> 0.953</td> <td>-6595.030</td> <td> 6992.607</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>New York</th>        <td>  -41.8870</td> <td> 3256.039</td> <td>   -0.013</td> <td> 0.990</td> <td>-6604.003</td> <td> 6520.229</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td>14.782</td> <th>  Durbin-Watson:     </th> <td>   1.283</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.001</td> <th>  Jarque-Bera (JB):  </th> <td>  21.266</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td>-0.948</td> <th>  Prob(JB):          </th> <td>2.41e-05</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 5.572</td> <th>  Cond. No.          </th> <td>1.45e+06</td>\n",
       "</tr>\n",
       "</table><br/><br/>Notes:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The condition number is large, 1.45e+06. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
      ],
      "text/latex": [
       "\\begin{center}\n",
       "\\begin{tabular}{lclc}\n",
       "\\toprule\n",
       "\\textbf{Dep. Variable:}    &      Profit      & \\textbf{  R-squared:         } &     0.951   \\\\\n",
       "\\textbf{Model:}            &       OLS        & \\textbf{  Adj. R-squared:    } &     0.945   \\\\\n",
       "\\textbf{Method:}           &  Least Squares   & \\textbf{  F-statistic:       } &     169.9   \\\\\n",
       "\\textbf{Date:}             & Mon, 27 Nov 2023 & \\textbf{  Prob (F-statistic):} &  1.34e-27   \\\\\n",
       "\\textbf{Time:}             &     08:41:08     & \\textbf{  Log-Likelihood:    } &   -525.38   \\\\\n",
       "\\textbf{No. Observations:} &          50      & \\textbf{  AIC:               } &     1063.   \\\\\n",
       "\\textbf{Df Residuals:}     &          44      & \\textbf{  BIC:               } &     1074.   \\\\\n",
       "\\textbf{Df Model:}         &           5      & \\textbf{                     } &             \\\\\n",
       "\\textbf{Covariance Type:}  &    nonrobust     & \\textbf{                     } &             \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "\\begin{tabular}{lcccccc}\n",
       "                         & \\textbf{coef} & \\textbf{std err} & \\textbf{t} & \\textbf{P$> |$t$|$} & \\textbf{[0.025} & \\textbf{0.975]}  \\\\\n",
       "\\midrule\n",
       "\\textbf{const}           &    5.013e+04  &     6884.820     &     7.281  &         0.000        &     3.62e+04    &      6.4e+04     \\\\\n",
       "\\textbf{R\\&D Spend}      &       0.8060  &        0.046     &    17.369  &         0.000        &        0.712    &        0.900     \\\\\n",
       "\\textbf{Administration}  &      -0.0270  &        0.052     &    -0.517  &         0.608        &       -0.132    &        0.078     \\\\\n",
       "\\textbf{Marketing Spend} &       0.0270  &        0.017     &     1.574  &         0.123        &       -0.008    &        0.062     \\\\\n",
       "\\textbf{Florida}         &     198.7888  &     3371.007     &     0.059  &         0.953        &    -6595.030    &     6992.607     \\\\\n",
       "\\textbf{New York}        &     -41.8870  &     3256.039     &    -0.013  &         0.990        &    -6604.003    &     6520.229     \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "\\begin{tabular}{lclc}\n",
       "\\textbf{Omnibus:}       & 14.782 & \\textbf{  Durbin-Watson:     } &    1.283  \\\\\n",
       "\\textbf{Prob(Omnibus):} &  0.001 & \\textbf{  Jarque-Bera (JB):  } &   21.266  \\\\\n",
       "\\textbf{Skew:}          & -0.948 & \\textbf{  Prob(JB):          } & 2.41e-05  \\\\\n",
       "\\textbf{Kurtosis:}      &  5.572 & \\textbf{  Cond. No.          } & 1.45e+06  \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "%\\caption{OLS Regression Results}\n",
       "\\end{center}\n",
       "\n",
       "Notes: \\newline\n",
       " [1] Standard Errors assume that the covariance matrix of the errors is correctly specified. \\newline\n",
       " [2] The condition number is large, 1.45e+06. This might indicate that there are \\newline\n",
       " strong multicollinearity or other numerical problems."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                 Profit   R-squared:                       0.951\n",
       "Model:                            OLS   Adj. R-squared:                  0.945\n",
       "Method:                 Least Squares   F-statistic:                     169.9\n",
       "Date:                Mon, 27 Nov 2023   Prob (F-statistic):           1.34e-27\n",
       "Time:                        08:41:08   Log-Likelihood:                -525.38\n",
       "No. Observations:                  50   AIC:                             1063.\n",
       "Df Residuals:                      44   BIC:                             1074.\n",
       "Df Model:                           5                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "===================================================================================\n",
       "                      coef    std err          t      P>|t|      [0.025      0.975]\n",
       "-----------------------------------------------------------------------------------\n",
       "const            5.013e+04   6884.820      7.281      0.000    3.62e+04     6.4e+04\n",
       "R&D Spend           0.8060      0.046     17.369      0.000       0.712       0.900\n",
       "Administration     -0.0270      0.052     -0.517      0.608      -0.132       0.078\n",
       "Marketing Spend     0.0270      0.017      1.574      0.123      -0.008       0.062\n",
       "Florida           198.7888   3371.007      0.059      0.953   -6595.030    6992.607\n",
       "New York          -41.8870   3256.039     -0.013      0.990   -6604.003    6520.229\n",
       "==============================================================================\n",
       "Omnibus:                       14.782   Durbin-Watson:                   1.283\n",
       "Prob(Omnibus):                  0.001   Jarque-Bera (JB):               21.266\n",
       "Skew:                          -0.948   Prob(JB):                     2.41e-05\n",
       "Kurtosis:                       5.572   Cond. No.                     1.45e+06\n",
       "==============================================================================\n",
       "\n",
       "Notes:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The condition number is large, 1.45e+06. This might indicate that there are\n",
       "strong multicollinearity or other numerical problems.\n",
       "\"\"\""
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import statsmodels.api as sm # menyediakan berbagai metode untuk melakukan analisis statistik dan regresi.\n",
    "\n",
    "# Menambahkan kolom konstanta ke array x. Kolom konstanta diperlukan untuk menghitung koefisien intersep dalam model regresi. \n",
    "# Fungsi add_constant dari statsmodels melakukan ini dengan menambahkan kolom berisi nilai konstan 1 ke array x.\n",
    "x_new = sm.add_constant(x)  \n",
    "\n",
    "# Memilih kolom-kolom yang akan digunakan dalam model regresi. \n",
    "#  Pada tahap awal, semua variabel independen mungkin dimasukkan ke dalam model. x_opt adalah array yang berisi kolom konstan dan kolom-kolom yang dipilih.\n",
    "x_opt = x_new.iloc[:, [0, 1, 2, 3, 4, 5]]\n",
    "\n",
    "# membuat model regresi linear berganda menggunakan metode Ordinary Least Squares (OLS) dari statsmodels\n",
    "# Parameter endog adalah variabel dependen (dalam hal ini, y), dan exog adalah variabel independen (dalam hal ini, x_opt)\n",
    "# metode fit() digunakan untuk menghitung koefisien regresi dan parameter lainnya.\n",
    "regressor_OLS = sm.OLS(endog = y, exog = x_opt).fit()\n",
    "\n",
    "# Menampilkan ringkasan hasil analisis regresi\n",
    "regressor_OLS.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b365ba5e-c766-49ac-8284-7956eda0e37a",
   "metadata": {},
   "source": [
    "**insight:**\n",
    "- P-value menyatakan adanya perbedaan dalam sebuah uji statistika. P-value kurang dari 0.005 maka akan semakin baik. \n",
    "- Pada nilai diatas p-value terbaik adalah kolom `R&D Spend`\n",
    "- sementara kolom `New York` adalah kolom tertinggi p-value nya"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "589b3843-9047-4701-aea8-b931997e9435",
   "metadata": {},
   "source": [
    "##### 7.2 Menggunakan 4 prediktor "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "ff020631-2f2f-43b2-ac51-cda7862c4c00",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>         <td>Profit</td>      <th>  R-squared:         </th> <td>   0.951</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.946</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   217.2</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Mon, 27 Nov 2023</td> <th>  Prob (F-statistic):</th> <td>8.49e-29</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>08:41:08</td>     <th>  Log-Likelihood:    </th> <td> -525.38</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>    50</td>      <th>  AIC:               </th> <td>   1061.</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>    45</td>      <th>  BIC:               </th> <td>   1070.</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>     4</td>      <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "         <td></td>            <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>const</th>           <td> 5.011e+04</td> <td> 6647.870</td> <td>    7.537</td> <td> 0.000</td> <td> 3.67e+04</td> <td> 6.35e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>R&D Spend</th>       <td>    0.8060</td> <td>    0.046</td> <td>   17.606</td> <td> 0.000</td> <td>    0.714</td> <td>    0.898</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Administration</th>  <td>   -0.0270</td> <td>    0.052</td> <td>   -0.523</td> <td> 0.604</td> <td>   -0.131</td> <td>    0.077</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Marketing Spend</th> <td>    0.0270</td> <td>    0.017</td> <td>    1.592</td> <td> 0.118</td> <td>   -0.007</td> <td>    0.061</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Florida</th>         <td>  220.1585</td> <td> 2900.536</td> <td>    0.076</td> <td> 0.940</td> <td>-5621.821</td> <td> 6062.138</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td>14.758</td> <th>  Durbin-Watson:     </th> <td>   1.282</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.001</td> <th>  Jarque-Bera (JB):  </th> <td>  21.172</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td>-0.948</td> <th>  Prob(JB):          </th> <td>2.53e-05</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 5.563</td> <th>  Cond. No.          </th> <td>1.40e+06</td>\n",
       "</tr>\n",
       "</table><br/><br/>Notes:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The condition number is large, 1.4e+06. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
      ],
      "text/latex": [
       "\\begin{center}\n",
       "\\begin{tabular}{lclc}\n",
       "\\toprule\n",
       "\\textbf{Dep. Variable:}    &      Profit      & \\textbf{  R-squared:         } &     0.951   \\\\\n",
       "\\textbf{Model:}            &       OLS        & \\textbf{  Adj. R-squared:    } &     0.946   \\\\\n",
       "\\textbf{Method:}           &  Least Squares   & \\textbf{  F-statistic:       } &     217.2   \\\\\n",
       "\\textbf{Date:}             & Mon, 27 Nov 2023 & \\textbf{  Prob (F-statistic):} &  8.49e-29   \\\\\n",
       "\\textbf{Time:}             &     08:41:08     & \\textbf{  Log-Likelihood:    } &   -525.38   \\\\\n",
       "\\textbf{No. Observations:} &          50      & \\textbf{  AIC:               } &     1061.   \\\\\n",
       "\\textbf{Df Residuals:}     &          45      & \\textbf{  BIC:               } &     1070.   \\\\\n",
       "\\textbf{Df Model:}         &           4      & \\textbf{                     } &             \\\\\n",
       "\\textbf{Covariance Type:}  &    nonrobust     & \\textbf{                     } &             \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "\\begin{tabular}{lcccccc}\n",
       "                         & \\textbf{coef} & \\textbf{std err} & \\textbf{t} & \\textbf{P$> |$t$|$} & \\textbf{[0.025} & \\textbf{0.975]}  \\\\\n",
       "\\midrule\n",
       "\\textbf{const}           &    5.011e+04  &     6647.870     &     7.537  &         0.000        &     3.67e+04    &     6.35e+04     \\\\\n",
       "\\textbf{R\\&D Spend}      &       0.8060  &        0.046     &    17.606  &         0.000        &        0.714    &        0.898     \\\\\n",
       "\\textbf{Administration}  &      -0.0270  &        0.052     &    -0.523  &         0.604        &       -0.131    &        0.077     \\\\\n",
       "\\textbf{Marketing Spend} &       0.0270  &        0.017     &     1.592  &         0.118        &       -0.007    &        0.061     \\\\\n",
       "\\textbf{Florida}         &     220.1585  &     2900.536     &     0.076  &         0.940        &    -5621.821    &     6062.138     \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "\\begin{tabular}{lclc}\n",
       "\\textbf{Omnibus:}       & 14.758 & \\textbf{  Durbin-Watson:     } &    1.282  \\\\\n",
       "\\textbf{Prob(Omnibus):} &  0.001 & \\textbf{  Jarque-Bera (JB):  } &   21.172  \\\\\n",
       "\\textbf{Skew:}          & -0.948 & \\textbf{  Prob(JB):          } & 2.53e-05  \\\\\n",
       "\\textbf{Kurtosis:}      &  5.563 & \\textbf{  Cond. No.          } & 1.40e+06  \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "%\\caption{OLS Regression Results}\n",
       "\\end{center}\n",
       "\n",
       "Notes: \\newline\n",
       " [1] Standard Errors assume that the covariance matrix of the errors is correctly specified. \\newline\n",
       " [2] The condition number is large, 1.4e+06. This might indicate that there are \\newline\n",
       " strong multicollinearity or other numerical problems."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                 Profit   R-squared:                       0.951\n",
       "Model:                            OLS   Adj. R-squared:                  0.946\n",
       "Method:                 Least Squares   F-statistic:                     217.2\n",
       "Date:                Mon, 27 Nov 2023   Prob (F-statistic):           8.49e-29\n",
       "Time:                        08:41:08   Log-Likelihood:                -525.38\n",
       "No. Observations:                  50   AIC:                             1061.\n",
       "Df Residuals:                      45   BIC:                             1070.\n",
       "Df Model:                           4                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "===================================================================================\n",
       "                      coef    std err          t      P>|t|      [0.025      0.975]\n",
       "-----------------------------------------------------------------------------------\n",
       "const            5.011e+04   6647.870      7.537      0.000    3.67e+04    6.35e+04\n",
       "R&D Spend           0.8060      0.046     17.606      0.000       0.714       0.898\n",
       "Administration     -0.0270      0.052     -0.523      0.604      -0.131       0.077\n",
       "Marketing Spend     0.0270      0.017      1.592      0.118      -0.007       0.061\n",
       "Florida           220.1585   2900.536      0.076      0.940   -5621.821    6062.138\n",
       "==============================================================================\n",
       "Omnibus:                       14.758   Durbin-Watson:                   1.282\n",
       "Prob(Omnibus):                  0.001   Jarque-Bera (JB):               21.172\n",
       "Skew:                          -0.948   Prob(JB):                     2.53e-05\n",
       "Kurtosis:                       5.563   Cond. No.                     1.40e+06\n",
       "==============================================================================\n",
       "\n",
       "Notes:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The condition number is large, 1.4e+06. This might indicate that there are\n",
       "strong multicollinearity or other numerical problems.\n",
       "\"\"\""
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_opt = x_new.iloc[:, [0, 1, 2, 3, 4]]\n",
    "regressor_OLS = sm.OLS(endog = y, exog = x_opt).fit()\n",
    "regressor_OLS.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2d31ebe-be2d-4dff-83a3-a27eaa75814a",
   "metadata": {},
   "source": [
    "**insight:**\n",
    "- kolom `Florida` perlu dihilangkan"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61c16233-df04-4341-9a9d-313d0800e339",
   "metadata": {
    "tags": []
   },
   "source": [
    "##### 7.3 Menggunakan 3 prediktor "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "4ae31958-43da-4045-8405-9f6269d7363f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>         <td>Profit</td>      <th>  R-squared:         </th> <td>   0.951</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.948</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   296.0</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Mon, 27 Nov 2023</td> <th>  Prob (F-statistic):</th> <td>4.53e-30</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>08:41:08</td>     <th>  Log-Likelihood:    </th> <td> -525.39</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>    50</td>      <th>  AIC:               </th> <td>   1059.</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>    46</td>      <th>  BIC:               </th> <td>   1066.</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>     3</td>      <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "         <td></td>            <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>const</th>           <td> 5.012e+04</td> <td> 6572.353</td> <td>    7.626</td> <td> 0.000</td> <td> 3.69e+04</td> <td> 6.34e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>R&D Spend</th>       <td>    0.8057</td> <td>    0.045</td> <td>   17.846</td> <td> 0.000</td> <td>    0.715</td> <td>    0.897</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Administration</th>  <td>   -0.0268</td> <td>    0.051</td> <td>   -0.526</td> <td> 0.602</td> <td>   -0.130</td> <td>    0.076</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Marketing Spend</th> <td>    0.0272</td> <td>    0.016</td> <td>    1.655</td> <td> 0.105</td> <td>   -0.006</td> <td>    0.060</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td>14.838</td> <th>  Durbin-Watson:     </th> <td>   1.282</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.001</td> <th>  Jarque-Bera (JB):  </th> <td>  21.442</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td>-0.949</td> <th>  Prob(JB):          </th> <td>2.21e-05</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 5.586</td> <th>  Cond. No.          </th> <td>1.40e+06</td>\n",
       "</tr>\n",
       "</table><br/><br/>Notes:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The condition number is large, 1.4e+06. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
      ],
      "text/latex": [
       "\\begin{center}\n",
       "\\begin{tabular}{lclc}\n",
       "\\toprule\n",
       "\\textbf{Dep. Variable:}    &      Profit      & \\textbf{  R-squared:         } &     0.951   \\\\\n",
       "\\textbf{Model:}            &       OLS        & \\textbf{  Adj. R-squared:    } &     0.948   \\\\\n",
       "\\textbf{Method:}           &  Least Squares   & \\textbf{  F-statistic:       } &     296.0   \\\\\n",
       "\\textbf{Date:}             & Mon, 27 Nov 2023 & \\textbf{  Prob (F-statistic):} &  4.53e-30   \\\\\n",
       "\\textbf{Time:}             &     08:41:08     & \\textbf{  Log-Likelihood:    } &   -525.39   \\\\\n",
       "\\textbf{No. Observations:} &          50      & \\textbf{  AIC:               } &     1059.   \\\\\n",
       "\\textbf{Df Residuals:}     &          46      & \\textbf{  BIC:               } &     1066.   \\\\\n",
       "\\textbf{Df Model:}         &           3      & \\textbf{                     } &             \\\\\n",
       "\\textbf{Covariance Type:}  &    nonrobust     & \\textbf{                     } &             \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "\\begin{tabular}{lcccccc}\n",
       "                         & \\textbf{coef} & \\textbf{std err} & \\textbf{t} & \\textbf{P$> |$t$|$} & \\textbf{[0.025} & \\textbf{0.975]}  \\\\\n",
       "\\midrule\n",
       "\\textbf{const}           &    5.012e+04  &     6572.353     &     7.626  &         0.000        &     3.69e+04    &     6.34e+04     \\\\\n",
       "\\textbf{R\\&D Spend}      &       0.8057  &        0.045     &    17.846  &         0.000        &        0.715    &        0.897     \\\\\n",
       "\\textbf{Administration}  &      -0.0268  &        0.051     &    -0.526  &         0.602        &       -0.130    &        0.076     \\\\\n",
       "\\textbf{Marketing Spend} &       0.0272  &        0.016     &     1.655  &         0.105        &       -0.006    &        0.060     \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "\\begin{tabular}{lclc}\n",
       "\\textbf{Omnibus:}       & 14.838 & \\textbf{  Durbin-Watson:     } &    1.282  \\\\\n",
       "\\textbf{Prob(Omnibus):} &  0.001 & \\textbf{  Jarque-Bera (JB):  } &   21.442  \\\\\n",
       "\\textbf{Skew:}          & -0.949 & \\textbf{  Prob(JB):          } & 2.21e-05  \\\\\n",
       "\\textbf{Kurtosis:}      &  5.586 & \\textbf{  Cond. No.          } & 1.40e+06  \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "%\\caption{OLS Regression Results}\n",
       "\\end{center}\n",
       "\n",
       "Notes: \\newline\n",
       " [1] Standard Errors assume that the covariance matrix of the errors is correctly specified. \\newline\n",
       " [2] The condition number is large, 1.4e+06. This might indicate that there are \\newline\n",
       " strong multicollinearity or other numerical problems."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                 Profit   R-squared:                       0.951\n",
       "Model:                            OLS   Adj. R-squared:                  0.948\n",
       "Method:                 Least Squares   F-statistic:                     296.0\n",
       "Date:                Mon, 27 Nov 2023   Prob (F-statistic):           4.53e-30\n",
       "Time:                        08:41:08   Log-Likelihood:                -525.39\n",
       "No. Observations:                  50   AIC:                             1059.\n",
       "Df Residuals:                      46   BIC:                             1066.\n",
       "Df Model:                           3                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "===================================================================================\n",
       "                      coef    std err          t      P>|t|      [0.025      0.975]\n",
       "-----------------------------------------------------------------------------------\n",
       "const            5.012e+04   6572.353      7.626      0.000    3.69e+04    6.34e+04\n",
       "R&D Spend           0.8057      0.045     17.846      0.000       0.715       0.897\n",
       "Administration     -0.0268      0.051     -0.526      0.602      -0.130       0.076\n",
       "Marketing Spend     0.0272      0.016      1.655      0.105      -0.006       0.060\n",
       "==============================================================================\n",
       "Omnibus:                       14.838   Durbin-Watson:                   1.282\n",
       "Prob(Omnibus):                  0.001   Jarque-Bera (JB):               21.442\n",
       "Skew:                          -0.949   Prob(JB):                     2.21e-05\n",
       "Kurtosis:                       5.586   Cond. No.                     1.40e+06\n",
       "==============================================================================\n",
       "\n",
       "Notes:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The condition number is large, 1.4e+06. This might indicate that there are\n",
       "strong multicollinearity or other numerical problems.\n",
       "\"\"\""
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_opt = x_new.iloc[:, [0, 1, 2, 3]]\n",
    "regressor_OLS = sm.OLS(endog = y, exog = x_opt).fit()\n",
    "regressor_OLS.summary()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2003eeb-3447-43f0-aab8-3d8deecf742b",
   "metadata": {},
   "source": [
    "**insight:**\n",
    "- kolom `Administration` dan `Marketing Spend` perlu dihilangkan"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "3140512a-d68f-4160-bcc8-55e7c1d98c6d",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>         <td>Profit</td>      <th>  R-squared:         </th> <td>   0.947</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.945</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   849.8</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Mon, 27 Nov 2023</td> <th>  Prob (F-statistic):</th> <td>3.50e-32</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>08:41:08</td>     <th>  Log-Likelihood:    </th> <td> -527.44</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>    50</td>      <th>  AIC:               </th> <td>   1059.</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>    48</td>      <th>  BIC:               </th> <td>   1063.</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>     1</td>      <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>const</th>     <td> 4.903e+04</td> <td> 2537.897</td> <td>   19.320</td> <td> 0.000</td> <td> 4.39e+04</td> <td> 5.41e+04</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>R&D Spend</th> <td>    0.8543</td> <td>    0.029</td> <td>   29.151</td> <td> 0.000</td> <td>    0.795</td> <td>    0.913</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td>13.727</td> <th>  Durbin-Watson:     </th> <td>   1.116</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.001</td> <th>  Jarque-Bera (JB):  </th> <td>  18.536</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td>-0.911</td> <th>  Prob(JB):          </th> <td>9.44e-05</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 5.361</td> <th>  Cond. No.          </th> <td>1.65e+05</td>\n",
       "</tr>\n",
       "</table><br/><br/>Notes:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The condition number is large, 1.65e+05. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
      ],
      "text/latex": [
       "\\begin{center}\n",
       "\\begin{tabular}{lclc}\n",
       "\\toprule\n",
       "\\textbf{Dep. Variable:}    &      Profit      & \\textbf{  R-squared:         } &     0.947   \\\\\n",
       "\\textbf{Model:}            &       OLS        & \\textbf{  Adj. R-squared:    } &     0.945   \\\\\n",
       "\\textbf{Method:}           &  Least Squares   & \\textbf{  F-statistic:       } &     849.8   \\\\\n",
       "\\textbf{Date:}             & Mon, 27 Nov 2023 & \\textbf{  Prob (F-statistic):} &  3.50e-32   \\\\\n",
       "\\textbf{Time:}             &     08:41:08     & \\textbf{  Log-Likelihood:    } &   -527.44   \\\\\n",
       "\\textbf{No. Observations:} &          50      & \\textbf{  AIC:               } &     1059.   \\\\\n",
       "\\textbf{Df Residuals:}     &          48      & \\textbf{  BIC:               } &     1063.   \\\\\n",
       "\\textbf{Df Model:}         &           1      & \\textbf{                     } &             \\\\\n",
       "\\textbf{Covariance Type:}  &    nonrobust     & \\textbf{                     } &             \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "\\begin{tabular}{lcccccc}\n",
       "                    & \\textbf{coef} & \\textbf{std err} & \\textbf{t} & \\textbf{P$> |$t$|$} & \\textbf{[0.025} & \\textbf{0.975]}  \\\\\n",
       "\\midrule\n",
       "\\textbf{const}      &    4.903e+04  &     2537.897     &    19.320  &         0.000        &     4.39e+04    &     5.41e+04     \\\\\n",
       "\\textbf{R\\&D Spend} &       0.8543  &        0.029     &    29.151  &         0.000        &        0.795    &        0.913     \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "\\begin{tabular}{lclc}\n",
       "\\textbf{Omnibus:}       & 13.727 & \\textbf{  Durbin-Watson:     } &    1.116  \\\\\n",
       "\\textbf{Prob(Omnibus):} &  0.001 & \\textbf{  Jarque-Bera (JB):  } &   18.536  \\\\\n",
       "\\textbf{Skew:}          & -0.911 & \\textbf{  Prob(JB):          } & 9.44e-05  \\\\\n",
       "\\textbf{Kurtosis:}      &  5.361 & \\textbf{  Cond. No.          } & 1.65e+05  \\\\\n",
       "\\bottomrule\n",
       "\\end{tabular}\n",
       "%\\caption{OLS Regression Results}\n",
       "\\end{center}\n",
       "\n",
       "Notes: \\newline\n",
       " [1] Standard Errors assume that the covariance matrix of the errors is correctly specified. \\newline\n",
       " [2] The condition number is large, 1.65e+05. This might indicate that there are \\newline\n",
       " strong multicollinearity or other numerical problems."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                 Profit   R-squared:                       0.947\n",
       "Model:                            OLS   Adj. R-squared:                  0.945\n",
       "Method:                 Least Squares   F-statistic:                     849.8\n",
       "Date:                Mon, 27 Nov 2023   Prob (F-statistic):           3.50e-32\n",
       "Time:                        08:41:08   Log-Likelihood:                -527.44\n",
       "No. Observations:                  50   AIC:                             1059.\n",
       "Df Residuals:                      48   BIC:                             1063.\n",
       "Df Model:                           1                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "const       4.903e+04   2537.897     19.320      0.000    4.39e+04    5.41e+04\n",
       "R&D Spend      0.8543      0.029     29.151      0.000       0.795       0.913\n",
       "==============================================================================\n",
       "Omnibus:                       13.727   Durbin-Watson:                   1.116\n",
       "Prob(Omnibus):                  0.001   Jarque-Bera (JB):               18.536\n",
       "Skew:                          -0.911   Prob(JB):                     9.44e-05\n",
       "Kurtosis:                       5.361   Cond. No.                     1.65e+05\n",
       "==============================================================================\n",
       "\n",
       "Notes:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The condition number is large, 1.65e+05. This might indicate that there are\n",
       "strong multicollinearity or other numerical problems.\n",
       "\"\"\""
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_opt = x_new.iloc[:, [0, 1]]\n",
    "regressor_OLS = sm.OLS(endog = y, exog = x_opt).fit()\n",
    "regressor_OLS.summary()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e037bf4-1032-432e-af82-84a39694cc4c",
   "metadata": {},
   "source": [
    "**insight:**\n",
    "- tersisa satu variable saja yaitu variable `R&D Spend` dimana memenuhi nilai p-values\n",
    "- persamaan yang dapat digunakan adalah y = 490303 + 0.8543X\n",
    "- artinya profit 50 perusahaan tergantung pada banyaknya biaya R&D. Semakin besar R&D maka semakin tinggi profit yang akan didapatkan "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91c9ae9b-9aa3-4692-a290-5dc5e6968c28",
   "metadata": {},
   "source": [
    "## Kelebihan dan Kekurangan Regresi"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d9645cc-73b0-4b2e-aec5-d3e26f07af23",
   "metadata": {},
   "source": [
    "### Kelebihan\n",
    "\n",
    "+ mudah diterapkan dan dijelaskan koefisiennya\n",
    "+ menjadi algoritma terbaik pada data linear karena kompleksitasnya yang lebih sedikit \n",
    "+ rentan terhadap outlier, namun dapat diatasi dengan standardisasi, dimensionality reduction, regularisasi atau cross validation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a53de85-4d81-4548-a077-f632a8333da9",
   "metadata": {},
   "source": [
    "### Kekurangan \n",
    "- jika gagal deteksi outlier maka sangat mempengaruhi model\n",
    "- algoritma ini tidak dapat menangkap seluruh variable prediktor yang mempengaruhi target"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "vulcan_da",
   "language": "python",
   "name": "vulcan_da"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}