[unit1] undertand loss function and Hinge Loss

Unit 1 Linear Classifiers and Generalizations/02 Perceptron.ipynb

@@ -13,6 +13,8 @@
    "source": [
     "The Perceptron is a simple algorithm for binary classification. It's a type of linear classifier that makes its predictions based on a linear predictor function combining a set of weights with the feature vector.\n",
     "\n",
+    "Perceptron is used on the dataset which is linearly seperable (line)\n",
+    "\n",
     "Mathematically, the prediction of a Perceptron for an input vector `x` is given by:\n",
     "\n",
     "$$y = sign(w.x + b)$$\n",
@@ -43,24 +45,39 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 8,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Final w: [0. 2.]\n",
-      "Final b: 0\n"
+      "Prediction: 1, Actual: 1, xi: [-4  2], w: [0. 0.], b: 0\n",
+      "Prediction: 1, Actual: 1, xi: [-2  1], w: [0. 0.], b: 0\n",
+      "Prediction: 1, Actual: -1, xi: [-1 -1], w: [0. 0.], b: 0\n",
+      "Prediction: 1, Actual: -1, xi: [2 2], w: [1. 1.], b: -1\n",
+      "Prediction: -1, Actual: -1, xi: [ 1 -2], w: [-1. -1.], b: -2\n",
+      "Prediction: 1, Actual: 1, xi: [-4  2], w: [-1. -1.], b: -2\n",
+      "Prediction: -1, Actual: 1, xi: [-2  1], w: [-1. -1.], b: -2\n",
+      "Prediction: 1, Actual: -1, xi: [-1 -1], w: [-3.  0.], b: -1\n",
+      "Prediction: -1, Actual: -1, xi: [2 2], w: [-2.  1.], b: -2\n",
+      "Prediction: -1, Actual: -1, xi: [ 1 -2], w: [-2.  1.], b: -2\n",
+      "Prediction: 1, Actual: 1, xi: [-4  2], w: [-2.  1.], b: -2\n",
+      "Prediction: 1, Actual: 1, xi: [-2  1], w: [-2.  1.], b: -2\n",
+      "Prediction: -1, Actual: -1, xi: [-1 -1], w: [-2.  1.], b: -2\n",
+      "Prediction: -1, Actual: -1, xi: [2 2], w: [-2.  1.], b: -2\n",
+      "Prediction: -1, Actual: -1, xi: [ 1 -2], w: [-2.  1.], b: -2\n",
+      "Final w: [-2.  1.]\n",
+      "Final b: -2\n"
      ]
     }
    ],
    "source": [
     "import numpy as np\n",
     "\n",
     "# Define a simple binary classification dataset\n",
-    "X = np.array([[1, 1], [1, -1], [-1, -1], [-1, 1]]) # feature vector\n",
-    "y = np.array([1, -1, -1, 1]) # corresponding labels\n",
+    "X = np.array([[-4, 2], [-2, 1], [-1, -1], [2, 2], [1, -2]]) # feature vector\n",
+    "y = np.array([1, 1, -1, -1, -1]) # corresponding labels\n",
     "\n",
     "# Initialize w and b to zero\n",
     "w = np.zeros(X.shape[1])\n",
@@ -71,10 +88,11 @@
     "    return 1 if x >= 0 else -1\n",
     "\n",
     "# Perceptron learning algorithm\n",
-    "for _ in range(100):  # Limit iterations to prevent infinite loop\n",
+    "for _ in range(200):  # Limit iterations to prevent infinite loop\n",
     "    error_count = 0\n",
     "    for xi, yi in zip(X, y):\n",
     "        prediction = sign(np.dot(w, xi) + b)\n",
+    "        print(f\"Prediction: {prediction}, Actual: {yi}, xi: {xi}, w: {w}, b: {b}\")\n",
     "        if prediction != yi:\n",
     "            w += yi * xi\n",
     "            b += yi\n",