Lab12_Task.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "415f193b",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "042f15a2",
   "metadata": {},
   "source": [
    "Linear Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "502f87da",
   "metadata": {},
   "outputs": [],
   "source": [
    "def hypothesis(X , thetas):\n",
    "    return np.dot(X , thetas)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "7a5d20fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "def loss_function(actual , predicted):\n",
    "    return np.mean((actual - predicted)**2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "c6c80901",
   "metadata": {},
   "outputs": [],
   "source": [
    "def derivative_loss(X, Y, thetas):\n",
    "    \n",
    "    h = hypothesis(X, thetas)\n",
    "    difference = Y - h\n",
    "    r = np.dot(difference.T, X)\n",
    "    r = -2*r / float(X.shape[0])\n",
    "    \n",
    "    return r"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "919be8e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "def gradient_descent(o_thetas, g_thetas, alpha):\n",
    "    return (o_thetas - alpha * g_thetas)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77d16a03",
   "metadata": {},
   "source": [
    "#### Question 1 : For the given dataset implement gradient descent from scratch and find the linear relationship between features and output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "206614c3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TV</th>\n",
       "      <th>radio</th>\n",
       "      <th>newspaper</th>\n",
       "      <th>sales</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>230.1</td>\n",
       "      <td>37.8</td>\n",
       "      <td>69.2</td>\n",
       "      <td>22.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>44.5</td>\n",
       "      <td>39.3</td>\n",
       "      <td>45.1</td>\n",
       "      <td>10.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>17.2</td>\n",
       "      <td>45.9</td>\n",
       "      <td>69.3</td>\n",
       "      <td>9.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>151.5</td>\n",
       "      <td>41.3</td>\n",
       "      <td>58.5</td>\n",
       "      <td>18.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>180.8</td>\n",
       "      <td>10.8</td>\n",
       "      <td>58.4</td>\n",
       "      <td>12.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>196</th>\n",
       "      <td>38.2</td>\n",
       "      <td>3.7</td>\n",
       "      <td>13.8</td>\n",
       "      <td>7.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>197</th>\n",
       "      <td>94.2</td>\n",
       "      <td>4.9</td>\n",
       "      <td>8.1</td>\n",
       "      <td>9.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>198</th>\n",
       "      <td>177.0</td>\n",
       "      <td>9.3</td>\n",
       "      <td>6.4</td>\n",
       "      <td>12.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>199</th>\n",
       "      <td>283.6</td>\n",
       "      <td>42.0</td>\n",
       "      <td>66.2</td>\n",
       "      <td>25.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>200</th>\n",
       "      <td>232.1</td>\n",
       "      <td>8.6</td>\n",
       "      <td>8.7</td>\n",
       "      <td>13.4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>200 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        TV  radio  newspaper  sales\n",
       "1    230.1   37.8       69.2   22.1\n",
       "2     44.5   39.3       45.1   10.4\n",
       "3     17.2   45.9       69.3    9.3\n",
       "4    151.5   41.3       58.5   18.5\n",
       "5    180.8   10.8       58.4   12.9\n",
       "..     ...    ...        ...    ...\n",
       "196   38.2    3.7       13.8    7.6\n",
       "197   94.2    4.9        8.1    9.7\n",
       "198  177.0    9.3        6.4   12.8\n",
       "199  283.6   42.0       66.2   25.5\n",
       "200  232.1    8.6        8.7   13.4\n",
       "\n",
       "[200 rows x 4 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv('Advertising.csv', index_col=0)\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "69ec6070",
   "metadata": {},
   "outputs": [],
   "source": [
    "alpha = 0.00000001\n",
    "thetas = np.array([20, 40, 60, 80])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4fe5a9cf",
   "metadata": {},
   "outputs": [],
   "source": [
    "# for aoiding space mismatch\n",
    "ones = np.ones(data.shape[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "64253c57",
   "metadata": {},
   "outputs": [],
   "source": [
    "# X-Y Split\n",
    "\n",
    "X = data.iloc[: , :-1]\n",
    "X['One'] = ones\n",
    "X = np.array(X)\n",
    "\n",
    "Y = np.array(data.iloc[: , -1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "92541ff1",
   "metadata": {},
   "outputs": [],
   "source": [
    "def LinearRegression(X, Y, thetas, alpha):\n",
    "    \n",
    "    predictions = hypothesis(X , thetas)\n",
    "    MSE = loss_function(Y , predictions)\n",
    "    \n",
    "    g_thetas = derivative_loss(X, Y, thetas).T\n",
    "    new_thetas = gradient_descent(thetas, g_thetas, alpha)\n",
    "    \n",
    "    return new_thetas, MSE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "6916fba2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "MSE :  39174098.65425\n",
      "Updates Thetas:  [19.97992071 39.99697596 59.99577458 79.99988459]\n"
     ]
    }
   ],
   "source": [
    "# one time\n",
    "new_thetas, MSE = LinearRegression(X, Y, thetas, alpha)\n",
    "print(\"\\nMSE : \", MSE)\n",
    "print(\"Updates Thetas: \", new_thetas)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "50cfc712",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "MSE :  39131092.55089692\n",
      "Updates Thetas:  [19.95985364 39.99395343 59.99155118 79.99976924]\n",
      "\n",
      "MSE :  39088138.14525302\n",
      "Updates Thetas:  [19.9397988  39.99093242 59.98732979 79.99965395]\n",
      "\n",
      "MSE :  39045235.375016235\n",
      "Updates Thetas:  [19.91975617 39.98791292 59.98311041 79.99953873]\n",
      "\n",
      "MSE :  39002384.17795962\n",
      "Updates Thetas:  [19.89972574 39.98489494 59.97889305 79.99942357]\n",
      "\n",
      "MSE :  38959584.49193122\n",
      "Updates Thetas:  [19.87970751 39.98187848 59.97467769 79.99930847]\n"
     ]
    }
   ],
   "source": [
    "# 20 times\n",
    "for i in range(5):\n",
    "    \n",
    "    new_thetas, MSE = LinearRegression(X, Y, new_thetas, alpha)\n",
    "    \n",
    "    print(\"\\nMSE : \", MSE)\n",
    "    print(\"Updates Thetas: \", new_thetas)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "01ec06e2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Relationship betwen input and output : \n",
      "Old thetas :  [20 40 60 80]\n",
      "New thetas :  [19.87970751 39.98187848 59.97467769 79.99930847]\n"
     ]
    }
   ],
   "source": [
    "print(\"Relationship betwen input and output : \")\n",
    "print(\"Old thetas : \", thetas)\n",
    "print(\"New thetas : \", new_thetas)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f767feb6",
   "metadata": {},
   "source": [
    "Question 2 : Write a code that finds out the relationship between two linear arrays eg"
   ]
  },
  {
   "cell_type": "raw",
   "id": "b9555c40",
   "metadata": {},
   "source": [
    "o Input array x=[1,2,3,4,5,…]\n",
    "o Output array y=[3,5,7,9,11,…]\n",
    "o The code should work for any given input/output, use gradient descent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "7d89b2ba",
   "metadata": {},
   "outputs": [],
   "source": [
    "def main(X, Y):\n",
    "    \n",
    "    one = np.ones(len(X))\n",
    "    X = np.array([X, one]).T\n",
    "    Y = np.array(Y)\n",
    "    \n",
    "    alpha = 0.079\n",
    "    new_thetas = np.array([20, 40,])\n",
    "    \n",
    "    for i in range(100):\n",
    "        new_thetas, MSE = LinearRegression(X, Y, new_thetas, alpha)\n",
    "        \n",
    "    print('Minimum Loss : ', MSE)\n",
    "    print('Best Thetas : ', new_thetas)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "08c61be6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum Loss :  0.8535584902612404\n",
      "Best Thetas :  [1.41623141 3.10767563]\n"
     ]
    }
   ],
   "source": [
    "X = [1, 2, 3, 4, 5]\n",
    "Y = [3, 5, 7, 9, 11]\n",
    "\n",
    "main(X, Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d157694",
   "metadata": {},
   "source": [
    "Question 3: Is linear regression good for classification? If yes, then why, and if no, then why?"
   ]
  },
  {
   "cell_type": "raw",
   "id": "6068967f",
   "metadata": {},
   "source": [
    "Linear regression is a great algorithm but works best for continous values prediction, it is highly impacted by outliers. Hence we cannot use it to solve a classification problem."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "804fff7f",
   "metadata": {},
   "source": [
    "Question 4: Why do we use mse (mean squared error) over squared error? What is the benefit of mse?"
   ]
  },
  {
   "cell_type": "raw",
   "id": "d5b4db6d",
   "metadata": {},
   "source": [
    "MSE calculates the average squared difference between the observed and predicted values. When a model has no error, the MSE equals zero. As model error increases, its value increases. So it has direct relation with the loss. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5be872b",
   "metadata": {},
   "source": [
    "Question 5: Can you find a better estimator than mse?"
   ]
  },
  {
   "cell_type": "raw",
   "id": "fc8625a4",
   "metadata": {},
   "source": [
    "Since MSE is a function of the parameter, there will not be one “best” estimator in terms of MSE."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}