Lasso and Inductive Conformal Prediction Algorithm.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "458b66d2",
   "metadata": {},
   "source": [
    "# Lasso and Inductive Conformal Prediction Algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f62c21c9",
   "metadata": {},
   "source": [
    "Lasso for sklearn.datasets - load_diabetes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0c98ddfc",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import Lasso\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets import load_diabetes\n",
    "\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8102c0d",
   "metadata": {},
   "source": [
    "load diabetes dataset and splitting dataset using train_test_split in scikit-learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a9b99a9c",
   "metadata": {},
   "outputs": [],
   "source": [
    "diabetes = load_diabetes()\n",
    "X = diabetes.data\n",
    "y = diabetes.target\n",
    "X1_train,X1_test,y1_train,y1_test = train_test_split(X,y,random_state=2610)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f5aa517",
   "metadata": {},
   "source": [
    "Calculating Training and Test_R2 for the Lasso model using the default parameters and different parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2b2c9353",
   "metadata": {},
   "outputs": [],
   "source": [
    "lasso = Lasso().fit(X1_train,y1_train)\n",
    "lasso00=Lasso(alpha=0.1,max_iter=100000).fit(X1_train,y1_train)\n",
    "lasso01=Lasso(alpha=0.01,max_iter=100000).fit(X1_train,y1_train)\n",
    "lasso02=Lasso(alpha=0.001,max_iter=100000).fit(X1_train,y1_train)\n",
    "lasso03=Lasso(alpha=0.0001,max_iter=100000).fit(X1_train,y1_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea58446c",
   "metadata": {},
   "source": [
    "The default parameter for Lasso is alpha=1.0 and based on this we need to select iterations which is also a parameter of lasso"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6e8afc6e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.31543808091364745"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s=[]\n",
    "s.append(lasso.score(X1_test,y1_test))\n",
    "s[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ba96fa8",
   "metadata": {},
   "source": [
    "Train R^2 for alpha=0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "3be37812",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.45447531348213155"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.append(lasso00.score(X1_test,y1_test))\n",
    "s[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9a1afab",
   "metadata": {},
   "source": [
    "Train R^2 for alpha=0.01"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "2ab92cd5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4360653072280051"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.append(lasso01.score(X1_test,y1_test))\n",
    "s[2]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b69043d4",
   "metadata": {},
   "source": [
    "Train R^2 for alpha=0.001"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "f935667e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4345922057665438"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.append(lasso02.score(X1_test,y1_test))\n",
    "s[3]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f77acc3",
   "metadata": {},
   "source": [
    "Train R^2 for alpha=0.0001"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b71dc7d1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4354764053262594"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.append(lasso03.score(X1_test,y1_test))\n",
    "s[4]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4972528",
   "metadata": {},
   "source": [
    "Calculating how many features does this model use for default parameter and which."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ad7e8d89",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Feature selected:  2\n",
      "Feature selected:  3\n",
      "Feature selected:  8\n"
     ]
    }
   ],
   "source": [
    "for i in range(X.shape[1]):\n",
    "    if lasso.coef_[i] != 0:\n",
    "        print(\"Feature selected: \",i)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa562766",
   "metadata": {},
   "source": [
    "Calculating how many features does this model use for different parameter values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "c087139b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of selected feature for each alpha=[defalt_parameter,0.1,0.01,0.001,0.0001]:  [3, 6, 9, 9, 10]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "nz=[]\n",
    "\n",
    "nz.append(np.sum(lasso.coef_ != 0))\n",
    "nz.append(np.sum(lasso00.coef_ != 0))\n",
    "nz.append(np.sum(lasso01.coef_ != 0))\n",
    "nz.append(np.sum(lasso02.coef_ != 0))\n",
    "nz.append(np.sum(lasso03.coef_ != 0))\n",
    "print('Number of selected feature for each alpha=[defalt_parameter,0.1,0.01,0.001,0.0001]: ',nz)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "220b9ebb",
   "metadata": {},
   "source": [
    "Test R2 vs total number of features used for different alpha "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ff9cf5e4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAQB0lEQVR4nO3db5BV9X3H8c9nubiLZBmpXDLiH4hthrRl6srcQoytE2LMiHFApxlrRNumf5hOUqvpA7AdZ9JOedC0eUB5ksCYfx1EJyXSdmy0KqktDwh6UUygSDdR3KA0exmbAEnZuO63D+5F1mUXds85d/fsj/drZmfvn3PO78Od+/tw9pz7xxEhAECaOqY6AACgfSh5AEgYJQ8ACaPkASBhlDwAJKwymYPNmzcvFi1aNJlDAsC0t3fv3mMRUc2y7qSW/KJFi1Sv1ydzSACY9my/lnVdDtcAQMIoeQBIGCUPAAmj5AEgYZQ8ACTsvCVv+yu2+23vH3bbL9h+2nZv6/fc9sYEgLP1Hz+lOzbvVv+JU1MdpbTGsyf/NUk3j7jtAUk7I+L9kna2rgPApNq0s1fPH35Tm57pneoopeXxfNSw7UWSHo+IJa3rhyR9OCKO2r5M0rMRsfh826nVasHr5AHktfjBJzQwOHTW7Z2VDh3asHIKErWX7b0RUcuybtZj8u+NiKOS1Po9/xzh1tqu2643Go2MwwHAGbvWrdCqngXqmtmssK6ZHVrds0C71q+Y4mTl0/YTrxGxJSJqEVGrVjO9KxcA3mX+nC51d1Y0MDikzkqHBgaH1N1Z0fzurqmOVjpZP9bgR7YvG3a4pr/IUABwPsdODmjN8oW6a9lV2vZcnxqcfB1V1pL/F0m/K+lvWr//ubBEADAOm+85c4h6w21LpjBJuY3nJZSPSNotabHtI7b/QM1yv8l2r6SbWtcBACVz3j35iPjkGHfdWHAWAEDBeMcrACSMkgeAhFHyAJAwSh4AEkbJA0DCKHkASBglDwAJo+QBIGGUPAAkjJIHgIRR8gCQMEoeABJGyQNAwih5AEgYJQ8ACaPkASBhlDwAJIySB4CEUfIAkLBcJW/7Ptv7bR+wfX9BmQDgnPqPn9Idm3er/8Spd13G2TKXvO0lkv5I0jJJ10i61fb7iwoGAGPZtLNXzx9+U5ue6X3XZZytkmPdX5b0nYj4mSTZ/g9Jt0v62yKCAcBIix98QgODQ+9c37qn712Xt+7pU2elQ4c2rJyKeKWU53DNfkk32L7U9sWSbpF05ciFbK+1XbddbzQaOYYDcKHbtW6FVvUsUNfMZnXNsNTh5n1dMzu0umeBdq1fMYUJyydzyUfEQUmfl/S0pCclvSRpcJTltkRELSJq1Wo1c1AAmD+nS92dFQ0MDqmz0qG3QxoKqbPSoYHBIXV3VjS/u2uqY5ZKrhOvEfHliFgaETdIelMSB8UAtNWxkwNas3yhdnz6el05d5aunDtLOz59vdYsX6jGyYGpjlc6jojsK9vzI6Lf9lWSnpJ0XUT871jL12q1qNfrmccDgAuR7b0RUcuybp4Tr5L0TduXSnpL0mfOVfAAgMmXq+Qj4jeLCgIAKB7veAWAhFHyAJAwSh4AEkbJA0DCKHkASBglDwAJo+QBIGGUPAAkjJIHgIRR8gCQMEoeABJGyQNAwih5AEgYJQ8ACaPkASBhlDwAJIySB4CEUfIAkDBKHgASRskDQMJylbztz9o+YHu/7UdsdxUVDACQX+aSt325pD+VVIuIJZJmSLqzqGAAgPzyHq6pSJpluyLpYklv5I8EAChK5pKPiNclfUFSn6Sjkn4SEU+NXM72Wtt12/VGo5E9KQBgwvIcrpkrabWk90laIGm27btHLhcRWyKiFhG1arWaPSkAYMLyHK75qKRXI6IREW9JekzSh4qJBQAoQp6S75P0QdsX27akGyUdLCYWAKAIeY7J75G0XdILkr7X2taWgnIBAApQybNyRHxO0ucKygIAKBjveAWAhFHyAJAwSh4AEkbJA0DCKHkASBglDwAJo+QBIGGUPAAkjJIHgIRR8gCQMEoeABJGyQNAwih5AEgYJQ8ACaPkASBhlDwAJIySB4CEUfIAkDBKHgASlrnkbS+2vW/Yz3Hb9xeYDQCQU+Yv8o6IQ5J6JMn2DEmvS9pRTCwAQBGKOlxzo6QfRMRrBW0PAFCAokr+TkmPjHaH7bW267brjUajoOEAAOORu+RtXyRplaR/HO3+iNgSEbWIqFWr1bzDAQAmoIg9+ZWSXoiIHxWwLQBAgYoo+U9qjEM1AICplavkbV8s6SZJjxUTBwBQpMwvoZSkiPiZpEsLygIAKBjveAWAhFHyAJAwSh4AEkbJA0DCKHkASBglDwAJo+QBIGGUPAAkjJIHgIRR8gCQMEoeABJGyQNAwih5AEgYJQ8ACaPkASBhlDwAJIySB4CEUfIAkDBKHgASlveLvC+xvd32y7YP2r6uqGAAkIr+46d0x+bd6j9xatLHzrsn//eSnoyID0i6RtLB/JEAIC2bdvbq+cNvatMzvZM+tiMi24r2HEkvSbo6xrmRWq0W9Xo903gAMN0sfvAJDQwOnXV7Z6VDhzasHPd2bO+NiFqWDHn25K+W1JD0Vdsv2n7I9uxRwq21XbddbzQaOYYDgOll17oVWtWzQF0zm1XbNbNDq3sWaNf6FZOWIU/JVyQtlfTFiLhW0k8lPTByoYjYEhG1iKhVq9UcwwHA9DJ/Tpe6OysaGBxSZ6VDA4ND6u6saH5316RlqORY94ikIxGxp3V9u0YpeQC4kB07OaA1yxfqrmVXadtzfWpM8snXzCUfEf9j+4e2F0fEIUk3Svqv4qIBwPS3+Z4zh9I33LZk0sfPsycvSfdKetj2RZJekfSp/JEAAEXJVfIRsU9SpjO+AID24x2vAJAwSh4AEkbJA0DCKHkASBglDwAJo+QBIGGUPAAkjJIHgIRR8gCQMEoeABJGyQNAwih5AEgYJQ8ACaPkASBhlDwAJIySB4CEUfIAkDBKHgASRskDQMJyfcer7cOSTkh6W9JgRPB9rwBQIrlKvmVFRBwrYDsAgIJxuAYAEpa35EPSU7b32l472gK219qu2643Go2cwwEAJiJvyV8fEUslrZT0Gds3jFwgIrZERC0iatVqNedwAICJyFXyEfFG63e/pB2SlhURCgBQjMwlb3u27e7TlyV9TNL+ooIBAPLL8+qa90raYfv0drZFxJOFpAIAFCJzyUfEK5KuKTALAKBgvIQSABJGyQNAwih5AEgYJQ8ACaPkASBhlDwAJIySB4CEUfIAkDBKHgASRskDQMIoeQBIGCUPAAmj5AEgYZQ8ACSMkgeAhFHyAJAwSh4AEkbJA0DCKHkASBglDwAJy13ytmfYftH240UEAgAUp4g9+fskHSxgOwCAguUqedtXSPq4pIeKiQMAKFLePfmNktZJGhprAdtrbddt1xuNRs7hAAATkbnkbd8qqT8i9p5ruYjYEhG1iKhVq9WswwEAMsizJ3+9pFW2D0t6VNJHbG8tJBUAoBCZSz4i/jwiroiIRZLulPTtiLi7sGQAgNx4nTwAJKxSxEYi4llJzxaxLQBAcdiTB4CEUfIAkDBKHgASRskDQMIoeQBIGCUPAAmj5AEgYZQ8ACSMkgeAhFHyAJAwSh4AEkbJA0DCKHkASBglDwAJo+QBIGGUPAAkjJIHgIRR8gCQMEoeABKWueRtd9l+zvZLtg/Y/qsigw3Xf/yU7ti8W/0nTrVrCABIUp49+QFJH4mIayT1SLrZ9gcLSTXCpp29ev7wm9r0TG87Ng8AyapkXTEiQtLJ1tWZrZ8oItRpix98QgODQ+9c37qnT1v39Kmz0qFDG1YWORQAJCnXMXnbM2zvk9Qv6emI2DPKMmtt123XG43GhLa/a90KrepZoK6ZzZhdMzu0umeBdq1fkSc2AFwwcpV8RLwdET2SrpC0zPaSUZbZEhG1iKhVq9UJbX/+nC51d1Y0MDikzkqHBgaH1N1Z0fzurjyxAeCCkflwzXAR8WPbz0q6WdL+IrZ52rGTA1qzfKHuWnaVtj3XpwYnXwFg3DKXvO2qpLdaBT9L0kclfb6wZC2b76m9c3nDbWf9oQAAOIc8e/KXSfq67RlqHvb5RkQ8XkwsAEAR8ry65ruSri0wCwCgYLzjFQASRskDQMIoeQBIGCUPAAlz89MJJmkwuyHptTYPM0/SsTaPUSTyts90yiqRt92mc96FETGxd5O2TGrJTwbb9YionX/JciBv+0ynrBJ52+1CzcvhGgBIGCUPAAlLseS3THWACSJv+0ynrBJ52+2CzJvcMXkAwBkp7skDAFooeQBI2LQqeds32z5k+/u2Hxjl/tW2v2t7X+vbqH6jdfuVtv/d9sHWl47fV9asw+6fYftF25PyyZ558tq+xPZ22y+3HuPrSp73s63nwX7bj9hu+7fQnC/vsOV+3fbbtj8x0XXLkLeMc22srMNuL9VcG7bcaM+Fic+1iJgWP5JmSPqBpKslXSTpJUm/MmKZ9+jMeYZfk/Ry6/Jlkpa2LndL+u+R65Yl67D7/0zSNkmPl/mxbV3/uqQ/bF2+SNIlZc0r6XJJr0qa1br+DUm/N9V5hy33bUnfkvSJiaxborylm2tjZR12X6nm2rnyZplr02lPfpmk70fEKxHxc0mPSlo9fIGIOBmtf72k2Wp9sXhEHI2IF1qXT0g6qOZkL11WSbJ9haSPS3qojRkLyWt7jqQbJH25tdzPI+LHZc3bUpE0y3ZF0sWS3pjqvC33Svqmmt+ZPNF1S5G3jHNtrKxSOeday1l5s8616VTyl0v64bDrRzTKk8f27bZflvSvkn5/lPsXqfk5+Gd96XiB8mbdKGmdpKE2ZhwuT96rJTUkfbX1J+9DtmeXNW9EvC7pC5L6JB2V9JOIeGqq89q+XNLtkr400XXbIE/e4cssUgnm2nmyblTJ5to58maaa9Op5D3KbWe9/jMidkTEByTdJumv37UB+z1q/u94f0Qcb0fI00ONctu4stq+VVJ/ROxtY76R8jy2FUlLJX0xIq6V9FNJ7T5unOfxnavmntP7JC2QNNv23e2LKml8eTdKWh8Rb2dYt2h58jY3UK65tlGjZC3xXNuo0R/bTHOtkC/yniRHJF057PoVOsef2RHxn7Z/0fa8iDhme6aaT7qHI+KxsmaVdL2kVbZvkdQlaY7trRHRziLKk/eIpCMRcXpvbbvaX/J58q6Q9GpENCTJ9mOSPiRp6xTnrUl61LbU/GCqW2wPjnPdomXOGxH/VMK5NtZju1zlnGtj5f2Ossy1dp9oKOpHzf+QXlFzD+z0CYtfHbHML+nMybalkl5X839OS/oHSRvLnnXEMh/W5JwMypVX0i5Ji1uX/1LS35U1r5oT+4Cax+Kt5omse6c674jlv6YzJzIntG4J8pZuro2VdcTtpZlr58qbZa5Nmz35iBi0/SeS/k3NM89fiYgDtv+4df+XJP2WpN+x/Zak/5P02xERrZfP3SPpe7b3tTb5FxHxrbJlbUeeSch7r6SHbV+k5hP4UyXOu8f2dkkvSBqU9KLa/Hb3cead0LplzavmX6Jlm2ulUUDeCc81PtYAABI2nU68AgAmiJIHgIRR8gCQMEoeABJGyQNAwih5AEgYJQ8ACft/zOQEmk2jVP8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(s,nz,'*')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07bfcf87",
   "metadata": {},
   "source": [
    "According to above graph the most suitable parameter will be for 0.0001 as it is selecting 10 features out of 10 features and gives the best accuracy of 0.4354764053262594"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6268ff8",
   "metadata": {},
   "source": [
    "# Lasso method for diabetes.data (tab separated)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "a964ecbb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(442, 11)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diabetes=pd.read_csv('diabetes.data',sep='\\t')\n",
    "diabetes.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "8180aaef",
   "metadata": {},
   "outputs": [],
   "source": [
    "Xd=diabetes.values[:,:-1]\n",
    "yd=diabetes.values[:,-1]\n",
    "X_train,X_test,y_train,y_test = train_test_split(Xd,yd,random_state=2610)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e304a052",
   "metadata": {},
   "source": [
    "Calculating Train and Test R2 for the Lasso model using default parameters and different values of alpha"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "30ae5c19",
   "metadata": {},
   "outputs": [],
   "source": [
    "lasso1 = Lasso().fit(X_train,y_train)\n",
    "lasso10=Lasso(alpha=0.1,max_iter=100000).fit(X_train,y_train)\n",
    "lasso11=Lasso(alpha=0.01,max_iter=100000).fit(X_train,y_train)\n",
    "lasso12=Lasso(alpha=0.001,max_iter=100000).fit(X_train,y_train)\n",
    "lasso13=Lasso(alpha=0.0001,max_iter=100000).fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33cc6bcd",
   "metadata": {},
   "source": [
    "The default parameter for Lasso is alpha=1.0 and based on this we need to select iterations which is also a parameter of lasso"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "a63186bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "s1=[]\n",
    "s1.append(lasso1.score(X_test,y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b832c901",
   "metadata": {},
   "source": [
    "Train R^2 for alpha=0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "1acf8a9f",
   "metadata": {},
   "outputs": [],
   "source": [
    "s1.append(lasso10.score(X_test,y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a26701b",
   "metadata": {},
   "source": [
    "Train R^2 for alpha=0.01"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "b5754bc0",
   "metadata": {},
   "outputs": [],
   "source": [
    "s1.append(lasso11.score(X_test,y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d119f01",
   "metadata": {},
   "source": [
    "Train R^2 for alpha=0.001"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "f78493cb",
   "metadata": {},
   "outputs": [],
   "source": [
    "s1.append(lasso12.score(X_test,y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40a4b7e8",
   "metadata": {},
   "source": [
    "Train R^2 for alpha=0.0001"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "923c910a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.41290721446465406,\n",
       " 0.43413994373110765,\n",
       " 0.4354643080478602,\n",
       " 0.43559404222290565,\n",
       " 0.4356068022574697]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s1.append(lasso13.score(X_test,y_test))\n",
    "s1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "755c4f11",
   "metadata": {},
   "source": [
    "Calculating how many features does this model use for default parameter and which"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "c5c46052",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Feature selected:  AGE\n",
      "Feature selected:  SEX\n",
      "Feature selected:  BMI\n",
      "Feature selected:  BP\n",
      "Feature selected:  S1\n",
      "Feature selected:  S2\n",
      "Feature selected:  S3\n",
      "Feature selected:  S4\n",
      "Feature selected:  S5\n",
      "Feature selected:  S6\n"
     ]
    }
   ],
   "source": [
    "for i in range(Xd.shape[1]):\n",
    "    if lasso1.coef_[i] != 0:\n",
    "        print(\"Feature selected: \",diabetes.columns[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c69efe4",
   "metadata": {},
   "source": [
    "Calculating how many features does this model use for different parameter values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "b96be7bc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of selected feature for each alpha=[defalt_parameter,0.1,0.01,0.001,0.0001]:  [10, 10, 10, 10, 10]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "nz1=[]\n",
    "\n",
    "nz1.append(np.sum(lasso1.coef_ != 0))\n",
    "nz1.append(np.sum(lasso10.coef_ != 0))\n",
    "nz1.append(np.sum(lasso11.coef_ != 0))\n",
    "nz1.append(np.sum(lasso12.coef_ != 0))\n",
    "nz1.append(np.sum(lasso13.coef_ != 0))\n",
    "print('Number of selected feature for each alpha=[defalt_parameter,0.1,0.01,0.001,0.0001]: ',nz1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bfbb984",
   "metadata": {},
   "source": [
    "Test R2 vs total number of features used for different alpha"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "4cbbd930",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAOeElEQVR4nO3cbYylZX3H8e+vLGZBpCzsLC4PdtqGUpEI0pHaGJFIqNstKWqsKVhLDJGSQANNG137plZfFCU2vrAt2QphmxqICaRausGsKFASsB0si0s3uhQV113ZoWt9qBpc/ffF3JMZD2eezpnDLNd8P8nknHOf69z3NVduvnP2PjOkqpAktesXVnsCkqTRMvSS1DhDL0mNM/SS1DhDL0mNW7faE+hn48aNNT4+vtrTkKQXjUcfffTZqhrr99xRGfrx8XEmJydXexqS9KKR5BvzPeelG0lqnKGXpMYZeklqnKGXpMYZeklqnKGXpMYZeklqnKGXpMYZeklqnKGXpMYZeklqnKGXpMYZeklqnKGXpMYZeklqnKGXpMYZeklqnKGXpMYZeklqnKGXpMYZeklqnKGXpMYZeklqnKGXpMYZeklq3KKhT3JbkkNJ9szZdnKSXUn2dbcbFnj9MUn+M8k9KzVpSdLSLeUd/e3Alp5t24D7quos4L7u8XxuAPYONDtJ0tAWDX1VPQgc7tl8ObCju78DeEu/1yY5A/hd4BODT1GSNIxBr9GfWlUHAbrbTfOM+xjwXuBni+0wyTVJJpNMTk1NDTgtSVKvkX0Ym+Qy4FBVPbqU8VW1vaomqmpibGxsVNOSpDVn0NA/k2QzQHd7qM+Y1wO/l+TrwJ3Am5L804DHkyQNaNDQfwa4qrt/FfDp3gFV9f6qOqOqxoE/AD5fVX844PEkSQNayq9X3gE8DJydZH+Sq4GbgEuT7AMu7R6T5LQkO0c5YUnS8qxbbEBVXTHPU5f0GXsA2Npn+/3A/cucmyRpBfiXsZLUOEMvSY0z9JLUOEMvSY0z9JLUOEMvSY0z9JLUOEMvSY0z9JLUOEMvSY0z9JLUOEMvSY0z9JLUOEMvSY0z9JLUOEMvSY0z9JLUOEMvSY0z9JLUOEMvSY0z9JLUOEMvSY0z9JLUOEMvSY0z9JLUOEMvSY0z9JLUOEMvSY0z9JLUOEMvSY1bNPRJbktyKMmeOdtOTrIryb7udkOf152Z5AtJ9iZ5IskNKz15SdLilvKO/nZgS8+2bcB9VXUWcF/3uNcR4M+q6pXA64DrkpwzxFwlSQNYNPRV9SBwuGfz5cCO7v4O4C19Xnewqr7U3f8+sBc4fZjJSpKWb9Br9KdW1UGYDjqwaaHBScaB1wBfHPB4kqQBjfzD2CQnAHcBN1bV9xYYd02SySSTU1NTo56WJK0Zg4b+mSSbAbrbQ/0GJTmW6ch/sqruXmiHVbW9qiaqamJsbGzAaUmSeg0a+s8AV3X3rwI+3TsgSYBbgb1V9TcDHkeSNKSl/HrlHcDDwNlJ9ie5GrgJuDTJPuDS7jFJTkuys3vp64F3AW9K8lj3tXUk34UkaV7rFhtQVVfM89QlfcYeALZ29x8CMtTsJElD8y9jJalxhl6SGmfoJalxhl6SGmfoJalxhl6SGmfoJalxhl6SGmfoJalxhl6SGmfoJalxhl6SGmfoJalxhl6SGmfoJalxhl6SGmfoJalxhl6SGmfoJalxhl6SGmfoJalxhl6SGmfoJalxhl6SGmfoJalxhl6SGmfoJalxhl6SGmfoJalxhl6SGmfoJalxi4Y+yW1JDiXZM2fbyUl2JdnX3W6Y57VbknwlyZNJtq3kxPs6eBDe+Eb49rdHfihJa9xK9WZmP5/7HJx0Ejz++IpMb66lvKO/HdjSs20bcF9VnQXc1z3+OUmOAf4W+B3gHOCKJOcMNdvFfOhD8NBD8MEPjvQwkrRivZnZzzveAd/9Llx55crMb45U1eKDknHgnqo6t3v8FeDiqjqYZDNwf1Wd3fOa3wI+UFVv7h6/H6Cq/nqx401MTNTk5OTSv4vjjoMf//j529evhx/9aOn7kaTFrFRv5tvPXEvo84wkj1bVRL/nBr1Gf2pVHZyeRx0ENvUZczrwzTmP93fb5pvkNUkmk0xOTU0tbzZPPTX9U/D446cfH388vPOd8LWvLW8/krSYlerNzH7Wr3/+c+PjsHv30FOdMcoPY9Nn27w/nqpqe1VNVNXE2NjY8o60eTOceOL0T8f166dvTzwRXv7yZU5ZkhaxUr2Z2c9zzz3/uZe+FF796pWZL4OH/pnukg3d7aE+Y/YDZ855fAZwYMDjLWFGz8C118Ijj0zf+oGspFFZqd7M7GfjRtiwAV77WnjVq+Dw4RWd7qDX6G8G/qeqbup+m+bkqnpvz2vWAV8FLgG+BfwHcGVVPbHY8ZZ9jV6S1rihrtEnuQN4GDg7yf4kVwM3AZcm2Qdc2j0myWlJdgJU1RHgeuCzwF7gU0uJvCRpZa1bbEBVXTHPU5f0GXsA2Drn8U5g58CzkyQNzb+MlaTGGXpJapyhl6TGGXpJapyhl6TGGXpJapyhl6TGGXpJapyhl6TGGXpJapyhl6TGGXpJapyhl6TGGXpJapyhl6TGGXpJapyhl6TGGXpJapyhl6TGGXpJapyhl6TGGXpJapyhl6TGGXpJapyhl6TGGXpJapyhl6TGGXpJapyhl6TGGXpJatxQoU9yQ5I9SZ5IcmOf538xyb8k2d2Nefcwx5MkLd/AoU9yLvAe4ELgPOCyJGf1DLsO+K+qOg+4GPhokpcMekxJ0vIN847+lcAjVfXDqjoCPAC8tWdMAS9LEuAE4DBwZIhjSpKWaZjQ7wEuSnJKkuOBrcCZPWM+zvQPhAPAl4EbqupnQxxTkrRMA4e+qvYCHwZ2AfcCu3n+u/U3A48BpwHnAx9PcmK//SW5JslkksmpqalBpyVJ6jHUh7FVdWtVXVBVFzF9WWZfz5B3A3fXtCeBrwG/Ps++tlfVRFVNjI2NDTMtSdIcw/7Wzabu9hXA24A7eoY8DVzSjTkVOBt4aphjSpKWZ92Qr78rySnAT4Drquo7Sa4FqKpbgA8Btyf5MhDgfVX17JDHlCQtw1Chr6o39Nl2y5z7B4DfHuYYkqTh+JexktQ4Qy9JjTP0ktQ4Qy9JjTP0ktQ4Qy9JjTP0ktQ4Qy9JjTP0ktQ4Qy9JjTP0ktQ4Qy9JjTP0ktQ4Qy9JjTP0ktQ4Qy9JjTP0ktQ4Qy9JjTP0ktQ4Qy9JjTP0ktQ4Qy9JjTP0ktQ4Qy9JjTP0ktQ4Qy9JjTP0ktQ4Qy9JjTP0ktQ4Qy9JjTP0ktS4oUKf5IYke5I8keTGecZcnOSxbswDwxxPkrR86wZ9YZJzgfcAFwLPAfcm+deq2jdnzEnA3wFbqurpJJuGnK8kaZmGeUf/SuCRqvphVR0BHgDe2jPmSuDuqnoaoKoODXE8SdIAhgn9HuCiJKckOR7YCpzZM+bXgA1J7k/yaJI/mm9nSa5JMplkcmpqaohpSZLmGvjSTVXtTfJhYBfwA2A3cKTP/n8DuAQ4Dng4ySNV9dU++9sObAeYmJioQeclSfp5Q30YW1W3VtUFVXURcBjY1zNkP3BvVf1fVT0LPAicN8wxJUnLM+xv3Wzqbl8BvA24o2fIp4E3JFnXXd75TWDvMMeUJC3PwJduOnclOQX4CXBdVX0nybUAVXVLd3nnXuBx4GfAJ6pqz5DHlCQtw1Chr6o39Nl2S8/jm4GbhzmOJGlw/mWsJDXO0EtS4wy9JDXO0EtS4wy9JDXO0EtS4wy9JDXO0EtS4wy9JDXO0EtS4wy9JDXO0EtS4wy9JDXO0EtS4wy9JDXO0EtS4wy9JDXO0EtS4wy9JDXO0EtS4wy9JDXO0EtS4wy9JDXO0EtS41JVqz2H50kyBXxjteexRBuBZ1d7EkcJ12KWazHLtZg1yrX4paoa6/fEURn6F5Mkk1U1sdrzOBq4FrNci1muxazVWgsv3UhS4wy9JDXO0A9v+2pP4CjiWsxyLWa5FrNWZS28Ri9JjfMdvSQ1ztBLUuMM/RxJtiT5SpInk2xbYNxrk/w0ydvnbLstyaEke3rGfiDJt5I81n1tHeX3sFIGXYskZyb5QpK9SZ5IcsOcsScn2ZVkX3e74YX4XoY1orVYa+fF+iT/nmR3txZ/NWfsWjsvFlqL0ZwXVeXX9OcUxwD/DfwK8BJgN3DOPOM+D+wE3j5n+0XABcCenvEfAP58tb+/F2otgM3ABd39lwFfnXkt8BFgW3d/G/Dh1f5eV3Et1tp5EeCE7v6xwBeB163R82KhtRjJeeE7+lkXAk9W1VNV9RxwJ3B5n3F/AtwFHJq7saoeBA6PfJYvjIHXoqoOVtWXuvvfB/YCp3dPXw7s6O7vAN4yktmvrFGtxYvRMGtRVfWD7uGx3dfMb4KstfNiobUYCUM/63Tgm3Me76fnP8okpwNvBW5Z5r6vT/J4d3nnxfDP0hVZiyTjwGuYfscCcGpVHYTpCAKbVm7KIzOqtYA1dl4kOSbJY0xHb1dVrdnzYoG1gBGcF4Z+Vvps6/0p+zHgfVX102Xs9++BXwXOBw4CHx1kci+wodciyQlMv5O5saq+t7LTe0GNai3W3HlRVT+tqvOBM4ALk5y74jN84YxqLUZyXqxbiZ00Yj9w5pzHZwAHesZMAHcmgen/OdHWJEeq6p/n22lVPTNzP8k/APes1IRHaKi1SHIs02H7ZFXdPec1zyTZXFUHk2ym5/LXUWoka7EWz4uZAVX1v0nuB7YAe1iD58XMgN61GNl5sdofahwtX0z/0HsK+GVmP1x51QLjb2fOh7HdtnGe/2Hs5jn3/xS4c7W/11GuBdPvdP4R+FifcTfz8x+6fWS1v9dVXIu1dl6MASd1948D/g24bI2eFwutxUjOC9/Rd6rqSJLrgc8y/Un5bVX1RJJru+cXvC6f5A7gYmBjkv3AX1bVrcBHkpzP9D/rvg788ci+iRUy5Fq8HngX8OXuGiTAX1TVTuAm4FNJrgaeBn5/VN/DShnhWqy182IzsCPJMUxfMv5UVc28W11r58VCazGS88L/BYIkNc4PYyWpcYZekhpn6CWpcYZekhpn6CWpcYZekhpn6CWpcf8P8/TM2JKD4rMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(s1,nz1,'r*')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1929a78",
   "metadata": {},
   "source": [
    "According to above graph the all suitable parameter but 0.0001 is best as it selecting 10 features out of 10 features and gives the best accuracy of 0.4356068022574697]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "537059a9",
   "metadata": {},
   "source": [
    "# Preprocess the training and test sets using StandardScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "0f727483",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "scaler = StandardScaler()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "3a55ebfe",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "StandardScaler()"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scaler.fit(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "713d6a98",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_scaled = scaler.transform(X_train)\n",
    "X_test_scaled = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "4a821f75",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5337864254042757"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lasso2 = Lasso().fit(X_train_scaled,y_train)\n",
    "lasso2.score(X_train_scaled,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "586c45da",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.44905822102312876"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lasso2.score(X_test_scaled,y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aee28caf",
   "metadata": {},
   "source": [
    "# Choose the regularization parameter for the Lasso using cross-validation on the training set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "4fb543cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "param_grid = {'alpha': [0.001, 0.01, 0.1, 1, 10, 100]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "d8c20502",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "grid_search = GridSearchCV(Lasso(), param_grid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "2739fef6",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "E:\\app\\New folder\\lib\\site-packages\\sklearn\\linear_model\\_coordinate_descent.py:647: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 5.768e+03, tolerance: 1.587e+02\n",
      "  model = cd_fast.enet_coordinate_descent(\n",
      "E:\\app\\New folder\\lib\\site-packages\\sklearn\\linear_model\\_coordinate_descent.py:647: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 2.314e+04, tolerance: 1.463e+02\n",
      "  model = cd_fast.enet_coordinate_descent(\n",
      "E:\\app\\New folder\\lib\\site-packages\\sklearn\\linear_model\\_coordinate_descent.py:647: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 1.517e+04, tolerance: 1.445e+02\n",
      "  model = cd_fast.enet_coordinate_descent(\n",
      "E:\\app\\New folder\\lib\\site-packages\\sklearn\\linear_model\\_coordinate_descent.py:647: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 1.438e+03, tolerance: 1.537e+02\n",
      "  model = cd_fast.enet_coordinate_descent(\n",
      "E:\\app\\New folder\\lib\\site-packages\\sklearn\\linear_model\\_coordinate_descent.py:647: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 3.752e+02, tolerance: 1.463e+02\n",
      "  model = cd_fast.enet_coordinate_descent(\n",
      "E:\\app\\New folder\\lib\\site-packages\\sklearn\\linear_model\\_coordinate_descent.py:647: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 2.553e+02, tolerance: 1.445e+02\n",
      "  model = cd_fast.enet_coordinate_descent(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(estimator=Lasso(),\n",
       "             param_grid={'alpha': [0.001, 0.01, 0.1, 1, 10, 100]})"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid_search.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "9b10fb7d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.536818362103612"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid_search.score(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "fdb93c45",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.43413994373110765"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid_search.score(X_test,y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "24867c0f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'alpha': 0.1}\n"
     ]
    }
   ],
   "source": [
    "print(grid_search.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e3b0ad2",
   "metadata": {},
   "source": [
    "# Inductive Conformal Predictor"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b3b6e49",
   "metadata": {},
   "source": [
    "Splitting original training data into new training and calibration data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "93b2ac36",
   "metadata": {},
   "outputs": [],
   "source": [
    "new_xtrain,xcal,new_ytrain,ycal = train_test_split(X_train,y_train,test_size=0.299,random_state=2610)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0da3ccd",
   "metadata": {},
   "source": [
    "Preprossing data using StandardScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "a8f918e8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "StandardScaler()"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scaler1 = StandardScaler()\n",
    "scaler1.fit(new_xtrain)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "d9651085",
   "metadata": {},
   "outputs": [],
   "source": [
    "xcal_scaled = scaler1.transform(xcal)\n",
    "newxtrain_scaled = scaler1.transform(new_xtrain)\n",
    "test_scaled = scaler1.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d30c205d",
   "metadata": {},
   "source": [
    "Finding best parameter for Lasso and predicting preprossed calibration data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "092a7320",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "E:\\app\\New folder\\lib\\site-packages\\sklearn\\linear_model\\_coordinate_descent.py:647: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 1.073e+03, tolerance: 1.027e+02\n",
      "  model = cd_fast.enet_coordinate_descent(\n",
      "E:\\app\\New folder\\lib\\site-packages\\sklearn\\linear_model\\_coordinate_descent.py:647: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 1.941e+02, tolerance: 9.702e+01\n",
      "  model = cd_fast.enet_coordinate_descent(\n"
     ]
    }
   ],
   "source": [
    "grid_search1 = GridSearchCV(Lasso(), param_grid)\n",
    "grid_search1.fit(newxtrain_scaled,new_ytrain)\n",
    "\n",
    "lasso =Lasso(alpha=grid_search1.best_params_[\"alpha\"])\n",
    "lasso.fit(newxtrain_scaled,new_ytrain)\n",
    "\n",
    "y_pred_cal=lasso.predict(xcal_scaled)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50959dee",
   "metadata": {},
   "source": [
    "Using non-conformity measure α=|y-y^|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "b8ae423c",
   "metadata": {},
   "outputs": [],
   "source": [
    "new=[]\n",
    "for i in range(len(ycal)):\n",
    "    x=abs(ycal[i]-y_pred_cal[i])\n",
    "    new.append(x)\n",
    "new.sort()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a36e0c1",
   "metadata": {},
   "source": [
    "k values for significance levels 5% & 20% "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "c50d481e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "95.0 80.0\n"
     ]
    }
   ],
   "source": [
    "e=0.05\n",
    "k=(1-e)*(len(xcal)+1)\n",
    "e1=0.2\n",
    "k1=(1-e1)*(len(xcal)+1)\n",
    "print(k,k1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54f73ef5",
   "metadata": {},
   "source": [
    "Length of the half of prediction intervals for the test samples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "c7848144",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100.67402150143079 72.76203709441307\n"
     ]
    }
   ],
   "source": [
    "c1=new[int(k-1)]\n",
    "c2=new[int(k1-1)]\n",
    "print(c1,c2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f6f1186",
   "metadata": {},
   "source": [
    "Predicting for scaled test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "eae5aeb9",
   "metadata": {},
   "outputs": [],
   "source": [
    "yt_pred=lasso.predict(test_scaled)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f44723f",
   "metadata": {},
   "source": [
    "Calculating prediction intervals for each predicted scaled test "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "b8e094d7",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_test_interval=[]\n",
    "y_test_interval1=[]\n",
    "\n",
    "for i in range(len(yt_pred)):\n",
    "    y_test_interval.append([yt_pred[i]-c1,yt_pred[i]+c1])\n",
    "    y_test_interval1.append([yt_pred[i]-c2,yt_pred[i]+c2])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0c8578e",
   "metadata": {},
   "source": [
    "Error rate for y predicted labels for significance level 5%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "3c1f021b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.09909909909909909\n"
     ]
    }
   ],
   "source": [
    "acc=0\n",
    "for j in range(len(y_test_interval)):\n",
    "    if y_test[j]<=y_test_interval[j][1] and y_test[j]>=y_test_interval[j][0]:\n",
    "            acc=acc+1\n",
    "accuracy=acc/len(y_test)\n",
    "error_c1=1-accuracy\n",
    "print(error_c1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed0fa521",
   "metadata": {},
   "source": [
    "Error rate for y predicted labels for significance level 20%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "fb30d89f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.23423423423423428\n"
     ]
    }
   ],
   "source": [
    "acc1=0\n",
    "for j in range(len(y_test_interval1)):\n",
    "    if y_test[j]<=y_test_interval1[j][1] and y_test[j]>=y_test_interval1[j][0]:\n",
    "            acc1=acc1+1\n",
    "\n",
    "accuracy2=acc1/len(y_test)\n",
    "error_c2=1-accuracy2\n",
    "print(error_c2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bfdb7a0d",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}