Testing_and_Remediating_Bias_constrained.ipynb

{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "view-in-github"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/anilkumarpanda/fairness_cb/blob/development/Testing_and_Remediating_Bias_constrained.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "2c98ab72-1169-40d7-bf5d-fd7c0eb12a14",
      "metadata": {
        "id": "2c98ab72-1169-40d7-bf5d-fd7c0eb12a14"
      },
      "source": [
        "# Fairness Code Breakfast : Testing and Remediating Bias in an XGBoost Credit Decision Model.\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/anilkumarpanda/fairness_cb/development/img/adog.JPG\" alt=\"Alt Text\" height=\"400\">\n",
        "\n",
        "This AI generated dog image has nothing to do with Fairness, but I hope I have your attention now.\n",
        "\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "0KGo0YNaMNB0",
      "metadata": {
        "id": "0KGo0YNaMNB0"
      },
      "source": [
        "# Fairness in Machine Learning ?\n",
        "\n",
        "1. Is it important to consider fairness while developing models ?\n",
        "\n",
        "2. Can it be achieved via a model ?"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "tQjNaztO_Gyn",
      "metadata": {
        "id": "tQjNaztO_Gyn"
      },
      "source": [
        "Code breakfast outline :\n",
        "\n",
        "1. We’ll start off by training XGBoost on a variant of the credit card data.\n",
        "2. We’ll then test for bias by checking for differences in performance and outcomes across demographic groups.\n",
        "3. Once we confirm the existence of measurable levels of bias in our model predictions, we’ll start trying to fix, or remediate, that bias.\n",
        "4. We employ pre-processing methods that attempt to fix the training data, model, and outcomes, respectively.\n",
        "5. We’ll finish off by conducting bias-aware model selection that leaves us with a model that is both performant and more fair than the original model."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "4f31d8e1-ca2f-4081-afc6-7332dcc43bc3",
      "metadata": {
        "id": "4f31d8e1-ca2f-4081-afc6-7332dcc43bc3"
      },
      "source": [
        "## 1. Setting the environment\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "cl-o2WSxfjcR",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cl-o2WSxfjcR",
        "outputId": "a5a9914e-9a6b-4923-959c-fc43d768ff1c",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# Installing the libraries\n",
        "%pip install -q shap\n",
        "%pip install -q 'XGBoost==1.6'"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "3415e2e2-657c-461f-96ba-8771f4173587",
      "metadata": {
        "id": "3415e2e2-657c-461f-96ba-8771f4173587",
        "tags": []
      },
      "source": [
        "## 2. Evaluating an XGBoost Model\n",
        "\n",
        "\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "6cce747b-126f-4f98-b3c6-7f6096ac00c5",
      "metadata": {
        "id": "6cce747b-126f-4f98-b3c6-7f6096ac00c5"
      },
      "source": [
        "### 2.1 Train a Credit Decision Model\n",
        "\n",
        "Run the code/cells till #3. Detecting Bias without thinking too much. We need a model to start thinking about fairness."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5fcddefc-4827-4b7c-810c-85231a48bf06",
      "metadata": {
        "id": "5fcddefc-4827-4b7c-810c-85231a48bf06",
        "tags": []
      },
      "outputs": [],
      "source": [
        "import xgboost as xgb\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "from scipy import stats\n",
        "import sklearn\n",
        "import shap\n",
        "np.random.seed(42)\n",
        "import warnings\n",
        "warnings.filterwarnings('ignore')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "Webdy5NoeD_e",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Webdy5NoeD_e",
        "outputId": "0bf712d0-c9b0-418b-885a-5fb799852c85"
      },
      "outputs": [],
      "source": [
        "!wget https://raw.githubusercontent.com/anilkumarpanda/fairness_cb/development/credit_line_increase.csv"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "0a5d6485-4c74-43f1-aa64-c14efe0bcb22",
      "metadata": {
        "id": "0a5d6485-4c74-43f1-aa64-c14efe0bcb22",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# Read the dataset and convert string values to numerical.\n",
        "data = pd.read_csv('/content/credit_line_increase.csv')\n",
        "#data = pd.read_csv('credit_line_increase.csv')\n",
        "data['SEX'] = np.where(data['SEX'] == 1, 'male', 'female')\n",
        "race_map = {1: 'hispanic', 2: 'black', 3: 'white', 4: 'asian'}\n",
        "data['RACE'] = data['RACE'].apply(lambda x: race_map[x])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "32daad4e-2e51-4924-aa02-70c3c56a32a0",
      "metadata": {
        "id": "32daad4e-2e51-4924-aa02-70c3c56a32a0",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# Modify the data so there is a distributional difference\n",
        "# between borrowers of different race/ethnicities.\n",
        "\n",
        "new_limit_bal = data['LIMIT_BAL'] - 20000*np.random.randn(len(data))\n",
        "new_limit_bal[new_limit_bal <= 10000] = 10000\n",
        "data['LIMIT_BAL'] = np.where((data['RACE'] == 'hispanic') | (data['RACE'] == 'black'),\n",
        "                             new_limit_bal,\n",
        "                             data['LIMIT_BAL'])\n",
        "\n",
        "for i in range(1, 7):\n",
        "    delta = 1000*np.random.randn(len(data))\n",
        "    new_pay = data[f'PAY_AMT{i}'] - delta\n",
        "    new_pay[new_pay < 0] = 0\n",
        "\n",
        "    new_bill = data[f'BILL_AMT{i}'] - delta\n",
        "    new_bill[new_bill < 0] = 0\n",
        "\n",
        "    data[f'PAY_AMT{i}'] = np.where((data['RACE'] == 'hispanic') | (data['RACE'] == 'black'),\n",
        "                                   new_pay,\n",
        "                                   data[f'PAY_AMT{i}'])\n",
        "    data[f'BILL_AMT{i}'] = np.where((data['RACE'] == 'hispanic') | (data['RACE'] == 'black'),\n",
        "                                    new_bill,\n",
        "                                    data[f'BILL_AMT{i}'])\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7556f587-e2bf-4c22-b468-edbbb33d4f05",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7556f587-e2bf-4c22-b468-edbbb33d4f05",
        "outputId": "3e677898-c929-4522-d200-9a05da16ed12",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# Split the data into train validation and test\n",
        "seed = 12345\n",
        "np.random.seed(seed)\n",
        "\n",
        "split_train_test = 2/3\n",
        "\n",
        "split = np.random.rand(len(data)) < split_train_test\n",
        "train = data[split].copy()\n",
        "test = data[~split].copy()\n",
        "\n",
        "split_test_valid = 1/2\n",
        "\n",
        "split = np.random.rand(len(test)) < split_test_valid\n",
        "valid = test[split].copy()\n",
        "test = test[~split].copy()\n",
        "\n",
        "del data\n",
        "\n",
        "print(f\"Train/Validation/Test sizes: {len(train)}/{len(valid)}/{len(test)}\")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "IC4jgQusEacB",
      "metadata": {
        "id": "IC4jgQusEacB"
      },
      "source": [
        "Generally speaking, for most business applications, it’s safest not to use demographic information as model inputs. Not only is this legally risky in spaces like consumer credit, housing, and employment, it also implies that business decisions should be based on race or gender—and that’s dangerous territory. We will also not use these features in our model.\n",
        "\n",
        "It’s also true, however, that using demographic data in model training can decrease bias e.g certain kinds of decisions that should be based on demographic information, such as those about medical treatments.\n",
        "But this is not the case here.\n",
        "\n",
        "However, having this information is important to measure bias."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f7d5b7ad-871f-4d27-ab8e-da35e042250f",
      "metadata": {
        "id": "f7d5b7ad-871f-4d27-ab8e-da35e042250f",
        "tags": []
      },
      "outputs": [],
      "source": [
        "id_col = 'ID'\n",
        "groups = ['SEX', 'RACE', 'EDUCATION', 'MARRIAGE', 'AGE']\n",
        "target = 'DELINQ_NEXT'\n",
        "features = [col for col in train.columns if col not in groups + [id_col, target]]\n",
        "\n",
        "dtrain = xgb.DMatrix(train[features],\n",
        "                     label=train[target])\n",
        "\n",
        "dvalid = xgb.DMatrix(valid[features],\n",
        "                     label=valid[target])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a585bb90-9e27-4e29-a3b0-2c7221abc771",
      "metadata": {
        "id": "a585bb90-9e27-4e29-a3b0-2c7221abc771",
        "tags": []
      },
      "outputs": [],
      "source": [
        "corr = pd.DataFrame(train[features + [target]].corr(method='spearman')[target]).iloc[:-1]\n",
        "\n",
        "def get_monotone_constraints(data, target, corr_threshold):\n",
        "    \"\"\"Calculate monotonic constraints.\n",
        "    Monotonic constraints help in explainability.\n",
        "    It enforces the model to maintain certain relationship.\n",
        "\n",
        "    Using a cutoff on Spearman correlation between features and target,\n",
        "    return a tuple ready to pass into XGBoost.\n",
        "\n",
        "    Spearman correlation is nice because it considers monotonicity rather than\n",
        "    linearity (as is the case with Pearson correlation coefficient).\n",
        "\n",
        "\n",
        "    Args:\n",
        "        data (pd.DataFrame): A DataFrame containing the features in the order they appear to XGBoost, as well as the target variable.\n",
        "        target (str): The name of the column with the target variable in 'data'.\n",
        "        corr_threshold (float): The Spearman correlation threshold.\n",
        "\n",
        "    Returns:\n",
        "        tuple: A tuple with values in {-1, 0, 1}, where each element corresponds to a column in data (excluding the target itself). Ready to pass into xgb.train()\n",
        "\n",
        "    \"\"\"\n",
        "\n",
        "    corr = pd.Series(data.corr(method='spearman')[target]).drop(target)\n",
        "    monotone_constraints = tuple(np.where(corr < -corr_threshold,\n",
        "                                          -1,\n",
        "                                          np.where(corr > corr_threshold,\n",
        "                                                   1,\n",
        "                                                   0)))\n",
        "    return monotone_constraints\n",
        "\n",
        "correlation_cutoff = 0.1\n",
        "\n",
        "monotone_constraints = get_monotone_constraints(train[features+[target]],\n",
        "                                                target,\n",
        "                                                correlation_cutoff)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d02293f2-848c-4f1b-a3cd-0e5b79c62d70",
      "metadata": {
        "id": "d02293f2-848c-4f1b-a3cd-0e5b79c62d70",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# Feed the model the global bias\n",
        "# refers to the initial prediction value assigned to all\n",
        "# instances before the boosting process begins.\n",
        "# It acts as a starting point for the gradient boosting algorithm.\n",
        "\n",
        "base_score = train[target].mean()\n",
        "\n",
        "params = {\n",
        "    'objective': 'binary:logistic',\n",
        "    'eval_metric': 'auc',\n",
        "    'eta': 0.05,\n",
        "    'subsample': 0.6,\n",
        "    'colsample_bytree': 1.0,\n",
        "    'max_depth': 5,\n",
        "    'base_score': base_score,\n",
        "    'monotone_constraints': dict(zip(features, monotone_constraints)),\n",
        "    'seed': seed\n",
        "}\n",
        "\n",
        "# Train using early stopping on the validation dataset.\n",
        "watchlist = [(dtrain, 'train'), (dvalid, 'eval')]\n",
        "\n",
        "model_constrained = xgb.train(params,\n",
        "                              dtrain,\n",
        "                              num_boost_round=200,\n",
        "                              evals=watchlist,\n",
        "                              early_stopping_rounds=10,\n",
        "                              verbose_eval=False)\n",
        "\n",
        "train[f'p_{target}'] = model_constrained.predict(dtrain)\n",
        "valid[f'p_{target}'] = model_constrained.predict(dvalid)\n",
        "test[f'p_{target}'] = model_constrained.predict(xgb.DMatrix(test[features], label=test[target]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4dbcc992-de01-4568-9081-439097a39a73",
      "metadata": {
        "id": "4dbcc992-de01-4568-9081-439097a39a73",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# Select the optimal probability cutoff by maximizing the F1 score on validation data.\n",
        "# You can choose any other method as well.\n",
        "\n",
        "def perf_metrics(y_true, y_score, pos=1, neg=0, res=0.01):\n",
        "    \"\"\"\n",
        "    Calculates precision, recall, and f1 given outcomes and probabilities.\n",
        "\n",
        "    Args:\n",
        "        y_true: Array of binary outcomes\n",
        "        y_score: Array of assigned probabilities.\n",
        "        pos: Primary target value, default 1.\n",
        "        neg: Secondary target value, default 0.\n",
        "        res: Resolution by which to loop through cutoffs, default 0.01.\n",
        "\n",
        "    Returns:\n",
        "        Pandas dataframe of precision, recall, and f1 values.\n",
        "    \"\"\"\n",
        "\n",
        "    eps = 1e-20 # for safe numerical operations\n",
        "\n",
        "    # init p-r roc frame\n",
        "    prauc_frame = pd.DataFrame(columns=['cutoff', 'recall', 'precision', 'f1'])\n",
        "\n",
        "    # loop through cutoffs to create p-r roc frame\n",
        "    for cutoff in np.arange(0, 1 + res, res):\n",
        "\n",
        "        # binarize decision to create confusion matrix values\n",
        "        decisions = np.where(y_score > cutoff , 1, 0)\n",
        "\n",
        "        # calculate confusion matrix values\n",
        "        tp = np.sum((decisions == pos) & (y_true == pos))\n",
        "        fp = np.sum((decisions == pos) & (y_true == neg))\n",
        "        tn = np.sum((decisions == neg) & (y_true == neg))\n",
        "        fn = np.sum((decisions == neg) & (y_true == pos))\n",
        "\n",
        "        # calculate precision, recall, and f1\n",
        "        recall = (tp + eps)/((tp + fn) + eps)\n",
        "        precision = (tp + eps)/((tp + fp) + eps)\n",
        "        f1 = 2/((1/(recall + eps)) + (1/(precision + eps)))\n",
        "\n",
        "\n",
        "        # add new values to frame\n",
        "        prauc_frame = prauc_frame.append({'cutoff': cutoff,\n",
        "                                          'recall': recall,\n",
        "                                          'precision': precision,\n",
        "                                          'f1': f1},\n",
        "                                          ignore_index=True)\n",
        "\n",
        "    return prauc_frame\n",
        "\n",
        "\n",
        "model_metrics = perf_metrics(y_true=valid[target], y_score=model_constrained.predict(dvalid))\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ca0a2c92-f87d-4d6a-b9d9-fd6d64d53540",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ca0a2c92-f87d-4d6a-b9d9-fd6d64d53540",
        "outputId": "6ac25ef3-5db8-40ad-f80b-1f703a2f0e51",
        "tags": []
      },
      "outputs": [],
      "source": [
        "model_metrics.loc[model_metrics['f1'].idxmax()]"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "MHk5ZkJZHzHe",
      "metadata": {
        "id": "MHk5ZkJZHzHe"
      },
      "source": [
        "As we set the cutoff to 0.26 , all predictions above 0.26 are not going to get the credit line increase on offer. All predictions that are 0.26 or below will be accepted."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e1ddf410-ee18-477b-b363-6e9ed17303fc",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 466
        },
        "id": "e1ddf410-ee18-477b-b363-6e9ed17303fc",
        "outputId": "625f8fe0-73da-4350-b381-cbab662a471b",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# Since the Disparate Impact Analysis(DIA) analysis will focus on model outcomes\n",
        "# (rather than scores), choose a cutoff in probability space.\n",
        "\n",
        "best_cut = model_metrics.loc[model_metrics['f1'].idxmax(), 'cutoff']\n",
        "best_cut_original = best_cut\n",
        "\n",
        "fig, ax = plt.subplots()\n",
        "ax.plot(model_metrics['cutoff'], model_metrics['precision'], label='Precision',linestyle='--')\n",
        "ax.plot(model_metrics['cutoff'], model_metrics['recall'], label='Recall',linestyle=':')\n",
        "ax.plot(model_metrics['cutoff'], model_metrics['f1'], label='F1 Score',linestyle='-.')\n",
        "ax.legend(loc=3)\n",
        "ax.set_xlabel('Score Cutoff')\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "364ef8b5-e55b-4f0a-9258-8f053997b098",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 112
        },
        "id": "364ef8b5-e55b-4f0a-9258-8f053997b098",
        "outputId": "7014c82f-bbaa-41b4-99e3-511408854440",
        "tags": []
      },
      "outputs": [],
      "source": [
        "def get_confusion_matrix(frame, y, yhat, by=None, level=None, cutoff=0.5):\n",
        "    \"\"\"\n",
        "    Creates confusion matrix from pandas dataframe of y and yhat values,\n",
        "    can be sliced by a variable and level.\n",
        "\n",
        "    Args:\n",
        "        frame: Pandas dataframe of actual (y) and predicted (yhat) values.\n",
        "        y: Name of actual value column.\n",
        "        yhat: Name of predicted value column.\n",
        "        by: By variable to slice frame before creating confusion matrix, default None.\n",
        "        level: Value of by variable to slice frame before creating confusion matrix, default None.\n",
        "        cutoff: Cutoff threshold for confusion matrix, default 0.5.\n",
        "\n",
        "    Returns:\n",
        "        Confusion matrix as pandas dataframe.\n",
        "    \"\"\"\n",
        "\n",
        "    # determine levels of target (y) variable\n",
        "    # sort for consistency\n",
        "    level_list = list(frame[y].unique())\n",
        "    level_list.sort(reverse=True)\n",
        "\n",
        "    # init confusion matrix\n",
        "    cm_frame = pd.DataFrame(columns=['actual: ' +  str(i) for i in level_list],\n",
        "                            index=['predicted: ' + str(i) for i in level_list])\n",
        "\n",
        "    # don't destroy original data\n",
        "    frame_ = frame.copy(deep=True)\n",
        "\n",
        "    # convert numeric predictions to binary decisions using cutoff\n",
        "    dname = 'd_' + str(y)\n",
        "    frame_[dname] = np.where(frame_[yhat] > cutoff , 1, 0)\n",
        "\n",
        "    # slice frame\n",
        "    if (by is not None) & (level is not None):\n",
        "        frame_ = frame_[frame[by] == level]\n",
        "\n",
        "    # calculate size of each confusion matrix value\n",
        "    for i, lev_i in enumerate(level_list):\n",
        "        for j, lev_j in enumerate(level_list):\n",
        "            cm_frame.iat[j, i] = frame_[(frame_[y] == lev_i) & (frame_[dname] == lev_j)].shape[0]\n",
        "\n",
        "    return cm_frame\n",
        "\n",
        "get_confusion_matrix(test, target, f'p_{target}', cutoff=best_cut)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "bHgxaXnOM78u",
      "metadata": {
        "id": "bHgxaXnOM78u"
      },
      "source": [
        "## 3. Detecting Bias\n",
        "\n",
        "\n",
        "Imagine that the model you created above will be used provide loans. So it is important to measure the models for fairness. Incase it is found that the model has bias, we can also look at some bias mitigating technique.\n",
        "\n",
        "Lets first start by measuring bias.\n",
        "In the domain of fairness, there a lot of metrics that can be considered depending on the problem.\n",
        "\n",
        "Some of them include :\n",
        "\n",
        "- Prevalence: '(tp + fn) / (tp + tn +fp + fn)', # How much default actually happens for this group\n",
        "\n",
        "- Accuracy: '(tp + tn) / (tp + tn + fp + fn)', # how often the model predicts default and non-default correctly for this group\n",
        "\n",
        "- True Positive Rate: 'tp / (tp + fn)',  # out of the people in the group *that did* default, how many the model predicted *correctly* would default\n",
        "\n",
        "- Precision: 'tp / (tp + fp)',  # out of the people in the group the model *predicted* would default, how many the model predicted *correctly* would default\n",
        "\n",
        "- Specificity: 'tn / (tn + fp)', # out of the people in the group *that did not* default, how many the model predicted *correctly* would not default\n",
        "\n",
        "- Negative Predicted Value: 'tn / (tn + fn)', # out of the people in the group the model *predicted* would not default, how many the model predicted *correctly* would not default\n",
        "\n",
        "- False Positive Rate: 'fp / (tn + fp)', # out of the people in the group *that did not* default, how many the model predicted *incorrectly* would default\n",
        "\n",
        "- False Discovery Rate: 'fp / (tp + fp)', # out of the people in the group the model *predicted* would default, how many the model predicted *incorrectly* would default\n",
        "\n",
        "- False Negative Rate: 'fn / (tp + fn)', # out of the people in the group *that did* default, how many the model predicted *incorrectly* would not default\n",
        "\n",
        "- False Omissions Rate: 'fn / (tn + fn)'  # out of the people in the group the model *predicted* would not default, how many the model predicted *incorrectly* would not default\n",
        "\n",
        "\n",
        "Are you feeling overwhelmed with all these metrics ??\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/anilkumarpanda/fairness_cb/development/img/blue_cat.JPG\" alt=\"Sad Bleu Cat\" height=\"400\">\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "<img src=\"https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTkPZ1nf-X_zq_stiiNqv7opkVZU7wEJsMcjAhJSrq2ug&s\" alt=\"Andrew Ng\" height=\"400\">\n",
        "\n",
        "\n",
        "The Fairness Decision Tree can be a good starting point. ![fairness decision tree](https://raw.githubusercontent.com/anilkumarpanda/fairness_cb/development/fairness_tree.png)\n",
        "\n",
        "\n",
        "Other more practical metrics include AIR,SMD etc. These are likely more aligned to legal standards and are used to analyze differences in outcomes across groups, using traditional measures of statistical and practical significance. \n",
        "We’ll pair two well-known practical bias-testing measures, AIR and SMD, with chi-squared and t-tests, respectively. \n",
        "\n",
        "AIR : Adverse Impact Ratio\n",
        "\n",
        "SMD : Statistical Measures of Disparate Impact.\n",
        "\n",
        "We will cover these definitions later in the notebook."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "88c65147-1edb-476d-adc6-40bc01ff1be9",
      "metadata": {
        "id": "88c65147-1edb-476d-adc6-40bc01ff1be9"
      },
      "source": [
        "#### Confusion Matrix with Disparity Metrics\n",
        "\n",
        "Similar to normal confusion matrix, we can also create a confusion matrix with various disparity metrics as well.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5ca6de54-cdab-48f0-9127-d25f786f2f07",
      "metadata": {
        "id": "5ca6de54-cdab-48f0-9127-d25f786f2f07",
        "tags": []
      },
      "outputs": [],
      "source": [
        "metric_dict = {\n",
        "'Prevalence': '(tp + fn) / (tp + tn +fp + fn)', # how much default actually happens for this group\n",
        "'Accuracy': '(tp + tn) / (tp + tn + fp + fn)', # how often the model predicts default and non-default correctly for this group\n",
        "'True Positive Rate': 'tp / (tp + fn)',  # out of the people in the group *that did* default, how many the model predicted *correctly* would default\n",
        "'Precision': 'tp / (tp + fp)',  # out of the people in the group the model *predicted* would default, how many the model predicted *correctly* would default\n",
        "'Specificity': 'tn / (tn + fp)', # out of the people in the group *that did not* default, how many the model predicted *correctly* would not default\n",
        "'Negative Predicted Value': 'tn / (tn + fn)', # out of the people in the group the model *predicted* would not default, how many the model predicted *correctly* would not default\n",
        "\n",
        "'False Positive Rate': 'fp / (tn + fp)', # out of the people in the group *that did not* default, how many the model predicted *incorrectly* would default\n",
        "'False Discovery Rate': 'fp / (tp + fp)', # out of the people in the group the model *predicted* would default, how many the model predicted *incorrectly* would default\n",
        "\n",
        "'False Negative Rate': 'fn / (tp + fn)', # out of the people in the group *that did* default, how many the model predicted *incorrectly* would not default\n",
        "'False Omissions Rate': 'fn / (tn + fn)'  # out of the people in the group the model *predicted* would not default, how many the model predicted *incorrectly* would not default\n",
        "}\n",
        "\n",
        "\n",
        "def confusion_matrix_parser(expression):\n",
        "\n",
        "    # tp | fp       cm_dict[level].iat[0, 0] | cm_dict[level].iat[0, 1]\n",
        "    # -------  ==>  --------------------------------------------\n",
        "    # fn | tn       cm_dict[level].iat[1, 0] | cm_dict[level].iat[1, 1]\n",
        "\n",
        "    expression = expression.replace('tp', 'cm_dict[level].iat[0, 0]')\\\n",
        "                           .replace('fp', 'cm_dict[level].iat[0, 1]')\\\n",
        "                           .replace('fn', 'cm_dict[level].iat[1, 0]')\\\n",
        "                           .replace('tn', 'cm_dict[level].iat[1, 1]')\n",
        "\n",
        "    return expression"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4b106abf-4ab1-4a4c-bedd-1421ecaaaee2",
      "metadata": {
        "id": "4b106abf-4ab1-4a4c-bedd-1421ecaaaee2",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# initialize dict of confusion matrices and corresponding rows of dataframe\n",
        "sex_confusion_mats = {'male': get_confusion_matrix(test, target,\n",
        "                                                   f'p_{target}', by='SEX',\n",
        "                                                   level='male', cutoff=best_cut),\n",
        "                      'female': get_confusion_matrix(test, target,\n",
        "                                                     f'p_{target}', by='SEX',\n",
        "                                                     level='female', cutoff=best_cut)}\n",
        "\n",
        "def confusion_matrix_metrics(cm_dict, metric_dict):\n",
        "    levels = list(cm_dict.keys())\n",
        "\n",
        "    metrics_frame = pd.DataFrame(index=levels) # frame for metrics\n",
        "\n",
        "    for level in levels:\n",
        "        for metric in metric_dict.keys():\n",
        "\n",
        "            # parse metric expressions into executable pandas statements\n",
        "            expression = confusion_matrix_parser(metric_dict[metric])\n",
        "\n",
        "            # dynamically evaluate metrics to avoid code duplication\n",
        "            metrics_frame.loc[level, metric] = eval(expression)\n",
        "\n",
        "    return metrics_frame\n",
        "\n",
        "sex_confusion_metrics = confusion_matrix_metrics(sex_confusion_mats, metric_dict)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "60585b3e-67cb-452f-8716-85d2c9388d39",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 211
        },
        "id": "60585b3e-67cb-452f-8716-85d2c9388d39",
        "outputId": "a8be3a84-8866-4a36-ef9e-14d8993ff938",
        "tags": []
      },
      "outputs": [],
      "source": [
        "sex_confusion_metrics"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "qZq1a43DVVTw",
      "metadata": {
        "id": "qZq1a43DVVTw"
      },
      "source": [
        "Let us create the confusion matrix with disparity metrics for Race groups.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b6f8825f-528b-4ad6-a05e-6f02c9554123",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 274
        },
        "id": "b6f8825f-528b-4ad6-a05e-6f02c9554123",
        "outputId": "3a67a54e-56b9-4467-a0a0-de111bc9efe3",
        "tags": []
      },
      "outputs": [],
      "source": [
        "race_levels = list(race_map.values())\n",
        "race_confusion_mats = {level: get_confusion_matrix(test, target, f'p_{target}', by='RACE',\n",
        "                                                   level=level, cutoff=best_cut) for level in race_levels}\n",
        "race_confusion_metrics = confusion_matrix_metrics(race_confusion_mats, metric_dict)\n",
        "race_confusion_metrics"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "2kPDUTu3NoP0",
      "metadata": {
        "id": "2kPDUTu3NoP0"
      },
      "source": [
        "The metrics are quite different across groups.\n",
        "E.g the precision rates.  White & Asian on one hand while hispanic and black on the other.\n",
        "\n",
        "we are starting to get a hint that our model is predicting more defaults for Black and Hispanic people, but it’s still hard to tell if it’s doing a good or equitable job.\n",
        "\n",
        "To understand if this is actually problematic ,we’ll follow methods from traditional bias testing and divide the value for each group by the corresponding value for the control group and apply the four-fifths rule as a guide. In this case, we assume the control group is white people.\n",
        "\n",
        "Strictly speaking,the control group is the most favored group in an analysis, not necessarily white people or males. There may also be other reasons to use control groups that are not white people or males. E.g teacher selection in educational institutions .Choosing the control or reference group for a bias-testing analysis is a difficult task, best done in concert with legal, compliance, social science experts, or stakeholders."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "VtHtzfcVrXUq",
      "metadata": {
        "id": "VtHtzfcVrXUq"
      },
      "source": [
        "Now that we have the confusion matrix, let us identify the metrics that \"problematic\" using the four-fifths rule.\n",
        "\n",
        "Four-fifths Rule :\n",
        "\n",
        "The four-fifths rule is a guideline released in the 1978 Uniform Guidelines on Employee Selection Procedures (UGESP) by the Equal Employment Opportunity Commission (EEOC). Part 1607.4 of the UGESP states that “a selection rate for any race, sex, or ethnic group which is less than four-fifths (4/5) (or eighty percent) of the rate for the group with the highest rate will generally be regarded by the Federal enforcement agencies as evidence of adverse impact.”\n",
        "\n",
        "For better or worse, the value of 0.8 for adverse impact —which compares event rates, like job selection or credit approval—has become a widespread benchmark for bias in ML systems.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "0aac8417-d6c0-479a-b763-13787da2d086",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 247
        },
        "id": "0aac8417-d6c0-479a-b763-13787da2d086",
        "outputId": "33462b38-5d9a-4eb8-8254-f92680177cb7",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# @title Confusion Matrix for Race Group : Highlight disparity\n",
        "\n",
        "race_disparity_frame = race_confusion_metrics/race_confusion_metrics.loc['white', :]\n",
        "race_disparity_frame.columns=[col + ' Disparity' for col in race_confusion_metrics.columns]\n",
        "\n",
        "# small utility function to format pandas table output\n",
        "def disparate_red(val, parity_threshold_low=0.8, parity_threshold_hi=1.20):\n",
        "    color = 'grey' if (parity_threshold_low < val < parity_threshold_hi) else 'red'\n",
        "    return 'color: %s' % color\n",
        "\n",
        "race_disparity_frame.style.applymap(disparate_red)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "HvHUoJrYQXt3",
      "metadata": {
        "id": "HvHUoJrYQXt3"
      },
      "source": [
        "We see no out-of-range values for Asian people. This means that the model performs fairly equitably across white and Asian people. However, we do see glaring out-of-range values for Hispanic and Black people for precision, false positive rate, false discovery rate, and false omissions rate disparities. While applying the four-fifths rule can help us flag these values, it really can’t help us interpret them. For this, we’ll have to rely on our human brains to think through these results.\n",
        "\n",
        "e.g Given that prevalence of defaults in the data is so much higher for Black and Hispanic people, one thing these results suggest is that our model learned more about defaults in these groups, and predicts defaults at a higher rate in these groups."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ab269477-90f9-4eb6-a072-6b2a5dfc46c2",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 184
        },
        "id": "ab269477-90f9-4eb6-a072-6b2a5dfc46c2",
        "outputId": "5bd21bc4-9ba7-40fe-d22a-eb17cfefa4a9",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# @title Confusion Matrix for Gender Group : Highlight disparity\n",
        "\n",
        "sex_disparity_frame = sex_confusion_metrics/sex_confusion_metrics.loc['male', :]\n",
        "sex_disparity_frame.columns=[col + ' Disparity' for col in sex_confusion_metrics.columns]\n",
        "\n",
        "# small utility function to format pandas table output\n",
        "def disparate_red(val, parity_threshold_low=0.8, parity_threshold_hi=1.20):\n",
        "    color = 'grey' if (parity_threshold_low < val < parity_threshold_hi) else 'red'\n",
        "    return 'color: %s' % color\n",
        "\n",
        "sex_disparity_frame.style.applymap(disparate_red)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "ee794dd6-c94d-4b4c-a888-6ba453b8952b",
      "metadata": {
        "id": "ee794dd6-c94d-4b4c-a888-6ba453b8952b"
      },
      "source": [
        "### Fair Lending Disparity Analysis\n",
        "\n",
        "Statistically significant differences in outcomes or mean scores is one of the most common legally recognized measures of discrimination, especially in areas like credit lending, where algorithmic decision making has been regulated for decades. \n",
        "\n",
        "By using practical tests and effect size measures, like AIR and SMD, with statistical significance tests, we get two pieces of information: the magnitude of the observed difference, and whether it’s statistically significant, i.e., likely to be seen again in other samples of data.\n",
        "\n",
        "AIR : Adverse Impact Ratio\n",
        "\n",
        "* AIR is often applied to categorical outcomes, like credit lending or hiring outcomes, where someone either receives a positive outcome or not. \n",
        "* AIR is defined as the rate of positive outcomes for a protected group, like minorities or women, divided by the same rate of positive outcomes for a control group, like white people or men. \n",
        "* According to the four-fifths rule, we look for the AIR to be above 0.8.  An AIR below 0.8 points to a serious problem. We then test whether this difference will probably be seen again or if it’s due to chance using a chi-squared test.\n",
        "\n",
        "SMD : Statistical Measures of Disparate Impact.\n",
        "\n",
        "SMD is defined as the mean score for a protected group minus the mean score for a control group, with that quantity divided by a measure of the standard deviation of the score.\n",
        "\n",
        "* SMD and t-tests are often used on predictions from regression models, or on numeric quantities like wages, salaries, or credit limits. We’ll apply SMD and t-tests to our model’s predicted probabilities for demonstration purposes and to get some extra information about bias in our model.\n",
        "\n",
        "* SMD has well-known cutoffs at magnitudes of 0.2, 0.5, and 0.8 for small, medium, and large differences, respectively. We’ll use a t-test to decide whether the effect size measured by SMD is statistically significant.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "80fe3d51-fce9-4f17-9c6b-b17cccb887d1",
      "metadata": {
        "id": "80fe3d51-fce9-4f17-9c6b-b17cccb887d1",
        "tags": []
      },
      "outputs": [],
      "source": [
        "from scipy.stats import ttest_ind, chisquare, fisher_exact, chi2_contingency\n",
        "\n",
        "\n",
        "def air_statistical_signif(group_count, group_favorable, reference_count, reference_favorable):\n",
        "    # Perform a chi-square test \n",
        "    # (or Fisher's exact when cells in the contingency test have less than 30 individuals in them).\n",
        "    \n",
        "    group_unfavorable = group_count - group_favorable\n",
        "    reference_unfavorable = reference_count - reference_favorable\n",
        "    \n",
        "    contingency_table = np.array([[group_favorable, group_unfavorable], \n",
        "                                  [reference_favorable, reference_unfavorable]])\n",
        "    \n",
        "    if np.min(contingency_table) < 30:\n",
        "        _, p = fisher_exact(contingency_table)\n",
        "    else:\n",
        "        _, p, _, _ = chi2_contingency(contingency_table)\n",
        "    \n",
        "    return p\n",
        "\n",
        "def smd_statistical_signif(group_scores, reference_scores):\n",
        "    # Perform a one-sided t-test. An outcome of 1 is assumed to be favorable. \n",
        "    \n",
        "    # We do not assume that the two scores have equal variance. Furthermore, we are testing \n",
        "    # against the alternative hypothesis that the group receives lower scores than the reference\n",
        "    # group.\n",
        "    _, p = ttest_ind(group_scores, reference_scores, equal_var=False, alternative='less')\n",
        "    return p\n",
        "\n",
        "def fair_lending_disparity(frame, y, yhat, demo_name, groups, reference_group, cutoff=0.5, favorable_outcome=0):\n",
        "    \"\"\" \n",
        "    Creates a table of fair lending disparity metrics (AIR and SMD).\n",
        "    \n",
        "    Args:\n",
        "        frame: Pandas dataframe of actual (y) and predicted (yhat) values with group membership.\n",
        "        y: Name of actual value column.\n",
        "        yhat: Name of predicted value column.\n",
        "        demo_name: The name of the column containing the group information\n",
        "        groups: The names of the groups in the demo_name column.\n",
        "        reference_group: The control group.\n",
        "        cutoff: Cutoff threshold for confusion matrix, default 0.5. \n",
        "        favorable_outcome: The value {0, 1} that corresponds to a desirable outcome.\n",
        "\n",
        "    Returns:\n",
        "        A DataFrame summarizing the fair lending metrics analysis\n",
        "    \"\"\"\n",
        "    \n",
        "    protected_groups = [group for group in groups if group != reference_group]\n",
        "    groups_ordered = protected_groups + [reference_group]   \n",
        "    \n",
        "    temp_frame = frame.copy()\n",
        "    temp_frame['model_outcome'] = np.where(temp_frame[yhat] <= cutoff, 0, 1)\n",
        "    temp_frame['fav_outcome'] = temp_frame['model_outcome'] == favorable_outcome\n",
        "    temp_frame['fav_score'] = temp_frame[yhat] if favorable_outcome else 1-temp_frame[yhat]\n",
        "    \n",
        "    disparity_table = pd.DataFrame(index=groups_ordered)\n",
        "    \n",
        "    disparity_table['Count'] = [len(temp_frame.loc[temp_frame[demo_name] == group]) for group in groups_ordered]\n",
        "    disparity_table['Favorable Outcomes'] = [temp_frame.loc[temp_frame[demo_name] == group]['fav_outcome'].sum() \n",
        "                                             for group in groups_ordered]\n",
        "    disparity_table['Favorable Rate'] = [disparity_table['Favorable Outcomes'][group]/disparity_table['Count'][group] \n",
        "                                         for group in groups_ordered]\n",
        "    disparity_table['Mean Score'] = [temp_frame.loc[temp_frame[demo_name] == group][yhat].mean() \n",
        "                                             for group in groups_ordered]\n",
        "    disparity_table['Std Score'] = [temp_frame.loc[temp_frame[demo_name].isin([reference_group, group])][yhat].std() \n",
        "                                             for group in groups_ordered]\n",
        "    try:\n",
        "        disparity_table['AIR'] = [disparity_table['Favorable Rate'][group]/disparity_table['Favorable Rate'][reference_group] \n",
        "                                  for group in groups_ordered]\n",
        "    except:\n",
        "        disparity_table['AIR'] = np.nan\n",
        "        \n",
        "    disparity_table['AIR p-value'] = [air_statistical_signif(disparity_table['Count'][group], \n",
        "                                                             disparity_table['Favorable Outcomes'][group],\n",
        "                                                             disparity_table['Count'][reference_group],\n",
        "                                                             disparity_table['Favorable Outcomes'][reference_group])\n",
        "                                      for group in groups_ordered]\n",
        "\n",
        "    disparity_table['SMD'] = [(disparity_table['Mean Score'][group] - \n",
        "                               disparity_table['Mean Score'][reference_group]) / \n",
        "                              disparity_table['Std Score'][group]\n",
        "                              for group in groups_ordered]\n",
        "    \n",
        "    disparity_table['SMD p-value'] = [smd_statistical_signif(temp_frame.loc[temp_frame[demo_name] == group]['fav_score'],\n",
        "                                                             temp_frame.loc[temp_frame[demo_name] == reference_group]['fav_score'])\n",
        "                                      for group in groups_ordered]\n",
        "\n",
        "    return disparity_table\n",
        "    "
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "JJ3DggFmQzUw",
      "metadata": {
        "id": "JJ3DggFmQzUw"
      },
      "source": [
        "Let us see how the disparity looks like for training set ?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "92f6081a-51be-4c72-b1f1-3f57b63f48a6",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "92f6081a-51be-4c72-b1f1-3f57b63f48a6",
        "outputId": "9f2fd265-33cf-4aec-bb51-d4125a633ee5",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# Calculate the AIR and SMD scores across Train, Test & Validation datasets.\n",
        "\n",
        "fair_lending_disparity(train, y=target, yhat=f'p_{target}',\n",
        "                       demo_name='RACE', groups=race_levels, reference_group='white',\n",
        "                       cutoff=best_cut)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "b491f76e",
      "metadata": {},
      "source": [
        "It clear that there is a big difference in scores for Black and Hispanic people versus scores for white and Asian people.\n",
        "\n",
        "While our data is simulated, very sadly, this is not atypical in consumer finance.\n",
        "\n",
        "It is immediately obvious that Black and Hispanic people have higher mean scores and lower favorable rates than white and Asian people, while all four groups have similar standard deviations for scores. \n",
        "\n",
        "Are these differences big enough to be a bias problem? \n",
        "\n",
        "That’s where our practical significance tests come in. AIR and SMD are both calculated in reference to white people. That’s why white people have scores of 1.0 and 0.0 for these, respectively.\n",
        "\n",
        "Looking at AIR, both Black and Hispanic AIRs are below 0.8. Big red flag! SMDs for those two groups are around 0.5, meaning a medium difference in scores between groups. That’s not a great sign either. We’d like for those SMD values to be below or around 0.2, signifying a small difference."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "L_oilVfQQ6U0",
      "metadata": {
        "id": "L_oilVfQQ6U0"
      },
      "source": [
        "Can you calculate the disparity for test and validation set ?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "90488eb6-92b6-4e6c-8de9-de78219bba1f",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "90488eb6-92b6-4e6c-8de9-de78219bba1f",
        "outputId": "79da34df-ed31-4427-fb52-4484c3e2bf37",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# @title Excercise : Disparity for validation set\n",
        "fair_lending_disparity(valid, y=target, yhat=f'p_{target}',\n",
        "                       demo_name='RACE', groups=race_levels, reference_group='white',\n",
        "                       cutoff=best_cut)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "50553a58-eab2-4ea8-9a59-5d3ca5cdc28f",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "50553a58-eab2-4ea8-9a59-5d3ca5cdc28f",
        "outputId": "915c042a-e16e-4db6-fc7d-4df7718ac65d",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# @title Excercise : Disparity for test set\n",
        "\n",
        "fair_lending_disparity(test, y=target, yhat=f'p_{target}',\n",
        "                       demo_name='RACE', groups=race_levels, reference_group='white',\n",
        "                       cutoff=best_cut)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "9735f488-77f4-41f8-9bd1-56dd31760351",
      "metadata": {
        "id": "9735f488-77f4-41f8-9bd1-56dd31760351"
      },
      "source": [
        "## 4. Remediating Model Bias\n",
        "\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/anilkumarpanda/fairness_cb/development/img/ds_cat.JPG\" alt=\"Alt Text\" height=\"400\">\n",
        "\n",
        "A cat data scientist at work.\n",
        "\n",
        "\n",
        "\n",
        "There are many potential ways to remediate bias. Some of the most common include pre-, in-, and postprocessing, and model selection:\n",
        "\n",
        "1. Preprocessing : Rebalancing, reweighing, or resampling training data so that demographic groups are better represented or positive outcomes are distributed more equitably.\n",
        "\n",
        "2. In-processing : Any number of alterations to ML training algorithms, including constraints, regularization and dual loss functions, or incorporation of adversarial modeling information, that attempt to generate more balanced outputs or performance across demographic groups.\n",
        "\n",
        "3. Postprocessing :Changing model predictions directly to create less biased outcomes.\n",
        "\n",
        "4. Model selection : Considering bias along with performance when selecting models. Typically, it’s possible to find a model with good performance and fairness characteristics if we measure bias and performance across a large set of hyperparameter settings and input features."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "d34102ca-88d9-4234-8801-f74ab803a83b",
      "metadata": {
        "id": "d34102ca-88d9-4234-8801-f74ab803a83b"
      },
      "source": [
        "### 4.1 Pre-Processing\n",
        "\n",
        "The first bias-remediation technique we’ll try is a preprocessing technique known as reweighing. \n",
        "\n",
        "It was published first by Faisal Kamiran and Toon Calders in their 2012 paper, “Data Preprocessing Techniques for Classification Without Discrimination”. \n",
        "\n",
        "The idea of reweighing is to make the average outcome across groups equal using observation weights and then retrain the model. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8e3681b8-3826-48e5-a41b-e0c8de234558",
      "metadata": {
        "id": "8e3681b8-3826-48e5-a41b-e0c8de234558",
        "tags": []
      },
      "outputs": [],
      "source": [
        "def reweight_dataset(frame, y, demo_name, groups):\n",
        "    \"\"\"\n",
        "    Generates a weight for each observation according to the reweighting algorithm of\n",
        "    Kamiran and Kalders 2012, Data preprocessing techniques for classification without discrimination.\n",
        "\n",
        "    Args:\n",
        "        frame: Pandas dataframe of actual (y) and group information.\n",
        "        y: Name of actual value column (assumed to be binary).\n",
        "        demo_name: The name of the column containing the group information\n",
        "        groups: The names of the groups in the demo_name column.\n",
        "\n",
        "    Returns:\n",
        "        A Series containing the new observation weights.\n",
        "    \"\"\"\n",
        "\n",
        "    n = len(frame)\n",
        "\n",
        "    freq_dict = {'pos': len(frame.loc[frame[y] == 1])/n,\n",
        "                 'neg': len(frame.loc[frame[y] == 0])/n}\n",
        "\n",
        "    freq_dict.update({group: frame[demo_name].value_counts()[group]/n for group in groups})\n",
        "\n",
        "    weights = pd.Series(np.ones(n), index=frame.index)\n",
        "\n",
        "    for label in [0, 1]:\n",
        "        for group in groups:\n",
        "            label_name = 'pos' if label == 1 else 'neg'\n",
        "            freq = frame.loc[frame[y] == label][demo_name].value_counts()[group]/n\n",
        "            weights[(frame[y] == label) & (frame[demo_name] == group)] *= freq_dict[group]*freq_dict[label_name]/freq\n",
        "\n",
        "    return weights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "710035da",
      "metadata": {},
      "outputs": [],
      "source": [
        "train_weights = reweight_dataset(train, target, 'RACE', race_levels)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "61aa0e96-bf79-4e63-a259-d139a92110c7",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "61aa0e96-bf79-4e63-a259-d139a92110c7",
        "outputId": "3669ebd2-0ced-4bc9-f7a4-8a3b2b507b84",
        "tags": []
      },
      "outputs": [],
      "source": [
        "for race in race_levels:\n",
        "    print(f\"Mean outcome for {race}: {np.round(train.loc[train['RACE'] == race][target].mean(), 3)}\")\n",
        "\n",
        "    weighted_target = np.multiply(train.loc[train['RACE'] == race][target],\n",
        "    train_weights.loc[train['RACE'] == race])\n",
        "\n",
        "    print(f\"Mean outcome for {race} - reweighted: {np.round(weighted_target.mean(), 3)} \\n\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "62333804-55b3-46ad-a941-20186e8376be",
      "metadata": {
        "id": "62333804-55b3-46ad-a941-20186e8376be",
        "tags": []
      },
      "outputs": [],
      "source": [
        "dtrain = xgb.DMatrix(train[features],\n",
        "                     label=train[target],\n",
        "                     weight=train_weights)\n",
        "\n",
        "\n",
        "model_reweighted = xgb.train(params,\n",
        "                             dtrain,\n",
        "                             num_boost_round=100,\n",
        "                             evals=watchlist,\n",
        "                             early_stopping_rounds=10,\n",
        "                             verbose_eval=False)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "96280603-a9e9-4106-87cb-0c85a9bee0c9",
      "metadata": {
        "id": "96280603-a9e9-4106-87cb-0c85a9bee0c9",
        "tags": []
      },
      "outputs": [],
      "source": [
        "reweighted_model_metrics = perf_metrics(y_true=valid[target], y_score=model_reweighted.predict(dvalid))\n",
        "reweighted_best_cut = reweighted_model_metrics.loc[reweighted_model_metrics['f1'].idxmax(), 'cutoff']"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a6b07bea-a72d-486e-8196-51c8163ff324",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        },
        "id": "a6b07bea-a72d-486e-8196-51c8163ff324",
        "outputId": "488ebbe4-98ac-4571-9947-6269c86cc35d",
        "tags": []
      },
      "outputs": [],
      "source": [
        "test[f'p_{target}_reweighted'] = model_reweighted.predict(xgb.DMatrix(test[features], label=test[target]))\n",
        "\n",
        "fair_lending_disparity(test, y=target, yhat=f'p_{target}_reweighted',\n",
        "                       demo_name='RACE', groups=race_levels, reference_group='white',\n",
        "                       cutoff=reweighted_best_cut)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "df38e4ef",
      "metadata": {},
      "source": [
        "When we trained XGBoost on the unweighted data, we saw some problematic AIR values. \n",
        "\n",
        "* Originally, the AIR was around 0.73 for Black and Hispanic people. These values are not great—signifying that for every 1,000 credit products the model extends to white people, this model only accepts applications from about 730 Hispanic or Black people.\n",
        "\n",
        "* This level of bias is ethically troubling, but it could also give rise to legal troubles in consumer finance, hiring, or other areas that rely on traditional legal standards for bias testing. The four-fifths rule—while flawed and imperfect—tells us we should not see values below 0.8 for AIR. \n",
        "\n",
        "Luckily, in our case, reweighing provides good remediation results.\n",
        "\n",
        "However , how does that affect the model performance ?\n",
        "\n",
        "We can check that by looking at F1 scores at diffrent re-weighting values.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6a127a3f",
      "metadata": {},
      "outputs": [],
      "source": [
        "# @title Excercise : Reweighting Scheme and its effect on AIR\n",
        "# Conduct a similar experiment with the magnitude of the reweighing scheme.\n",
        "\n",
        "magnitudes = np.arange(0, 1.5, 0.10)\n",
        "scores = []\n",
        "trees = []\n",
        "\n",
        "air_black_ii = []\n",
        "air_hispanic_ii = []\n",
        "air_asian_ii = []\n",
        "\n",
        "dtest = xgb.DMatrix(test[features], label=test[target])\n",
        "\n",
        "for mag in magnitudes:\n",
        "\n",
        "    weight_change = 1 - train_weights\n",
        "    new_train_weights = 1 - mag*weight_change\n",
        "\n",
        "    dtrain = xgb.DMatrix(train[features],\n",
        "                     label=train[target],\n",
        "                     weight=new_train_weights)\n",
        "\n",
        "\n",
        "    model = xgb.train(params,\n",
        "                      dtrain,\n",
        "                      num_boost_round=200,\n",
        "                      evals=watchlist,\n",
        "                      early_stopping_rounds=10,\n",
        "                      verbose_eval=False)\n",
        "\n",
        "    model_metrics = perf_metrics(y_true=valid[target],\n",
        "                                 y_score=model.predict(dvalid, iteration_range=(0, model.best_iteration)))\n",
        "    best_cut = model_metrics.loc[model_metrics['f1'].idxmax(), 'cutoff']\n",
        "\n",
        "    test[f'p_{target}_lam_test'] = model.predict(dtest, iteration_range=(0, model.best_iteration))\n",
        "\n",
        "    air_table = fair_lending_disparity(test, y=target, yhat=f'p_{target}_lam_test',\n",
        "                                       demo_name='RACE', groups=race_levels, reference_group='white',\n",
        "                                       cutoff=best_cut)\n",
        "\n",
        "    scores.append(model_metrics['f1'].max())\n",
        "    trees.append(model.best_ntree_limit)\n",
        "\n",
        "    air_black_ii.append(air_table.loc['black']['AIR'])\n",
        "    air_hispanic_ii.append(air_table.loc['hispanic']['AIR'])\n",
        "    air_asian_ii.append(air_table.loc['asian']['AIR'])\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "80a9e6d9",
      "metadata": {},
      "outputs": [],
      "source": [
        "# @title Excercise : Reweighting effect on model performance\n",
        "\n",
        "# This plot shows a relative drop in F1 score down to 94% of the original value.\n",
        "# Meanwhile, black and Hispanic AIRs go from 0.72 to 0.9!\n",
        "\n",
        "fig, axs = plt.subplots(1, 2, figsize=(10, 5))\n",
        "axs[0].plot(magnitudes, [score/scores[0] for score in scores])\n",
        "axs[0].set_xlabel('Reweighing Strength')\n",
        "axs[0].set_ylabel('F1 Score')\n",
        "\n",
        "axs[1].plot(magnitudes, trees)\n",
        "axs[1].set_xlabel('Reweighing Strength')\n",
        "axs[1].set_ylabel('Num Trees')\n",
        "\n",
        "#fig.savefig('../Data/Data/Figures/preprocessing_summary_i.svg')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ed9e472b",
      "metadata": {},
      "outputs": [],
      "source": [
        "# @title Excercise : Reweighting effect on AIR across Race groups\n",
        "\n",
        "fig, axs = plt.subplots(1, 3, figsize=(15, 5))\n",
        "axs[0].plot(magnitudes, air_black_ii)\n",
        "axs[0].set_xlabel('Reweighing Strength')\n",
        "axs[0].set_ylabel('AIR (black)')\n",
        "axs[0].set_ylim((air_black_ii[0]-0.05, 1.05))\n",
        "\n",
        "axs[1].plot(magnitudes, air_hispanic_ii)\n",
        "axs[1].set_xlabel('Reweighing Strength')\n",
        "axs[1].set_ylabel('AIR (Hispanic)')\n",
        "axs[1].set_ylim((air_hispanic_ii[0]-0.05, 1.05))\n",
        "\n",
        "axs[2].plot(magnitudes, air_asian_ii)\n",
        "axs[2].set_xlabel('Reweighing Strength')\n",
        "axs[2].set_ylabel('AIR (asian)')\n",
        "axs[2].set_ylim((air_asian_ii[0]-0.05, 1.05))\n",
        "\n",
        "#fig.savefig('../Data/Data/Figures/preprocessing_summary_ii.png')"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "YDhTF_3rjMyu",
      "metadata": {
        "id": "YDhTF_3rjMyu"
      },
      "source": [
        "#### Excersise:\n",
        "\n",
        "1. Can you think of possible scenarios where re-weighting the data would help ?"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "3946b4b0-f228-4359-a0fd-6e4375048148",
      "metadata": {
        "id": "3946b4b0-f228-4359-a0fd-6e4375048148"
      },
      "source": [
        "### 4.2 Model Selection\n",
        "\n",
        "The final technique we’ll discuss is fairness-aware model selection. \n",
        "\n",
        "To be exact, we’ll conduct simple feature selection and random hyperparameter tuning while keeping track of model performance and AIRs. \n",
        "\n",
        "We certainly already performing these steps when it comes to performance assessment, so this technique has fairly low overhead costs. \n",
        "\n",
        "Another advantage of model selection as a remediation technique is that it raises the fewest disparate treatment concerns. (On the other end of the spectrum is the reject option postprocessing, described in the previous section, wherein we literally changed model outcomes depending on the protected group status of each observation.)\n",
        "\n",
        "\n",
        "ML models seem to present more ways to fix them than traditional linear models. \n",
        "\n",
        "Due to the Rashomon effect—the fact that there are often many accurate ML models for any given training dataset, we simply have more levers to pull and switches to flip to find better options for decreased bias and sustained predictive performance in ML models versus simpler models.\n",
        "\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "fVJDWK6USI8t",
      "metadata": {
        "id": "fVJDWK6USI8t"
      },
      "source": [
        "#### 4.2.1 Feature Selection\n",
        "\n",
        "\n",
        "Feature selection can be a powerful remediation technique, but it works best when guided by subject matter experts and when alternative sources of data are available. \n",
        "For example, a compliance expert at a bank may know that a feature in a lending model can be swapped out with an alternative feature that encodes less historical bias. \n",
        "\n",
        "We don’t have the luxury of accessing these alternative features, so for our example data we’ll only have the option of dropping features from our model, and we’ll test the effect of dropping each feature individually while maintaining the original hyperparameters.\n",
        "\n",
        "Between feature selection and hyperparameter tuning, we’re about to train a lot of different models, so we’ll employ five-fold cross-validation using our original training data. \n",
        "\n",
        "We are reaching the end , Stay Strong !!\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/anilkumarpanda/fairness_cb/development/img/red_cat.JPG\" alt=\"Alt Text\" height=\"400\">\n",
        "\n",
        "### Excercise\n",
        "1. This code takes a while a run, in the meanwhile can you come up with an\n",
        "easier way to identify features that can be adding addtional bias to the model ?\n",
        "\n",
        "Hint : Adversarial Models ?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "16f2a138-b820-45ca-83e8-2ac0988308da",
      "metadata": {
        "id": "16f2a138-b820-45ca-83e8-2ac0988308da",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# We'll examine the effect of dropping each feature individually on model performance and adverse impact ratios.\n",
        "features_to_drop = ['original model'] + features"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "afc683a9-938b-409a-8c5e-e9910962630d",
      "metadata": {
        "id": "afc683a9-938b-409a-8c5e-e9910962630d",
        "tags": []
      },
      "outputs": [],
      "source": [
        "num_cv_folds = 5\n",
        "\n",
        "# Build the custom cross-validation iterable.\n",
        "all_indices = np.arange(0, len(train))\n",
        "all_indices = np.random.permutation(all_indices)\n",
        "splits = np.array([int(np.floor(len(train)/num_cv_folds)) for _ in range(num_cv_folds-1)])\n",
        "splits = np.append(splits, len(train) - splits.sum())\n",
        "\n",
        "test_indices = np.split(all_indices, splits.cumsum())[:-1]\n",
        "test_groups = [train.iloc[test_ind]['RACE'].values for test_ind in test_indices]\n",
        "\n",
        "train_indices = [np.array([i for i in all_indices if i not in test_ind]) for test_ind in test_indices]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4ifQ766Gk6EB",
      "metadata": {
        "id": "4ifQ766Gk6EB"
      },
      "outputs": [],
      "source": [
        "feature_selection_results = pd.DataFrame(index=features_to_drop,\n",
        "                                         columns=['AUC', 'Black AIR', 'Hispanic AIR'])\n",
        "\n",
        "for dropped_feature in features_to_drop:\n",
        "\n",
        "    new_features = list(set(features).difference(set([dropped_feature])))\n",
        "    cv_auc = []\n",
        "    cv_black_air = []\n",
        "    cv_hispanic_air = []\n",
        "\n",
        "    for fold_num, (train_ind, test_ind) in enumerate(zip(train_indices, test_indices)):\n",
        "\n",
        "        new_monotone_constraints={k: v for k, v in dict(zip(features, monotone_constraints)).items() if k != dropped_feature}\n",
        "\n",
        "        cv_model = xgb.XGBClassifier(n_estimators=150,\n",
        "                                     max_depth=5,\n",
        "                                     learning_rate=0.05,\n",
        "                                     subsample=0.6,\n",
        "                                     colsample_bytree=1.0,\n",
        "                                     monotone_constraints=new_monotone_constraints,\n",
        "                                     random_state=12345,\n",
        "                                     use_label_encoder=False,\n",
        "                                     base_score=params['base_score'],\n",
        "                                     eval_metric='logloss',n_jobs=8)\n",
        "\n",
        "        train_slice = train.reset_index(drop=True).iloc[train_ind].copy()\n",
        "        test_slice = train.reset_index(drop=True).iloc[test_ind].copy()\n",
        "      \n",
        "        cv_model = cv_model.fit(train_slice[new_features],\n",
        "                                train_slice[target])\n",
        "        y_pred = cv_model.predict_proba(test_slice[new_features])[:, 1]\n",
        "\n",
        "\n",
        "        model_metrics = perf_metrics(y_true=train[target].values[test_ind], y_score=y_pred)\n",
        "        best_cut = model_metrics.loc[model_metrics['f1'].idxmax(), 'cutoff']\n",
        "\n",
        "        cv_auc.append(sklearn.metrics.roc_auc_score(y_true=train[target].values[test_ind],\n",
        "                                                    y_score=y_pred))\n",
        "\n",
        "        test_slice['pred'] = y_pred\n",
        "        disparity_table = fair_lending_disparity(test_slice, y=target, yhat=f'pred',\n",
        "                           demo_name='RACE', groups=race_levels, reference_group='white',\n",
        "                           cutoff=best_cut)\n",
        "\n",
        "        cv_black_air.append(disparity_table.loc['black']['AIR'])\n",
        "        cv_hispanic_air.append(disparity_table.loc['hispanic']['AIR'])\n",
        "\n",
        "    feature_selection_results.loc[dropped_feature] = [np.mean(cv_auc),\n",
        "                                                      np.mean(cv_black_air),\n",
        "                                                      np.mean(cv_hispanic_air)]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "D10vBcPJqo61",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "D10vBcPJqo61",
        "outputId": "fb483979-3a81-4c60-e058-ced49768b935",
        "tags": []
      },
      "outputs": [],
      "source": [
        "feature_to_drop = feature_selection_results.sort_values('Black AIR').index[-1]\n",
        "feature_selection_results.sort_values('Black AIR')\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "d30d766f",
      "metadata": {},
      "source": [
        "The above table can be read as follows :\n",
        "Fitting the model without feature e.g PAY_AMT3 results in AUC of 0.79 and Black AIR of 0.75 etc."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f9704501",
      "metadata": {},
      "outputs": [],
      "source": [
        "print(f'Feature to drop : {feature_to_drop}')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2881d68e-e5da-4ea8-b9a2-2efa2f6b351c",
      "metadata": {
        "id": "2881d68e-e5da-4ea8-b9a2-2efa2f6b351c",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# Drop the most 'problematic feature' calculate model scores.\n",
        "new_features = list(set(features).difference(set([feature_to_drop])))\n",
        "new_monotone_constraints={k: v for k, v in dict(zip(features, monotone_constraints)).items() if k in new_features}"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "8e22ff40-f598-4cbd-9222-3b445f5b9a0f",
      "metadata": {
        "id": "8e22ff40-f598-4cbd-9222-3b445f5b9a0f"
      },
      "source": [
        "\n",
        "We will use a random grid search using the scikit-learn API.\n",
        "Since we want to cross-validate AIRs throughout this process, we have to put together a scoring function to pass into scikit-learn.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7ec1b684-a223-4980-acad-cbb6f2d35ac3",
      "metadata": {
        "id": "7ec1b684-a223-4980-acad-cbb6f2d35ac3",
        "tags": []
      },
      "outputs": [],
      "source": [
        "parameter_distributions = {\n",
        "    'n_estimators': np.arange(10, 221, 30),\n",
        "    'max_depth': [3, 4, 5, 6, 7],\n",
        "    'learning_rate': stats.uniform(0.01, 0.1),\n",
        "    'subsample': stats.uniform(0.7, 0.3),\n",
        "    'colsample_bytree': stats.uniform(0.5, 0.5),\n",
        "    'reg_lambda': stats.uniform(0.1, 50),\n",
        "    'monotone_constraints': [new_monotone_constraints],\n",
        "    'base_score': [params['base_score']]\n",
        "    }"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4fd339e8-39d6-448c-9d3e-4f7751737359",
      "metadata": {
        "id": "4fd339e8-39d6-448c-9d3e-4f7751737359",
        "tags": []
      },
      "outputs": [],
      "source": [
        "fold_number = -1\n",
        "\n",
        "def black_air(y_true, y_pred):\n",
        "\n",
        "\n",
        "    global fold_number\n",
        "    fold_number = (fold_number + 1) % num_cv_folds\n",
        "\n",
        "    model_metrics = perf_metrics(y_true, y_score=y_pred)\n",
        "    best_cut = model_metrics.loc[model_metrics['f1'].idxmax(), 'cutoff']\n",
        "\n",
        "    data = pd.DataFrame({'RACE': test_groups[fold_number],\n",
        "                        'y_true': y_true,\n",
        "                        'y_pred': y_pred},\n",
        "                        index=np.arange(len(y_pred)))\n",
        "\n",
        "    disparity_table = fair_lending_disparity(data, y='y_true', yhat='y_pred',\n",
        "                       demo_name='RACE', groups=race_levels, reference_group='white',\n",
        "                       cutoff=best_cut)\n",
        "\n",
        "    return disparity_table.loc['black']['AIR']\n",
        "\n",
        "\n",
        "def hispanic_air(y_true, y_pred):\n",
        "\n",
        "\n",
        "    global fold_number\n",
        "\n",
        "    model_metrics = perf_metrics(y_true, y_score=y_pred)\n",
        "    best_cut = model_metrics.loc[model_metrics['f1'].idxmax(), 'cutoff']\n",
        "\n",
        "    data = pd.DataFrame({'RACE': test_groups[fold_number],\n",
        "                        'y_true': y_true,\n",
        "                        'y_pred': y_pred},\n",
        "                        index=np.arange(len(y_pred)))\n",
        "\n",
        "    disparity_table = fair_lending_disparity(data, y='y_true', yhat='y_pred',\n",
        "                       demo_name='RACE', groups=race_levels, reference_group='white',\n",
        "                       cutoff=best_cut)\n",
        "\n",
        "    return disparity_table.loc['hispanic']['AIR']\n",
        "\n",
        "scoring = {\n",
        "        'AUC': 'roc_auc',\n",
        "        'Black AIR': sklearn.metrics.make_scorer(black_air, needs_proba=True),\n",
        "        'Hispanic AIR': sklearn.metrics.make_scorer(hispanic_air, needs_proba=True)\n",
        "    }"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "26FFPsfdmBhe",
      "metadata": {
        "id": "26FFPsfdmBhe"
      },
      "outputs": [],
      "source": [
        "# @title Excercise : Increase the no.of iterations.\n",
        "# Increase the no.of iterations to 50."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "54512802-7fdc-4039-a168-78efd84c4170",
      "metadata": {
        "id": "54512802-7fdc-4039-a168-78efd84c4170",
        "tags": []
      },
      "outputs": [],
      "source": [
        "grid_search = sklearn.model_selection.RandomizedSearchCV(xgb.XGBClassifier(random_state=12345,\n",
        "                                                                           use_label_encoder=False,\n",
        "                                                                           eval_metric='logloss'),\n",
        "                                                         parameter_distributions,\n",
        "                                                         n_iter=10,\n",
        "                                                         scoring=scoring,\n",
        "                                                         cv=zip(train_indices, test_indices),\n",
        "                                                         refit=False,\n",
        "                                                         error_score='raise').fit(train[new_features], train[target].values)\n",
        "results = pd.DataFrame(grid_search.cv_results_)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "007d3e14-d6ce-4b6d-afb8-05c5a9c9641b",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 466
        },
        "id": "007d3e14-d6ce-4b6d-afb8-05c5a9c9641b",
        "outputId": "54ab3660-426e-4b15-896b-2bc4ecf74a17",
        "tags": []
      },
      "outputs": [],
      "source": [
        "original_auc = feature_selection_results.loc['original model']['AUC']\n",
        "original_black_air = feature_selection_results.loc['original model']['Black AIR']\n",
        "\n",
        "new_auc = feature_selection_results.loc[feature_to_drop]['AUC']\n",
        "new_black_air = feature_selection_results.loc[feature_to_drop]['Black AIR']\n",
        "fig, ax = plt.subplots()\n",
        "ax.scatter(original_black_air, 1.0, s=50, color='red', label='original model')\n",
        "ax.scatter(results['mean_test_Black AIR'], results['mean_test_AUC']/original_auc, label='post-hyperparameter tuning', marker='x')\n",
        "ax.scatter(new_black_air, new_auc/original_auc, color='orange', label='post-feature selection', marker='v')\n",
        "ax.legend(loc='lower left')\n",
        "ax.set_xlabel('Black AIR')\n",
        "ax.set_ylabel(f'AUC (normalized)')\n",
        "\n",
        "#fig.savefig('../Data/Data/Figures/model_tuning_scatter.svg', dpi=300)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c086344e",
      "metadata": {},
      "outputs": [],
      "source": [
        "# @title Excercise : Can you plot the results for Hispanic AIR\n",
        "original_auc = feature_selection_results.loc['original model']['AUC']\n",
        "original_black_air = feature_selection_results.loc['original model']['Hispanic AIR']\n",
        "\n",
        "new_auc = feature_selection_results.loc[feature_to_drop]['AUC']\n",
        "new_black_air = feature_selection_results.loc[feature_to_drop]['Hispanic AIR']\n",
        "fig, ax = plt.subplots()\n",
        "ax.scatter(original_black_air, 1.0, s=50, color='red', label='original model')\n",
        "ax.scatter(results['mean_test_Hispanic AIR'], results['mean_test_AUC']/original_auc, label='post-hyperparameter tuning', marker='x')\n",
        "ax.scatter(new_black_air, new_auc/original_auc, color='orange', label='post-feature selection', marker='v')\n",
        "ax.legend(loc='lower left')\n",
        "ax.set_xlabel('Hispanic AIR')\n",
        "ax.set_ylabel(f'AUC (normalized)')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "08976d5d-9fbe-44d9-89c2-4d77569de9d7",
      "metadata": {
        "id": "08976d5d-9fbe-44d9-89c2-4d77569de9d7",
        "tags": []
      },
      "outputs": [],
      "source": [
        "results[['mean_test_Black AIR', 'mean_test_Hispanic AIR', 'mean_test_AUC']].sort_values(['mean_test_Black AIR', 'mean_test_AUC'], ascending=False)\n",
        "highest_air_idx = results.loc[results['mean_test_Black AIR'] == results['mean_test_Black AIR'].max()].index[0]\n",
        "\n",
        "# We can also choose the fairest model that demonstrates no more than a 1% decrease in AUC from the original model.\n",
        "business_viable_models = results.loc[results['mean_test_AUC'] >= 0.98*original_auc]\n",
        "alternative_model_idx = business_viable_models.loc[business_viable_models['mean_test_Black AIR'] == business_viable_models['mean_test_Black AIR'].max()].index[0]\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "069ea30d-cf0d-4041-bfb0-b2040be1a6b6",
      "metadata": {
        "id": "069ea30d-cf0d-4041-bfb0-b2040be1a6b6",
        "tags": []
      },
      "outputs": [],
      "source": [
        "new_hyperparameter_idx = [highest_air_idx, alternative_model_idx]\n",
        "tuned_params = ['_'.join(col.split('_')[1:]) for col in results.columns if col.startswith('param_')]\n",
        "new_hyperparameters = [dict(zip(tuned_params, [row[f'param_{param}'] for param in tuned_params])) for _, row in results.loc[new_hyperparameter_idx].iterrows()]\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "e7ff7fb3-d12d-47fc-9a90-58a94b907877",
      "metadata": {
        "id": "e7ff7fb3-d12d-47fc-9a90-58a94b907877"
      },
      "source": [
        "## 5.Conclusion"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7fcf5043-68cc-4b72-bcde-873ae6925656",
      "metadata": {
        "id": "7fcf5043-68cc-4b72-bcde-873ae6925656",
        "tags": []
      },
      "outputs": [],
      "source": [
        "def model_summary(y_true, y_pred, group_info, reference_group, metric_dict, cutoff,\n",
        "                  confusion_metrics_to_show=['False Positive Rate']):\n",
        "    \"\"\"\n",
        "    Function to put all things together.\n",
        "\n",
        "    \"\"\"\n",
        "\n",
        "    groups = np.unique(group_info)\n",
        "    protected_groups = [group for group in groups if group != reference_group]\n",
        "\n",
        "    model_metrics = perf_metrics(y_true, y_score=y_pred)\n",
        "    # best_cut = model_metrics.loc[model_metrics['f1'].idxmax(), 'cutoff']\n",
        "    # f1 = model_metrics['f1'].max()\n",
        "\n",
        "    f1 = model_metrics.loc[model_metrics['cutoff'] == cutoff, 'f1'].values[0]\n",
        "\n",
        "    auc = sklearn.metrics.roc_auc_score(y_true, y_score=y_pred)\n",
        "\n",
        "    data = pd.DataFrame({'demo': group_info,\n",
        "                        'y_true': y_true,\n",
        "                        'y_pred': y_pred},\n",
        "                        index=np.arange(len(y_pred)))\n",
        "\n",
        "    disparity_table = fair_lending_disparity(data, y='y_true', yhat='y_pred',\n",
        "                       demo_name='demo', groups=groups, reference_group=reference_group,\n",
        "                       cutoff=cutoff)\n",
        "\n",
        "    airs = dict(zip([f'{group} AIR' for group in protected_groups],\n",
        "                    [disparity_table.loc[group]['AIR'] for group in protected_groups]))\n",
        "\n",
        "    confusion_mats = {level: get_confusion_matrix(data, 'y_true', 'y_pred', by='demo',\n",
        "                                                  level=level, cutoff=cutoff) for level in groups}\n",
        "    confusion_metrics = confusion_matrix_metrics(confusion_mats, metric_dict)\n",
        "    confusion_disparity_frame = confusion_metrics/confusion_metrics.loc[reference_group, :]\n",
        "\n",
        "    confusion_metric_disparities = dict()\n",
        "    for metric in confusion_metrics_to_show:\n",
        "        confusion_metric_disparities.update(dict(zip([f\"{group} {metric} Disparity\" for group in protected_groups],\n",
        "                                               [confusion_disparity_frame.loc[group][metric] for group in protected_groups])))\n",
        "\n",
        "    output = {'AUC': auc, 'F1': f1}\n",
        "    output.update(airs)\n",
        "    output.update(confusion_metric_disparities)\n",
        "\n",
        "    return pd.Series(output)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1e486922-b7fb-4c8c-9f63-6eca9c0485e6",
      "metadata": {
        "id": "1e486922-b7fb-4c8c-9f63-6eca9c0485e6",
        "tags": []
      },
      "outputs": [],
      "source": [
        "\n",
        "feature_selection_model = xgb.XGBClassifier(n_estimators=150,\n",
        "                                     max_depth=5,\n",
        "                                     learning_rate=0.05,\n",
        "                                     subsample=0.6,\n",
        "                                     colsample_bytree=1.0,\n",
        "                                     monotone_constraints=new_monotone_constraints,\n",
        "                                     random_state=12345,\n",
        "                                     use_label_encoder=False,\n",
        "                                     base_score=params['base_score'],\n",
        "                                     eval_metric='logloss').fit(train[new_features], train[target])\n",
        "\n",
        "model_metrics = perf_metrics(y_true=valid[target],\n",
        "                             y_score=feature_selection_model.predict_proba(valid[new_features])[:, 1])\n",
        "best_cut_feature_selection = model_metrics.loc[model_metrics['f1'].idxmax(), 'cutoff']\n",
        "\n",
        "hyp_tuning_model_1 = xgb.XGBClassifier(random_state=12345,\n",
        "                                     use_label_encoder=False,\n",
        "                                     eval_metric='logloss', **new_hyperparameters[0]).fit(train[new_features], train[target])\n",
        "\n",
        "model_metrics = perf_metrics(y_true=valid[target],\n",
        "                             y_score=hyp_tuning_model_1.predict_proba(valid[new_features])[:, 1])\n",
        "best_cut_hyp_1 = model_metrics.loc[model_metrics['f1'].idxmax(), 'cutoff']\n",
        "\n",
        "\n",
        "hyp_tuning_model_2 = xgb.XGBClassifier(random_state=12345, \n",
        "                                     use_label_encoder=False,\n",
        "                                     eval_metric='logloss', **new_hyperparameters[1]).fit(train[new_features], train[target])\n",
        "\n",
        "\n",
        "model_metrics = perf_metrics(y_true=valid[target], \n",
        "                             y_score=hyp_tuning_model_2.predict_proba(valid[new_features])[:, 1])\n",
        "best_cut_hyp_2 = model_metrics.loc[model_metrics['f1'].idxmax(), 'cutoff']"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d9844eac-d5df-44a8-9c24-676023479c57",
      "metadata": {
        "id": "d9844eac-d5df-44a8-9c24-676023479c57",
        "tags": []
      },
      "outputs": [],
      "source": [
        "model_predictions = {'Original Model': (test[f\"p_{target}\"].values, best_cut_original),\n",
        "                    'Pre-processing (reweighting)': (test[f'p_{target}_reweighted'].values, reweighted_best_cut),\n",
        "                    'Feature selection': (feature_selection_model.predict_proba(test[new_features])[:, 1], best_cut_feature_selection),\n",
        "                    'Fairest Model': (hyp_tuning_model_1.predict_proba(test[new_features])[:, 1], best_cut_hyp_1),\n",
        "                    'Fairer - Business Viable Model': (hyp_tuning_model_2.predict_proba(test[new_features])[:, 1], best_cut_hyp_2)\n",
        "                    }"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8832a731-cf3d-486a-ac39-e6087191a9cd",
      "metadata": {
        "id": "8832a731-cf3d-486a-ac39-e6087191a9cd",
        "tags": []
      },
      "outputs": [],
      "source": [
        "model_summaries = {name: model_summary(test[target].values, pred, test['RACE'].values, 'white', metric_dict, cutoff) for name, (pred, cutoff) in model_predictions.items()}"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "Bu6MiWebsvgW",
      "metadata": {
        "id": "Bu6MiWebsvgW"
      },
      "source": [
        "Put your money on the models!\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/anilkumarpanda/fairness_cb/development/img/banker_cat.JPG\" alt=\"Alt Text\" height=\"400\">"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c9789fc9-51a6-4fe3-b26e-0022522442e4",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 483
        },
        "collapsed": true,
        "id": "c9789fc9-51a6-4fe3-b26e-0022522442e4",
        "outputId": "fe7e797d-1575-4a65-c3a7-2bbdee614dca",
        "tags": []
      },
      "outputs": [],
      "source": [
        "pd.DataFrame(model_summaries)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "bYPPnBPgClhL",
      "metadata": {
        "id": "bYPPnBPgClhL"
      },
      "source": [
        "However we’ll need to remember that legal liability can come into play with ML bias issues.\n",
        "\n",
        "There are many legal liabilities associated with bias in ML systems, and since we’re not lawyers (and likely neither are you), we need to be humble about the complexity of law, not let the Dunning-Kruger effect take over, and defer to actual experts on nondiscrimination law.\n",
        "\n",
        "If we have any concerns about legal problems in our ML systems, now is the time to reach out to your managers or our legal department."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "id": "224cb364",
      "metadata": {
        "id": "224cb364"
      },
      "source": [
        "\n",
        "\n",
        "## 6.References\n",
        "\n",
        "Most of the code and knowledge is borrowed from the following sources :\n",
        "\n",
        "1. [Fairness Tree](https://docs.google.com/presentation/d/1ycNhDJr5uQiBhWvdlNArnv7ojydW73qryOjRBB30Z44/edit#slide=id.g8d5290cc44_0_417)\n",
        "2. [Machine Learning for High-Risk Applications](https://learning.oreilly.com/library/view/machine-learning-for/9781098102425/)\n",
        "3. [SHAP for Fairness](https://shap.readthedocs.io/en/latest/example_notebooks/overviews/Explaining%20quantitative%20measures%20of%20fairness.html)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1Qp290ypsp-u",
      "metadata": {
        "id": "1Qp290ypsp-u"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "colab": {
      "include_colab_link": true,
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.11"
    },
    "vscode": {
      "interpreter": {
        "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}