01-Introduction.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4e93aafa-d72b-4660-a13e-1e65a18efbf8",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99f97733-3be3-42cf-aff8-148c8bf7d185",
   "metadata": {},
   "source": [
    "# Introduction to LMI\n",
    "- **Goal:** Get familiar with the basic operations, get a feel for how the training / searching process operates.\n",
    "- **Table of Contents:**\n",
    "    - [1. Load the data](#1.-Load-the-data)\n",
    "    - [2. Build the index](#2.-Build-the-index)\n",
    "    - [3. Load additional data for search](#3.-Load-search-data)\n",
    "    - [4. Search in the index](#4.-Search-in-the-index)\n",
    "- **Author:** T. Slanináková, `xslanin@fi.muni.cz`\n",
    "- **Date:** 2023-09-12"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "4e8783fd-4590-4665-9beb-85adf879f6ae",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import logging\n",
    "import numpy as np\n",
    "np.random.seed(2023)\n",
    "\n",
    "logging.basicConfig(\n",
    "    level=logging.INFO,\n",
    "    format='[%(asctime)s][%(levelname)-5.5s][%(name)-.20s] %(message)s'\n",
    ")\n",
    "LOG = logging.getLogger(__name__)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98460326-c0a2-4309-a147-e7b5e2699746",
   "metadata": {},
   "source": [
    "# 1. Load the data\n",
    "The data are from SISAP 2023 indexing challenge (LAION dataset). There are `100K`, `300K`, and `10M` versions (also `100M`, but that one wasn't tested with LMI). The queries are not included in the data (they are outside of the dataset)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "5a368ad6-f582-4013-b092-1f135740f042",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "from urllib.request import urlretrieve\n",
    "from pathlib import Path\n",
    "import h5py\n",
    "\n",
    "def download(src, dst):\n",
    "    if not os.path.exists(dst):\n",
    "        os.makedirs(Path(dst).parent, exist_ok=True)\n",
    "        LOG.info('downloading %s -> %s...' % (src, dst))\n",
    "        urlretrieve(src, dst)\n",
    "\n",
    "def prepare(kind, size):\n",
    "    url = \"https://sisap-23-challenge.s3.amazonaws.com/SISAP23-Challenge\"\n",
    "    task = {\n",
    "        \"query\": f\"{url}/public-queries-10k-{kind}.h5\",\n",
    "        \"dataset\": f\"{url}/laion2B-en-{kind}-n={size}.h5\",\n",
    "    }\n",
    "\n",
    "    for version, url in task.items():\n",
    "        target_path = os.path.join(\"data\", kind, size, f\"{version}.h5\")\n",
    "        download(url, target_path)\n",
    "        assert os.path.exists(target_path), f\"Failed to download {url}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "30603999-c63c-4b96-b4a0-7902f207baf9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "config = {\n",
    "    # get the smallest version of the LAION dataset\n",
    "    'dataset': 'pca32v2',\n",
    "    'emb': 'pca32',\n",
    "    'size': '100K',\n",
    "    # n. of nearest neighbors\n",
    "    'k': 10,\n",
    "    # normalize the data to be able to use K-Means\n",
    "    'preprocess': True\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "19ec8167-261d-473b-b363-26727c5caeef",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((100000, 32), (10000, 32))"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# download the data\n",
    "prepare(config['dataset'], config['size'])\n",
    "\n",
    "def get_data(data_part, **config):\n",
    "    return np.array(\n",
    "        h5py.File(\n",
    "            os.path.join(\n",
    "                'data',\n",
    "                config['dataset'],\n",
    "                config['size'],\n",
    "                data_part\n",
    "            ),\n",
    "            'r'\n",
    "        )[config['emb']]\n",
    "    )\n",
    "\n",
    "# load the data    \n",
    "data = get_data(\"dataset.h5\", **config)\n",
    "queries = get_data(\"query.h5\", **config)\n",
    "data.shape, queries.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a24a39e7-fbc9-404d-84ce-5e31e223b23a",
   "metadata": {},
   "source": [
    "## 1.2. Pre-process the data\n",
    "The default distance metric for LAION dataset is the cosine distance. In order for us to use K-Means for partitioning (which operates only with Euclidean distances), we need to **normalize the data to unit length** (i.e., a single vector will sum up to 1). Data normalized like this can continue to be used with euclidean distance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "43066688-f16a-4131-a98b-f1e043d88b74",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4463985259644687"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# data characteristic before:\n",
    "sum(data[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ffcdd826-9e23-4259-8d1d-0d29a5337c03",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn import preprocessing\n",
    "if config['preprocess']:\n",
    "    data = preprocessing.normalize(data)\n",
    "    queries = preprocessing.normalize(queries)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "75e923c4-952f-47ec-adff-a13f66f07de2",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.004468702711165"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# data characteristics after\n",
    "sum(data[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b23007bd-4b5e-4096-b1bf-a3d3a688e9e1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "# data to pandas\n",
    "data = pd.DataFrame(data)\n",
    "# index from one (needed to fit the evaluation procedure later)\n",
    "data.index += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3f9e9bc-c91a-442d-b68f-7cfbaa5a22a1",
   "metadata": {},
   "source": [
    "# 2. Build the index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "83796d0d-04ca-4317-98f2-afe6529a14a6",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2023-10-04 09:30:44,803][INFO ][faiss.loader] Loading faiss with AVX2 support.\n",
      "[2023-10-04 09:30:47,167][INFO ][faiss.loader] Successfully loaded faiss with AVX2 support.\n",
      "[2023-10-04 09:32:33,885][INFO ][numexpr.utils] Note: detected 128 virtual cores but NumExpr set to maximum of 64, check \"NUMEXPR_MAX_THREADS\" environment variable.\n",
      "[2023-10-04 09:32:33,886][INFO ][numexpr.utils] Note: NumExpr detected 128 cores but \"NUMEXPR_MAX_THREADS\" not set, so enforcing safe limit of 8.\n"
     ]
    }
   ],
   "source": [
    "from li.BuildConfiguration import BuildConfiguration\n",
    "from li.clustering import algorithms\n",
    "from li.LearnedIndexBuilder import LearnedIndexBuilder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "92b4c129-bd5e-49b3-b19d-349d350a77f8",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "n_categories = [10, 10]\n",
    "\n",
    "build_config = BuildConfiguration(\n",
    "    # which clustering algorithm to use\n",
    "    algorithms['faiss_kmeans'],\n",
    "    # how many epochs to train for\n",
    "    200,\n",
    "    # what model to use (see li/model.py\n",
    "    'MLP',\n",
    "    # what learning rate to use\n",
    "    0.01,\n",
    "    # how many categories at what level to build LMI for\n",
    "    # 10, 10 results in 100 buckets in total\n",
    "    n_categories\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "540f2938-8cb8-48da-b056-04c12c3d9d5a",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [02:54<00:00, 17.42s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 5min 38s, sys: 1.46 s, total: 5min 40s\n",
      "Wall time: 5min 52s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "builder = LearnedIndexBuilder(data, build_config)\n",
    "li, data_prediction, n_buckets_in_index, build_t, cluster_t = builder.build()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "2db0754c-6322-4e48-90b6-1a394dfd0935",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2023-10-04 09:58:00,083][INFO ][__main__] Total number of buckets in the index: 100\n",
      "[2023-10-04 09:58:00,085][INFO ][__main__] Cluster time: 0.9519636631011963\n",
      "[2023-10-04 09:58:00,086][INFO ][__main__] Pure build time: 352.65329146385193\n"
     ]
    }
   ],
   "source": [
    "LOG.info(f\"Total number of buckets in the index: {n_buckets_in_index}\")\n",
    "LOG.info(f\"Cluster time: {cluster_t}\")\n",
    "LOG.info(f\"Pure build time: {build_t}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "6a003c2f-4e5d-4162-a3e4-7668e2248d4e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# n_levels == len(n_categories)\n",
    "# `data_prediction` (dimensions: n of data points x n_levels)\n",
    "#   stores assignment of every object to a bucket\n",
    "n_levels = len(n_categories)\n",
    "assert data_prediction.shape == ((data.shape[0], len(n_categories)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "afd9fa57-cb37-43aa-aaa0-e65c9e71feed",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[8, 7],\n",
       "       [3, 2]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_prediction[:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dff5af95-e70b-4742-a6e4-0fb597e18fbd",
   "metadata": {},
   "source": [
    "# 3. Load search data\n",
    "In this specific dataset, we use data that has been subject to dimensionality reduction (hence `pca32v2` in `config['dataset']`). This is nice, because it allows us to train faster (on 32-dim. vectors vs. 768) and have faster navigation during search. However during the sequential search phase on the candidate set, we reach better accuracy when searching with the non-reduced set.\n",
    "\n",
    "Let's load the full dataset now:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "625d2eff-13f3-483c-926a-1baa3db99234",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2023-10-04 09:59:08,370][INFO ][__main__] downloading https://sisap-23-challenge.s3.amazonaws.com/SISAP23-Challenge/public-queries-10k-clip768v2.h5 -> data/clip768v2/100K/query.h5...\n",
      "[2023-10-04 09:59:11,056][INFO ][__main__] downloading https://sisap-23-challenge.s3.amazonaws.com/SISAP23-Challenge/laion2B-en-clip768v2-n=100K.h5 -> data/clip768v2/100K/dataset.h5...\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "((100000, 768), (10000, 768))"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "config['dataset'] = 'clip768v2'\n",
    "config['emb'] = 'emb'\n",
    "\n",
    "prepare(config['dataset'], config['size'])\n",
    "data_search = get_data(\"dataset.h5\", **config)\n",
    "queries_search = get_data(\"query.h5\", **config)\n",
    "\n",
    "data_search = pd.DataFrame(data_search)\n",
    "data_search.index += 1\n",
    "\n",
    "data_search.shape, queries_search.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbd59797-8dc9-46c8-bd6c-28f85c933bc2",
   "metadata": {},
   "source": [
    "# 4. Search in the index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "f92c58de-8939-4648-97fb-bca324633ac0",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# specify the stop condition\n",
    "bucket=10\n",
    "# specify the n. of neighbors\n",
    "k=10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "69146a6e-c4fe-4c82-a01c-cc0f83139a6e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2023-10-04 10:07:00,406][INFO ][li.LearnedIndex.Lear] Precomputed bucket order time: 0.4756293296813965\n",
      "100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 170.06it/s]\n",
      "100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 174.12it/s]\n",
      "100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 177.48it/s]\n",
      "100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 177.95it/s]\n",
      "100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 179.18it/s]\n",
      "100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 180.52it/s]\n",
      "100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 180.18it/s]\n",
      "100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 181.15it/s]\n",
      "100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 181.54it/s]\n",
      "100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 180.81it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 7.44 s, sys: 55.5 ms, total: 7.49 s\n",
      "Wall time: 7.75 s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "dists, nns, measured_time = li.search(\n",
    "    # the 'navigation' data\n",
    "    data_navigation=data,\n",
    "    queries_navigation=queries,\n",
    "    # the 'sequential filtering' data\n",
    "    data_search=data_search,\n",
    "    queries_search=queries_search,\n",
    "    # mapping of object -> bucket\n",
    "    data_prediction=data_prediction,\n",
    "    # n. of categories present in index\n",
    "    n_categories=n_categories,\n",
    "    # stop condition for the search\n",
    "    n_buckets=bucket,\n",
    "    # number of nearest neighbors we're interested in\n",
    "    k=k\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "65440198-8335-430a-9936-e500fa3ab143",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defaultdict(float,\n",
       "            {'inference': 0.05044746398925781,\n",
       "             'search_within_buckets': 7.249154806137085,\n",
       "             'seq_search': 4.535206317901611,\n",
       "             'sort': 0.0,\n",
       "             'search': 7.75262188911438})"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Time to search (broken down into various search parts)\n",
    "measured_time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "af84d793-a15f-4322-83c9-ae1fc85f1042",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 10)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# matrix of the nearest neighbors (`k` for each query)\n",
    "nns.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "eb0a131a-c02f-4f48-bc9c-b665f8d7dde8",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[79172, 15735, 22337, 74173, 41079, 38159, 69015, 92811, 79896,\n",
       "        13236],\n",
       "       [14347, 82848, 79302, 85923,  6016, 67067, 54566, 34591, 11620,\n",
       "        53783]], dtype=uint32)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nns[:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "8018f307-21b4-4365-8fca-869f4f3ce97b",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 10)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# matrix of distances to the closest neighbors (`k` for each query)\n",
    "dists.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "13aa6923-3fce-4cf2-9694-192204015dae",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.27291209, 0.30623567, 0.3131932 , 0.32404494, 0.33161247,\n",
       "        0.33278447, 0.34032881, 0.34535122, 0.35354602, 0.36600691],\n",
       "       [0.19766825, 0.21139383, 0.22871637, 0.23902297, 0.25272477,\n",
       "        0.25969118, 0.2700808 , 0.2767331 , 0.27809215, 0.28464031]])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dists[:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b53c9269-32cb-43b2-94a7-3cac3765b01d",
   "metadata": {},
   "source": [
    "# 5. Evaluate the search performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "6a6c721c-925a-4c3e-a290-d0e58c6fa84c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2023-10-04 10:03:59,376][INFO ][__main__] downloading https://sisap-23-challenge.s3.amazonaws.com/SISAP23-Challenge/laion2B-en-public-gold-standard-v2-100K.h5 -> data/groundtruth-100K.h5...\n"
     ]
    }
   ],
   "source": [
    "def get_groundtruth(size=\"100K\"):\n",
    "    url = f\"https://sisap-23-challenge.s3.amazonaws.com/SISAP23-Challenge/laion2B-en-public-gold-standard-v2-{size}.h5\"\n",
    "\n",
    "    out_fn = os.path.join(\"data\", f\"groundtruth-{size}.h5\")\n",
    "    download(url, out_fn)\n",
    "    gt_f = h5py.File(out_fn, \"r\")\n",
    "    true_I = np.array(gt_f['knns'])\n",
    "    gt_f.close()\n",
    "    return true_I\n",
    "\n",
    "gt = get_groundtruth(config['size'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "dbbb8831-8ae8-4dd8-a850-b7147febd6a5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def get_recall(I, gt, k):\n",
    "    assert k <= I.shape[1]\n",
    "    assert len(I) == len(gt)\n",
    "\n",
    "    n = len(I)\n",
    "    recall = 0\n",
    "    for i in range(n):\n",
    "        recall += len(set(I[i, :k]) & set(gt[i, :k]))\n",
    "    return recall / (n * k)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "7df2cb73-e965-4b9b-831d-cd24ca1a59b9",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.87099"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recall = get_recall(nns, gt, k)\n",
    "recall"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.17"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}