gpu.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c5d60b39-6161-4fd3-90fb-eaf488bee240",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ['MKL_NUM_THREADS'] = '1'\n",
    "os.environ['NUMEXPR_NUM_THREADS'] = '1'\n",
    "os.environ['OMP_NUM_THREADS'] = '1'\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import glob\n",
    "import math\n",
    "from datetime import date\n",
    "from functools import partial\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib import colors\n",
    "%config InlineBackend.figure_format = 'svg'\n",
    "plt.rcParams.update({'font.size': '16'})\n",
    "import warnings; warnings.simplefilter('ignore')\n",
    "# Load CUPY module\n",
    "import cupy as cp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffd14b11-4fad-4913-9a6c-ceb44bc0db6a",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# GPU Parallelism"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd642c04-a14e-4925-9533-5eb56d0cf461",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## CPU vs GPU\n",
    "\n",
    "- CPU (Central Processing Unit) \n",
    "    - highly flexible computing elements, capable of doing any task required\n",
    "    - Few very fast cores with large slow-access memory\n",
    "\n",
    "- GPU (Graphics Processing Unit):\n",
    "    - developed to handle very specific tasks, like ray-tracing and image rendering\n",
    "    - 1000s of cores with small high-speed memory\n",
    "\n",
    "- Some problems can be effectively split across the GPU cores for incredible speed-ups"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4ba377f-4122-4745-8d17-a203f130e866",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Vectorized Functions\n",
    "\n",
    "- Vectorization: Applying the same function to every element of an array\n",
    "- Example 1: Operate a function on each element of an array\n",
    "- Example 2: matrix multiplication with large matrices (10k x 10k)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5fce55a-6218-40b2-9e1f-f1f73b78b63c",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## GPUs on the Clusters\n",
    "\n",
    "- We have a collection of GPUs available on Farnam and Grace\n",
    "- Requesting these resources is straightforward ([YCRC docs](http://docs.ycrc.yale.edu/clusters-at-yale/job-scheduling/resource-requests/#request-gpus)), and only envolves adding a few flags to your `salloc` or `sbatch` commands:\n",
    "\n",
    "`salloc --x11 -p gpu_devel -t 2:00:00 --gpus=1`\n",
    "\n",
    "- This will request one GPU (a \"general resource\" or `gres`) from the `gpu_devel` partition\n",
    "- Similar commands can be added to batch SLURM scripts and run on the `gpu` partition"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f7b9df0-6ecc-4b78-a3de-0667c734cce9",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## PyCUDA\n",
    "_https://documen.tician.de/pycuda/_\n",
    "\n",
    "- Python connection to NVIDIA's CUDA GPU framework\n",
    "- Low-level code written in C++, but all the mess is abstracted away\n",
    "- Still rather complex to work with, but very powerful"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fd5affe-a75d-42a0-b4ba-5ac5abb0c6ce",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Cupy\n",
    "\n",
    "_https://docs-cupy.chainer.org/en/stable/_\n",
    "\n",
    "- Drop-in replacement for numpy (fully compatible API)\n",
    "- Allows for near seamless GPU-based computation \n",
    "- Matrix multiplication, vector operations, etc.\n",
    "\n",
    "Easily installed via `conda` after loading the `CUDA` module on the clusters\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d3494078-7a3e-4f46-9f1b-b5e911589eab",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Load CUPY module\n",
    "import cupy as cp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36218934-30bd-4111-956e-c492b7489458",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "First, let's define a test routine with `numpy`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d6fc7b75-f674-4f9b-9be5-113eb42f04fd",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "%%timeit\n",
    "\n",
    "# Create 2D numpy arrays\n",
    "a = np.random.random(25000000)\n",
    "a = a.reshape(5000,5000)\n",
    "\n",
    "b = np.random.random(25000000)\n",
    "b = b.reshape(5000,5000)\n",
    "\n",
    "# Matrix Mult\n",
    "out = np.matmul(a,b)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6abe3d4-3c15-43cb-be54-45c3addd5ca4",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Now, let's perform the same code but running the multiplication on the GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e4ebdf1-0577-4bd2-abc5-cb29dfa1786a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "%%timeit\n",
    "\n",
    "# Create 2D numpy arrays\n",
    "a = np.random.random(25000000)\n",
    "a = a.reshape(5000,5000)\n",
    "\n",
    "b = np.random.random(25000000)\n",
    "b = b.reshape(5000,5000)\n",
    "\n",
    "# Move to GPU\n",
    "g = cp.asarray(a)\n",
    "h = cp.asarray(b)\n",
    "\n",
    "# Matrix Mult\n",
    "out = cp.matmul(g,h)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79261ea7-1c3c-4295-95a4-51a7dbe3f5db",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Considerablly faster matrix multiplication without any complicated parallel work!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efb63dc0-e2c7-4a0a-b481-e28da3e2be6c",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Example 4: NYC Taxi Cab Data (again)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d05cadf-b3a9-4823-9a31-c7e0023e51e7",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Load data using Pandas\n",
    "Pandas has very friendly tools for reading data, we will use the `read_csv` method to read our Taxi Cab data before converting it to numpy arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "35f3523e-01ad-45e1-8865-fbf643ac1208",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "january = pd.read_parquet('../taxi/yellow_tripdata_2022-01.parquet')\n",
    "july = pd.read_parquet('../taxi/yellow_tripdata_2022-07.parquet')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3258ceff-e497-4c9c-a181-97656bf90932",
   "metadata": {},
   "outputs": [],
   "source": [
    "tip_jan = np.array(january['tip_amount'])\n",
    "distance_jan = np.array(january['trip_distance'])\n",
    "\n",
    "tip_jul = np.array(july['tip_amount'])\n",
    "distance_jul = np.array(july['trip_distance'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c6c4f73-b921-465a-bc60-9e320fc5758d",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Move data to GPU\n",
    "\n",
    "Cupy has built-in tools to move data to and from GPUs, `cp.asarray()` and `cp.asnumpy`. We will use these to analyze data from the Taxi Cab dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ea078f08-65f3-483c-93de-aa556c178876",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "gpu_tip_jan = cp.asarray(tip_jan)\n",
    "gpu_dist_jan = cp.asarray(distance_jan)\n",
    "\n",
    "gpu_tip_jul = cp.asarray(tip_jul)\n",
    "gpu_dist_jul = cp.asarray(distance_jul)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4c5e3c2-713b-4a48-bc22-cfdd2912940e",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Comparison of CPU and GPU performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1c755c3d-5872-4c09-bc8f-7766fb8868a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%timeit\n",
    "np.divide(tip_jan, distance_jan)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80507665-01ad-492f-af3d-24e733aded8f",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%timeit \n",
    "gpu_tip_per_mile = cp.divide(gpu_tip_jan, gpu_dist_jan)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b94026ff-ccdf-4f16-871e-2afb98b858dd",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Visualizing Results\n",
    "\n",
    "Data have to be pulled off the GPU to be able to visualize them. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "90e998ac-f924-4e71-ac5a-3ae6394bd118",
   "metadata": {},
   "outputs": [],
   "source": [
    "gpu_tip_per_mile = cp.divide(gpu_tip_jan, gpu_dist_jan)\n",
    "tpm = cp.asnumpy(gpu_tip_per_mile)[(tip_jan > 0) & (distance_jan > 1)]\n",
    "\n",
    "plt.hist(tpm, bins=200, range=(0.1,10), histtype='step', label='January');\n",
    "print(f'January Average: {np.mean(tpm[(tpm>0)&(tpm<10)]):.3f}')\n",
    "\n",
    "gpu_tip_per_mile = cp.divide(gpu_tip_jul, gpu_dist_jul)\n",
    "tpm = cp.asnumpy(gpu_tip_per_mile)[(tip_jul > 0) & (distance_jul > 1)]\n",
    "\n",
    "plt.hist(tpm, bins=200, range=(0.1,10), histtype='step', label='July');\n",
    "print(f'July Average: {np.mean(tpm[(tpm>0)&(tpm<10)]):.3f}')\n",
    "\n",
    "plt.xlabel('Tip Efficiency (Dollar/Mile)');plt.ylabel('Rides');plt.yscale('log');plt.legend();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a266bd4-b98a-4e80-9094-c5c72a938794",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## GPU Summary\n",
    "\n",
    "- There are a ton of exciting projects that are starting to utilize GPUs. \n",
    "- Having python connection to these tools enable rapid work with machine learning or other computationally intensive tasks\n",
    "- Make use of the GPUs in the clusters to get started with this kind of tool"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}