04_dataloader_speed_issues.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exploring TS definitions...\n",
    "> Evaluating the performance of iterating over a bucnh of timeseries."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will explore different possibilities to construct a data pipeline that is fast and simple using fastai building blocks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Disclaimer: Ignacio's tsai solves this issue doing something very similar to the solution proposed at the end of this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "import pandas as pd\n",
    "from fastcore.all import *\n",
    "from scipy.io import arff\n",
    "\n",
    "import fastai\n",
    "from fastcore.test import *\n",
    "from fastai.basics import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fastai: 2.1.6 \n",
      "torch:  1.7.0\n"
     ]
    }
   ],
   "source": [
    "print(f'fastai: {fastai.__version__} \\ntorch:  {torch.__version__}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TESTS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = np.random.rand(10000, 1000)\n",
    "y = np.random.randint(0,10,10000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading from arrays:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The simples possible idea, would be to build a Numpy (kinda of scipy) type of Dataset. We only need to define how to `__getitem__` and `__len__`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class NaiveNumpyDataset(torch.utils.data.Dataset):\n",
    "    \"A Dataset that stores numpy arrays\"\n",
    "    def __init__(self, X, y=None):\n",
    "        self.X, self.y = X, y\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        if self.y is None: return (self.X[idx], )\n",
    "        else: return (self.X[idx], self.y[idx])\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds = NaiveNumpyDataset(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we can convert this to a dataloader directly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dls_torch = torch.utils.data.DataLoader(dataset=ds, batch_size=8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The fastai's DataLoaders (it is just thin wrapper around pytorch's one, or not?)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dls = DataLoaders.from_dsets(ds, bs=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cycle_dl(dl):\n",
    "    for _ in iter(dl):\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "87.1 ms ± 841 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit cycle_dl(dls_torch)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "827 ms ± 15.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "%timeit cycle_dl(dls.train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "it is 10 times slower!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "in real life, we would like to iterate on torch Tensors and send the output to GPU... `TSeries.create` does this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class NumpyDataset():\n",
    "    \"Tensor aware implementation\"\n",
    "    def __init__(self, X, y=None):\n",
    "        self.X, self.y = X, y\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        if self.y is None: return (self.X[idx], )\n",
    "        else: return (TSeries.create(self.X[idx]), TensorCategory(self.y[idx]))\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds = NumpyDataset(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(TSeries(ch=1, len=1000), TensorCategory(5))"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dls = DataLoaders.from_dsets(ds, bs=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(TSeries(ch=8, len=1000), TensorCategory([5, 1, 6, 5, 4, 4, 4, 7]))"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dls.train.one_batch()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.1 s ± 24.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "%timeit cycle_dl(dls.train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "the casting is painfully slow..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TSTransform(Transform):\n",
    "    def __init__(self, x, y):\n",
    "        self.x, self.y = x, y\n",
    "        \n",
    "    def encodes(self, i):\n",
    "        return (TSeries.create(self.x[i]), TensorCategory(self.y[i]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tl = TfmdLists(range_of(X), TSTransform(X, y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(#5) [(TSeries(ch=1, len=1000), TensorCategory(5)),(TSeries(ch=1, len=1000), TensorCategory(1)),(TSeries(ch=1, len=1000), TensorCategory(6)),(TSeries(ch=1, len=1000), TensorCategory(4)),(TSeries(ch=1, len=1000), TensorCategory(5))]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tl[0:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dls = DataLoaders.from_dsets(tl, bs=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bx, by = dls.one_batch()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.13 s ± 21.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "%timeit cycle_dl(dls.train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dl =TfmdDL(tl, bs=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.12 s ± 12.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "%timeit cycle_dl(dl)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using TfmLists is not better..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The `contigous memory solution`\n",
    "> Fixing the issue of slicing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Slicer:\n",
    "    \"slice numpy ds\"\n",
    "    def __init__(self,to): self.to = to\n",
    "    def __getitem__(self, idxs):\n",
    "        return self.to.new(*self.to[idxs])\n",
    "\n",
    "class NumpyDataset2():\n",
    "    def __init__(self, X, y=None):\n",
    "        self.X, self.y = X, y\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        if self.y is None: return (self.X[idx], )\n",
    "        else: return (self.X[idx], self.y[idx])\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.X)\n",
    "\n",
    "    @property\n",
    "    def slicer(self):\n",
    "        return Slicer(self)\n",
    "\n",
    "    def new(self, X, y):\n",
    "        return type(self)(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds = NumpyDataset2(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<__main__.NumpyDataset2 at 0x7f81d9d2e160>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds.slicer[0:4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ReadTSBatch(ItemTransform):\n",
    "    def __init__(self, to): self.to = to\n",
    "\n",
    "    def encodes(self, to):\n",
    "        res = (tensor(to.X).float(), )\n",
    "        res = res + (tensor(to.y),)\n",
    "        # if to.device is not None: res = to_device(res, to.device)\n",
    "        return res"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[0.2080, 0.8743, 0.1296,  ..., 0.8668, 0.8287, 0.5834],\n",
       "         [0.4801, 0.7196, 0.1208,  ..., 0.8895, 0.9759, 0.4256],\n",
       "         [0.7388, 0.8696, 0.8988,  ..., 0.8905, 0.6833, 0.7819],\n",
       "         [0.6905, 0.0943, 0.7811,  ..., 0.4363, 0.2213, 0.9070]]),\n",
       " tensor([5, 1, 6, 4]))"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rtb = ReadTSBatch(ds) \n",
    "rtb.encodes(ds.slicer[0:4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TSDataloader(TfmdDL):\n",
    "    do_item = noops\n",
    "    def __init__(self, dataset, bs=16, shuffle=False, after_batch=None, num_workers=0, **kwargs):\n",
    "        if after_batch is None: after_batch = L(TransformBlock().batch_tfms)+ReadTSBatch(dataset)\n",
    "        super().__init__(dataset, bs=bs, shuffle=shuffle, after_batch=after_batch, num_workers=num_workers, **kwargs)\n",
    "\n",
    "    def create_batch(self, b): return self.dataset.slicer[b]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dl = TSDataloader(ds, bs=128)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20.6 ms ± 587 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit cycle_dl(dl)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}