04b_datetime.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Times and Dates in Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `datetime` module\n",
    "\n",
    "The `datetime` module handles time with the Gregorian calendar (the calendar we, as humans, are familiar with); it is independent of Unix time. The `datetime` module uses an [object-oriented](#Thirty-second-introduction-to-Object-Oriented-programming) approach; it contains the `date`, `time`, `datetime`, `timedelta`, and `tzinfo` classes.\n",
    "\n",
    "-   `date` class represents the day, month, and year\n",
    "-   `time` class represents the time of day\n",
    "-   `datetime` class is a combination of the `date` and `time` classes\n",
    "-   `timedelta` class represents a time duration\n",
    "-   `tzinfo` class represents time zones, and is an abstract class.\n",
    "\n",
    "The `datetime` module is effective for:\n",
    "\n",
    "-   performing date and time arithmetic and calculating time duration\n",
    "-   reading and writing date and time strings with various formats\n",
    "-   handling time zones (with the help of third-party libraries)\n",
    "\n",
    "The `time` and `datetime` modules overlap in functionality, but in your geoscientific work, you will probably be using the `datetime` module more than the `time` module."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[The full range of formatting options for strftime() and strptime() is described in the Python documentation](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior). In most cases, finding the correct formatting options inherently takes some degree of experimentation to get right. This is a situation where Python shines; you can use the IPython interpreter, or a Jupyter notebook, to quickly test numerous formatting options. Beyond the official documentation, Google and [Stack Overflow](https://stackoverflow.com/) are your friends in this process. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime as dt\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021-03-14 15:09:26\n"
     ]
    }
   ],
   "source": [
    "pisecond = dt.datetime(2021, 3, 14, 15, 9, 26)\n",
    "print(pisecond)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "03/14/2021\n"
     ]
    }
   ],
   "source": [
    "print(pisecond.strftime(format='%m/%d/%Y'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2007-06-27 16:18:21.898000\n"
     ]
    }
   ],
   "source": [
    "strike_time = dt.datetime.strptime('06/27/07 16:18:21.898', '%m/%d/%y %H:%M:%S.%f')\n",
    "print(strike_time)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16h 18m 21s\n"
     ]
    }
   ],
   "source": [
    "print(strike_time.strftime(format='%Hh %Mm %Ss'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2007"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "strike_time.year"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This works because the `datetime` object stores the data as individual attributes: \n",
    "`year`, `month`, `day`, `hour`, `minute`, `second`, `microsecond`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### See how many days have elapsed since the strike:\n",
    "\n",
    "This example shows how to find the number of days since an event; in this case, the lightning strike described earlier:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5956"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(dt.datetime.now() - strike_time).days"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To illustrate that the difference of two times yields a `timedelta` object, we can use a built-in Python function called `type()`, which returns the type of its argument.  In the above example, we call `type()` in the last `print` statement, and it returns the type of `timedelta`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(dt.datetime.now() - strike_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following table, drawn from the [NumPy datetime64 documentation](http://docs.scipy.org/doc/numpy/reference/arrays.datetime.html), lists the available format codes along with the relative and absolute timespans that they can encode:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "|Code    | Meaning     | Time span (relative) | Time span (absolute)   |\n",
    "|--------|-------------|----------------------|------------------------|\n",
    "| ``Y``  | Year\t       | ± 9.2e18 years       | [9.2e18 BC, 9.2e18 AD] |\n",
    "| ``M``  | Month       | ± 7.6e17 years       | [7.6e17 BC, 7.6e17 AD] |\n",
    "| ``W``  | Week\t       | ± 1.7e17 years       | [1.7e17 BC, 1.7e17 AD] |\n",
    "| ``D``  | Day         | ± 2.5e16 years       | [2.5e16 BC, 2.5e16 AD] |\n",
    "| ``h``  | Hour        | ± 1.0e15 years       | [1.0e15 BC, 1.0e15 AD] |\n",
    "| ``m``  | Minute      | ± 1.7e13 years       | [1.7e13 BC, 1.7e13 AD] |\n",
    "| ``s``  | Second      | ± 2.9e12 years       | [ 2.9e9 BC, 2.9e9 AD]  |\n",
    "| ``ms`` | Millisecond | ± 2.9e9 years        | [ 2.9e6 BC, 2.9e6 AD]  |\n",
    "| ``us`` | Microsecond | ± 2.9e6 years        | [290301 BC, 294241 AD] |\n",
    "| ``ns`` | Nanosecond  | ± 292 years          | [ 1678 AD, 2262 AD]    |\n",
    "| ``ps`` | Picosecond  | ± 106 days           | [ 1969 AD, 1970 AD]    |\n",
    "| ``fs`` | Femtosecond | ± 2.6 hours          | [ 1969 AD, 1970 AD]    |\n",
    "| ``as`` | Attosecond  | ± 9.2 seconds        | [ 1969 AD, 1970 AD]    |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the types of data we see in the real world, a useful default is ``datetime64[ns]``, as it can encode a useful range of modern dates with a suitably fine precision."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dates and times in pandas\n",
    "\n",
    "Pandas builds upon all the tools just discussed to provide a ``Timestamp`` object, which combines the ease-of-use of ``datetime`` and ``dateutil`` with the efficient storage and vectorized interface of ``numpy.datetime64``.\n",
    "From a group of these ``Timestamp`` objects, Pandas can construct a ``DatetimeIndex`` that can be used to index data in a ``Series`` or ``DataFrame``; we'll see many examples of this below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2015-07-04 04:30:00')"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# date = pd.to_datetime(\"4th of July, 2015\")\n",
    "date = pd.to_datetime(\"04-07-2015 04:30\", dayfirst=True)\n",
    "date"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Saturday'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "date.strftime('%A')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TimedeltaIndex([ '0 days',  '1 days',  '2 days',  '3 days',  '4 days',\n",
       "                 '5 days',  '6 days',  '7 days',  '8 days',  '9 days',\n",
       "                '10 days', '11 days'],\n",
       "               dtype='timedelta64[ns]', freq=None)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.to_timedelta(np.arange(12), 'D')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "date + pd.to_timedelta(np.arange(12), 'D')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.to_timedelta(np.arange(12), 'D')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pandas Time Series Data Structures\n",
    "\n",
    "This section will introduce the fundamental Pandas data structures for working with time series data:\n",
    "\n",
    "- For *time stamps*, Pandas provides the ``Timestamp`` type. As mentioned before, it is essentially a replacement for Python's native ``datetime``, but is based on the more efficient ``numpy.datetime64`` data type. The associated Index structure is ``DatetimeIndex``.\n",
    "- For *time Periods*, Pandas provides the ``Period`` type. This encodes a fixed-frequency interval based on ``numpy.datetime64``. The associated index structure is ``PeriodIndex``.\n",
    "- For *time deltas* or *durations*, Pandas provides the ``Timedelta`` type. ``Timedelta`` is a more efficient replacement for Python's native ``datetime.timedelta`` type, and is based on ``numpy.timedelta64``. The associated index structure is ``TimedeltaIndex``."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The most fundamental of these date/time objects are the ``Timestamp`` and ``DatetimeIndex`` objects.\n",
    "While these class objects can be invoked directly, it is more common to use the ``pd.to_datetime()`` function, which can parse a wide variety of formats.\n",
    "Passing a single date to ``pd.to_datetime()`` yields a ``Timestamp``; passing a series of dates by default yields a ``DatetimeIndex``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2015-07-03', '2015-07-04', '2015-07-06', '2015-07-07',\n",
       "               '2015-07-08'],\n",
       "              dtype='datetime64[ns]', freq=None)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dates = pd.to_datetime([dt.datetime(2015, 7, 3), '4th of July, 2015',\n",
    "                       '2015-Jul-6', '07-07-2015', '20150708'])\n",
    "dates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A ``TimedeltaIndex`` is created, for example, when a date is subtracted from another:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TimedeltaIndex(['1 days', '2 days', '1 days', '1 days'], dtype='timedelta64[ns]', freq=None)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dates[1:] - dates[0:-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Regular sequences: ``pd.date_range()``\n",
    "\n",
    "To make the creation of regular date sequences more convenient, Pandas offers a few functions for this purpose: ``pd.date_range()`` for timestamps, ``pd.period_range()`` for periods, and ``pd.timedelta_range()`` for time deltas.\n",
    "We've seen that Python's ``range()`` and NumPy's ``np.arange()`` turn a startpoint, endpoint, and optional stepsize into a sequence.\n",
    "Similarly, ``pd.date_range()`` accepts a start date, an end date, and an optional frequency code to create a regular sequence of dates.\n",
    "By default, the frequency is one day:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.arange(0, 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2015-07-03', '2015-07-04', '2015-07-05', '2015-07-06',\n",
       "               '2015-07-07', '2015-07-08', '2015-07-09', '2015-07-10',\n",
       "               '2015-07-11', '2015-07-12',\n",
       "               ...\n",
       "               '2020-06-24', '2020-06-25', '2020-06-26', '2020-06-27',\n",
       "               '2020-06-28', '2020-06-29', '2020-06-30', '2020-07-01',\n",
       "               '2020-07-02', '2020-07-03'],\n",
       "              dtype='datetime64[ns]', length=1828, freq='D')"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.date_range('2015-07-03', '2020-07-03')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, the date range can be specified not with a start and endpoint, but with a startpoint and a number of periods:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2015-07-03', '2015-07-04', '2015-07-05', '2015-07-06',\n",
       "               '2015-07-07', '2015-07-08', '2015-07-09', '2015-07-10'],\n",
       "              dtype='datetime64[ns]', freq='D')"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.date_range('2015-07-03', periods=8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The spacing can be modified by altering the ``freq`` argument, which defaults to ``D``.\n",
    "For example, here we will construct a range of hourly timestamps:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2015-08-01', '2015-09-01', '2015-10-01', '2015-11-01',\n",
       "               '2015-12-01', '2016-01-01', '2016-02-01', '2016-03-01'],\n",
       "              dtype='datetime64[ns]', freq='MS')"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.date_range('2015-07-03', periods=8, freq='MS')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create regular sequences of ``Period`` or ``Timedelta`` values, the very similar ``pd.period_range()`` and ``pd.timedelta_range()`` functions are useful.\n",
    "Here are some monthly periods:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.period_range('2015-07', periods=8, freq='M')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And a sequence of durations increasing by an hour:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.timedelta_range('1 days', periods=10, freq='H')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Frequencies and Offsets\n",
    "\n",
    "Fundamental to these Pandas time series tools is the concept of a frequency or date offset.\n",
    "Just as we saw the ``D`` (day) and ``H`` (hour) codes above, we can use such codes to specify any desired frequency spacing.\n",
    "The following table summarizes the main codes available:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Code   | Description         | Code   | Description          |\n",
    "|--------|---------------------|--------|----------------------|\n",
    "| ``D``  | Calendar day        | ``B``  | Business day         |\n",
    "| ``W``  | Weekly              |        |                      |\n",
    "| ``M``  | Month end           | ``BM`` | Business month end   |\n",
    "| ``Q``  | Quarter end         | ``BQ`` | Business quarter end |\n",
    "| ``A``  | Year end            | ``BA`` | Business year end    |\n",
    "| ``H``  | Hours               | ``BH`` | Business hours       |\n",
    "| ``T``  | Minutes             |        |                      |\n",
    "| ``S``  | Seconds             |        |                      |\n",
    "| ``L``  | Milliseonds         |        |                      |\n",
    "| ``U``  | Microseconds        |        |                      |\n",
    "| ``N``  | nanoseconds         |        |                      |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The monthly, quarterly, and annual frequencies are all marked at the end of the specified period.\n",
    "By adding an ``S`` suffix to any of these, they instead will be marked at the beginning:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Code    | Description            || Code    | Description            |\n",
    "|---------|------------------------||---------|------------------------|\n",
    "| ``MS``  | Month start            ||``BMS``  | Business month start   |\n",
    "| ``QS``  | Quarter start          ||``BQS``  | Business quarter start |\n",
    "| ``AS``  | Year start             ||``BAS``  | Business year start    |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Additionally, you can change the month used to mark any quarterly or annual code by adding a three-letter month code as a suffix:\n",
    "\n",
    "- ``Q-JAN``, ``BQ-FEB``, ``QS-MAR``, ``BQS-APR``, etc.\n",
    "- ``A-JAN``, ``BA-FEB``, ``AS-MAR``, ``BAS-APR``, etc.\n",
    "\n",
    "In the same way, the split-point of the weekly frequency can be modified by adding a three-letter weekday code:\n",
    "\n",
    "- ``W-SUN``, ``W-MON``, ``W-TUE``, ``W-WED``, etc.\n",
    "\n",
    "On top of this, codes can be combined with numbers to specify other frequencies.\n",
    "For example, for a frequency of 2 hours 30 minutes, we can combine the hour (``H``) and minute (``T``) codes as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.to_datetime('2022-07-01') + pd.timedelta_range(0, periods=9, freq=\"2H30T\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All of these short codes refer to specific instances of Pandas time series offsets, which can be found in the ``pd.tseries.offsets`` module.\n",
    "For example, we can create a business day offset directly as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pandas.tseries.offsets import BDay\n",
    "pd.date_range('2015-07-01', periods=5, freq=BDay())"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}