03_functional_programming.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Functional Programming"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions Are Just Objects"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What's the difference between an integer and a function?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = 42\n",
    "def add_one(x):\n",
    "    return x + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A function has a type, just like an integer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(a), type(add_one)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A function has an identity, just like an integer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "id(a), id(add_one)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`add_one` is just a _name_ of a function object, just as `a` is the _name_ of the integer object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "add_one, add_one(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A function has attributes, like other objects:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dir(add_one)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note the `__call__()` magic method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "add_one.__call__(42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dataclasses import dataclass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can define classes having a `__call__()` method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@dataclass\n",
    "class AddN:\n",
    "    n: int\n",
    "    \n",
    "    def __call__(self, x) -> int:\n",
    "        return x + self.n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... and call objects of these classes as if they were functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "add_two = AddN(2)\n",
    "add_two(42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So a function is nothing else than an object with a `__call__()` method implemented. Every mention of _callable_ in documentation or type hints means that the \"callable\" object has a `__call__()` implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "callable(add_one), callable(add_two)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Discussion__: What would be a good use case for having an (obviously more complex) class with a `__call__()` method instead of a simple function definition?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions as Arguments"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because functions are objects, we can pass them as arguments to other functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Callable, Iterable, TypeVar, Any\n",
    "\n",
    "T = TypeVar('T')\n",
    "\n",
    "def mapper(fn: Callable[[T], T], l: Iterable[T]) -> list[T]:\n",
    "    return [fn(x) for x in l]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice the `TypeVar` `T`, which allows us to specify that the exact types of elements of the iterables and callable signature are unknown, but they all must be of the same type. If we had used `Any` instead of `T`, there would be no way to check if elements of `l` match the type of `fn`'s arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mapper(add_one, [1, 10, 100])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def is_even(n: int) -> bool:\n",
    "    return n % 2 == 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Exercise:__ Create a function `filters(fn, l)` that accepts a callable and iterable, just as `mapper()` above, and returns the elements of the iterable for which `fn` returns `True`. You can use the function `is_even()` to test your solution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your solution:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert filters(is_even, [1, 2, 3, 4]) == [2, 4]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our `mapper()` and `filters()` functions were just there for illustration purposes: Python has the built-in functions [`map()`](https://docs.python.org/3/library/functions.html#map) and [`filter()`](https://docs.python.org/3/library/functions.html#filter)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transaction_df = pd.DataFrame({\n",
    "    'amount': [42., 100., 999.],\n",
    "    'from': ['bob', 'alice', 'bob'],\n",
    "    'to': ['alice', 'bob', 'alice']\n",
    "})\n",
    "transaction_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's define a function that indicates if the amount of a transaction is larger than 100:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def select_large_transactions(transaction_df):\n",
    "    return transaction_df['amount'] > 100  # Where does transaction_df refer to?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since `.loc[]` accepts a _callable_ as input, we can pass our function to it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transaction_df.loc[select_large_transactions]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we didn't _call_ the function ourselves. It was `.loc[]` that called our function, and it was `loc[]` that passed whatever DataFrame it was bound to at that moment as the first argument to our function."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can similarly pass functions to `assign()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_commission(transaction_df):\n",
    "    return transaction_df['amount'] * 0.05"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transaction_df.assign(commission=get_commission)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also _freeze_ some arguments of a function before passing it using [`functools.partial()`](https://docs.python.org/3/library/functools.html#functools.partial) from the standard library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from functools import partial"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_n(x: int, n: int = 1):\n",
    "    return x + n\n",
    "\n",
    "add_three = partial(add_n, n=3)\n",
    "add_three(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_commission(transaction_df, commission_percent=5):\n",
    "    return transaction_df['amount'] * (commission_percent / 100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transaction_df.assign(commission=partial(get_commission, commission_percent=10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions Returning Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If functions can accept other functions as arguments, they surely can also _return_ functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_n(n: int) -> Callable[[int], int]:\n",
    "    def adder(x: int) -> int:\n",
    "        return x + n\n",
    "    return adder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "add_two = add_n(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "add_two, type(add_two)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "add_two(42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mapper(add_two, [1, 42, 100])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mapper(add_two, [1, None, 100])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There's a problem when we pass `None` (or other unexpected types) to the function that we apply on our iterable. We can fix it by modifying the adder function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_one(x: int) -> int:\n",
    "    if x is not None:\n",
    "        return x + 1\n",
    "    else:\n",
    "        return None\n",
    "\n",
    "mapper(add_one, [1, None, 100])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why is this sub-optimal? What if there's an `add_two()` etc? Do we need to repeat all the checking for None everywhere?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def skip_None(fn: Callable[[int | None], int | None]) -> Callable[[int | None], int | None]:\n",
    "    def fn_wrapper(n: int | None) -> int | None:\n",
    "        if n is not None:\n",
    "            return fn(n)\n",
    "        else:\n",
    "            return None\n",
    "    \n",
    "    return fn_wrapper"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A solution is to define a _wrapper_ function `fn_wrapper()` that intercepts calls to the wrapped function `fn`, and only calls the wrapped function (and returns its result) if the arguments passed to the wrapped function are valid. Note that this wrapper function is defined inside the body of another function `skip_None()`, which accepts the actual function to be wrapped."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_one(x: int) -> int:\n",
    "    return x + 1\n",
    "\n",
    "def add_two(x: int) -> int:\n",
    "    return x + 2\n",
    "\n",
    "add_one_wrapped = skip_None(add_one)\n",
    "add_two_wrapped = skip_None(add_two)\n",
    "mapper(add_one_wrapped, [1, None, 100]), mapper(add_two_wrapped, [1, None, 100])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instead of manually calling `skip_None()` to wrap our adder functions as above, Python has the `@` _decorator_ construct, which is just a syntactic shortcut to help us write more concise code. The following snippets are all equivalent:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_one(x: int) -> int:\n",
    "    return x + 1\n",
    "\n",
    "skip_None(add_one)(1), skip_None(add_one)(None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_one(x: int) -> int:\n",
    "    return x + 1\n",
    "\n",
    "add_one = skip_None(add_one)\n",
    "add_one(1), add_one(None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@skip_None\n",
    "def add_one(x: int) -> int:\n",
    "    return x + 1\n",
    "\n",
    "add_one(1), add_one(None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also write decorators that accept arguments. These add a second layer of inner function nesting:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def check_value(\n",
    "    max_expected: float\n",
    ") -> Callable[[Callable[[float], float]], Callable[[float], float]]:  # 🤯\n",
    "    def value_check_decorator(\n",
    "        fn: Callable[[float], float]\n",
    "    ) -> Callable[[float], float]:\n",
    "        def fn_wrapper(n: float) -> float:\n",
    "            if n > max_expected:\n",
    "                print(f'Unusual value {n}, expected a maximum of {max_expected}')\n",
    "            return fn(n)\n",
    "        return fn_wrapper\n",
    "    return value_check_decorator\n",
    "\n",
    "@check_value(max_expected=42)\n",
    "def add_one(x: int) -> int:\n",
    "    return x + 1\n",
    "\n",
    "mapper(add_one, [1, 42, 100])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Seeing decorators like these for the first time may be confusing. The following snippets clarify the nesting of functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_one(x: int) -> int:\n",
    "    return x + 1\n",
    "\n",
    "max_41_checker = check_value(max_expected=41)\n",
    "add_one_with_max_41_checking = max_41_checker(add_one)\n",
    "add_one_with_max_41_checking(40), add_one_with_max_41_checking(42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    check_value(max_expected=41)(add_one)(40),\n",
    "    check_value(max_expected=41)(add_one)(42)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Anonymous Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we intend to use a one-off function that consists of returning the evaluation of one expression, we can use _lambda expressions_ instead of defining a (named) function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mapper(lambda x: x + 10, [1, 42, 100])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even a lambda expression ... is just an object (of type `function`)!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(lambda x: x + 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "id(lambda x: x + 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dir(lambda x: x + 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(lambda x: x + 10)(42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can even assign a lambda expression to a variable, but this is discouraged in the \"official\" [Python Style Guide](https://www.python.org/dev/peps/pep-0008/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Don't do this IRL\n",
    "add_ten = lambda x: x + 10\n",
    "add_ten(42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lambda expressions can have more than one argument:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from functools import reduce"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "reduce(lambda x, y: x + y, [1, 2, 3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look back at our Vector class from the previous module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dataclasses import dataclass\n",
    "from __future__ import annotations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@dataclass\n",
    "class Vector:\n",
    "    values: list[float]\n",
    "    \n",
    "    def __getitem__(self, index: int) -> float:\n",
    "        return self.values[index]\n",
    "    \n",
    "    def __len__(self) -> int:\n",
    "        return len(self.values)\n",
    "    \n",
    "    def __mul__(self, scalar: float) -> Vector:\n",
    "        return Vector([v * scalar for v in self.values])\n",
    "    \n",
    "    def __add__(self, other: Vector) -> Vector:\n",
    "        return Vector([self[i] + other[i] for i in range(len(self))])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Exercise__: Add a method `pipe()` to our `Vector` class, which accepts as argument a function that transforms its input vector argument to another vector. As an example of such transformation functions, consider `rotate_right()` and `rotate_left()` given below. Test your implementation with the assertions below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rotate_right(v: Vector) -> Vector:\n",
    "    # rotates input by 90 degrees clockwise\n",
    "    return Vector([v[1], -v[0]])\n",
    "\n",
    "def rotate_left(v: Vector) -> Vector:\n",
    "    # rotates input by 90 degrees counterclockwise\n",
    "    return Vector([-v[1], v[0]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your solution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# %load solutions/vector_pipe.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert Vector([1, 1]).pipe(rotate_right) == Vector([1, -1])\n",
    "assert Vector([1, 1]).pipe(rotate_left).pipe(rotate_left) == Vector([-1, -1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Bonus Exercise__: Make sure that our `pipe()` method can accept additional arguments: create a generic `rotate()` function that accepts an optional `direction` keyword argument with possible values `'right'`, `'clockwise'`, `'left'`, `'counterclockwise'`. If an invalid or no `direction` keyword argument is provided, `rotate()` should return the original vector. See the assertions below for the expected behavior of the solution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your solution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# %load solutions/vector_pipe_vararg.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# bonus, have one generic rotation function\n",
    "assert Vector([1, 1]).pipe(rotate) == Vector([1, 1])\n",
    "assert Vector([1, 1]).pipe(rotate, direction='right').pipe(rotate, direction='counterclockwise') == Vector([1, 1])\n",
    "assert Vector([1, 1]).pipe(rotate, direction='clockwise').pipe(rotate, direction='left') == Vector([1, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# bonus, deal with any Vector manipulation function\n",
    "assert Vector([1, 1]).pipe(lambda v: Vector([v[0] * 42, v[1] * 99])) == Vector([42, 99])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By now, the pandas `pipe()` method should have no secrets anymore:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transaction_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def select_amount_greater_than(tx_df, amount=100):\n",
    "    return tx_df.loc[lambda df: df['amount'] > amount]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transaction_df.pipe(select_amount_greater_than)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transaction_df.pipe(select_amount_greater_than, amount=99)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transaction_df.pipe(lambda df: df.loc[df['to'].isin(['bob', 'carol'])])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}