lectures/lecture06/lecture06.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![logo](https://www.python.org/static/community_logos/python-logo-master-v3-TM.png)\n",
    "\n",
    "# Python is a powerful, expressive, and easy to learn programming language. It is widely used in computational biology. There are a wealth of packages (numpy, scipy, pandas, statsmodels, scikit-learn) for data analysis in python. So it's **totally** worth learning!\n",
    "\n",
    "[Official python tutorial](https://docs.python.org/3/tutorial/)\n",
    "\n",
    "\n",
    "As we work through this notebook, you might want to open up a second one for trying things out. You can also fiddle with the code here.\n",
    "\n",
    "Here are a few tips for working with notebooks:\n",
    "\n",
    "* There are two modes: *command mode* and *edit mode*. \n",
    "* In edit mode you can edit the contents of individual cells. The selected cell is typically marked by a colored border. \n",
    "* In command mode you can move up and down between cells, add and delete cells, switch a cell's *type* between code and markdown, etc. \n",
    "* To enter command mode, hit `Esc` (the Escape key).\n",
    "* to enter edit mode with the selected cell (the one with the blue border or bar on the left-hand side), hit `Enter`. The border color will change to green (viewed in a browser, or become blue in vscode).\n",
    "* To run (execute) a cell (or selected cells), type `Ctrl-Enter`. `Shift-Enter` will run the current cell and select the next one.\n",
    "* To see the other keyboard shortcuts, click on the `Help` menu.\n",
    "* If you get stuck executing an infinite loop, choose `Interrupt` from the `Kernel` menu. \n",
    "* If you are in command mode, hitting the `a` key will add a new cell above the currently selected cell; `b` will add a new cell below the current cell.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# numeric types (int, float, complex)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In python, everything on a given line that follows a # is a COMMENT\n",
    "#\n",
    "# create an integer variable called 'a' and assign it a value of 4\n",
    "a = 4\n",
    "\n",
    "# the output or result of the last command we type in a notebook cell will appear below the\n",
    "# cell when we execute it\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**DIGRESSION on NAMES and OBJECTS**\n",
    "\n",
    "It's worth digging into this simple command `a = 4` a little more and learning some special\n",
    "python terms. What this command actually does is create an OBJECT of TYPE `int` (i.e., an integer)\n",
    "with the value of `4` and also a NAME (`a`) which points to the OBJECT.\n",
    "\n",
    "In python, objects exist and have individuality (they live somewhere in a specific place in the\n",
    "computer's memory). Multiple names can point to the same object. For simple objects like integers\n",
    "this doesn't matter so much, but we will see later that having two names referring ('pointing') to the\n",
    "same object can occasionally create confusion.\n",
    "\n",
    "It's OK to think of names as variables, if you are comfortable with the notion of a variable from\n",
    "another programming language or from math. As you become more proficient with python programming the\n",
    "NAME versus OBJECT distinction will be more important."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# type is a python FUNCTION that will tell us the python type of an object\n",
    "#\n",
    "# FUNCTIONs are little snippets of previously defined python code that have a name\n",
    "# and can be executed by typing that name followed by parentheses. FUNCTIONs can\n",
    "# do stuff (like printing output) and also RETURN VALUES for us to use. FUNCTIONS\n",
    "# may take ARGUMENTS that are passed in to the function to control their behavior. \n",
    "# ARGUMENTS (like 'a' below) go inside the parentheses. We will see later how to \n",
    "# define our own functions.\n",
    "\n",
    "# (Like everything else in python, FUNCTIONs are actually just OBJECTs)\n",
    "#\n",
    "type(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = 4\n",
    "a += 3 # += adds the value (or thing) on the right to the variable on the left; shorthand for \"a = a+3\"\n",
    "\n",
    "# print is a handy python FUNCTION that prints output to the terminal/notebook\n",
    "# Note that the output of print() appears in a different place from the last value in the cell\n",
    "# so we can have multiple outputs printed in a single cell\n",
    "print(a)\n",
    "\n",
    "# print can take multiple ARGUMENTS (things we pass in to FUNCTIONs to modify their behavior)\n",
    "# FUNCTION ARGUMENTS are separated by commas\n",
    "print('a+10 =', a+10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# a decimal point tells python this is an object of type float\n",
    "b = 3.5\n",
    "type(b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a/3 # In python 3, integer division returns floats rather than truncating"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a//3 # this is what a/3 does in python2: the fractional part is discarded, so we always get an integer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## a % b gives the remainder from dividing a by b (aka modulus)\n",
    "a = 17\n",
    "b = 3\n",
    "print(a, 'divided by', b, 'is', a//b, 'remainder', a%b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a=3\n",
    "b=0.1\n",
    "a+b # integer + float = float"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "c = 3 - 4j # we can do complex numbers, too. j is the special symbol for the square root of -1\n",
    "\n",
    "# abs() is a built-in python FUNCTION that works with all numeric types\n",
    "# abs of a complex number is the length of the hypotenuse when we plot the number in 2D,\n",
    "# that's why this comes out so nicely (3-4-5 triangle)\n",
    "print(c, 'absolute value of c=', abs(c))\n",
    "type(c)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# bools and comparisons"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# One equals sign for assignment, two for comparison\n",
    "a = 2\n",
    "b = 4\n",
    "a==b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "c = ( a == b ) # we can store the result of a comparison in a variable of type 'bool'\n",
    "print(c)\n",
    "type(c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print ( a > b )  # greater than\n",
    "print ( a <= b ) # less than OR equal to"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `strings`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = 'hello'\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# len is a SUPER IMPORTANT built-in python FUNCTION that tells us the length of OBJECTs like strings\n",
    "print(len(s))\n",
    "\n",
    "# strings have TYPE str\n",
    "type(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print (s) # print doesn't show the enclosing quotes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = \"ain't isn't a word\" # double quotes work fine too and are useful for strings with single quotes\n",
    "print (s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = \"\\\"ain't\\\" isn't a word\" # we can use backslash to escape quotes \n",
    "print(s)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = \"two-line\\nstring\" # \\n is a newline\n",
    "print (s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**OBJECTS CAN HAVE FUNCTIONS \"ATTACHED\" TO THEM**\n",
    "\n",
    "These special functions are called *methods*. To call a method, we use the '.' symbol\n",
    "after the object name, like: `object.method()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a new string object with the text 'hi there', assign the name 's' to point to it\n",
    "s = 'hi there'\n",
    "\n",
    "# call the upper() method of s. It returns an all-upper-case version of s\n",
    "s.upper()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**INDEXING**\n",
    "\n",
    "Access individual characters in a string using square braces ([]). **The numbering for indices starts at 0, not 1**. Negative indices count back from the end, with `-1` being the last element."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s='rhino'\n",
    "print ('s[0]=',s[0]) # As we saw above, the print FUNCTION can take multiple ARGUMENTS separated by commas\n",
    "print('s[2]=', s[2]) # the python language doesn't care about spacing around and within the parentheses\n",
    "print('s[-2]=', s[-2]) # there are style suggestions here: https://www.python.org/dev/peps/pep-0008/\n",
    "print('s[-1]=', s[-1]) # spaces after commas make it more readable\n",
    "\n",
    "# indexing into a string returns a new string object containing the requested character\n",
    "type(s[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**SLICING**\n",
    "\n",
    "Strings can be *sliced* to generate sub-strings, as shown in the next cell.\n",
    "Slicing can be applied to all sorts of python objects that are made of multiple\n",
    "individual elements (strings, lists, tuples, ...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s='peter pan'\n",
    "print ( s[0:5] ) # 0 is the start; slice runs from first index up to but not including the second index\n",
    "print ( len( s[2:-1]) ) # len tells us the length of objects like strings\n",
    "print ( s[:5] ) # python assumes a missing first index is 0 (ie slice starts at the beginning)\n",
    "print ( s[6:] ) # a missing second index means go all the way to the end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Strings can be *added* and *multiplied*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "'hello'+' goodbye'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "'whoah!'*3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Strings are immutable, ie they can't be changed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = 'hello'\n",
    "#s[0] = 'j' #this would give an error (try it)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to change a string, make a new one"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = 'hello'\n",
    "print(s)\n",
    "s  = 'j' + s[1:]\n",
    "print(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Converting between different types of objects.\n",
    "\n",
    "If we want to turn a string into an integer or vice versa, or more generally create an object of one type from an object of another, we can use the name of the desired TYPE as a FUNCTION and pass it the OBJECT of the other TYPE, and it will return a new OBJECT of the desired TYPE. Which sounds kind of complicated but is actually pretty simple:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create integer object \n",
    "a = 4\n",
    "\n",
    "# create string from 'a' -- see how this looks like we are calling a function called \"str\"\n",
    "b = str(a)\n",
    "\n",
    "b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create string object \n",
    "a = '57'\n",
    "\n",
    "# create integer from 'a' -- looks like we are calling the 'int' function\n",
    "b = int(a)\n",
    "\n",
    "b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### fstrings\n",
    "\n",
    "python3 gave us a SUPER convenient way of creating nicely formatted strings from other objects called `f-strings`. You don't need to know the details but it's helpful to know that `f-strings` exist."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# f-strings start with an f before the quotes, and contain {} expressions that are replaced with variables\n",
    "#  or other strings. The {} expressions can contain a ':' that is followed by cryptic instructions for\n",
    "#  how python should format the thing before the ':'\n",
    "\n",
    "name = 'Phil'\n",
    "age = 29\n",
    "\n",
    "message = f'My name is {name}, and I am {age} years old.' # yeah right\n",
    "message"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# f-strings are useful for formatting numeric values nicely, among many other things\n",
    "# here the :5d tells python to make the length 5, by padding with whitespace if necessary\n",
    "# and :9.2e says use scientific notation (e) with total length of 9 and 2 digits after the \n",
    "# decimal point\n",
    "#\n",
    "p_value = 0.000003545\n",
    "num_trials = 154\n",
    "print(f'number of trials: {num_trials:5d} P-value: {p_value:9.2e}')\n",
    "\n",
    "p_value = 0.000243\n",
    "num_trials = 12\n",
    "print(f'number of trials: {num_trials:5d} P-value: {p_value:9.2e}')\n",
    "\n",
    "p_value = 1.7456e-20\n",
    "num_trials = 12043\n",
    "print(f'number of trials: {num_trials:5d} P-value: {p_value:9.2e}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Practice time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In one line, create the string 'elephant' by combining slices from the two strings a and b below \n",
    "# Try doing it two different ways, once with and once without negative indices\n",
    "\n",
    "a = 'telephone'\n",
    "b = 'santa'\n",
    "#c = ?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In one line, create a boolean variable that will be True if the dna sequence below \n",
    "#  preserves the reading frame when inserted into a transcript\n",
    "#\n",
    "# Use the function len() which returns the length of a string. ie, len('hello') returns the integer 5,\n",
    "#  as well as the % operator introduced above.\n",
    "#\n",
    "dna_seq = 'TTATACGCGACTATCATATCGCCAGCCTTTGGAGTGTCAC'\n",
    "\n",
    "#in_frame = ?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `lists`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# I am in the habit of using the name 'l' for lists.\n",
    "# unfortunately an 'l' looks a lot like a '1' -- apologies in advance!\n",
    "#\n",
    "l = [1,2,3,4]\n",
    "print (l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Like strings, lists are python objects that have some useful *methods* (functions attached to objects). To see these methods interactively, try creating a list called `l` and typing `l.` and then pressing tab.\n",
    "\n",
    "By doing so you will see, for example, that `l` has a method called `append` which will add things to the end of the list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This shows that lists, unlike strings, are MUTABLE OBJECTS, which means that their contents\n",
    "# can be modified during the lifetime of the object\n",
    "#\n",
    "l.append( 5 )\n",
    "l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can get information about a function or object in the notebook by typing `<name>?` and running the cell. For example, try typing `l.append?` and pressing control+enter. This will work for functions you write in the future too, provided you write a little help message called a *docstring* at the beginning (more on this later)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "l.append?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that you can also use tab-completion: `l.appe[TAB]` will complete to `l.append`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lists can contain a mixture of types:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "l = [1,2,3,4,5]\n",
    "l.append( 'horse') # like strings\n",
    "print(l)\n",
    "l.append( [11,13,15] ) # or even other lists\n",
    "print(l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Indexing works pretty much the same for lists as for strings (but note that you *can* change elements of a list):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('l[0]=',l[0])\n",
    "print('l[-1]=',l[-1])\n",
    "l[2] = 'pony'\n",
    "print('now, l=',l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another very useful list method is `sort`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "l = [3,2,11,5,10,-15]\n",
    "l.sort()\n",
    "print(l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A useful `string` method is `split` -- it returns a `list` of strings generated by splitting the initial string, by default on whitespace but we can ask it to split on any string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = 'hi   there tim' # note multiple spaces between hi and there\n",
    "print(s.split()) # no whitespace in the output strings\n",
    "\n",
    "filename = '/Users/pbradley/gitrepos/tfcb_2021/lectures/lecture06/lecture06.pynb'\n",
    "print(filename.split('/'))\n",
    "\n",
    "shortname = filename.split('/')[-1]\n",
    "print(shortname)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# the string join method is like the opposite of split\n",
    "\n",
    "sep = '/'\n",
    "filename_pieces = ['', 'Users', 'pbradley', 'Documents', 'boring.docx']\n",
    "print(sep.join(filename_pieces))\n",
    "\n",
    "#or \n",
    "print('/'.join(filename_pieces))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Danger\n",
    "\n",
    "Assignment (e.g., setting `x=y`) doesn't generally make a new copy of the object that `y` points to. Use slicing (`x=y[:]`) or explicit creation of a new object with the object typename (e.g. `x=list(y)`) to create a copy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a new object of type list with the contents [1,2,3,4,5] and assign the name 'a' to point to it\n",
    "a = [1,2,3,4,5]\n",
    "print('a=',a)\n",
    "b = a # create a new name 'b' and point it at the same object that a points to\n",
    "b[0] = 1000\n",
    "print('b=',b)\n",
    "print('a=',a) # a was changed, too"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "c = a[:] # slicing a list makes a copy\n",
    "c[0] = 0\n",
    "print('c=',c)\n",
    "print('a=',a) # a still the same as before we modified c\n",
    "\n",
    "d = list(a)\n",
    "d[0] = -1\n",
    "print('d=',d)\n",
    "print('a=',a) # a still the same as before we modified d"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `dictionaries`\n",
    "Python provides an associative mapping object called a *dictionary* to easily hold key-value pairs. The type name is `dict`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we define dictionaries with curly braces {}\n",
    "base_partner = {'A':'T', 'T':'A', 'C':'G', 'G':'C'}\n",
    "\n",
    "# we look up the values for keys using square braces []\n",
    "print('A pairs with:', base_partner['A'])\n",
    "\n",
    "# trying to look up a key that's not already in the dictionary gives an error\n",
    "#base_partner['X']\n",
    "\n",
    "# we can assign new key,value pairs using [] notation:\n",
    "base_partner['N'] = 'N'\n",
    "\n",
    "print('base_partner=', base_partner)\n",
    "type(base_partner)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# the `in` keyword\n",
    "\n",
    "Python provides a handy way of checking whether something \"is contained in\" something else: the `in` keyword. It works with lots of different types: strings, lists, dictionaries, ... As with len(), indexing, and slicing, it's easy to make your new special python class behave nicely with `in` as well. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for strings, \"a in b\" checks if a is a substring of b \n",
    "s = 'ACTGATCGTA'\n",
    "print('G' in s)\n",
    "print('X' in s)\n",
    "print('TGA' in s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for lists, \"a in b\" checks if a is an element of b \n",
    "alist = [1,3,6]\n",
    "print(1 in alist)\n",
    "print(7 in alist)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for dictionaries, \"a in b\" checks if a is one of the keys of b\n",
    "d = {'a':1, 'b':2, 'c':3}\n",
    "print('a' in d)\n",
    "print(2 in d)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Practice time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# How many methods like \"append\" does a list have? Create a list and type l.[TAB] and count how many there are. \n",
    "\n",
    "#ans=?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Can you find the string method that returns a lower-case copy of the string?\n",
    "#\n",
    "# Again, you can define a string s and then type s.[TAB] to see the method names.\n",
    "# IMPORTANT: Note that this method does not change the starting string. strings are 'immutable' so their\n",
    "# methods will not change them, rather they return new copies. \n",
    "#\n",
    "# This is different from how many of the list methods behave.\n",
    "#"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What is the difference between list.append() and list.extend()?\n",
    "# First read the help messages for each using, for example, l.append?[CTRL+ENTER]\n",
    "# Then use trial and error by appending / extending different things onto a list and seeing what happens. \n",
    "# Does extend work with all input types?\n",
    "#"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a new list called l2 which has the same elements as the list l1 below but in decreasing\n",
    "#  order, using two list methods: one we've talked about and one we haven't\n",
    "#\n",
    "# Don't change l1 in the process! This will take a couple lines.\n",
    "\n",
    "l1 = [5, 1, 1, 13, 9, 3, 18, 15, 10, 8, 15, 7, 15, 10, 1, 13, 4, 15, 18, 9, 7, 3, 19, 8, 0, 14, 11, 0, 15, 16]\n",
    "\n",
    "#l2 = ... #\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OPTIONAL DIGRESSION: object introspection\n",
    "\n",
    "In programming, introspection refers to the ability to investigate the type and/or other properties of objects at runtime. Remember that just about everything in python is an object! Two useful functions for introspection are `dir` and `id`.\n",
    "\n",
    "`dir` shows all the methods (built-in functions) and attributes (accessible internal data) of an object.\n",
    "\n",
    "`id` returns a unique identifier for the given object (for example, the location in memory where it is stored). No two objects will have the same `id`. `id` can be useful for figuring out whether two names point to the same underlying object.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create list\n",
    "l = [1,2]\n",
    "\n",
    "# show attributes and methods of l\n",
    "print(dir(l))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = [4,5,6]\n",
    "b = a # did we make a copy or not??\n",
    "b.append(3)\n",
    "\n",
    "print('id(a)=', id(a))\n",
    "print('id(b)=', id(b))\n",
    "\n",
    "# here we store the result of a comparison (checking whether two things are equal) in a boolean variable (ie, object)\n",
    "the_same = id(a) == id(b)\n",
    "print('a and b point to the same object:', the_same)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# flow of control: `for` loops\n",
    "Python `for` loops let you repeat a block of code while changing the value of a LOOPING VARIABLE. The code to be repeated is INDENTED. This is a really important and distinctive thing about python: the formatting of the code, specifically the level of indenting, actually affects the syntax. Some people hate this. Some people love it. I like it and think it makes code more readable. But it takes some getting used to if you are coming from other languages like C/C++, java, etc. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# a simple for-loop over the elements of the list\n",
    "\n",
    "pets = ['dog', 'cat', 'goldfish']\n",
    "\n",
    "# Below, 'pet' is the looping variable (name)\n",
    "# it is equal to each of the elements of the list, in turn. First 'dog', then 'cat', then 'goldfish'\n",
    "# In other words, the first time through the loop, pet is a name that points to the first object in the list\n",
    "# the second time through the loop, pet points to the second object in the list. And so on.\n",
    "#\n",
    "for pet in pets:\n",
    "    print('A', pet, 'is a great pet, if you happen to like pets.')\n",
    "print('Those are the only pets I know.') # this only gets printed at the end; try indenting it and re-running...\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#  range(num) gives us a loop from 0 to num-1\n",
    "#  It's kind of like having a list of numbers running from 0 to num-1 (but not exactly the same)\n",
    "\n",
    "range(5)\n",
    "list(range(5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "range?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "product = 1\n",
    "\n",
    "## here i is the looping variable (name) that has a different value each time through the loop\n",
    "## \n",
    "for i in range(10):\n",
    "    print('2 to the', i, 'power=',product)\n",
    "    product *= 2\n",
    "    \n",
    "print('At the end, product=', product)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `for` is an incredibly versatile statement: you can loop over almost any kind of object that has multiple items or components\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we can loop over the letters in a string\n",
    "s = 'string'\n",
    "for a in s:\n",
    "    print(a) #try using the optional end= argument"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we can loop over the keys of a dictionary\n",
    "base_partner = {'A':'T', 'T':'A', 'C':'G', 'G':'C'}\n",
    "\n",
    "for a in base_partner:\n",
    "    print( a, 'pairs with', base_partner[a] )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we can loop over the elements of a list\n",
    "words = 'the quick brown fox jumps over the lazy dog'.split() # create a list by splitting a string\n",
    "\n",
    "counter=0\n",
    "# we can loop over the elements in a  list\n",
    "for word in words:\n",
    "    print('The',counter,'word is',word)\n",
    "    counter +=1\n",
    "    \n",
    "# we can loop over a range of numbers\n",
    "print('\\nAgain,') # using \\n to add some space\n",
    "for i in range(len(words)):\n",
    "    print('The',i,'word is',words[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `if` statements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# if statements let you choose between two (or more) outcomes based on a boolean expression \n",
    "#\n",
    "for i in range(10):\n",
    "    if i%2==0:\n",
    "        print(i,'is even')\n",
    "    else:\n",
    "        print(i,'is odd')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# if statements let you choose between two (or more) outcomes based on a boolean expression \n",
    "#\n",
    "# Notice what happens for 6:\n",
    "#\n",
    "\n",
    "for i in range(10):\n",
    "    if i%2==0:\n",
    "        print(i,'is even')\n",
    "    elif i%3==0:\n",
    "        print(i,'is divisible by 3')\n",
    "    else:\n",
    "        print(i,'is not')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## the `break` statement gets us out of a loop, while `continue` jumps directly to the next cycle through."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(1000):\n",
    "    if i == 7:\n",
    "        continue # 7 is a bad number\n",
    "\n",
    "    print('3 to the',i,'power=',3**i)\n",
    "    \n",
    "    if i >= 9: # we don't actually want that much output\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `while` loops\n",
    "\n",
    "`while` loops repeat a block of indented code as long as a boolean expression evaluates to `True`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# SUPER-HANDY TIP:\n",
    "# you can assign two (or more) variables at the same time. This is called 'unpacking' in python.\n",
    "a, b = 0, 1\n",
    "\n",
    "# generate the fibonacci sequence\n",
    "while a<1000:\n",
    "    print('a=',a)\n",
    "    # Here unpacking saves us from having to use a temporary variable\n",
    "    a, b = b, a+b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Practice time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write a little program to generate the list of all the even, non-negative integers less than 100.\n",
    "#\n",
    "# Do this two ways,\n",
    "#  1) by using if, %, and append inside a for loop up to 100\n",
    "#  2) just by using append and a shorter loop (even numbers are all multiples of...)\n",
    "#\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write a little program to generate the reverse complement of the following nucleotide sequence\n",
    "# Recall that A pairs with T and C pairs with G\n",
    "# \n",
    "\n",
    "forward_dna_seq = 'TTATACGCGACTATCATATCGCCAGCCTTTGGAGTGTCAC'\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write a little program to generate the list of all the prime numbers less than 100.\n",
    "# Again, % is your friend here. You will probably need to do a for loop inside a for loop!\n",
    "#\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Defining new functions\n",
    "We use the `def` keyword to define a new function. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def say_hello():\n",
    "    print('Hello!')\n",
    "\n",
    "say_hello()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# functions can take inputs, called \"arguments\" or \"parameters\"\n",
    "def greet(name):\n",
    "    print('Nice to meet you,', name)\n",
    "\n",
    "greet('phil')\n",
    "greet('charlie')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Functions can return results to the caller, using the `return` statement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "base_partner = {'A':'T', 'T':'A', 'C':'G', 'G':'C'}\n",
    "num_hbonds = {'A':2, 'T':2, 'C':3, 'G':3}\n",
    "\n",
    "\n",
    "def reverse_complement(seq):\n",
    "    \"returns the reverse complement of a nucleic acid sequence\"\n",
    "    bwd = ''\n",
    "    for base in seq:\n",
    "        bwd = base_partner[base] + bwd\n",
    "    return bwd\n",
    "\n",
    "def total_hbonds(seq):\n",
    "    'returns the total number of watson-crick hbonds between seq and its reverse complement'\n",
    "    total=0\n",
    "    for a in seq:\n",
    "        total += num_hbonds[a]\n",
    "    return total\n",
    "\n",
    "fwd = 'ACGGTAATGATCCTCAG'\n",
    "rev = reverse_complement( fwd )\n",
    "\n",
    "print ('fwd=',fwd,'rev=',rev,'num_hbonds=',total_hbonds(fwd))\n",
    "\n",
    "# an assert statement checks to see if something is true, and will interrupt the program if it's not.\n",
    "assert reverse_complement(reverse_complement(fwd)) == fwd\n",
    "assert total_hbonds(fwd) == total_hbonds(rev)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Functions can have OPTIONAL ARGUMENTS whose DEFAULT VALUES are pre-specified in the function definition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def reverse_complement(seq, unk_partner='N'):\n",
    "    \"\"\"Returns the reverse complement of a nucleic acid sequence\n",
    "    \n",
    "    Uses unk_partner as the partner of unrecognized letters\n",
    "    \"\"\"\n",
    "\n",
    "    rseq = ''\n",
    "    for a in seq:\n",
    "        if a in base_partner:\n",
    "            rseq = base_partner[a] + rseq\n",
    "        else:\n",
    "            rseq = unk_partner + rseq\n",
    "    return rseq\n",
    "\n",
    "fwd = 'ACTGTAGCxGAcTNCGAC'\n",
    "print(reverse_complement(fwd))\n",
    "print(reverse_complement(fwd, unk_partner='-'))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Functions can call other functions, even themselves:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def factorial(n):\n",
    "    \"\"\"Calculate the factorial of a number recursively. Bad things will happen \n",
    "    if the number is negative or not an integer \"\"\"\n",
    "    if n==0:\n",
    "        return 1\n",
    "    else:\n",
    "        return n * factorial(n-1) # this is called \"recursion\"\n",
    "    \n",
    "# the help function prints the docstring\n",
    "help(factorial)\n",
    "\n",
    "for i in range(10):\n",
    "    print(i,'factorial =',factorial(i))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try using `factorial?` and `factorial??` to see the docstring and source code of our new function. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "factorial??\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using a function in a built-in module\n",
    "\n",
    "Python has an extensive collection of built-in **modules** which contain all sorts of useful special purpose functions and objects. A few favorites:\n",
    "\n",
    "* `math`: mathematical functions and constants\n",
    "* `re`: regular expression searches.\n",
    "* `os`: access to operating system routines (e.g., `os.path.exists` function to check if a file exists)\n",
    "* `csv`: routines for reading/writing delimitted data\n",
    "* `sys`: special variables used or maintained by the interpreter (`sys.path`, `sys.stdout`, `sys.argv`, ...)\n",
    "* `random`: random number generators\n",
    "* `glob`: file-finding functions with wild-cards (like using `*` on the command line)\n",
    "* `pdb`: python debugger\n",
    "* `timeit`: for profiling (timing) code snippets\n",
    "* `tkinter`: python interface to tcl/tk graphics library\n",
    "* `xml`: xml processing modules\n",
    "* `string`: common string operations and constants\n",
    "\n",
    "\n",
    "[The full list is here](https://docs.python.org/3/library/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Here we are \"importing\" a module called math which contains some math-y functions like sqrt and floor\n",
    "import math\n",
    "\n",
    "\n",
    "def is_prime( num ):\n",
    "    \"\"\"Figure out whether num is prime\"\"\"\n",
    "    assert type(num) is int\n",
    "    assert num >= 1\n",
    "    \n",
    "    max_factor = math.floor( math.sqrt( num ) )\n",
    "    #max_factor = num-1\n",
    "    \n",
    "    for i in range(2,max_factor+1):\n",
    "        if num%i == 0:\n",
    "            return False\n",
    "    return True\n",
    "\n",
    "for i in range(1,50):\n",
    "    if is_prime(i):\n",
    "        print(i)\n",
    "        \n",
    "# look at the distribution of primes less than 1000\n",
    "\n",
    "prime_counts_per_100 = [0]*10 # we need 10 bins for this\n",
    "\n",
    "for i in range(1,1000):\n",
    "    if is_prime(i):\n",
    "        counts_bin = i//100\n",
    "        prime_counts_per_100[ counts_bin ] += 1\n",
    "        \n",
    "print( 'prime_counts_per_100:',prime_counts_per_100 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## try this to see how long the is_prime function takes (remove the #):\n",
    "# %timeit is_prime(997)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def is_prime_v2( num, smaller_primes ):\n",
    "    \"\"\"Figure out if num is prime based on the set of smaller primes\"\"\"\n",
    "    max_factor = math.floor( math.sqrt( num ) )\n",
    "    for p in smaller_primes:\n",
    "        if p>max_factor: break\n",
    "        if num%p == 0:\n",
    "            return False\n",
    "    return True\n",
    "\n",
    "\n",
    "# test the prime number theorem which says that the number of primes less than N\n",
    "# asymptotically approaches N/log(N)\n",
    "#\n",
    "\n",
    "primes = []\n",
    "\n",
    "for num in range(2,10000):\n",
    "    if is_prime_v2( num, primes ):\n",
    "        primes.append( num )\n",
    "    if num%100==0:\n",
    "        actual = len(primes)\n",
    "        estimate = num / math.log(num)\n",
    "        ratio = actual / estimate\n",
    "        print( num, actual, estimate, ratio )\n",
    "\n",
    "\n",
    "        \n",
    "            \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# dictionary `.items()` for looping\n",
    "\n",
    "We can loop over key,value pairs easily"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "base_partner = {'A':'T', 'T':'A', 'C':'G', 'G':'C'}\n",
    "\n",
    "for a, a_partner in base_partner.items():\n",
    "    print( a, 'pairs with', a_partner )\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# list comprehension\n",
    "Python gives us a super-compact way of creating lists called a *list comprehension*. It looks kind of like a for-loop inside square brackets.\n",
    "\n",
    "The general syntax is `[<value> for <var> in <list> if <condition>]` where the `if <condition>` part is optional.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "even_numbers = [2*i for i in range(50)]\n",
    "print(even_numbers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "even_numbers = [i for i in range(100) if i%2==0]\n",
    "print(even_numbers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "base_partner = {'A':'T', 'T':'A', 'C':'G', 'G':'C'}\n",
    "\n",
    "fwd = 'ACATGTCGATCGTAGCTGCA'\n",
    "\n",
    "# strings have a .join method that joins together a list of strings into a single string.\n",
    "# it's like the opposite of split\n",
    "rev = ''.join([base_partner[x] for x in reversed(fwd)]) # reversed lets us loop over the elements of a list/string/etc in reverse order\n",
    "rev"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can make dictionaries this way, too!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "amino_acids = 'ACDEFGHIKLMNPQRSTVWY'\n",
    "\n",
    "#enumerate is a wonderful builtin function that lets you get BOTH the index AND the item in a for-loop \n",
    "aa_to_index = {aa:i for i,aa in enumerate(amino_acids)}\n",
    "print(aa_to_index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `tuples`\n",
    "\n",
    "Dictionary keys can be any *immutable* objects, where immutable objects are ones (like integers, floats, strings) that can't be changed after they are created. So lists are no good. But, there's something called a `tuple` which looks a lot like a list and can be used as a dictionary key, and in lots of other places where multiple items are passed around."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a tuple\n",
    "t = (4,5,'a')\n",
    "\n",
    "# create an empty dictionary\n",
    "D = {}\n",
    "\n",
    "# set the value 4 for the key t in the dictionary D\n",
    "D[t] = 4\n",
    "\n",
    "# print stuff\n",
    "print('t=',t)\n",
    "print('D=',D)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`tuples` can be **unpacked** by assigning to them a sequence of variables matching their length: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nums = ( 4, 11, 17 )\n",
    "\n",
    "a,b,c = nums\n",
    "\n",
    "print('nums=',nums)\n",
    "print('a=',a)\n",
    "print('b=',b)\n",
    "print('c=',c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# running shell commands in notebooks and getting the results\n",
    "We can run (most) shell commands from within the notebook by preceding them with `!`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pwd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even cooler, we can get the results of these commands back in python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "l = !pwd\n",
    "print(l)\n",
    "print(l[0].split('/'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "l = !find ../\n",
    "print( len(l), l)"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "5f6714e2aa3121d756e42b722ecdf75e634d845b1bdb25747afcfc86db983945"
  },
  "kernelspec": {
   "display_name": "Python 3.8.8 64-bit ('base': conda)",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}