SenatorRepresentations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A bit more interesting\n",
    "\n",
    "I want to breifly talk about two other ways to use representational learning in addtion to the first:\n",
    "\n",
    "1. Representations as a byproduct of prediction\n",
    "2. Hand crafted representations\n",
    "3. Prediction as a byproduct of representations\n",
    "\n",
    "We covered the first in WordRepresentations notebook. The second we won't cover here, but I took inspiration from this homework from a class I took a long time ago: https://github.com/cs109/content/blob/master/HW5_solutions.ipynb\n",
    "\n",
    "In this homework we used graph algorithms to hand craft a representation of senators, and lo and behold they turned out to be quite partison. So if you want a great example of tactic 2, then check that out.\n",
    "\n",
    "The third tactic is what we will do below. We will create a prediciton problem that we don't really care about. But this problem if solved by representations will create quite useful ones. The classic example of this is word2vec.\n",
    "\n",
    "We will walk through much more of the details this time because we are fresh ground."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import requests\n",
    "from pattern import web\n",
    "import json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def get_senate_vote(congress, year, vote):\n",
    "    url = 'http://www.govtrack.us/data/congress/{}/votes/{}/s{}/data.json'.format(congress, year, vote)\n",
    "    page = requests.get(url).text\n",
    "    return json.loads(page)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def get_all_votes(congress, year):\n",
    "    page = requests.get('https://www.govtrack.us/data/congress/{}/votes/{}/'.format(congress, year)).text\n",
    "    dom = web.Element(page)\n",
    "    votes = [a.attr['href'] for a in dom.by_tag('a') \n",
    "             if a.attr.get('href', '').startswith('s')]\n",
    "    n_votes = len(votes)\n",
    "    votes_on_bills = []\n",
    "    for i in range(1, n_votes + 1):\n",
    "        vote = get_senate_vote(congress, year, i)\n",
    "        if 'bill' in vote:\n",
    "            votes_on_bills.append(vote)\n",
    "    \n",
    "    return votes_on_bills"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above two functions will scrape a website that keeps track of how US government votes go. I have already scraped it, but for people curious as to how I got the data, you can check out above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# vote_data_113_2013 = get_all_votes(113, 2013)\n",
    "# vote_data_113_2014 = get_all_votes(113, 2014)\n",
    "# vote_data_114_2015 = get_all_votes(114, 2015)\n",
    "# vote_data_114_2016= get_all_votes(114, 2016)\n",
    "\n",
    "# all_vote_data = vote_data_113_2013 + vote_data_113_2014 + vote_data_114_2015 + vote_data_114_2016\n",
    "\n",
    "# with open('all_vote_data.json', 'w') as outfile:\n",
    "#     json.dump(all_vote_data, outfile)\n",
    "    \n",
    "all_vote_data = json.load(open('all_vote_data.json'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{u'bill': {u'congress': 113,\n",
       "  u'number': 15,\n",
       "  u'title': u'A resolution to improve procedures for the consideration of legislation and nominations in the Senate.',\n",
       "  u'type': u'sres'},\n",
       " u'category': u'passage',\n",
       " u'chamber': u's',\n",
       " u'congress': 113,\n",
       " u'date': u'2013-01-24T19:54:00-05:00',\n",
       " u'number': 1,\n",
       " u'question': u'On the Resolution S.Res. 15',\n",
       " u'record_modified': u'2013-01-24T20:38:00-05:00',\n",
       " u'requires': u'3/5',\n",
       " u'result': u'Resolution Agreed to',\n",
       " u'result_text': u'Resolution Agreed to (78-16, 3/5 majority required)',\n",
       " u'session': u'2013',\n",
       " u'source_url': u'http://www.senate.gov/legislative/LIS/roll_call_votes/vote1131/vote_113_1_00001.xml',\n",
       " u'subject': u'S. Res. 15',\n",
       " u'type': u'On the Resolution',\n",
       " u'updated_at': u'2016-12-25T10:01:28-05:00',\n",
       " u'vote_id': u's1-113.2013',\n",
       " u'votes': {u'Nay': [{u'display_name': u'Crapo (R-ID)',\n",
       "    u'first_name': u'Mike',\n",
       "    u'id': u'S266',\n",
       "    u'last_name': u'Crapo',\n",
       "    u'party': u'R',\n",
       "    u'state': u'ID'},\n",
       "   {u'display_name': u'Cruz (R-TX)',\n",
       "    u'first_name': u'Ted',\n",
       "    u'id': u'S355',\n",
       "    u'last_name': u'Cruz',\n",
       "    u'party': u'R',\n",
       "    u'state': u'TX'},\n",
       "   {u'display_name': u'Flake (R-AZ)',\n",
       "    u'first_name': u'Jeff',\n",
       "    u'id': u'S358',\n",
       "    u'last_name': u'Flake',\n",
       "    u'party': u'R',\n",
       "    u'state': u'AZ'},\n",
       "   {u'display_name': u'Hatch (R-UT)',\n",
       "    u'first_name': u'Orrin',\n",
       "    u'id': u'S118',\n",
       "    u'last_name': u'Hatch',\n",
       "    u'party': u'R',\n",
       "    u'state': u'UT'},\n",
       "   {u'display_name': u'Heller (R-NV)',\n",
       "    u'first_name': u'Dean',\n",
       "    u'id': u'S352',\n",
       "    u'last_name': u'Heller',\n",
       "    u'party': u'R',\n",
       "    u'state': u'NV'},\n",
       "   {u'display_name': u'Johnson (R-WI)',\n",
       "    u'first_name': u'Ron',\n",
       "    u'id': u'S345',\n",
       "    u'last_name': u'Johnson',\n",
       "    u'party': u'R',\n",
       "    u'state': u'WI'},\n",
       "   {u'display_name': u'Lee (R-UT)',\n",
       "    u'first_name': u'Mike',\n",
       "    u'id': u'S346',\n",
       "    u'last_name': u'Lee',\n",
       "    u'party': u'R',\n",
       "    u'state': u'UT'},\n",
       "   {u'display_name': u'Paul (R-KY)',\n",
       "    u'first_name': u'Rand',\n",
       "    u'id': u'S348',\n",
       "    u'last_name': u'Paul',\n",
       "    u'party': u'R',\n",
       "    u'state': u'KY'},\n",
       "   {u'display_name': u'Risch (R-ID)',\n",
       "    u'first_name': u'James',\n",
       "    u'id': u'S323',\n",
       "    u'last_name': u'Risch',\n",
       "    u'party': u'R',\n",
       "    u'state': u'ID'},\n",
       "   {u'display_name': u'Rubio (R-FL)',\n",
       "    u'first_name': u'Marco',\n",
       "    u'id': u'S350',\n",
       "    u'last_name': u'Rubio',\n",
       "    u'party': u'R',\n",
       "    u'state': u'FL'},\n",
       "   {u'display_name': u'Sanders (I-VT)',\n",
       "    u'first_name': u'Bernie',\n",
       "    u'id': u'S313',\n",
       "    u'last_name': u'Sanders',\n",
       "    u'party': u'I',\n",
       "    u'state': u'VT'},\n",
       "   {u'display_name': u'Scott (R-SC)',\n",
       "    u'first_name': u'Timothy',\n",
       "    u'id': u'S365',\n",
       "    u'last_name': u'Scott',\n",
       "    u'party': u'R',\n",
       "    u'state': u'SC'},\n",
       "   {u'display_name': u'Sessions (R-AL)',\n",
       "    u'first_name': u'Jeff',\n",
       "    u'id': u'S261',\n",
       "    u'last_name': u'Sessions',\n",
       "    u'party': u'R',\n",
       "    u'state': u'AL'},\n",
       "   {u'display_name': u'Shelby (R-AL)',\n",
       "    u'first_name': u'Richard',\n",
       "    u'id': u'S184',\n",
       "    u'last_name': u'Shelby',\n",
       "    u'party': u'R',\n",
       "    u'state': u'AL'},\n",
       "   {u'display_name': u'Toomey (R-PA)',\n",
       "    u'first_name': u'Pat',\n",
       "    u'id': u'S351',\n",
       "    u'last_name': u'Toomey',\n",
       "    u'party': u'R',\n",
       "    u'state': u'PA'},\n",
       "   {u'display_name': u'Vitter (R-LA)',\n",
       "    u'first_name': u'David',\n",
       "    u'id': u'S299',\n",
       "    u'last_name': u'Vitter',\n",
       "    u'party': u'R',\n",
       "    u'state': u'LA'}],\n",
       "  u'Not Voting': [{u'display_name': u'Burr (R-NC)',\n",
       "    u'first_name': u'Richard',\n",
       "    u'id': u'S300',\n",
       "    u'last_name': u'Burr',\n",
       "    u'party': u'R',\n",
       "    u'state': u'NC'},\n",
       "   {u'display_name': u'Chambliss (R-GA)',\n",
       "    u'first_name': u'Saxby',\n",
       "    u'id': u'S290',\n",
       "    u'last_name': u'Chambliss',\n",
       "    u'party': u'R',\n",
       "    u'state': u'GA'},\n",
       "   {u'display_name': u'Coats (R-IN)',\n",
       "    u'first_name': u'Dan',\n",
       "    u'id': u'S212',\n",
       "    u'last_name': u'Coats',\n",
       "    u'party': u'R',\n",
       "    u'state': u'IN'},\n",
       "   {u'display_name': u'Coburn (R-OK)',\n",
       "    u'first_name': u'Tom',\n",
       "    u'id': u'S301',\n",
       "    u'last_name': u'Coburn',\n",
       "    u'party': u'R',\n",
       "    u'state': u'OK'},\n",
       "   {u'display_name': u'Graham (R-SC)',\n",
       "    u'first_name': u'Lindsey',\n",
       "    u'id': u'S293',\n",
       "    u'last_name': u'Graham',\n",
       "    u'party': u'R',\n",
       "    u'state': u'SC'},\n",
       "   {u'display_name': u'Landrieu (D-LA)',\n",
       "    u'first_name': u'Mary',\n",
       "    u'id': u'S258',\n",
       "    u'last_name': u'Landrieu',\n",
       "    u'party': u'D',\n",
       "    u'state': u'LA'}],\n",
       "  u'Present': [],\n",
       "  u'Yea': [{u'display_name': u'Alexander (R-TN)',\n",
       "    u'first_name': u'Lamar',\n",
       "    u'id': u'S289',\n",
       "    u'last_name': u'Alexander',\n",
       "    u'party': u'R',\n",
       "    u'state': u'TN'},\n",
       "   {u'display_name': u'Ayotte (R-NH)',\n",
       "    u'first_name': u'Kelly',\n",
       "    u'id': u'S340',\n",
       "    u'last_name': u'Ayotte',\n",
       "    u'party': u'R',\n",
       "    u'state': u'NH'},\n",
       "   {u'display_name': u'Baldwin (D-WI)',\n",
       "    u'first_name': u'Tammy',\n",
       "    u'id': u'S354',\n",
       "    u'last_name': u'Baldwin',\n",
       "    u'party': u'D',\n",
       "    u'state': u'WI'},\n",
       "   {u'display_name': u'Barrasso (R-WY)',\n",
       "    u'first_name': u'John',\n",
       "    u'id': u'S317',\n",
       "    u'last_name': u'Barrasso',\n",
       "    u'party': u'R',\n",
       "    u'state': u'WY'},\n",
       "   {u'display_name': u'Baucus (D-MT)',\n",
       "    u'first_name': u'Max',\n",
       "    u'id': u'S127',\n",
       "    u'last_name': u'Baucus',\n",
       "    u'party': u'D',\n",
       "    u'state': u'MT'},\n",
       "   {u'display_name': u'Begich (D-AK)',\n",
       "    u'first_name': u'Mark',\n",
       "    u'id': u'S319',\n",
       "    u'last_name': u'Begich',\n",
       "    u'party': u'D',\n",
       "    u'state': u'AK'},\n",
       "   {u'display_name': u'Bennet (D-CO)',\n",
       "    u'first_name': u'Michael',\n",
       "    u'id': u'S330',\n",
       "    u'last_name': u'Bennet',\n",
       "    u'party': u'D',\n",
       "    u'state': u'CO'},\n",
       "   {u'display_name': u'Blumenthal (D-CT)',\n",
       "    u'first_name': u'Richard',\n",
       "    u'id': u'S341',\n",
       "    u'last_name': u'Blumenthal',\n",
       "    u'party': u'D',\n",
       "    u'state': u'CT'},\n",
       "   {u'display_name': u'Blunt (R-MO)',\n",
       "    u'first_name': u'Roy',\n",
       "    u'id': u'S342',\n",
       "    u'last_name': u'Blunt',\n",
       "    u'party': u'R',\n",
       "    u'state': u'MO'},\n",
       "   {u'display_name': u'Boozman (R-AR)',\n",
       "    u'first_name': u'John',\n",
       "    u'id': u'S343',\n",
       "    u'last_name': u'Boozman',\n",
       "    u'party': u'R',\n",
       "    u'state': u'AR'},\n",
       "   {u'display_name': u'Boxer (D-CA)',\n",
       "    u'first_name': u'Barbara',\n",
       "    u'id': u'S223',\n",
       "    u'last_name': u'Boxer',\n",
       "    u'party': u'D',\n",
       "    u'state': u'CA'},\n",
       "   {u'display_name': u'Brown (D-OH)',\n",
       "    u'first_name': u'Sherrod',\n",
       "    u'id': u'S307',\n",
       "    u'last_name': u'Brown',\n",
       "    u'party': u'D',\n",
       "    u'state': u'OH'},\n",
       "   {u'display_name': u'Cantwell (D-WA)',\n",
       "    u'first_name': u'Maria',\n",
       "    u'id': u'S275',\n",
       "    u'last_name': u'Cantwell',\n",
       "    u'party': u'D',\n",
       "    u'state': u'WA'},\n",
       "   {u'display_name': u'Cardin (D-MD)',\n",
       "    u'first_name': u'Ben',\n",
       "    u'id': u'S308',\n",
       "    u'last_name': u'Cardin',\n",
       "    u'party': u'D',\n",
       "    u'state': u'MD'},\n",
       "   {u'display_name': u'Carper (D-DE)',\n",
       "    u'first_name': u'Thomas',\n",
       "    u'id': u'S277',\n",
       "    u'last_name': u'Carper',\n",
       "    u'party': u'D',\n",
       "    u'state': u'DE'},\n",
       "   {u'display_name': u'Casey (D-PA)',\n",
       "    u'first_name': u'Bob',\n",
       "    u'id': u'S309',\n",
       "    u'last_name': u'Casey',\n",
       "    u'party': u'D',\n",
       "    u'state': u'PA'},\n",
       "   {u'display_name': u'Cochran (R-MS)',\n",
       "    u'first_name': u'Thad',\n",
       "    u'id': u'S136',\n",
       "    u'last_name': u'Cochran',\n",
       "    u'party': u'R',\n",
       "    u'state': u'MS'},\n",
       "   {u'display_name': u'Collins (R-ME)',\n",
       "    u'first_name': u'Susan',\n",
       "    u'id': u'S252',\n",
       "    u'last_name': u'Collins',\n",
       "    u'party': u'R',\n",
       "    u'state': u'ME'},\n",
       "   {u'display_name': u'Coons (D-DE)',\n",
       "    u'first_name': u'Christopher',\n",
       "    u'id': u'S337',\n",
       "    u'last_name': u'Coons',\n",
       "    u'party': u'D',\n",
       "    u'state': u'DE'},\n",
       "   {u'display_name': u'Corker (R-TN)',\n",
       "    u'first_name': u'Bob',\n",
       "    u'id': u'S310',\n",
       "    u'last_name': u'Corker',\n",
       "    u'party': u'R',\n",
       "    u'state': u'TN'},\n",
       "   {u'display_name': u'Cornyn (R-TX)',\n",
       "    u'first_name': u'John',\n",
       "    u'id': u'S287',\n",
       "    u'last_name': u'Cornyn',\n",
       "    u'party': u'R',\n",
       "    u'state': u'TX'},\n",
       "   {u'display_name': u'Donnelly (D-IN)',\n",
       "    u'first_name': u'Joe',\n",
       "    u'id': u'S356',\n",
       "    u'last_name': u'Donnelly',\n",
       "    u'party': u'D',\n",
       "    u'state': u'IN'},\n",
       "   {u'display_name': u'Durbin (D-IL)',\n",
       "    u'first_name': u'Richard',\n",
       "    u'id': u'S253',\n",
       "    u'last_name': u'Durbin',\n",
       "    u'party': u'D',\n",
       "    u'state': u'IL'},\n",
       "   {u'display_name': u'Enzi (R-WY)',\n",
       "    u'first_name': u'Michael',\n",
       "    u'id': u'S254',\n",
       "    u'last_name': u'Enzi',\n",
       "    u'party': u'R',\n",
       "    u'state': u'WY'},\n",
       "   {u'display_name': u'Feinstein (D-CA)',\n",
       "    u'first_name': u'Dianne',\n",
       "    u'id': u'S221',\n",
       "    u'last_name': u'Feinstein',\n",
       "    u'party': u'D',\n",
       "    u'state': u'CA'},\n",
       "   {u'display_name': u'Fischer (R-NE)',\n",
       "    u'first_name': u'Deb',\n",
       "    u'id': u'S357',\n",
       "    u'last_name': u'Fischer',\n",
       "    u'party': u'R',\n",
       "    u'state': u'NE'},\n",
       "   {u'display_name': u'Franken (D-MN)',\n",
       "    u'first_name': u'Al',\n",
       "    u'id': u'S332',\n",
       "    u'last_name': u'Franken',\n",
       "    u'party': u'D',\n",
       "    u'state': u'MN'},\n",
       "   {u'display_name': u'Gillibrand (D-NY)',\n",
       "    u'first_name': u'Kirsten',\n",
       "    u'id': u'S331',\n",
       "    u'last_name': u'Gillibrand',\n",
       "    u'party': u'D',\n",
       "    u'state': u'NY'},\n",
       "   {u'display_name': u'Grassley (R-IA)',\n",
       "    u'first_name': u'Chuck',\n",
       "    u'id': u'S153',\n",
       "    u'last_name': u'Grassley',\n",
       "    u'party': u'R',\n",
       "    u'state': u'IA'},\n",
       "   {u'display_name': u'Hagan (D-NC)',\n",
       "    u'first_name': u'Kay',\n",
       "    u'id': u'S320',\n",
       "    u'last_name': u'Hagan',\n",
       "    u'party': u'D',\n",
       "    u'state': u'NC'},\n",
       "   {u'display_name': u'Harkin (D-IA)',\n",
       "    u'first_name': u'Tom',\n",
       "    u'id': u'S172',\n",
       "    u'last_name': u'Harkin',\n",
       "    u'party': u'D',\n",
       "    u'state': u'IA'},\n",
       "   {u'display_name': u'Heinrich (D-NM)',\n",
       "    u'first_name': u'Martin',\n",
       "    u'id': u'S359',\n",
       "    u'last_name': u'Heinrich',\n",
       "    u'party': u'D',\n",
       "    u'state': u'NM'},\n",
       "   {u'display_name': u'Heitkamp (D-ND)',\n",
       "    u'first_name': u'Heidi',\n",
       "    u'id': u'S360',\n",
       "    u'last_name': u'Heitkamp',\n",
       "    u'party': u'D',\n",
       "    u'state': u'ND'},\n",
       "   {u'display_name': u'Hirono (D-HI)',\n",
       "    u'first_name': u'Mazie',\n",
       "    u'id': u'S361',\n",
       "    u'last_name': u'Hirono',\n",
       "    u'party': u'D',\n",
       "    u'state': u'HI'},\n",
       "   {u'display_name': u'Hoeven (R-ND)',\n",
       "    u'first_name': u'John',\n",
       "    u'id': u'S344',\n",
       "    u'last_name': u'Hoeven',\n",
       "    u'party': u'R',\n",
       "    u'state': u'ND'},\n",
       "   {u'display_name': u'Inhofe (R-OK)',\n",
       "    u'first_name': u'Jim',\n",
       "    u'id': u'S236',\n",
       "    u'last_name': u'Inhofe',\n",
       "    u'party': u'R',\n",
       "    u'state': u'OK'},\n",
       "   {u'display_name': u'Isakson (R-GA)',\n",
       "    u'first_name': u'Johnny',\n",
       "    u'id': u'S305',\n",
       "    u'last_name': u'Isakson',\n",
       "    u'party': u'R',\n",
       "    u'state': u'GA'},\n",
       "   {u'display_name': u'Johanns (R-NE)',\n",
       "    u'first_name': u'Mike',\n",
       "    u'id': u'S321',\n",
       "    u'last_name': u'Johanns',\n",
       "    u'party': u'R',\n",
       "    u'state': u'NE'},\n",
       "   {u'display_name': u'Johnson (D-SD)',\n",
       "    u'first_name': u'Tim',\n",
       "    u'id': u'S257',\n",
       "    u'last_name': u'Johnson',\n",
       "    u'party': u'D',\n",
       "    u'state': u'SD'},\n",
       "   {u'display_name': u'Kaine (D-VA)',\n",
       "    u'first_name': u'Timothy',\n",
       "    u'id': u'S362',\n",
       "    u'last_name': u'Kaine',\n",
       "    u'party': u'D',\n",
       "    u'state': u'VA'},\n",
       "   {u'display_name': u'Kerry (D-MA)',\n",
       "    u'first_name': u'John',\n",
       "    u'id': u'S173',\n",
       "    u'last_name': u'Kerry',\n",
       "    u'party': u'D',\n",
       "    u'state': u'MA'},\n",
       "   {u'display_name': u'King (I-ME)',\n",
       "    u'first_name': u'Angus',\n",
       "    u'id': u'S363',\n",
       "    u'last_name': u'King',\n",
       "    u'party': u'I',\n",
       "    u'state': u'ME'},\n",
       "   {u'display_name': u'Kirk (R-IL)',\n",
       "    u'first_name': u'Mark',\n",
       "    u'id': u'S339',\n",
       "    u'last_name': u'Kirk',\n",
       "    u'party': u'R',\n",
       "    u'state': u'IL'},\n",
       "   {u'display_name': u'Klobuchar (D-MN)',\n",
       "    u'first_name': u'Amy',\n",
       "    u'id': u'S311',\n",
       "    u'last_name': u'Klobuchar',\n",
       "    u'party': u'D',\n",
       "    u'state': u'MN'},\n",
       "   {u'display_name': u'Lautenberg (D-NJ)',\n",
       "    u'first_name': u'Frank',\n",
       "    u'id': u'S166',\n",
       "    u'last_name': u'Lautenberg',\n",
       "    u'party': u'D',\n",
       "    u'state': u'NJ'},\n",
       "   {u'display_name': u'Leahy (D-VT)',\n",
       "    u'first_name': u'Patrick',\n",
       "    u'id': u'S057',\n",
       "    u'last_name': u'Leahy',\n",
       "    u'party': u'D',\n",
       "    u'state': u'VT'},\n",
       "   {u'display_name': u'Levin (D-MI)',\n",
       "    u'first_name': u'Carl',\n",
       "    u'id': u'S131',\n",
       "    u'last_name': u'Levin',\n",
       "    u'party': u'D',\n",
       "    u'state': u'MI'},\n",
       "   {u'display_name': u'Manchin (D-WV)',\n",
       "    u'first_name': u'Joe',\n",
       "    u'id': u'S338',\n",
       "    u'last_name': u'Manchin',\n",
       "    u'party': u'D',\n",
       "    u'state': u'WV'},\n",
       "   {u'display_name': u'McCain (R-AZ)',\n",
       "    u'first_name': u'John',\n",
       "    u'id': u'S197',\n",
       "    u'last_name': u'McCain',\n",
       "    u'party': u'R',\n",
       "    u'state': u'AZ'},\n",
       "   {u'display_name': u'McCaskill (D-MO)',\n",
       "    u'first_name': u'Claire',\n",
       "    u'id': u'S312',\n",
       "    u'last_name': u'McCaskill',\n",
       "    u'party': u'D',\n",
       "    u'state': u'MO'},\n",
       "   {u'display_name': u'McConnell (R-KY)',\n",
       "    u'first_name': u'Mitch',\n",
       "    u'id': u'S174',\n",
       "    u'last_name': u'McConnell',\n",
       "    u'party': u'R',\n",
       "    u'state': u'KY'},\n",
       "   {u'display_name': u'Menendez (D-NJ)',\n",
       "    u'first_name': u'Robert',\n",
       "    u'id': u'S306',\n",
       "    u'last_name': u'Menendez',\n",
       "    u'party': u'D',\n",
       "    u'state': u'NJ'},\n",
       "   {u'display_name': u'Merkley (D-OR)',\n",
       "    u'first_name': u'Jeff',\n",
       "    u'id': u'S322',\n",
       "    u'last_name': u'Merkley',\n",
       "    u'party': u'D',\n",
       "    u'state': u'OR'},\n",
       "   {u'display_name': u'Mikulski (D-MD)',\n",
       "    u'first_name': u'Barbara',\n",
       "    u'id': u'S182',\n",
       "    u'last_name': u'Mikulski',\n",
       "    u'party': u'D',\n",
       "    u'state': u'MD'},\n",
       "   {u'display_name': u'Moran (R-KS)',\n",
       "    u'first_name': u'Jerry',\n",
       "    u'id': u'S347',\n",
       "    u'last_name': u'Moran',\n",
       "    u'party': u'R',\n",
       "    u'state': u'KS'},\n",
       "   {u'display_name': u'Murkowski (R-AK)',\n",
       "    u'first_name': u'Lisa',\n",
       "    u'id': u'S288',\n",
       "    u'last_name': u'Murkowski',\n",
       "    u'party': u'R',\n",
       "    u'state': u'AK'},\n",
       "   {u'display_name': u'Murphy (D-CT)',\n",
       "    u'first_name': u'Christopher',\n",
       "    u'id': u'S364',\n",
       "    u'last_name': u'Murphy',\n",
       "    u'party': u'D',\n",
       "    u'state': u'CT'},\n",
       "   {u'display_name': u'Murray (D-WA)',\n",
       "    u'first_name': u'Patty',\n",
       "    u'id': u'S229',\n",
       "    u'last_name': u'Murray',\n",
       "    u'party': u'D',\n",
       "    u'state': u'WA'},\n",
       "   {u'display_name': u'Nelson (D-FL)',\n",
       "    u'first_name': u'Bill',\n",
       "    u'id': u'S282',\n",
       "    u'last_name': u'Nelson',\n",
       "    u'party': u'D',\n",
       "    u'state': u'FL'},\n",
       "   {u'display_name': u'Portman (R-OH)',\n",
       "    u'first_name': u'Rob',\n",
       "    u'id': u'S349',\n",
       "    u'last_name': u'Portman',\n",
       "    u'party': u'R',\n",
       "    u'state': u'OH'},\n",
       "   {u'display_name': u'Pryor (D-AR)',\n",
       "    u'first_name': u'Mark',\n",
       "    u'id': u'S295',\n",
       "    u'last_name': u'Pryor',\n",
       "    u'party': u'D',\n",
       "    u'state': u'AR'},\n",
       "   {u'display_name': u'Reed (D-RI)',\n",
       "    u'first_name': u'John',\n",
       "    u'id': u'S259',\n",
       "    u'last_name': u'Reed',\n",
       "    u'party': u'D',\n",
       "    u'state': u'RI'},\n",
       "   {u'display_name': u'Reid (D-NV)',\n",
       "    u'first_name': u'Harry',\n",
       "    u'id': u'S198',\n",
       "    u'last_name': u'Reid',\n",
       "    u'party': u'D',\n",
       "    u'state': u'NV'},\n",
       "   {u'display_name': u'Roberts (R-KS)',\n",
       "    u'first_name': u'Pat',\n",
       "    u'id': u'S260',\n",
       "    u'last_name': u'Roberts',\n",
       "    u'party': u'R',\n",
       "    u'state': u'KS'},\n",
       "   {u'display_name': u'Rockefeller (D-WV)',\n",
       "    u'first_name': u'Jay',\n",
       "    u'id': u'S176',\n",
       "    u'last_name': u'Rockefeller',\n",
       "    u'party': u'D',\n",
       "    u'state': u'WV'},\n",
       "   {u'display_name': u'Schatz (D-HI)',\n",
       "    u'first_name': u'Brian',\n",
       "    u'id': u'S353',\n",
       "    u'last_name': u'Schatz',\n",
       "    u'party': u'D',\n",
       "    u'state': u'HI'},\n",
       "   {u'display_name': u'Schumer (D-NY)',\n",
       "    u'first_name': u'Charles',\n",
       "    u'id': u'S270',\n",
       "    u'last_name': u'Schumer',\n",
       "    u'party': u'D',\n",
       "    u'state': u'NY'},\n",
       "   {u'display_name': u'Shaheen (D-NH)',\n",
       "    u'first_name': u'Jeanne',\n",
       "    u'id': u'S324',\n",
       "    u'last_name': u'Shaheen',\n",
       "    u'party': u'D',\n",
       "    u'state': u'NH'},\n",
       "   {u'display_name': u'Stabenow (D-MI)',\n",
       "    u'first_name': u'Debbie',\n",
       "    u'id': u'S284',\n",
       "    u'last_name': u'Stabenow',\n",
       "    u'party': u'D',\n",
       "    u'state': u'MI'},\n",
       "   {u'display_name': u'Tester (D-MT)',\n",
       "    u'first_name': u'Jon',\n",
       "    u'id': u'S314',\n",
       "    u'last_name': u'Tester',\n",
       "    u'party': u'D',\n",
       "    u'state': u'MT'},\n",
       "   {u'display_name': u'Thune (R-SD)',\n",
       "    u'first_name': u'John',\n",
       "    u'id': u'S303',\n",
       "    u'last_name': u'Thune',\n",
       "    u'party': u'R',\n",
       "    u'state': u'SD'},\n",
       "   {u'display_name': u'Udall (D-CO)',\n",
       "    u'first_name': u'Mark',\n",
       "    u'id': u'S325',\n",
       "    u'last_name': u'Udall',\n",
       "    u'party': u'D',\n",
       "    u'state': u'CO'},\n",
       "   {u'display_name': u'Udall (D-NM)',\n",
       "    u'first_name': u'Tom',\n",
       "    u'id': u'S326',\n",
       "    u'last_name': u'Udall',\n",
       "    u'party': u'D',\n",
       "    u'state': u'NM'},\n",
       "   {u'display_name': u'Warner (D-VA)',\n",
       "    u'first_name': u'Mark',\n",
       "    u'id': u'S327',\n",
       "    u'last_name': u'Warner',\n",
       "    u'party': u'D',\n",
       "    u'state': u'VA'},\n",
       "   {u'display_name': u'Warren (D-MA)',\n",
       "    u'first_name': u'Elizabeth',\n",
       "    u'id': u'S366',\n",
       "    u'last_name': u'Warren',\n",
       "    u'party': u'D',\n",
       "    u'state': u'MA'},\n",
       "   {u'display_name': u'Whitehouse (D-RI)',\n",
       "    u'first_name': u'Sheldon',\n",
       "    u'id': u'S316',\n",
       "    u'last_name': u'Whitehouse',\n",
       "    u'party': u'D',\n",
       "    u'state': u'RI'},\n",
       "   {u'display_name': u'Wicker (R-MS)',\n",
       "    u'first_name': u'Roger',\n",
       "    u'id': u'S318',\n",
       "    u'last_name': u'Wicker',\n",
       "    u'party': u'R',\n",
       "    u'state': u'MS'},\n",
       "   {u'display_name': u'Wyden (D-OR)',\n",
       "    u'first_name': u'Ron',\n",
       "    u'id': u'S247',\n",
       "    u'last_name': u'Wyden',\n",
       "    u'party': u'D',\n",
       "    u'state': u'OR'}]}}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_vote_data[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see that we have the bill and all the votes that it got from various senators. In addition to this information we will want to find out one more bit of info, who sponsored the bill?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def get_senate_bill(bill_type, bill_number):\n",
    "    url = 'http://www.govtrack.us/data/congress/113/bills/{}/{}{}/data.json'.format(bill_type, bill_type, bill_number)\n",
    "    page = requests.get(url).text\n",
    "    return json.loads(page)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# bill_data = []\n",
    "# for vote in all_vote_data:\n",
    "#     if 'bill' in vote:\n",
    "#         bill_type = vote['bill']['type']\n",
    "#         bill_number = vote['bill']['number']\n",
    "#         bill = get_senate_bill(bill_type, bill_number)\n",
    "#         bill['id'] = '{}{}'.format(bill_type, bill_number)\n",
    "#         bill_data.append(bill)\n",
    "        \n",
    "# with open('bill_data.json', 'w') as outfile:\n",
    "#     json.dump(bill_data, outfile)\n",
    "    \n",
    "bill_data = json.load(open('bill_data.json'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{u'actions': [{u'acted_at': u'2013-01-24',\n",
       "   u'references': [{u'reference': u'CR S293',\n",
       "     u'type': u'text of measure as introduced'}],\n",
       "   u'text': u'Submitted in the Senate.',\n",
       "   u'type': u'action'},\n",
       "  {u'acted_at': u'2013-01-24',\n",
       "   u'references': [{u'reference': u'CR S270-274', u'type': u'consideration'},\n",
       "    {u'reference': u'CR S293', u'type': u'text of measure as introduced'}],\n",
       "   u'text': u'Measure laid before Senate by unanimous consent.',\n",
       "   u'type': u'action'},\n",
       "  {u'acted_at': u'2013-01-24',\n",
       "   u'how': u'roll',\n",
       "   u'references': [{u'reference': u'CR S272', u'type': u'text'}],\n",
       "   u'result': u'pass',\n",
       "   u'roll': u'1',\n",
       "   u'status': u'PASSED:SIMPLERES',\n",
       "   u'text': u'Resolution agreed to in Senate, under the order of 1/24/2012, having achieved 60 votes in the affirmative, without amendment by Yea-Nay Vote. 78 - 16. Record Vote Number: 1.',\n",
       "   u'type': u'vote',\n",
       "   u'vote_type': u'vote',\n",
       "   u'where': u's'}],\n",
       " u'amendments': [{u'amendment_id': u'samdt3-113',\n",
       "   u'amendment_type': u'samdt',\n",
       "   u'chamber': u's',\n",
       "   u'number': u'3'}],\n",
       " u'bill_id': u'sres15-113',\n",
       " u'bill_type': u'sres',\n",
       " u'by_request': False,\n",
       " u'committees': [],\n",
       " u'congress': u'113',\n",
       " u'cosponsors': [{u'district': None,\n",
       "   u'name': u'Levin, Carl',\n",
       "   u'sponsored_at': u'2013-01-24',\n",
       "   u'state': u'MI',\n",
       "   u'thomas_id': u'01384',\n",
       "   u'title': u'Sen',\n",
       "   u'withdrawn_at': None},\n",
       "  {u'district': None,\n",
       "   u'name': u'McCain, John',\n",
       "   u'sponsored_at': u'2013-01-24',\n",
       "   u'state': u'AZ',\n",
       "   u'thomas_id': u'00754',\n",
       "   u'title': u'Sen',\n",
       "   u'withdrawn_at': None}],\n",
       " u'enacted_as': None,\n",
       " u'history': {u'active': True,\n",
       "  u'active_at': u'2013-01-24',\n",
       "  u'awaiting_signature': False,\n",
       "  u'enacted': False,\n",
       "  u'senate_passage_result': u'pass',\n",
       "  u'senate_passage_result_at': u'2013-01-24',\n",
       "  u'vetoed': False},\n",
       " u'id': u'sres15',\n",
       " u'introduced_at': u'2013-01-24',\n",
       " u'number': u'15',\n",
       " u'official_title': u'A resolution to improve procedures for the consideration of legislation and nominations in the Senate.',\n",
       " u'popular_title': None,\n",
       " u'related_bills': [],\n",
       " u'short_title': None,\n",
       " u'sponsor': {u'district': None,\n",
       "  u'name': u'Reid, Harry',\n",
       "  u'state': u'NV',\n",
       "  u'thomas_id': u'00952',\n",
       "  u'title': u'Sen',\n",
       "  u'type': u'person'},\n",
       " u'status': u'PASSED:SIMPLERES',\n",
       " u'status_at': u'2013-01-24',\n",
       " u'subjects': [u'Congress',\n",
       "  u'Federal district courts',\n",
       "  u'Federal officials',\n",
       "  u'Judges',\n",
       "  u'Legislative rules and procedure',\n",
       "  u'Senate'],\n",
       " u'subjects_top_term': u'Congress',\n",
       " u'summary': {u'as': u'Passed Senate without amendment',\n",
       "  u'date': u'2013-01-24',\n",
       "  u'text': u'Limits to four hours any debate on a motion to proceed to the consideration of a measure or matter. Applies the following conditions on the proposal of amendments if the motion to proceed is agreed to:\\n\\n the first amendments in order shall be one first-degree amendment each offered by the minority, the majority, the minority, and the majority (with forfeiture of the right to offer an amendment if it is not offered in its designated order); makes it out of order, if a cloture motion has been filed, for: (1) the minority to propose its first amendment unless it has been submitted to the Senate Journal Clerk by 1:00 p.m. on the day following the filing of the cloture motion, (2) the majority to propose its first amendment unless it has been submitted by 3:00 p.m. on such day, (3) the minority to propose its second amendment unless it has been submitted by 5:00 p.m on such day, or (4) the majority to propose its second amendment unless it has been submitted by 7:00 p.m. on such day; forfeiture of the right to offer any amendment that is not timely submitted; disposal of the first-degree amendment offered must occur before the next first-degree amendment in order may be offered; an amendment is not divisible or subject to amendment while pending; a first-degree amendment, if adopted, shall be considered original text for the purpose of further amendment; no points of order shall be waived by virtue of this resolution; and no motion to commit or recommit shall be in order during the pendency of any first-degree amendment. Declares that if cloture is invoked on a measure or matter before all first-degree amendments are disposed of: (1) any such amendment in order but not actually pending upon the expiration of post-cloture time may be offered and may be debated for up to one hour, equally divided in the usual form; and (2) any such amendment ruled non-germane on a point of order shall not fall upon that ruling, but instead remain pending and require 60 affirmative votes to be adopted.\\n\\nLimits post-cloture consideration to: (1) eight hours for any nomination except one to a position at level I of the Executive Schedule or to serve as a federal judge or justice appointed to hold office during good behavior, and (2) two hours for any U.S. district court nominations.\\n\\nRequires consideration of legislation and of nominations to be equally divided in the usual form.'},\n",
       " u'titles': [{u'as': u'introduced',\n",
       "   u'is_for_portion': False,\n",
       "   u'title': u'A resolution to improve procedures for the consideration of legislation and nominations in the Senate.',\n",
       "   u'type': u'official'}],\n",
       " u'updated_at': u'2016-06-29T09:49:08-04:00',\n",
       " u'url': u'http://thomas.loc.gov/cgi-bin/bdquery/z?d113:SE15:@@@L&summ2=m&'}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bill_data[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again we get a ton of information. But we are just interested in who sponsored it.\n",
    "\n",
    "We will then map each senator to an ID, just like we did with words:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "senators = []\n",
    "for vote in all_vote_data:\n",
    "    for sen in vote['votes']['Nay']:\n",
    "        senators.append(sen['last_name'] + ', ' + sen['state'])\n",
    "        \n",
    "    for sen in vote['votes']['Yea']:\n",
    "        senators.append(sen['last_name'] + ', ' + sen['state'])\n",
    "        \n",
    "# leave the first two blank for padding and not senators\n",
    "senator_to_id = {k: v + 2 for v, k in enumerate(set(senators))}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will convert all the sponsors and cosponsors into IDs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# for each bill, pull out it's sponsor and co-sponsors\n",
    "def unique_name(senator):\n",
    "    last_name = senator['name'].split(',')[0]\n",
    "    return '{}, {}'.format(last_name, senator['state'])\n",
    "\n",
    "def get_sen_id(sen):\n",
    "    if sen not in senator_to_id:\n",
    "        return 1\n",
    "    return senator_to_id[sen]\n",
    "\n",
    "bill_dict = {}\n",
    "for bill in bill_data:\n",
    "    bill_dict[bill['id']] = {\n",
    "        'sponsor': get_sen_id(unique_name(bill['sponsor'])),\n",
    "        'cosponsors': [get_sen_id(unique_name(cosponsor)) for cosponsor in bill['cosponsors']]\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'cosponsors': [56, 47], 'sponsor': 24}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bill_dict[u'sres15']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And finally we will make our data. So we are really interested in representing our senators, but for an ML algorithm to learn that, it needs a goal to acheive with the representations aka a procedure to determine if the representation is good. \n",
    "\n",
    "Our prediciton problem will be: can we predict a senator's vote based on who sponsored it?\n",
    "\n",
    "Notice the prediciton problem is composed of an interaction of representations (if representations don't interact the problem becomes too simple). Now if we were truely interested in the prediciton problem we would include a ton more features: the age of the sentator, whether they are rep or a dem. But we are interested in the representation. So let's get our data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "senator_vote_data = []\n",
    "id_to_displayname = {}\n",
    "\n",
    "for vote in all_vote_data:\n",
    "    bill_type = vote['bill']['type']\n",
    "    bill_number = vote['bill']['number']\n",
    "    bill_id = '{}{}'.format(bill_type, bill_number)\n",
    "    if bill_id in bill_dict:\n",
    "        bill_sponsors = bill_dict[bill_id]\n",
    "        sponsor = bill_sponsors['sponsor']\n",
    "        cosponsors = bill_sponsors['cosponsors']\n",
    "    else:\n",
    "        continue\n",
    "    \n",
    "    for sen in vote['votes']['Nay']:\n",
    "        senator_id = get_sen_id(sen['last_name'] + ', ' + sen['state'])\n",
    "        id_to_displayname[senator_id] = sen[u'display_name']\n",
    "        senator_vote_data.append((0, senator_id, sponsor, cosponsors)) \n",
    "        \n",
    "    for sen in vote['votes']['Yea']:\n",
    "        senator_id = get_sen_id(sen['last_name'] + ', ' + sen['state'])\n",
    "        id_to_displayname[senator_id] = sen[u'display_name']\n",
    "        senator_vote_data.append((1, senator_id, sponsor, cosponsors)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "69856\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(0, 106, 24, [56, 47])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print len(senator_vote_data)\n",
    "senator_vote_data[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "~70k examples of (vote, senator voting, sponsor, cosponsor) tuples is pretty good (we could of course scrape more). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# we extract our target \n",
    "y = [d[0] for d in senator_vote_data]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# we extract our features\n",
    "import numpy as np\n",
    "\n",
    "# again we pad\n",
    "def pad_or_crop(lst, l=10):\n",
    "    return (lst + [0] * l)[:10]\n",
    "\n",
    "x_1 = np.array(map(lambda x: x[1], senator_vote_data))\n",
    "x_2 = np.array(map(lambda x: x[2], senator_vote_data))\n",
    "x_3 = np.array(map(lambda x: pad_or_crop(x[3]), senator_vote_data))\n",
    "x = [x_1, x_2, x_3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# we add in padding and unknown senators\n",
    "id_to_displayname[0] = '<PAD>'\n",
    "id_to_displayname[1] = '<NOT A SENATOR>'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "120"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# this gives us how many representations:\n",
    "len(id_to_displayname)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# we again need to write down the metadata\n",
    "import csv\n",
    "\n",
    "with open('senator_metadata.csv', 'wb') as csvfile:\n",
    "    writer = csv.writer(csvfile, delimiter='\\t')\n",
    "       \n",
    "    for key, value in sorted(id_to_displayname.items()):\n",
    "        writer.writerow([value.encode('utf8')])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "# finally we build our model\n",
    "from keras.layers import concatenate\n",
    "from keras.layers import Dense, Input, Flatten\n",
    "from keras.layers import MaxPooling1D, Embedding\n",
    "\n",
    "embedding_layer = Embedding(len(id_to_displayname), 100)\n",
    "\n",
    "# train a 1D convnet with global maxpooling\n",
    "voting = voting_input = Input(shape=(1,), dtype='int32')\n",
    "voting = embedding_layer(voting)\n",
    "voting = Dense(32, activation='relu')(voting)\n",
    "voting = Dense(32, activation='relu')(voting)\n",
    "\n",
    "sponsor = sponsor_input = Input(shape=(1,), dtype='int32')\n",
    "sponsor = embedding_layer(sponsor)\n",
    "sponsor = Dense(32, activation='relu')(sponsor)\n",
    "sponsor = Dense(32, activation='relu')(sponsor)\n",
    "\n",
    "cosponsor = cosponsor_input = Input(shape=(10,), dtype='int32')\n",
    "cosponsor = embedding_layer(cosponsor)\n",
    "cosponsor = MaxPooling1D(10)(cosponsor)\n",
    "cosponsor = Dense(32, activation='relu')(cosponsor)\n",
    "cosponsor = Dense(32, activation='relu')(cosponsor)\n",
    "\n",
    "combined = concatenate([voting, sponsor, cosponsor])\n",
    "combined = Dense(32, activation='relu')(combined)\n",
    "combined = Dense(1, activation='sigmoid')(combined)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 55884 samples, validate on 13972 samples\n",
      "Epoch 1/10\n",
      "55884/55884 [==============================] - 5s 84us/step - loss: 0.6487 - acc: 0.6229 - val_loss: 0.6545 - val_acc: 0.6452\n",
      "Epoch 2/10\n",
      "55884/55884 [==============================] - 4s 80us/step - loss: 0.6054 - acc: 0.6594 - val_loss: 0.6513 - val_acc: 0.6454\n",
      "Epoch 3/10\n",
      "55884/55884 [==============================] - 4s 71us/step - loss: 0.5926 - acc: 0.6652 - val_loss: 0.6643 - val_acc: 0.6417\n",
      "Epoch 4/10\n",
      "55884/55884 [==============================] - 3s 57us/step - loss: 0.5892 - acc: 0.6688 - val_loss: 0.6721 - val_acc: 0.5994\n",
      "Epoch 5/10\n",
      "55884/55884 [==============================] - 3s 62us/step - loss: 0.5873 - acc: 0.6679 - val_loss: 0.6720 - val_acc: 0.5865\n",
      "Epoch 6/10\n",
      "55884/55884 [==============================] - 3s 59us/step - loss: 0.5865 - acc: 0.6696 - val_loss: 0.6843 - val_acc: 0.5731\n",
      "Epoch 7/10\n",
      "55884/55884 [==============================] - 3s 59us/step - loss: 0.5850 - acc: 0.6706 - val_loss: 0.6815 - val_acc: 0.5910\n",
      "Epoch 8/10\n",
      "55884/55884 [==============================] - 3s 56us/step - loss: 0.5836 - acc: 0.6721 - val_loss: 0.6802 - val_acc: 0.6185\n",
      "Epoch 9/10\n",
      "55884/55884 [==============================] - 3s 62us/step - loss: 0.5833 - acc: 0.6725 - val_loss: 0.7072 - val_acc: 0.5530\n",
      "Epoch 10/10\n",
      "55884/55884 [==============================] - 3s 60us/step - loss: 0.5827 - acc: 0.6715 - val_loss: 0.6993 - val_acc: 0.5658\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x18265cab10>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from keras.models import Model\n",
    "from keras.callbacks import TensorBoard\n",
    "\n",
    "model = Model([voting_input, sponsor_input, cosponsor_input], combined)\n",
    "\n",
    "model.compile(loss='binary_crossentropy',\n",
    "              optimizer='rmsprop',\n",
    "              metrics=['acc'])\n",
    "\n",
    "embedding_metadata = {\n",
    "    embedding_layer.name: 'senator_metadata.csv'\n",
    "}\n",
    "\n",
    "model.fit([x_1, x_2, x_3], np.array(y).reshape(-1, 1, 1),\n",
    "          batch_size=128,\n",
    "          epochs=10,\n",
    "          validation_split=0.2,\n",
    "          callbacks=[TensorBoard(log_dir='senator_reps', embeddings_freq=1, embeddings_metadata=embedding_metadata)])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}