Practice_crime.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Module 2 practice  - Minneapolis crime data\n",
    "\n",
    "The dataset used in the notebook (curtesy of Open Data Minneapolis) includes information about 311 calls and crimes committed between 2010 to 2016. \n",
    "\n",
    "We will use the data to do some association rule mining for finding frequent patterns. Read the data from /dsa/data/DSA-8630/minneapolis_crimedata/crimes.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "crimes_data = read.csv('/dsa/data/DSA-8630/minneapolis_crimedata/crimes.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "head(crimes_data,2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "dim(crimes_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The columns \n",
    "- `controlnbr`\n",
    "- `CCN`\n",
    "- `Time`\n",
    "- `ReportedDate`\n",
    "- `Offense`\n",
    "- `UCRCode`\n",
    "- `EnteredDate`\n",
    "- `x`\n",
    "- `y`\n",
    "- `lastchanged`\n",
    "- `LastUpdateDate`\n",
    "- `OBJECTID`\n",
    "- `ESRI_OID` \n",
    "\n",
    "are not helpful or interpretable anyway. So lets just delete them from dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Activity 1: ** Remove the columns listed avove from dataframe. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code for activity 1 goes here..\n",
    "\n",
    "crimes_data = crimes_data[,-c(2, 3, 5, 7, 8, 10, 11,12, 13, 14, 15, 17, 18, 19, 20)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Activity 2: ** The BeginDate columnis of type factor. Convert its type to POSIXlt using strptime() function,. Also, replace the character \"T\" in the column with a white space \" \". \n",
    "\n",
    "We are trying to format the date and time values appropriately using strptime() function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Your code for activity 2 goes here..\n",
    "\n",
    "crimes_data$BeginDate = strptime(sub(\"T\",\" \", crimes_data$BeginDate), format = \"%Y-%m-%d %X\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first 6 characters in address doesn't make any sense. \n",
    "\n",
    "**Activity 3: ** Strip the first 7 characters or extract rest of the characters from the publicaddress column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code for activity 3 goes here..\n",
    "\n",
    "crimes_data$publicaddress = substr(crimes_data$publicaddress,7,(length(crimes_data$publicaddress)-7))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Activity 4:** Extract the date from BeginDate column and stored it as a new column called date. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code for activity 4 goes here..\n",
    "\n",
    "library(lubridate)\n",
    "crimes_data$date = as.Date(format(crimes_data$BeginDate,\"%Y-%m-%d\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class(crimes_data$date)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Activity 5:** Extract weekday from date and hour from from BeginDate column and stored them as new columns called weekday and hour respectively. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Your code for activity 5 goes here..\n",
    "\n",
    "crimes_data$weekday = weekdays(crimes_data$date)\n",
    "crimes_data$hour = hour(crimes_data$BeginDate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We dont need the BeginDate column any more. So lets just delete it from dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "crimes_data = crimes_data[,names(crimes_data)!=\"BeginDate\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert the hour variable into an ordered factor with levels \"mid night\", \"morning\", \"noon\",\"night\" for different hours of the day. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "crimes_data[[\"hour\"]] <- ordered(cut(crimes_data[[\"hour\"]], c(1,6,12,18,24)), labels = c(\"mid night\", \"morning\", \"noon\",\"night\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**Activity 6: ** Convert the columns \"publicaddress\", \"Precinct\", \"weekday\", \"date\" into factor type. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code for activity 6 goes here..\n",
    "\n",
    "crimes_data[\"publicaddress\"] = as.factor(crimes_data[[\"publicaddress\"]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "crimes_data[\"Precinct\"] = as.factor(crimes_data[[\"Precinct\"]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "crimes_data[\"weekday\"] = as.factor(crimes_data[[\"weekday\"]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "crimes_data[\"date\"] = as.factor(crimes_data[[\"date\"]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "library(\"arules\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "str(crimes_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Activity 7: ** Now, coerce the data set into transactions. Save this transactions to crimes_trans variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code for activity 7 goes here..\n",
    "\n",
    "crimes_trans <- as(crimes_data, \"transactions\")\n",
    "crimes_trans"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "summary(crimes_trans)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Activity 8: ** Generate association rules for the transactions in crimes_trans with support of 0.01 and confidence of 0.6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code for activity 8 goes here..\n",
    "\n",
    "rules <- apriori(crimes_trans, parameter = list(support = 0.01, confidence = 0.6))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Activity 9: ** Display the generated rules using inspect(). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code for activity 9 goes here..\n",
    "\n",
    "inspect(rules)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "3.3.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}