Slithering into GIS.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Slithering into GIS - An intro to making maps with Python\n",
    "by Clinton Dow\n",
    "![Geo Python](./img/python_geospatial.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## An introduction to GIS concepts and how Python fits in to the picture\n",
    "- Many different backgrounds and experience levels\n",
    "- Not much code, introducing the concepts instead!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- About me\n",
    "- What is GIS Anyways?\n",
    "- Geospatial Developers\n",
    "- Python in GIS\n",
    "- Python Geospatial Libraries\n",
    "- Geospatial Data\n",
    "- Databases\n",
    "- On the web\n",
    "- Geospatial is just getting started!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## About me"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Computer nerd for basically my whole life\n",
    "- Python Programmer for approximately 5 years\n",
    "    - Started because of Civilization IV & V video game\n",
    "        - Programming for fun becomes programming for profit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Python product engineer\n",
    "    - Works with Python nearly 40 hours per week"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Attends numerous tutorials, conventions etc\n",
    "    - Great way to build your skills quickly\n",
    "    - Meet like-minded people who will inspire you!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## What is GIS anyways?\n",
    "![GIS](./img/GIS1.jpg)\n",
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Geographic Information Systems \n",
    "- An evolving field of study\n",
    "- Not any one specific software platform, job or technique\n",
    "- How to represent geospatial data as information\n",
    "- Kind of like defining 'Programming' "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Some concepts that fall under Geographic Information Systems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Data Collection\n",
    "- Geoprocessing\n",
    "- Geocoding\n",
    "- Cloud Computing\n",
    "- Cartography\n",
    "- Data Visualization\n",
    "- Big Data\n",
    "- Many more..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "GIS is found in, or supports every major (and many minor) organization and business"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Government \n",
    "- Armed Forces\n",
    "- Natural Resources\n",
    "- Utilities\n",
    "- Financial\n",
    "- Engineering and Construction\n",
    "- Environmental\n",
    "- Marketing\n",
    "- Insurance\n",
    "- Real Estate\n",
    "- Supply Chain Management\n",
    "- Telecom"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "A Geographic Information System\n",
    "- An implementation of one or more concepts from GIS\n",
    "- Commonly refered to as GIS Tools or GIS Applications\n",
    "![GIS App](./img/gis_phone.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Many different GIS applications have become part of our lives"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- News\n",
    "- Traffic\n",
    "- Weather\n",
    "- Social Networking\n",
    "- Dating"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "GIS Tools often work together as a GIS platform."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "GIS Platforms are like IDEs for GIS.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "![GIS Platform](./img/city.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- ArcGIS\n",
    "- QGIS\n",
    "- Mapbox\n",
    "- PostGIS\n",
    "- Google Earth\n",
    "- ENVI\n",
    "- Feature Manipulation Engine (FME)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "All major GIS platforms have Python API/SDKs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Many GIS Developers customize GIS Platforms.\n",
    "- Scripting GIS Tools\n",
    "- Modifying functionality\n",
    "- Developing plugins\n",
    "- Combining multiple platforms' functionality"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## GIS/Geospatial Developers\n",
    "![GisDev](./img/GisDev.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Are in high demand!! Especially those with demonstratable Python skills."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Python can be found in almost all facets of Geospatial Development:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Web GIS \n",
    "- App development\n",
    "- Embedded systems\n",
    "- IoT\n",
    "- Data Dashboards\n",
    "- Geoprocessing\n",
    "- etc.."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "It can be a demanding role, but very rewarding!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Requires knowledge of geography and programming, often additional skills\n",
    "- Constantly evolving field\n",
    "- Supports many different industries and organizations\n",
    "- Vaguely defined role at many companies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Many new and exciting opportunities\n",
    "- Never boring\n",
    "- Outstanding potential for entrepeneurs\n",
    "- Define your own role!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# How does Python fit into all of this?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- A powerful and robust language out of the box\n",
    "- Many open-source geospatial libraries\n",
    "- Supported by every major GIS Platform (API/SDK)\n",
    "- Supported by every major Database Platform\n",
    "- Powerful web frameworks\n",
    "    - Django for fully featured web applications\n",
    "    - Flask for service-oriented web APIs\n",
    "- Involved, passionate community"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Python Open Source Geospatial Libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Reading and Writing Geospatial Data\n",
    "- GDAL/OGR\n",
    "- Fiona"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "These libraries allow you to work with various data types that are not accessible in pure Python.\n",
    "![Convert Data](./img/convert.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Projections\n",
    "- pyproj"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "A projection defines how the spherical surface of the earth is displayed on a flat surface\n",
    "- Paper maps\n",
    "- Computer Screens\n",
    "![Projections](./img/proj.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Analyzing and Manipulating Geospatial Data \n",
    "- Shapely\n",
    "- PySAL\n",
    "- Pandas/Geopandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Perform Geoprocessing, joins and relates and map algebra on geospatial data.\n",
    "![gp](./img/gp.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Plotting Geospatial Data\n",
    "- Mapnik\n",
    "- matplotlib w/ Basemap Toolkit & pyplot\n",
    "- Descartes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Draw the actual map on the screen.\n",
    "![MTL](./img/mtl.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Geospatial Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Two main categories of Geospatial Data\n",
    "![vecras](./img/Vec_ras.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Vector Data\n",
    "- Points, Lines and Polygons\n",
    "- Individual features hold values\n",
    "- Locations of addresses\n",
    "- Road Networks\n",
    "- Boundaries\n",
    "- Discrete data\n",
    "\n",
    "![Vector](./img/v.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Raster Data\n",
    "- 'Grids' of data\n",
    "- Cells hold values\n",
    "- Images\n",
    "- Height maps\n",
    "- Temperature maps\n",
    "- Continuous data\n",
    "![Raster](./img/rasterdatamodel.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Sources of Vector Data:\n",
    "OpenStreetMap\n",
    "World Borders Dataset\n",
    "Twitter (Location-enabled Tweets)\n",
    "Government open data portals\n",
    "GPS Points\n",
    "Survey Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Sources of Raster Data:\n",
    "Satellite Photography\n",
    "LandSat\n",
    "Natural Earth\n",
    "Aerial Photography\n",
    "Sensor Arrays (ie: Weather Sensors)\n",
    "National Elevation Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Databases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Geospatial Databases are just regular databases with geospatial querying/indexing capabilities.\n",
    "Every major geospatial database has a Python API"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Examples of Python APIs for geospatial databases:\n",
    "    - SQLAlchemy\n",
    "    - arcpy\n",
    "    - psycopg\n",
    "    - cartodb-python\n",
    "    - cloudant-python\n",
    "    - etc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Two main things that make a database 'geospatial':\n",
    "- Spatial Indexes\n",
    "- GIS Metadata\n",
    "- Can be a relational or NoSQL database\n",
    "![Spatial Index](./img/spatial_index.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Relational\n",
    "- MySQL\n",
    "- Postgre/PostGIS\n",
    "- Oracle\n",
    "- etc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "NoSQL\n",
    "- CartoDB\n",
    "- CouchDB\n",
    "- Cloudant\n",
    "- etc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## On the web"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Web GIS is a huge area of growth in the geospatial industry. \n",
    "- Not just webmaps\n",
    "- Apps\n",
    "- Dashboards"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Web frameworks allow you to build your own web GIS tools and platforms.\n",
    "![webgis](./img/webgis.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Web GIS tools are typically offered as endpoints in a Service Oriented Architecture (SOA) model\n",
    "- REST \n",
    "- SOAP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Frameworks\n",
    "- GeoDjango\n",
    "- Folium\n",
    "- Papyrus"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Web GIS Platforms offer pre-created services for use with your data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Platforms\n",
    "- ArcGIS Online\n",
    "- Boundless (OpenGeo)\n",
    "- Mapbox\n",
    "- Google Maps\n",
    "- GeoServer\n",
    "- Many more..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Wrapping it all up"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Geographic Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The 'Geography' in Geographic Information Systems\n",
    "- Data acquisition\n",
    "    - Surveying\n",
    "    - GPS\n",
    "    - Location-enabled Apps\n",
    "- Data representation\n",
    "    - Vectors\n",
    "    - Rasters\n",
    "    - Projections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Data != Information"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The 'Information' in Geographic Information Systems\n",
    "- Converting raw data into meaningful information\n",
    "    - Databases\n",
    "    - Services\n",
    "- Using techniques such as\n",
    "    - Geoprocessing\n",
    "    - Spatial statistics\n",
    "    - Map Algebra\n",
    "    - Joins & relates\n",
    "    - Queries\n",
    "- Represented via\n",
    "    - Charts and graphs\n",
    "    - Cartography"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Content Delivery Systems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The 'Systems' in Geographic Information Systems\n",
    "- How do you deliver the information to a user?\n",
    "    - Paper maps/atlases\n",
    "    - Webmaps\n",
    "    - Dashboards\n",
    "    - Applications\n",
    "- Supported by\n",
    "    - Improved publishing process\n",
    "    - Web server administration\n",
    "        - DevOps\n",
    "    - System Security\n",
    "    - Wearables\n",
    "    - Ubiquitous computing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The future of GIS is very promising!\n",
    "- Proliferation of location-enabled devices\n",
    "- Big Data\n",
    "- Cloud computing\n",
    "- Machine learning\n",
    "- Virtual Reality/Augumented Reality\n",
    "- Ubiquitous Computing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Plenty of opportunities in GIS Development\n",
    "- One of the largest growth occupations in recent years\n",
    "- Innovative products and ideas\n",
    "- Tons of open source projects\n",
    "- Opportunities in all aspects of I.T. and Development."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Finding a job as a GIS Developer\n",
    "- Meetups\n",
    "    - Try to meet at least one new person at each meetup! \n",
    "- Open source\n",
    "- Conferences\n",
    "    - Esri UC\n",
    "    - PyCon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Thank you for coming!\n",
    "\n",
    "Slides available at - https://github.com/data-navigator/python-ie-gis-presentation\n",
    "\n",
    "Contact Information:\n",
    "email   - clint@datanavigator.ca\n",
    "twitter - @DataNavigator\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}