notebooks/02.00-Working-with-Data-in-OpenCV.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--BOOK_INFORMATION-->\n",
    "<a href=\"https://www.packtpub.com/big-data-and-business-intelligence/machine-learning-opencv\" target=\"_blank\"><img align=\"left\" src=\"data/cover.jpg\" style=\"width: 76px; height: 100px; background: white; padding: 1px; border: 1px solid black; margin-right:10px;\"></a>\n",
    "*This notebook contains an excerpt from the book [Machine Learning for OpenCV](https://www.packtpub.com/big-data-and-business-intelligence/machine-learning-opencv) by Michael Beyeler.\n",
    "The code is released under the [MIT license](https://opensource.org/licenses/MIT),\n",
    "and is available on [GitHub](https://github.com/mbeyeler/opencv-machine-learning).*\n",
    "\n",
    "*Note that this excerpt contains only the raw code - the book is rich with additional explanations and illustrations.\n",
    "If you find this content useful, please consider supporting the work by\n",
    "[buying the book](https://www.packtpub.com/big-data-and-business-intelligence/machine-learning-opencv)!*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--NAVIGATION-->\n",
    "< [A Taste of Machine Learning](01.00-A-Taste-of-Machine-Learning.ipynb) | [Contents](../README.md) | [Dealing with Data Using Python's NumPy Package](02.01-Dealing-with-Data-Using-Python-NumPy.ipynb) >"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Working with Data in OpenCV\n",
    "\n",
    "Now that we have whetted our appetite for machine learning, it is time to delve a little\n",
    "deeper into the different parts that make up a typical machine learning system.\n",
    "\n",
    "Machine learning is all about building mathematical models in order\n",
    "to understand data. The learning aspect enters this process when we give a machine\n",
    "learning model the capability to adjust its **internal parameters**; we can tweak these\n",
    "parameters so that the model explains the data better. In a sense, this can be understood as\n",
    "the model learning from the data. Once the model has learned enough—whatever that\n",
    "means—we can ask it to explain newly observed data.\n",
    "\n",
    "Hence machine learning problems are always split into (at least) two\n",
    "distinct phases:\n",
    "- A **training phase**, during which we aim to train a machine learning model on a set of data that we call the **training dataset**.\n",
    "- A **test phase**, during which we evaluate the learned (or finalized) machine learning model on a new set of never-before-seen data that we call the **test dataset**.\n",
    "\n",
    "The importance of splitting our data into a training set and test set cannot be understated.\n",
    "We always evaluate our models on an independent test set because we are interested in\n",
    "knowing how well our models generalize to new data. In the end, isn't this what learning is\n",
    "all about—be it machine learning or human learning?\n",
    "\n",
    "Machine learning is also all about the **data**.\n",
    "Data can be anything from images and movies to text\n",
    "documents and audio files. Therefore, in its raw form, data might be made of pixels, letters,\n",
    "words, or even worse: pure bits. It is easy to see that data in such a raw form might not be\n",
    "very convenient to work with. Instead, we have to find ways to **preprocess** the data in order\n",
    "to bring it into a form that is easy to parse.\n",
    "\n",
    "In this chapter, we want to learn how data fits in with machine learning, and how to work with data using the tools of our choice: OpenCV and Python.\n",
    "\n",
    "In specific, we want to address the following questions:\n",
    "- What does a typical machine learning workflow look like?\n",
    "- What are training data, validation data, and test data - and what are they good for?\n",
    "- How do I load, store, and work with such data in OpenCV using Python?\n",
    "\n",
    "## Outline\n",
    "\n",
    "- [Dealing with Data Using Python's NumPy Package](02.01-Dealing-with-Data-Using-Python-NumPy.ipynb)\n",
    "- [Loading External Datasets in Python](02.02-Loading-External-Datasets-in-Python.ipynb)\n",
    "- [Visualizing Data Using Matplotlib](02.03-Visualizing-Data-Using-Matplotlib.ipynb)\n",
    "- [Dealing with Data Using OpenCV's TrainData container in C++](02.05-Dealing-with-Data-Using-the-OpenCV-TrainData-Container-in-C%2B%2B.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Starting a new IPython or Jupyter session\n",
    "\n",
    "Before we can get started, we need to open an IPython shell or start a Jupyter Notebook:\n",
    "\n",
    "1. Open a terminal like we did in the previous chapter, and navigate to the `opencv-machine-learning` directory:\n",
    "\n",
    "   ```\n",
    "    $ cd Desktop/opencv-machine-learning\n",
    "   ```\n",
    "\n",
    "2. Activate the conda environment we created in the previous chapter:\n",
    "\n",
    "    ```\n",
    "    $ source activate Python3 # Mac OS X / Linux\n",
    " $ activate Python3 # Windows\n",
    "    ```\n",
    "\n",
    "3. Start a new IPython or Jupyter session:\n",
    "\n",
    "   ```\n",
    "    $ ipython # for an IPython session\n",
    " $ jupyter notebook # for a Jupyter session\n",
    "   ```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you chose to start an IPython session, the program should have greeted you with a\n",
    "welcome message such as the following:\n",
    "    \n",
    "    $ ipython\n",
    "    Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul 5 2016, 11:41:13)\n",
    "    [MSC v.1900 64 bit (AMD64)]\n",
    "    Type \"copyright\", \"credits\" or \"license\" for more information.\n",
    "    IPython 3.5.0 -- An enhanced Interactive Python.\n",
    "    ? -> Introduction and overview of IPython's features.\n",
    "    %quickref -> Quick reference.\n",
    "    help -> Python's own help system.\n",
    "    object? -> Details about 'object', use 'object??' for extra details.\n",
    "    \n",
    "    In [1]:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The line starting with `In [1]` is where you type in your regular Python commands. In\n",
    "addition, you can also use the Tab key while typing the names of variables and functions in\n",
    "order to have IPython automatically complete them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you chose to start a Jupyter session, a new window should have opened in your web\n",
    "browser that is pointing to http://localhost:8888. You want to create a new notebook by\n",
    "clicking on New in the top-right corner and selecting Notebooks (Python3).\n",
    "\n",
    "This will open a new window that contains an empty page with the same command line as in an IPython session:\n",
    "    \n",
    "    In [ ]:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--NAVIGATION-->\n",
    "< [A Taste of Machine Learning](01.00-A-Taste-of-Machine-Learning.ipynb) | [Contents](../README.md) | [Dealing with Data Using Python's NumPy Package](02.01-Dealing-with-Data-Using-Python-NumPy.ipynb) >"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}