aula07-preprocessamento.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Aula 07 &mdash; Pré-processamento de dados\n",
    "\n",
    "Renato Vimieiro\n",
    "rv2 {em} cin.ufpe.br\n",
    "\n",
    "abril 2017"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Tópicos da aula\n",
    "\n",
    "1. [tipos de atributos](#tipos-atributos)\n",
    "2. [ajustar tipos de dados em pandas](#tipos-em-pandas)\n",
    "5. [tratamento de dados ausentes](#dados-ausentes)\n",
    "4. [discretização](#discretizacao)\n",
    "3. [normalização](#normalizacao)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Como discutimos no início do curso, frequentemente trabalharemos com dados heterogêneos obtidos de diversas fontes. É importante ter claro em mente o conceito de objetos, atributos e os diversos tipos de dados que podemos nos deparar. Além disso, devemos dedicar uma parte considerável do tempo para formatação, limpeza e transformação dos dados obtidos. Um dos dizeres mais populares em análise de dados é *Garbage in, Garbage out*. Em outras palavras, não podemos esperar bons resultados de dados 'ruins'. Logo, essa é, de fato, uma das tarefas mais importantes em data science.\n",
    "\n",
    "Antes de iniciarmos nossos estudos sobre as tarefas de pré-processamento dos dados, precisamos esclarecer alguns termos que são frequentemente usados. O primeiro deles é o conceito de **objetos**. Um objeto é a unidade básica da coleta dos dados. Objetos podem ser itens de uma loja, pacientes numa base de dados médicos, clientes, estudantes, documentos, páginas web, etc. Os objetos de um conjunto de dados também são referidos por outros termos como *amostras*, *exemplos* e *instâncias*. O uso desses termos vária de acordo com a origem do profissional. Pessoas de aprendizado de máquina tendem a usar mais o termo *exemplo*, já outras da área de bioinformática, por exemplo, usam o termo *amostra*.\n",
    "\n",
    "Objetos são descritos por **atributos**. Os atributos representam as características de um objeto. Novamente o termo atributo não é unânime. Estatísticos, por exemplo, adotam o termo variável. Pessoas de aprendizado de máquina usam característica ou *feature*. Outras ainda utilizam o termo *dimensão*. De fato, esse termo é muito utilizado quando se deseja referenciar a quantidade de atributos no conjunto de dados (e.g. conjuntos de alta/baixa dimensionalidade).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"tipos-atributos\"></a> Tipos de atributos"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "O conceito de tipos de dados não é estranho para nós da computação. Podemos classificar os atributos de acordo com os valores que eles podem assumir. Os tipos possíveis são:\n",
    "\n",
    "- nominal\n",
    "- binário\n",
    "- ordinal\n",
    "- numérico\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Atributos nominais\n",
    "\n",
    "Os atributos nominais ou categóricos são descrições **qualitativas** de objetos. Eles são nomes ou categorias que são atribuídas a um objeto. Por exemplo, a *cor_de_cabelo* é um atributo categórico para uma pessoa, podendo assumir os valores *ruivo*, *louro*, *preto*, *castanho*, *grisalho*, *branco*. Frequentemente escolhemos associar rótulos numéricos a esses valores. Contudo, tais rótulos são apenas artifícios de representação e não representam quantidades. Dessa forma, não podemos calcular nenhuma medida de tendência para esses atributos a não ser a frequência de ocorrência. Também não podemos estabelecer quaisquer comparações entre tais valores, exceto igualdade."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Atributos binários\n",
    "\n",
    "Atributos binários são um caso particular de atributos nominais em que existem apenas duas categorias. Em geral representamos tais atributos por **0** e **1**. Também frequentemente associamos 0 com a ausência do atributo e 1 com a presença. Exemplos de atributos binários são: *fumante* (numa base de dados médicos, indicando se a pessoa é ou não fumante), genêro (0 indicando feminino e 1 masculino). Esses atributos podem ser **simétricos** ou **assimétricos**. Em atributos simétricos ambos os códigos possuem o mesmo 'peso' e são igualmente prováveis. Por exemplo, o gênero masculino e feminino são igualmente prováveis e, nesse caso, tanto faz associar masculino a 0 ou 1. Atributos assimétricos, por outro lado, não são igualmente prováveis. Por exemplo, a indicação de um resultado de um teste de câncer. Proporcionalmente temos mais eventos negativos que positivos. Isto é, os eventos positivos são mais raros que os negativos, logo associamos 1 à presença (evento positivo) e 0 à ausência (evento negativo)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Atributos ordinais\n",
    "\n",
    "Atributos ordinais são aqueles em que existe uma ordem (total) definida entre suas possíveis categorias. Por exemplo, tamanho de roupas divididos P, M, e G. Sabemos que existe uma ordem entre as possíveis categorias, mas não sabemos quantificar tal diferença. Não sabemos quão grande é a diferença de tamanho entre o P e o G. Atributos desse tipo surgem com frequência em questionários associados a números (por exemplo, pesquisa de satisfação). Embora exista uma ordem entre as categorias e essas estejam relacionadas a números, não podemos executar certas operações algébricas com tais atributos. Por exemplo, não faz sentido subtrair duas categorias ou, ainda, calcular a média de um atributo ordinal. Em outras palavras, esses ainda continuam sendo descrições qualitativas de objetos."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Atributos numéricos\n",
    "\n",
    "Esses atributos representam quantidades mensuradas. Os atributos numéricos podem ser grandezas reais ou inteiras. Eles estão subdivididos em dados **intervalares** e de **razão**. \n",
    "\n",
    "Nos dados intervalares existe uma relação de ordem entre os diversos valores, e além disso a diferença entre dois valores possui significado. O exemplo clássico é o atributo temperatura. Sabemos que 40 graus é mais quente que 30 (uma diferença de 10 graus). O mesmo pode ser dito em relação a 20 e 10 graus (novamente uma diferença de 10 graus). Contudo, não podemos dizer que um dia em que foi registrado 40 graus é 2x mais quente que um dia que foi registrado 20 graus. Isso é bastante evidente se mudarmos a escala de Celsius para Fahrenheit. 40 graus C é equivalente a 104 F, e 20C equivalente a 68F. Nesse caso, temos uma diferença de aproximadamente 1,5x. O mesmo ocorre quando comparamos anos (em datas). Existe uma diferença de 5 anos entre 1940 e 1945, mas não podemos dizer que 2016 é 2x 1008. A razão é simples: não sabemos a localização exata do zero. O que definimos como zero tanto na escala de temperatura quanto no calendário é uma mera convenção. Mesmo assim, podemos calcular médias, desvios e outras estatísticas de dados intervalares.\n",
    "\n",
    "Os dados de razão são aqueles em que o zero absoluto é conhecido. Esse ponto representa a total ausência do atributo. A temperatura medida em Kelvin, ao contrário da escala de Celsius e Fahrenheit, é uma escala de razão; 20K é duas vezes mais quente que 10K. O mesmo ocorre, por exemplo, quando comparamos uma pessoa que recebe um salário de R\\$20\\.000,00, recebe cinco vezes mais que alguém que receba R\\$4\\.000,00."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"tipos-em-pandas\"></a> Ajustar tipos de dados em Pandas\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Muitas vezes quando carregamos dados com Pandas o tipo do atributo pode não ser assinalado corretamente. Nesses casos, temos de ajustá-los antes de iniciarmos a análise. Já tivemos um breve contato com ajuste dos tipos em aulas anteriores, mas iremos revisitar e explorar mais a fundo esse conceito nessa aula. Vamos utilizar os dados do MovieLens novamente como exemplo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Primeiro carregamos os dados."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "usuarios = pd.read_csv(\n",
    "    \"http://files.grouplens.org/datasets/movielens/ml-100k/u.user\",\n",
    "    sep='|',header=None, names=[\"user_id\", \"age\", \"gender\", \"occupation\", \"zip_code\"])\n",
    "filmes = pd.read_csv(\n",
    "    \"http://files.grouplens.org/datasets/movielens/ml-100k/u.item\",\n",
    "    sep='|',header=None, names=[\"movie_id\", \"movie_title\",  \"release_date\", \"video_release_date\", \"IMDb_URL\", \"unknown\", \"Action\", \"Adventure\", \"Animation\", \n",
    "        \"Children\", \"Comedy\", \"Crime\", \"Documentary\", \"Drama\", \"Fantasy\",\"FilmNoir\", \n",
    "                                \"Horror\", \"Musical\", \"Mystery\", \"Romance\", \"SciFi\",\"Thriller\", \"War\", \"Western\"],\n",
    "encoding='latin1')\n",
    "avaliacoes = pd.read_csv(\n",
    "    \"http://files.grouplens.org/datasets/movielens/ml-100k/u.data\",\n",
    "    sep='\\t',header=None, names=[\"user_id\", \"movie_id\", \"rating\", \"timestamp\"])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Agora verificamos se os dados foram carregados corretamente, assim como se os tipos dos atributos estão condizentes com o que esperamos. O atributo `dtypes` revela os tipos assinalados às colunas de um data frame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "user_id        int64\n",
      "age            int64\n",
      "gender        object\n",
      "occupation    object\n",
      "zip_code      object\n",
      "dtype: object\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>occupation</th>\n",
       "      <th>zip_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>24</td>\n",
       "      <td>M</td>\n",
       "      <td>technician</td>\n",
       "      <td>85711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>53</td>\n",
       "      <td>F</td>\n",
       "      <td>other</td>\n",
       "      <td>94043</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>23</td>\n",
       "      <td>M</td>\n",
       "      <td>writer</td>\n",
       "      <td>32067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>24</td>\n",
       "      <td>M</td>\n",
       "      <td>technician</td>\n",
       "      <td>43537</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>33</td>\n",
       "      <td>F</td>\n",
       "      <td>other</td>\n",
       "      <td>15213</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id  age gender  occupation zip_code\n",
       "0        1   24      M  technician    85711\n",
       "1        2   53      F       other    94043\n",
       "2        3   23      M      writer    32067\n",
       "3        4   24      M  technician    43537\n",
       "4        5   33      F       other    15213"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(usuarios.dtypes)\n",
    "usuarios.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Analisando a tabela de usuário, vemos que `user_id` e `age` foram assinalados corretamente, porém os demais não foram. O gênero, como discutimos anteriormente, é categórico. Pandas possui um tipo `category`, mas ainda assim assinalou o tipo `object`. Podemos corrigir transformando os valores com a função `astype`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      M\n",
       "1      F\n",
       "2      M\n",
       "3      M\n",
       "4      F\n",
       "5      M\n",
       "6      M\n",
       "7      M\n",
       "8      M\n",
       "9      M\n",
       "10     F\n",
       "11     F\n",
       "12     M\n",
       "13     M\n",
       "14     F\n",
       "15     M\n",
       "16     M\n",
       "17     F\n",
       "18     M\n",
       "19     F\n",
       "20     M\n",
       "21     M\n",
       "22     F\n",
       "23     F\n",
       "24     M\n",
       "25     M\n",
       "26     F\n",
       "27     M\n",
       "28     M\n",
       "29     M\n",
       "      ..\n",
       "913    F\n",
       "914    M\n",
       "915    M\n",
       "916    F\n",
       "917    M\n",
       "918    M\n",
       "919    F\n",
       "920    F\n",
       "921    F\n",
       "922    M\n",
       "923    M\n",
       "924    F\n",
       "925    M\n",
       "926    M\n",
       "927    M\n",
       "928    M\n",
       "929    F\n",
       "930    M\n",
       "931    M\n",
       "932    M\n",
       "933    M\n",
       "934    M\n",
       "935    M\n",
       "936    M\n",
       "937    F\n",
       "938    F\n",
       "939    M\n",
       "940    M\n",
       "941    F\n",
       "942    M\n",
       "Name: gender, Length: 943, dtype: category\n",
       "Categories (2, object): [F, M]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "usuarios['gender'] = usuarios.gender.astype('category')\n",
    "usuarios.gender"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pandas irá tratar atributos categóricos de forma muito similar a objetos. No entanto, algumas funções estarão indisponíveis pelo tipo do dado. Além disso, podemos adicionar outras categorias ao tipo, ainda que elas não estejam presentes nos dados. Podemos também alterar os rótulos das categorias."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      Masculino\n",
       "1       Feminino\n",
       "2      Masculino\n",
       "3      Masculino\n",
       "4       Feminino\n",
       "5      Masculino\n",
       "6      Masculino\n",
       "7      Masculino\n",
       "8      Masculino\n",
       "9      Masculino\n",
       "10      Feminino\n",
       "11      Feminino\n",
       "12     Masculino\n",
       "13     Masculino\n",
       "14      Feminino\n",
       "15     Masculino\n",
       "16     Masculino\n",
       "17      Feminino\n",
       "18     Masculino\n",
       "19      Feminino\n",
       "20     Masculino\n",
       "21     Masculino\n",
       "22      Feminino\n",
       "23      Feminino\n",
       "24     Masculino\n",
       "25     Masculino\n",
       "26      Feminino\n",
       "27     Masculino\n",
       "28     Masculino\n",
       "29     Masculino\n",
       "         ...    \n",
       "913     Feminino\n",
       "914    Masculino\n",
       "915    Masculino\n",
       "916     Feminino\n",
       "917    Masculino\n",
       "918    Masculino\n",
       "919     Feminino\n",
       "920     Feminino\n",
       "921     Feminino\n",
       "922    Masculino\n",
       "923    Masculino\n",
       "924     Feminino\n",
       "925    Masculino\n",
       "926    Masculino\n",
       "927    Masculino\n",
       "928    Masculino\n",
       "929     Feminino\n",
       "930    Masculino\n",
       "931    Masculino\n",
       "932    Masculino\n",
       "933    Masculino\n",
       "934    Masculino\n",
       "935    Masculino\n",
       "936    Masculino\n",
       "937     Feminino\n",
       "938     Feminino\n",
       "939    Masculino\n",
       "940    Masculino\n",
       "941     Feminino\n",
       "942    Masculino\n",
       "Name: gender, Length: 943, dtype: category\n",
       "Categories (2, object): [Feminino, Masculino]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "usuarios.gender.cat.categories = ['Feminino', 'Masculino']\n",
    "usuarios.gender"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "O mesmo pode ser feito com a ocupação e CEP."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "student          196\n",
      "other            105\n",
      "educator          95\n",
      "administrator     79\n",
      "engineer          67\n",
      "programmer        66\n",
      "librarian         51\n",
      "writer            45\n",
      "executive         32\n",
      "scientist         31\n",
      "artist            28\n",
      "technician        27\n",
      "marketing         26\n",
      "entertainment     18\n",
      "healthcare        16\n",
      "retired           14\n",
      "lawyer            12\n",
      "salesman          12\n",
      "none               9\n",
      "homemaker          7\n",
      "doctor             7\n",
      "Name: occupation, dtype: int64\n",
      "count       943\n",
      "unique      795\n",
      "top       55414\n",
      "freq          9\n",
      "Name: zip_code, dtype: object\n"
     ]
    }
   ],
   "source": [
    "usuarios.occupation = usuarios.occupation.astype('category')\n",
    "usuarios.zip_code = usuarios.zip_code.astype('category')\n",
    "\n",
    "print(usuarios.occupation.value_counts())\n",
    "print(usuarios.zip_code.describe())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Repetimos o processo com as outras tabelas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "movie_id                int64\n",
      "movie_title            object\n",
      "release_date           object\n",
      "video_release_date    float64\n",
      "IMDb_URL               object\n",
      "unknown                 int64\n",
      "Action                  int64\n",
      "Adventure               int64\n",
      "Animation               int64\n",
      "Children                int64\n",
      "Comedy                  int64\n",
      "Crime                   int64\n",
      "Documentary             int64\n",
      "Drama                   int64\n",
      "Fantasy                 int64\n",
      "FilmNoir                int64\n",
      "Horror                  int64\n",
      "Musical                 int64\n",
      "Mystery                 int64\n",
      "Romance                 int64\n",
      "SciFi                   int64\n",
      "Thriller                int64\n",
      "War                     int64\n",
      "Western                 int64\n",
      "dtype: object\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movie_id</th>\n",
       "      <th>movie_title</th>\n",
       "      <th>release_date</th>\n",
       "      <th>video_release_date</th>\n",
       "      <th>IMDb_URL</th>\n",
       "      <th>unknown</th>\n",
       "      <th>Action</th>\n",
       "      <th>Adventure</th>\n",
       "      <th>Animation</th>\n",
       "      <th>Children</th>\n",
       "      <th>...</th>\n",
       "      <th>Fantasy</th>\n",
       "      <th>FilmNoir</th>\n",
       "      <th>Horror</th>\n",
       "      <th>Musical</th>\n",
       "      <th>Mystery</th>\n",
       "      <th>Romance</th>\n",
       "      <th>SciFi</th>\n",
       "      <th>Thriller</th>\n",
       "      <th>War</th>\n",
       "      <th>Western</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>01-Jan-1995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>http://us.imdb.com/M/title-exact?Toy%20Story%2...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>GoldenEye (1995)</td>\n",
       "      <td>01-Jan-1995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>http://us.imdb.com/M/title-exact?GoldenEye%20(...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Four Rooms (1995)</td>\n",
       "      <td>01-Jan-1995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>http://us.imdb.com/M/title-exact?Four%20Rooms%...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Get Shorty (1995)</td>\n",
       "      <td>01-Jan-1995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>http://us.imdb.com/M/title-exact?Get%20Shorty%...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Copycat (1995)</td>\n",
       "      <td>01-Jan-1995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>http://us.imdb.com/M/title-exact?Copycat%20(1995)</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   movie_id        movie_title release_date  video_release_date  \\\n",
       "0         1   Toy Story (1995)  01-Jan-1995                 NaN   \n",
       "1         2   GoldenEye (1995)  01-Jan-1995                 NaN   \n",
       "2         3  Four Rooms (1995)  01-Jan-1995                 NaN   \n",
       "3         4  Get Shorty (1995)  01-Jan-1995                 NaN   \n",
       "4         5     Copycat (1995)  01-Jan-1995                 NaN   \n",
       "\n",
       "                                            IMDb_URL  unknown  Action  \\\n",
       "0  http://us.imdb.com/M/title-exact?Toy%20Story%2...        0       0   \n",
       "1  http://us.imdb.com/M/title-exact?GoldenEye%20(...        0       1   \n",
       "2  http://us.imdb.com/M/title-exact?Four%20Rooms%...        0       0   \n",
       "3  http://us.imdb.com/M/title-exact?Get%20Shorty%...        0       1   \n",
       "4  http://us.imdb.com/M/title-exact?Copycat%20(1995)        0       0   \n",
       "\n",
       "   Adventure  Animation  Children   ...     Fantasy  FilmNoir  Horror  \\\n",
       "0          0          1         1   ...           0         0       0   \n",
       "1          1          0         0   ...           0         0       0   \n",
       "2          0          0         0   ...           0         0       0   \n",
       "3          0          0         0   ...           0         0       0   \n",
       "4          0          0         0   ...           0         0       0   \n",
       "\n",
       "   Musical  Mystery  Romance  SciFi  Thriller  War  Western  \n",
       "0        0        0        0      0         0    0        0  \n",
       "1        0        0        0      0         1    0        0  \n",
       "2        0        0        0      0         1    0        0  \n",
       "3        0        0        0      0         0    0        0  \n",
       "4        0        0        0      0         1    0        0  \n",
       "\n",
       "[5 rows x 24 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(filmes.dtypes)\n",
    "filmes.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Vemos que as colunas de gênero do filme e a data não estão compatíveis com o que esparávamos. Logo, devemos ajustar os tipos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0   1995-01-01\n",
      "1   1995-01-01\n",
      "2   1995-01-01\n",
      "3   1995-01-01\n",
      "4   1995-01-01\n",
      "Name: release_date, dtype: datetime64[ns]\n",
      "count                    1681\n",
      "unique                    240\n",
      "top       1995-01-01 00:00:00\n",
      "freq                      215\n",
      "first     1922-01-01 00:00:00\n",
      "last      1998-10-23 00:00:00\n",
      "Name: release_date, dtype: object\n"
     ]
    }
   ],
   "source": [
    "filmes.release_date = pd.to_datetime(filmes.release_date, format=\"%d-%b-%Y\")\n",
    "print(filmes.release_date.head())\n",
    "print(filmes.release_date.describe())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unknown</th>\n",
       "      <th>Action</th>\n",
       "      <th>Adventure</th>\n",
       "      <th>Animation</th>\n",
       "      <th>Children</th>\n",
       "      <th>Comedy</th>\n",
       "      <th>Crime</th>\n",
       "      <th>Documentary</th>\n",
       "      <th>Drama</th>\n",
       "      <th>Fantasy</th>\n",
       "      <th>FilmNoir</th>\n",
       "      <th>Horror</th>\n",
       "      <th>Musical</th>\n",
       "      <th>Mystery</th>\n",
       "      <th>Romance</th>\n",
       "      <th>SciFi</th>\n",
       "      <th>Thriller</th>\n",
       "      <th>War</th>\n",
       "      <th>Western</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>False</th>\n",
       "      <td>1680</td>\n",
       "      <td>1431</td>\n",
       "      <td>1547</td>\n",
       "      <td>1640</td>\n",
       "      <td>1560</td>\n",
       "      <td>1177</td>\n",
       "      <td>1573</td>\n",
       "      <td>1632</td>\n",
       "      <td>957</td>\n",
       "      <td>1660</td>\n",
       "      <td>1658</td>\n",
       "      <td>1590</td>\n",
       "      <td>1626</td>\n",
       "      <td>1621</td>\n",
       "      <td>1435</td>\n",
       "      <td>1581</td>\n",
       "      <td>1431</td>\n",
       "      <td>1611</td>\n",
       "      <td>1655</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>True</th>\n",
       "      <td>2</td>\n",
       "      <td>251</td>\n",
       "      <td>135</td>\n",
       "      <td>42</td>\n",
       "      <td>122</td>\n",
       "      <td>505</td>\n",
       "      <td>109</td>\n",
       "      <td>50</td>\n",
       "      <td>725</td>\n",
       "      <td>22</td>\n",
       "      <td>24</td>\n",
       "      <td>92</td>\n",
       "      <td>56</td>\n",
       "      <td>61</td>\n",
       "      <td>247</td>\n",
       "      <td>101</td>\n",
       "      <td>251</td>\n",
       "      <td>71</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       unknown  Action  Adventure  Animation  Children  Comedy  Crime  \\\n",
       "False     1680    1431       1547       1640      1560    1177   1573   \n",
       "True         2     251        135         42       122     505    109   \n",
       "\n",
       "       Documentary  Drama  Fantasy  FilmNoir  Horror  Musical  Mystery  \\\n",
       "False         1632    957     1660      1658    1590     1626     1621   \n",
       "True            50    725       22        24      92       56       61   \n",
       "\n",
       "       Romance  SciFi  Thriller   War  Western  \n",
       "False     1435   1581      1431  1611     1655  \n",
       "True       247    101       251    71       27  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filmes.iloc[:,5:] = filmes.iloc[:,5:].astype('bool')\n",
    "filmes.iloc[:,5:].apply(pd.value_counts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finalmente ajustamos a tabela de avaliações. Nesse caso, temos dois possíveis problemas. A avaliação e o horário estão como numéricos. A avaliação pode ser interpretada tanto como um atributo numérico nesse contexto quanto categórico. Em geral, não faz muito sentido falar de uma avaliação fracionária (ainda que possa ser interessante calcular a média de avaliações). Para praticar a transformação dos dados, vamos optar por codificar a avaliação como um atributo ordinal. O timestamp, por outro lado, deve ser transformado em data para facilitar a visualização."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "user_id      int64\n",
      "movie_id     int64\n",
      "rating       int64\n",
      "timestamp    int64\n",
      "dtype: object\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>movie_id</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>196</td>\n",
       "      <td>242</td>\n",
       "      <td>3</td>\n",
       "      <td>881250949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>186</td>\n",
       "      <td>302</td>\n",
       "      <td>3</td>\n",
       "      <td>891717742</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>22</td>\n",
       "      <td>377</td>\n",
       "      <td>1</td>\n",
       "      <td>878887116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>244</td>\n",
       "      <td>51</td>\n",
       "      <td>2</td>\n",
       "      <td>880606923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>166</td>\n",
       "      <td>346</td>\n",
       "      <td>1</td>\n",
       "      <td>886397596</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id  movie_id  rating  timestamp\n",
       "0      196       242       3  881250949\n",
       "1      186       302       3  891717742\n",
       "2       22       377       1  878887116\n",
       "3      244        51       2  880606923\n",
       "4      166       346       1  886397596"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(avaliacoes.dtypes)\n",
    "avaliacoes.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count     100000\n",
       "unique         5\n",
       "top            4\n",
       "freq       34174\n",
       "Name: rating, dtype: int64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from pandas.api.types import CategoricalDtype\n",
    "\n",
    "avaliacoes.rating = avaliacoes.rating.astype(CategoricalDtype(categories=avaliacoes.rating.unique().sort(),\n",
    "                                                                  ordered=True))\n",
    "avaliacoes.rating.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        3\n",
       "1        3\n",
       "2        1\n",
       "3        2\n",
       "4        1\n",
       "5        4\n",
       "6        2\n",
       "7        5\n",
       "8        3\n",
       "9        3\n",
       "10       2\n",
       "11       5\n",
       "12       5\n",
       "13       3\n",
       "14       3\n",
       "15       3\n",
       "16       5\n",
       "17       2\n",
       "18       4\n",
       "19       2\n",
       "20       4\n",
       "21       4\n",
       "22       4\n",
       "23       2\n",
       "24       4\n",
       "25       2\n",
       "26       5\n",
       "27       2\n",
       "28       4\n",
       "29       5\n",
       "        ..\n",
       "99970    1\n",
       "99971    2\n",
       "99972    3\n",
       "99973    4\n",
       "99974    3\n",
       "99975    3\n",
       "99976    2\n",
       "99977    3\n",
       "99978    5\n",
       "99979    2\n",
       "99980    4\n",
       "99981    3\n",
       "99982    1\n",
       "99983    3\n",
       "99984    2\n",
       "99985    4\n",
       "99986    3\n",
       "99987    2\n",
       "99988    4\n",
       "99989    4\n",
       "99990    4\n",
       "99991    4\n",
       "99992    3\n",
       "99993    2\n",
       "99994    3\n",
       "99995    3\n",
       "99996    5\n",
       "99997    1\n",
       "99998    2\n",
       "99999    3\n",
       "Name: rating, Length: 100000, dtype: category\n",
       "Categories (5, int64): [1 < 2 < 3 < 4 < 5]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "avaliacoes.rating"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count                  100000\n",
       "unique                  49282\n",
       "top       1998-03-27 21:20:06\n",
       "freq                       12\n",
       "first     1997-09-20 03:05:10\n",
       "last      1998-04-22 23:10:38\n",
       "Name: timestamp, dtype: object"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "avaliacoes.timestamp = pd.to_datetime(avaliacoes.timestamp,unit='s')\n",
    "avaliacoes.timestamp.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"dados-ausentes\"></a> Tratamento de dados ausentes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Outro passo importante para limpeza dos dados é o tratamento de dados ausentes. É quase impossível não nos depararmos com a falta de algum dado para algum atributo. Também devemos tratar essa situação antes de iniciarmos a análise.\n",
    "\n",
    "Temos algumas alternativas para tratar os dados ausentes:\n",
    "\n",
    "1. Ignorar os objetos com dados ausentes\n",
    "2. Preenchê-los manualmente\n",
    "3. Preenchê-los com um valor único global\n",
    "4. Usar alguma medida de tendência para atribuir o valor central\n",
    "5. Usar uma medida de tendência calculado sobre o grupo ao qual o objeto pertence\n",
    "6. Usar estatística inferencial ou aprendizado de máquina para inferir o valor\n",
    "\n",
    "Com exceção do último, veremos como executar essa tarefa em Pandas.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Ignorar objetos com dados ausentes\n",
    "\n",
    "Antes de tratar os dados ausentes, precisamos descobrir se eles existem ou não. Pandas possui a função `isnull` que retorna um valor boolean para cada linha/coluna indicando se o objeto possui ou não o dado. Logo, podemos contar quantos dados estão ausentes em cada coluna para sabermos a ausência nesses atributos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "movie_id                 0\n",
       "movie_title              0\n",
       "release_date             1\n",
       "video_release_date    1682\n",
       "IMDb_URL                 3\n",
       "unknown                  0\n",
       "Action                   0\n",
       "Adventure                0\n",
       "Animation                0\n",
       "Children                 0\n",
       "Comedy                   0\n",
       "Crime                    0\n",
       "Documentary              0\n",
       "Drama                    0\n",
       "Fantasy                  0\n",
       "FilmNoir                 0\n",
       "Horror                   0\n",
       "Musical                  0\n",
       "Mystery                  0\n",
       "Romance                  0\n",
       "SciFi                    0\n",
       "Thriller                 0\n",
       "War                      0\n",
       "Western                  0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filmes.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A função para descartar linhas ou colunas com dados ausentes é `dropna`. Podemos especificar se queremos descartar linhas/colunas com pelo menos um dado ausente, ou se ela deve ser descarta somente se todos o são. Também podemos direcionar a consulta a elementos específicos. Por exemplo, vimos acima que *video_release_date* é ausente para todos os dados, portanto, não queremos descartar objetos que não possuam esse atributo, pois eliminaríamos todos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "movie_id                 0\n",
      "movie_title              0\n",
      "release_date             0\n",
      "video_release_date    1679\n",
      "IMDb_URL                 0\n",
      "unknown                  0\n",
      "Action                   0\n",
      "Adventure                0\n",
      "Animation                0\n",
      "Children                 0\n",
      "Comedy                   0\n",
      "Crime                    0\n",
      "Documentary              0\n",
      "Drama                    0\n",
      "Fantasy                  0\n",
      "FilmNoir                 0\n",
      "Horror                   0\n",
      "Musical                  0\n",
      "Mystery                  0\n",
      "Romance                  0\n",
      "SciFi                    0\n",
      "Thriller                 0\n",
      "War                      0\n",
      "Western                  0\n",
      "dtype: int64\n",
      "movie_id                 0\n",
      "movie_title              0\n",
      "release_date             1\n",
      "video_release_date    1682\n",
      "IMDb_URL                 3\n",
      "unknown                  0\n",
      "Action                   0\n",
      "Adventure                0\n",
      "Animation                0\n",
      "Children                 0\n",
      "Comedy                   0\n",
      "Crime                    0\n",
      "Documentary              0\n",
      "Drama                    0\n",
      "Fantasy                  0\n",
      "FilmNoir                 0\n",
      "Horror                   0\n",
      "Musical                  0\n",
      "Mystery                  0\n",
      "Romance                  0\n",
      "SciFi                    0\n",
      "Thriller                 0\n",
      "War                      0\n",
      "Western                  0\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(filmes.dropna(subset=['release_date','IMDb_URL']).isnull().sum())\n",
    "print(filmes.isnull().sum())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "movie_id        0\n",
       "movie_title     0\n",
       "release_date    1\n",
       "IMDb_URL        3\n",
       "unknown         0\n",
       "Action          0\n",
       "Adventure       0\n",
       "Animation       0\n",
       "Children        0\n",
       "Comedy          0\n",
       "Crime           0\n",
       "Documentary     0\n",
       "Drama           0\n",
       "Fantasy         0\n",
       "FilmNoir        0\n",
       "Horror          0\n",
       "Musical         0\n",
       "Mystery         0\n",
       "Romance         0\n",
       "SciFi           0\n",
       "Thriller        0\n",
       "War             0\n",
       "Western         0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filmes.dropna(axis=1,how='all').isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Preencher manualmente valores ausentes\n",
    "\n",
    "Vimos que somente um filme não possui o *release_date*. Podemos querer consultar manualmente a data de lançamento desse filme e incluí-la no data frame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([266]),)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.where(filmes.release_date.isnull())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "movie_id                  267\n",
       "movie_title           unknown\n",
       "release_date              NaT\n",
       "video_release_date        NaN\n",
       "IMDb_URL                  NaN\n",
       "unknown                  True\n",
       "Action                  False\n",
       "Adventure               False\n",
       "Animation               False\n",
       "Children                False\n",
       "Comedy                  False\n",
       "Crime                   False\n",
       "Documentary             False\n",
       "Drama                   False\n",
       "Fantasy                 False\n",
       "FilmNoir                False\n",
       "Horror                  False\n",
       "Musical                 False\n",
       "Mystery                 False\n",
       "Romance                 False\n",
       "SciFi                   False\n",
       "Thriller                False\n",
       "War                     False\n",
       "Western                 False\n",
       "Name: 266, dtype: object"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filmes.iloc[266]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nesse caso específico, percebemos que se trata de uma inconsistência e, portanto, é descartá-la."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Preencher dados ausentes com valor (global ou estatística)\n",
    "\n",
    "Podemos preencher todos os dados ausentes com um único valor. Isso é feito em Pandas através da função `fillna`, tal como havíamos feito anteriormente quando extraímos o ano da data de lançamento e o convertemos para inteiro. A função espera um valor que será assinalado ao atributo. Esse valor pode ser arbitrário ou uma medida de tendência computada sobre os dados presentes. Por exemplo, podemos preencher a data de lançamento do filme descoberto acima com a moda."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('1995-01-01 00:00:00')"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filmes.release_date.fillna(value=filmes.release_date.mode()[0])[266]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Preencher dados ausentes com estatística computada por grupo\n",
    "\n",
    "Esse caso requer o particionamento dos dados usando `groupby`. Em seguida, usamos a função `transform`. O tutorial de [*split-apply-combine*](http://pandas.pydata.org/pandas-docs/stable/groupby.html#transformation) mostra um exemplo de como executar essa transformação."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"discretizacao\"></a> Discretização"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A depender do tipo de análise que se queira fazer com os dados, será necessária a conversão de dados do tipo numérico para categórico, ou até mesmo mapear o conjunto de categorias para um outro espectro. Por exemplo, em aulas anteriores dividimos os filmes por décadas para contar a quantidade de filmes de cada gênero por década. Podemos discretizar o atributo *release_date*, mapeando para um novo atributo década.\n",
    "\n",
    "Existem de forma geral dois tipos de discretização: não-supervisionada, e supervisionada. Os métodos não-supervisionados assumem que os objetos não possuem rótulos e, portanto, levam em consideração apenas os valores do atributo para discretizá-lo. As duas técnicas mais comuns de discretização não-supervisionada são divisão em intervalos de mesma **largura**, e de mesma **frequência**. As técnicas supervisionadas, por outro lado, levam em consideração o rótulo dos objetos ao buscar os intervalos. A técnica mais comum é a baseada em entropia proposta por [Fayyad e Irani (1993)](http://web.donga.ac.kr/kjunwoo/files/Multi%20interval%20discretization%20of%20continuous%20valued%20attributes%20for%20classification%20learning.pdf). A ideia básica desse método é encontrar intervalos de tal forma que a proporção de objetos com um mesmo rótulo seja maximizada.\n",
    "\n",
    "Python/Pandas possui suporte para discretização não-supervisionada. Contudo, não existe um módulo (oficial) que forneça suporte a discretização por entropia. É possível fazê-la utilizando as bibliotecas de R atrvés do módulo `rpy2`. Dessa forma, restringiremos nossos exemplos à discretização não-supervisionada."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Discretização por intervalos com mesma largura\n",
    "\n",
    "A discretização por intervalos com mesma largura é feita através da função `cut` de Pandas.\n",
    "\n",
    "```python\n",
    "pandas.cut(x, bins, right=True, labels=None, retbins=False, precision=3, include_lowest=False)\n",
    "```\n",
    "\n",
    "O número de intervalos é definido pelo parâmetro `bins`. Esse parâmetro pode ser tanto um inteiro quanto uma sequência de valores escalares. No caso de inteiro, a função ordena os valores a serem discretizados e determina os intervalos igualmente espaçados. No caso de ser uma sequência, a função utiliza os valores como limites dos intervalos. O parâmetro `labels` permite ainda que os rótulos das categorias sejam informados. Caso contrário a função gerará rótulos do tipo `(inicio, fim]`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       90s\n",
       "1       90s\n",
       "2       90s\n",
       "3       90s\n",
       "4       90s\n",
       "5       90s\n",
       "6       90s\n",
       "7       90s\n",
       "8       90s\n",
       "9       90s\n",
       "10      90s\n",
       "11      90s\n",
       "12      90s\n",
       "13      90s\n",
       "14      90s\n",
       "15      90s\n",
       "16      90s\n",
       "17      90s\n",
       "18      90s\n",
       "19      90s\n",
       "20      90s\n",
       "21      90s\n",
       "22      90s\n",
       "23      90s\n",
       "24      90s\n",
       "25      90s\n",
       "26      90s\n",
       "27      90s\n",
       "28      90s\n",
       "29      60s\n",
       "       ... \n",
       "1652    90s\n",
       "1653    90s\n",
       "1654    90s\n",
       "1655    90s\n",
       "1656    90s\n",
       "1657    90s\n",
       "1658    90s\n",
       "1659    90s\n",
       "1660    90s\n",
       "1661    90s\n",
       "1662    90s\n",
       "1663    90s\n",
       "1664    90s\n",
       "1665    90s\n",
       "1666    90s\n",
       "1667    90s\n",
       "1668    90s\n",
       "1669    90s\n",
       "1670    90s\n",
       "1671    90s\n",
       "1672    90s\n",
       "1673    60s\n",
       "1674    90s\n",
       "1675    90s\n",
       "1676    90s\n",
       "1677    90s\n",
       "1678    90s\n",
       "1679    90s\n",
       "1680    90s\n",
       "1681    90s\n",
       "Name: decada, Length: 1682, dtype: category\n",
       "Categories (7, object): [20s < 40s < 50s < 60s < 70s < 80s < 90s]"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filmes['decada'] = pd.cut(filmes.release_date.map(lambda x: x.year),\n",
    "       bins=np.insert(np.linspace(1940,2000,7,dtype=np.int32),0,1920),\n",
    "        labels=[\"20s\"]+list(map(lambda x: '{}s'.format(x%100),\n",
    "       np.linspace(1940,2000,7,dtype=np.int32)[:6])))\n",
    "\n",
    "filmes.decada\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Discretização por intervalos com mesma frequência\n",
    "\n",
    "A discretização por intervalos com mesma frequência é feita em Pandas através da função `qcut`:\n",
    "\n",
    "```python\n",
    "pandas.qcut(x, q, labels=None, retbins=False, precision=3)\n",
    "```\n",
    "\n",
    "A discretização nesse caso é feita por quantis. O parâmetro `q` pode ser tanto um inteiro quanto uma lista de quantis. No caso de inteiro, esse é referente ao número de quantis; por exemplo, 4 faz com que a discretização seja feita com base nos quartis. Caso seja uma lista de quantis, eles serão usados para discretizar. É importante ressaltar que os quantis são sempre calculados sobre a amostra (dados).\n",
    "\n",
    "Podemos usar essa função para discretizar as avaliações (tomadas como números) em *ruim*, *média*, e *boa*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        (0.999, 3.0]\n",
       "1        (0.999, 3.0]\n",
       "2        (0.999, 3.0]\n",
       "3        (0.999, 3.0]\n",
       "4        (0.999, 3.0]\n",
       "5          (3.0, 4.0]\n",
       "6        (0.999, 3.0]\n",
       "7          (4.0, 5.0]\n",
       "8        (0.999, 3.0]\n",
       "9        (0.999, 3.0]\n",
       "10       (0.999, 3.0]\n",
       "11         (4.0, 5.0]\n",
       "12         (4.0, 5.0]\n",
       "13       (0.999, 3.0]\n",
       "14       (0.999, 3.0]\n",
       "15       (0.999, 3.0]\n",
       "16         (4.0, 5.0]\n",
       "17       (0.999, 3.0]\n",
       "18         (3.0, 4.0]\n",
       "19       (0.999, 3.0]\n",
       "20         (3.0, 4.0]\n",
       "21         (3.0, 4.0]\n",
       "22         (3.0, 4.0]\n",
       "23       (0.999, 3.0]\n",
       "24         (3.0, 4.0]\n",
       "25       (0.999, 3.0]\n",
       "26         (4.0, 5.0]\n",
       "27       (0.999, 3.0]\n",
       "28         (3.0, 4.0]\n",
       "29         (4.0, 5.0]\n",
       "             ...     \n",
       "99970    (0.999, 3.0]\n",
       "99971    (0.999, 3.0]\n",
       "99972    (0.999, 3.0]\n",
       "99973      (3.0, 4.0]\n",
       "99974    (0.999, 3.0]\n",
       "99975    (0.999, 3.0]\n",
       "99976    (0.999, 3.0]\n",
       "99977    (0.999, 3.0]\n",
       "99978      (4.0, 5.0]\n",
       "99979    (0.999, 3.0]\n",
       "99980      (3.0, 4.0]\n",
       "99981    (0.999, 3.0]\n",
       "99982    (0.999, 3.0]\n",
       "99983    (0.999, 3.0]\n",
       "99984    (0.999, 3.0]\n",
       "99985      (3.0, 4.0]\n",
       "99986    (0.999, 3.0]\n",
       "99987    (0.999, 3.0]\n",
       "99988      (3.0, 4.0]\n",
       "99989      (3.0, 4.0]\n",
       "99990      (3.0, 4.0]\n",
       "99991      (3.0, 4.0]\n",
       "99992    (0.999, 3.0]\n",
       "99993    (0.999, 3.0]\n",
       "99994    (0.999, 3.0]\n",
       "99995    (0.999, 3.0]\n",
       "99996      (4.0, 5.0]\n",
       "99997    (0.999, 3.0]\n",
       "99998    (0.999, 3.0]\n",
       "99999    (0.999, 3.0]\n",
       "Name: rating, Length: 100000, dtype: category\n",
       "Categories (3, interval[float64]): [(0.999, 3.0] < (3.0, 4.0] < (4.0, 5.0]]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.qcut(avaliacoes.rating.astype(int),3)\n",
    "#pd.qcut(avaliacoes.rating.astype(int),3).cat.rename_categories(\n",
    "#    {pd.Interval(left=0.999, right=3.0,closed='right'):'ruim'})\n",
    "#pd.qcut(avaliacoes.rating.astype(int),3).cat.rename_categories(['ruim','media','boa'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"normalizacao\"></a> Normalização\n",
    "\n",
    "Em várias situações devemos ajustar valores em diferentes escalas para uma escala comum (em geral [0,1] ou [-1,1]). Isso ocorre porque os algoritmos de mineração/aprendizado podem atribuir pesos maiores aos atributos com valores maiores. Por exemplo, algoritmos de clustering usam distância euclidiana entre objetos computada sobre os atributos. Atributos com grandezas maiores podem 'dominar' outros com grandezas menores.\n",
    "\n",
    "O processo de normalização é uma tentativa de conferir pesos iguais a todos os atributos. As técnicas mais comuns de normalização são *min-max* e *z-score*.\n",
    "\n",
    "Usaremos, como exemplo de normalização, os dados sobre qualidade de vinhos disponível em http://mlr.cs.umass.edu/ml/machine-learning-databases/wine-quality/winequality-white.csv.\n",
    "Veremos como as duas técnicas podem ser facilmente computadas em Python, usando Pandas e Scikit-Learn.\n",
    "\n",
    "Logo, antes de iniciarmos as discussões sobre as técnicas, precisamos carregar os dados."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 4898 entries, 0 to 4897\n",
      "Data columns (total 12 columns):\n",
      "fixed acidity           4898 non-null float64\n",
      "volatile acidity        4898 non-null float64\n",
      "citric acid             4898 non-null float64\n",
      "residual sugar          4898 non-null float64\n",
      "chlorides               4898 non-null float64\n",
      "free sulfur dioxide     4898 non-null float64\n",
      "total sulfur dioxide    4898 non-null float64\n",
      "density                 4898 non-null float64\n",
      "pH                      4898 non-null float64\n",
      "sulphates               4898 non-null float64\n",
      "alcohol                 4898 non-null float64\n",
      "quality                 4898 non-null int64\n",
      "dtypes: float64(11), int64(1)\n",
      "memory usage: 459.3 KB\n",
      "None\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "      <th>quality</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7.0</td>\n",
       "      <td>0.27</td>\n",
       "      <td>0.36</td>\n",
       "      <td>20.7</td>\n",
       "      <td>0.045</td>\n",
       "      <td>45.0</td>\n",
       "      <td>170.0</td>\n",
       "      <td>1.0010</td>\n",
       "      <td>3.00</td>\n",
       "      <td>0.45</td>\n",
       "      <td>8.8</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>6.3</td>\n",
       "      <td>0.30</td>\n",
       "      <td>0.34</td>\n",
       "      <td>1.6</td>\n",
       "      <td>0.049</td>\n",
       "      <td>14.0</td>\n",
       "      <td>132.0</td>\n",
       "      <td>0.9940</td>\n",
       "      <td>3.30</td>\n",
       "      <td>0.49</td>\n",
       "      <td>9.5</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8.1</td>\n",
       "      <td>0.28</td>\n",
       "      <td>0.40</td>\n",
       "      <td>6.9</td>\n",
       "      <td>0.050</td>\n",
       "      <td>30.0</td>\n",
       "      <td>97.0</td>\n",
       "      <td>0.9951</td>\n",
       "      <td>3.26</td>\n",
       "      <td>0.44</td>\n",
       "      <td>10.1</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7.2</td>\n",
       "      <td>0.23</td>\n",
       "      <td>0.32</td>\n",
       "      <td>8.5</td>\n",
       "      <td>0.058</td>\n",
       "      <td>47.0</td>\n",
       "      <td>186.0</td>\n",
       "      <td>0.9956</td>\n",
       "      <td>3.19</td>\n",
       "      <td>0.40</td>\n",
       "      <td>9.9</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7.2</td>\n",
       "      <td>0.23</td>\n",
       "      <td>0.32</td>\n",
       "      <td>8.5</td>\n",
       "      <td>0.058</td>\n",
       "      <td>47.0</td>\n",
       "      <td>186.0</td>\n",
       "      <td>0.9956</td>\n",
       "      <td>3.19</td>\n",
       "      <td>0.40</td>\n",
       "      <td>9.9</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n",
       "0            7.0              0.27         0.36            20.7      0.045   \n",
       "1            6.3              0.30         0.34             1.6      0.049   \n",
       "2            8.1              0.28         0.40             6.9      0.050   \n",
       "3            7.2              0.23         0.32             8.5      0.058   \n",
       "4            7.2              0.23         0.32             8.5      0.058   \n",
       "\n",
       "   free sulfur dioxide  total sulfur dioxide  density    pH  sulphates  \\\n",
       "0                 45.0                 170.0   1.0010  3.00       0.45   \n",
       "1                 14.0                 132.0   0.9940  3.30       0.49   \n",
       "2                 30.0                  97.0   0.9951  3.26       0.44   \n",
       "3                 47.0                 186.0   0.9956  3.19       0.40   \n",
       "4                 47.0                 186.0   0.9956  3.19       0.40   \n",
       "\n",
       "   alcohol  quality  \n",
       "0      8.8        6  \n",
       "1      9.5        6  \n",
       "2     10.1        6  \n",
       "3      9.9        6  \n",
       "4      9.9        6  "
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wine = pd.read_csv(\"http://mlr.cs.umass.edu/ml/machine-learning-databases/wine-quality/winequality-white.csv\",\n",
    "                   sep=';')\n",
    "print(wine.info())\n",
    "wine.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Podemos observar que os atributos numéricos possuem escalas bastante diferentes. Por exemplo, o teor alcoólico, dióxido de enxofre e cloretos variam em escalas bem diferentes. Nesse caso, precisamos ajustar os valores a uma escala comum."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Normalização por *min-max*\n",
    "\n",
    "Os valores de um atributo são ajustados ao novo intervalo $[a,b]$ pela seguinte fórmula:\n",
    "\n",
    "$$\n",
    "v[i] = \\frac{v[i] - min(v)}{max(v) - min(v)}(b-a)+a\n",
    "$$\n",
    "\n",
    "Podemos aplicar a fórmula usando a função `apply` de Pandas, ou usar [MinMaxScaler](http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html#sklearn.preprocessing.MinMaxScaler) de `sklearn`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.307692</td>\n",
       "      <td>0.186275</td>\n",
       "      <td>0.216867</td>\n",
       "      <td>0.308282</td>\n",
       "      <td>0.106825</td>\n",
       "      <td>0.149826</td>\n",
       "      <td>0.373550</td>\n",
       "      <td>0.267785</td>\n",
       "      <td>0.254545</td>\n",
       "      <td>0.267442</td>\n",
       "      <td>0.129032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.240385</td>\n",
       "      <td>0.215686</td>\n",
       "      <td>0.204819</td>\n",
       "      <td>0.015337</td>\n",
       "      <td>0.118694</td>\n",
       "      <td>0.041812</td>\n",
       "      <td>0.285383</td>\n",
       "      <td>0.132832</td>\n",
       "      <td>0.527273</td>\n",
       "      <td>0.313953</td>\n",
       "      <td>0.241935</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.413462</td>\n",
       "      <td>0.196078</td>\n",
       "      <td>0.240964</td>\n",
       "      <td>0.096626</td>\n",
       "      <td>0.121662</td>\n",
       "      <td>0.097561</td>\n",
       "      <td>0.204176</td>\n",
       "      <td>0.154039</td>\n",
       "      <td>0.490909</td>\n",
       "      <td>0.255814</td>\n",
       "      <td>0.338710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.326923</td>\n",
       "      <td>0.147059</td>\n",
       "      <td>0.192771</td>\n",
       "      <td>0.121166</td>\n",
       "      <td>0.145401</td>\n",
       "      <td>0.156794</td>\n",
       "      <td>0.410673</td>\n",
       "      <td>0.163678</td>\n",
       "      <td>0.427273</td>\n",
       "      <td>0.209302</td>\n",
       "      <td>0.306452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.326923</td>\n",
       "      <td>0.147059</td>\n",
       "      <td>0.192771</td>\n",
       "      <td>0.121166</td>\n",
       "      <td>0.145401</td>\n",
       "      <td>0.156794</td>\n",
       "      <td>0.410673</td>\n",
       "      <td>0.163678</td>\n",
       "      <td>0.427273</td>\n",
       "      <td>0.209302</td>\n",
       "      <td>0.306452</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n",
       "0       0.307692          0.186275     0.216867        0.308282   0.106825   \n",
       "1       0.240385          0.215686     0.204819        0.015337   0.118694   \n",
       "2       0.413462          0.196078     0.240964        0.096626   0.121662   \n",
       "3       0.326923          0.147059     0.192771        0.121166   0.145401   \n",
       "4       0.326923          0.147059     0.192771        0.121166   0.145401   \n",
       "\n",
       "   free sulfur dioxide  total sulfur dioxide   density        pH  sulphates  \\\n",
       "0             0.149826              0.373550  0.267785  0.254545   0.267442   \n",
       "1             0.041812              0.285383  0.132832  0.527273   0.313953   \n",
       "2             0.097561              0.204176  0.154039  0.490909   0.255814   \n",
       "3             0.156794              0.410673  0.163678  0.427273   0.209302   \n",
       "4             0.156794              0.410673  0.163678  0.427273   0.209302   \n",
       "\n",
       "    alcohol  \n",
       "0  0.129032  \n",
       "1  0.241935  \n",
       "2  0.338710  \n",
       "3  0.306452  \n",
       "4  0.306452  "
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wine.drop('quality',axis=1).apply(lambda x: (x-x.min())/(x.max()-x.min())).head()\n",
    "#wine.iloc[:,:-1].apply(lambda x: (x-x.min())/(x.max()-x.min()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler, MinMaxScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.307692</td>\n",
       "      <td>0.186275</td>\n",
       "      <td>0.216867</td>\n",
       "      <td>0.308282</td>\n",
       "      <td>0.106825</td>\n",
       "      <td>0.149826</td>\n",
       "      <td>0.373550</td>\n",
       "      <td>0.267785</td>\n",
       "      <td>0.254545</td>\n",
       "      <td>0.267442</td>\n",
       "      <td>0.129032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.240385</td>\n",
       "      <td>0.215686</td>\n",
       "      <td>0.204819</td>\n",
       "      <td>0.015337</td>\n",
       "      <td>0.118694</td>\n",
       "      <td>0.041812</td>\n",
       "      <td>0.285383</td>\n",
       "      <td>0.132832</td>\n",
       "      <td>0.527273</td>\n",
       "      <td>0.313953</td>\n",
       "      <td>0.241935</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.413462</td>\n",
       "      <td>0.196078</td>\n",
       "      <td>0.240964</td>\n",
       "      <td>0.096626</td>\n",
       "      <td>0.121662</td>\n",
       "      <td>0.097561</td>\n",
       "      <td>0.204176</td>\n",
       "      <td>0.154039</td>\n",
       "      <td>0.490909</td>\n",
       "      <td>0.255814</td>\n",
       "      <td>0.338710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.326923</td>\n",
       "      <td>0.147059</td>\n",
       "      <td>0.192771</td>\n",
       "      <td>0.121166</td>\n",
       "      <td>0.145401</td>\n",
       "      <td>0.156794</td>\n",
       "      <td>0.410673</td>\n",
       "      <td>0.163678</td>\n",
       "      <td>0.427273</td>\n",
       "      <td>0.209302</td>\n",
       "      <td>0.306452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.326923</td>\n",
       "      <td>0.147059</td>\n",
       "      <td>0.192771</td>\n",
       "      <td>0.121166</td>\n",
       "      <td>0.145401</td>\n",
       "      <td>0.156794</td>\n",
       "      <td>0.410673</td>\n",
       "      <td>0.163678</td>\n",
       "      <td>0.427273</td>\n",
       "      <td>0.209302</td>\n",
       "      <td>0.306452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.413462</td>\n",
       "      <td>0.196078</td>\n",
       "      <td>0.240964</td>\n",
       "      <td>0.096626</td>\n",
       "      <td>0.121662</td>\n",
       "      <td>0.097561</td>\n",
       "      <td>0.204176</td>\n",
       "      <td>0.154039</td>\n",
       "      <td>0.490909</td>\n",
       "      <td>0.255814</td>\n",
       "      <td>0.338710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.230769</td>\n",
       "      <td>0.235294</td>\n",
       "      <td>0.096386</td>\n",
       "      <td>0.098160</td>\n",
       "      <td>0.106825</td>\n",
       "      <td>0.097561</td>\n",
       "      <td>0.294664</td>\n",
       "      <td>0.150183</td>\n",
       "      <td>0.418182</td>\n",
       "      <td>0.290698</td>\n",
       "      <td>0.258065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.307692</td>\n",
       "      <td>0.186275</td>\n",
       "      <td>0.216867</td>\n",
       "      <td>0.308282</td>\n",
       "      <td>0.106825</td>\n",
       "      <td>0.149826</td>\n",
       "      <td>0.373550</td>\n",
       "      <td>0.267785</td>\n",
       "      <td>0.254545</td>\n",
       "      <td>0.267442</td>\n",
       "      <td>0.129032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.240385</td>\n",
       "      <td>0.215686</td>\n",
       "      <td>0.204819</td>\n",
       "      <td>0.015337</td>\n",
       "      <td>0.118694</td>\n",
       "      <td>0.041812</td>\n",
       "      <td>0.285383</td>\n",
       "      <td>0.132832</td>\n",
       "      <td>0.527273</td>\n",
       "      <td>0.313953</td>\n",
       "      <td>0.241935</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.413462</td>\n",
       "      <td>0.137255</td>\n",
       "      <td>0.259036</td>\n",
       "      <td>0.013804</td>\n",
       "      <td>0.103858</td>\n",
       "      <td>0.090592</td>\n",
       "      <td>0.278422</td>\n",
       "      <td>0.128976</td>\n",
       "      <td>0.454545</td>\n",
       "      <td>0.267442</td>\n",
       "      <td>0.483871</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.413462</td>\n",
       "      <td>0.186275</td>\n",
       "      <td>0.246988</td>\n",
       "      <td>0.013037</td>\n",
       "      <td>0.071217</td>\n",
       "      <td>0.031359</td>\n",
       "      <td>0.125290</td>\n",
       "      <td>0.071139</td>\n",
       "      <td>0.245455</td>\n",
       "      <td>0.395349</td>\n",
       "      <td>0.645161</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.461538</td>\n",
       "      <td>0.147059</td>\n",
       "      <td>0.240964</td>\n",
       "      <td>0.055215</td>\n",
       "      <td>0.077151</td>\n",
       "      <td>0.052265</td>\n",
       "      <td>0.232019</td>\n",
       "      <td>0.146327</td>\n",
       "      <td>0.381818</td>\n",
       "      <td>0.360465</td>\n",
       "      <td>0.274194</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0.394231</td>\n",
       "      <td>0.098039</td>\n",
       "      <td>0.222892</td>\n",
       "      <td>0.009202</td>\n",
       "      <td>0.091988</td>\n",
       "      <td>0.048780</td>\n",
       "      <td>0.153132</td>\n",
       "      <td>0.094274</td>\n",
       "      <td>0.418182</td>\n",
       "      <td>0.476744</td>\n",
       "      <td>0.451613</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0.269231</td>\n",
       "      <td>0.078431</td>\n",
       "      <td>0.240964</td>\n",
       "      <td>0.013804</td>\n",
       "      <td>0.103858</td>\n",
       "      <td>0.160279</td>\n",
       "      <td>0.310905</td>\n",
       "      <td>0.078851</td>\n",
       "      <td>0.745455</td>\n",
       "      <td>0.348837</td>\n",
       "      <td>0.709677</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.432692</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.373494</td>\n",
       "      <td>0.286043</td>\n",
       "      <td>0.091988</td>\n",
       "      <td>0.135889</td>\n",
       "      <td>0.378190</td>\n",
       "      <td>0.252362</td>\n",
       "      <td>0.236364</td>\n",
       "      <td>0.523256</td>\n",
       "      <td>0.274194</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0.269231</td>\n",
       "      <td>0.088235</td>\n",
       "      <td>0.228916</td>\n",
       "      <td>0.013804</td>\n",
       "      <td>0.068249</td>\n",
       "      <td>0.090592</td>\n",
       "      <td>0.238979</td>\n",
       "      <td>0.082707</td>\n",
       "      <td>0.481818</td>\n",
       "      <td>0.383721</td>\n",
       "      <td>0.548387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0.240385</td>\n",
       "      <td>0.392157</td>\n",
       "      <td>0.024096</td>\n",
       "      <td>0.007669</td>\n",
       "      <td>0.109792</td>\n",
       "      <td>0.097561</td>\n",
       "      <td>0.208817</td>\n",
       "      <td>0.109697</td>\n",
       "      <td>0.472727</td>\n",
       "      <td>0.162791</td>\n",
       "      <td>0.258065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0.230769</td>\n",
       "      <td>0.568627</td>\n",
       "      <td>0.289157</td>\n",
       "      <td>0.009202</td>\n",
       "      <td>0.059347</td>\n",
       "      <td>0.094077</td>\n",
       "      <td>0.153132</td>\n",
       "      <td>0.040293</td>\n",
       "      <td>0.554545</td>\n",
       "      <td>0.197674</td>\n",
       "      <td>0.774194</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0.346154</td>\n",
       "      <td>0.254902</td>\n",
       "      <td>0.253012</td>\n",
       "      <td>0.007669</td>\n",
       "      <td>0.071217</td>\n",
       "      <td>0.052265</td>\n",
       "      <td>0.375870</td>\n",
       "      <td>0.088490</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.360465</td>\n",
       "      <td>0.532258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>0.259615</td>\n",
       "      <td>0.225490</td>\n",
       "      <td>0.084337</td>\n",
       "      <td>0.105828</td>\n",
       "      <td>0.103858</td>\n",
       "      <td>0.111498</td>\n",
       "      <td>0.287703</td>\n",
       "      <td>0.161751</td>\n",
       "      <td>0.454545</td>\n",
       "      <td>0.325581</td>\n",
       "      <td>0.241935</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0.230769</td>\n",
       "      <td>0.568627</td>\n",
       "      <td>0.289157</td>\n",
       "      <td>0.009202</td>\n",
       "      <td>0.059347</td>\n",
       "      <td>0.094077</td>\n",
       "      <td>0.153132</td>\n",
       "      <td>0.040293</td>\n",
       "      <td>0.554545</td>\n",
       "      <td>0.197674</td>\n",
       "      <td>0.774194</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0.250000</td>\n",
       "      <td>0.225490</td>\n",
       "      <td>0.228916</td>\n",
       "      <td>0.035276</td>\n",
       "      <td>0.086053</td>\n",
       "      <td>0.059233</td>\n",
       "      <td>0.215777</td>\n",
       "      <td>0.078851</td>\n",
       "      <td>0.409091</td>\n",
       "      <td>0.151163</td>\n",
       "      <td>0.483871</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>0.288462</td>\n",
       "      <td>0.176471</td>\n",
       "      <td>0.253012</td>\n",
       "      <td>0.016871</td>\n",
       "      <td>0.118694</td>\n",
       "      <td>0.135889</td>\n",
       "      <td>0.262181</td>\n",
       "      <td>0.113553</td>\n",
       "      <td>0.681818</td>\n",
       "      <td>0.302326</td>\n",
       "      <td>0.403226</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>0.365385</td>\n",
       "      <td>0.578431</td>\n",
       "      <td>0.084337</td>\n",
       "      <td>0.013804</td>\n",
       "      <td>0.192878</td>\n",
       "      <td>0.080139</td>\n",
       "      <td>0.368910</td>\n",
       "      <td>0.127048</td>\n",
       "      <td>0.300000</td>\n",
       "      <td>0.337209</td>\n",
       "      <td>0.209677</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>0.269231</td>\n",
       "      <td>0.186275</td>\n",
       "      <td>0.246988</td>\n",
       "      <td>0.010736</td>\n",
       "      <td>0.127596</td>\n",
       "      <td>0.048780</td>\n",
       "      <td>0.308585</td>\n",
       "      <td>0.154039</td>\n",
       "      <td>0.636364</td>\n",
       "      <td>0.290698</td>\n",
       "      <td>0.322581</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>0.307692</td>\n",
       "      <td>0.166667</td>\n",
       "      <td>0.192771</td>\n",
       "      <td>0.128834</td>\n",
       "      <td>0.109792</td>\n",
       "      <td>0.188153</td>\n",
       "      <td>0.547564</td>\n",
       "      <td>0.161751</td>\n",
       "      <td>0.481818</td>\n",
       "      <td>0.325581</td>\n",
       "      <td>0.387097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>0.298077</td>\n",
       "      <td>0.156863</td>\n",
       "      <td>0.210843</td>\n",
       "      <td>0.006135</td>\n",
       "      <td>0.127596</td>\n",
       "      <td>0.114983</td>\n",
       "      <td>0.317865</td>\n",
       "      <td>0.113553</td>\n",
       "      <td>0.663636</td>\n",
       "      <td>0.255814</td>\n",
       "      <td>0.322581</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>0.307692</td>\n",
       "      <td>0.196078</td>\n",
       "      <td>0.234940</td>\n",
       "      <td>0.124233</td>\n",
       "      <td>0.124629</td>\n",
       "      <td>0.104530</td>\n",
       "      <td>0.306265</td>\n",
       "      <td>0.173318</td>\n",
       "      <td>0.600000</td>\n",
       "      <td>0.360465</td>\n",
       "      <td>0.403226</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0.346154</td>\n",
       "      <td>0.186275</td>\n",
       "      <td>0.289157</td>\n",
       "      <td>0.007669</td>\n",
       "      <td>0.112760</td>\n",
       "      <td>0.052265</td>\n",
       "      <td>0.285383</td>\n",
       "      <td>0.082707</td>\n",
       "      <td>0.427273</td>\n",
       "      <td>0.313953</td>\n",
       "      <td>0.580645</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>0.326923</td>\n",
       "      <td>0.235294</td>\n",
       "      <td>0.216867</td>\n",
       "      <td>0.021472</td>\n",
       "      <td>0.071217</td>\n",
       "      <td>0.121951</td>\n",
       "      <td>0.243619</td>\n",
       "      <td>0.067284</td>\n",
       "      <td>0.345455</td>\n",
       "      <td>0.569767</td>\n",
       "      <td>0.693548</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4868</th>\n",
       "      <td>0.192308</td>\n",
       "      <td>0.147059</td>\n",
       "      <td>0.186747</td>\n",
       "      <td>0.059816</td>\n",
       "      <td>0.109792</td>\n",
       "      <td>0.139373</td>\n",
       "      <td>0.266821</td>\n",
       "      <td>0.118180</td>\n",
       "      <td>0.536364</td>\n",
       "      <td>0.488372</td>\n",
       "      <td>0.451613</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4869</th>\n",
       "      <td>0.269231</td>\n",
       "      <td>0.156863</td>\n",
       "      <td>0.198795</td>\n",
       "      <td>0.145706</td>\n",
       "      <td>0.068249</td>\n",
       "      <td>0.020906</td>\n",
       "      <td>0.167053</td>\n",
       "      <td>0.176403</td>\n",
       "      <td>0.427273</td>\n",
       "      <td>0.337209</td>\n",
       "      <td>0.290323</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4870</th>\n",
       "      <td>0.221154</td>\n",
       "      <td>0.235294</td>\n",
       "      <td>0.168675</td>\n",
       "      <td>0.092025</td>\n",
       "      <td>0.035608</td>\n",
       "      <td>0.094077</td>\n",
       "      <td>0.285383</td>\n",
       "      <td>0.091961</td>\n",
       "      <td>0.390909</td>\n",
       "      <td>0.162791</td>\n",
       "      <td>0.556452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4871</th>\n",
       "      <td>0.115385</td>\n",
       "      <td>0.117647</td>\n",
       "      <td>0.240964</td>\n",
       "      <td>0.019939</td>\n",
       "      <td>0.017804</td>\n",
       "      <td>0.062718</td>\n",
       "      <td>0.206497</td>\n",
       "      <td>0.049933</td>\n",
       "      <td>0.590909</td>\n",
       "      <td>0.383721</td>\n",
       "      <td>0.653226</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4872</th>\n",
       "      <td>0.211538</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.246988</td>\n",
       "      <td>0.180982</td>\n",
       "      <td>0.068249</td>\n",
       "      <td>0.167247</td>\n",
       "      <td>0.394432</td>\n",
       "      <td>0.175631</td>\n",
       "      <td>0.381818</td>\n",
       "      <td>0.441860</td>\n",
       "      <td>0.274194</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4873</th>\n",
       "      <td>0.182692</td>\n",
       "      <td>0.127451</td>\n",
       "      <td>0.192771</td>\n",
       "      <td>0.015337</td>\n",
       "      <td>0.062315</td>\n",
       "      <td>0.108014</td>\n",
       "      <td>0.262181</td>\n",
       "      <td>0.064199</td>\n",
       "      <td>0.554545</td>\n",
       "      <td>0.348837</td>\n",
       "      <td>0.629032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4874</th>\n",
       "      <td>0.173077</td>\n",
       "      <td>0.117647</td>\n",
       "      <td>0.216867</td>\n",
       "      <td>0.029141</td>\n",
       "      <td>0.115727</td>\n",
       "      <td>0.048780</td>\n",
       "      <td>0.269142</td>\n",
       "      <td>0.110083</td>\n",
       "      <td>0.700000</td>\n",
       "      <td>0.313953</td>\n",
       "      <td>0.322581</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4875</th>\n",
       "      <td>0.346154</td>\n",
       "      <td>0.137255</td>\n",
       "      <td>0.156627</td>\n",
       "      <td>0.009202</td>\n",
       "      <td>0.077151</td>\n",
       "      <td>0.055749</td>\n",
       "      <td>0.204176</td>\n",
       "      <td>0.102950</td>\n",
       "      <td>0.363636</td>\n",
       "      <td>0.220930</td>\n",
       "      <td>0.274194</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4876</th>\n",
       "      <td>0.230769</td>\n",
       "      <td>0.294118</td>\n",
       "      <td>0.253012</td>\n",
       "      <td>0.029141</td>\n",
       "      <td>0.086053</td>\n",
       "      <td>0.111498</td>\n",
       "      <td>0.250580</td>\n",
       "      <td>0.081164</td>\n",
       "      <td>0.581818</td>\n",
       "      <td>0.430233</td>\n",
       "      <td>0.580645</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4877</th>\n",
       "      <td>0.201923</td>\n",
       "      <td>0.450980</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.003067</td>\n",
       "      <td>0.068249</td>\n",
       "      <td>0.034843</td>\n",
       "      <td>0.169374</td>\n",
       "      <td>0.110854</td>\n",
       "      <td>0.481818</td>\n",
       "      <td>0.162791</td>\n",
       "      <td>0.129032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4878</th>\n",
       "      <td>0.230769</td>\n",
       "      <td>0.441176</td>\n",
       "      <td>0.012048</td>\n",
       "      <td>0.004601</td>\n",
       "      <td>0.077151</td>\n",
       "      <td>0.013937</td>\n",
       "      <td>0.167053</td>\n",
       "      <td>0.100829</td>\n",
       "      <td>0.472727</td>\n",
       "      <td>0.151163</td>\n",
       "      <td>0.241935</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4879</th>\n",
       "      <td>0.269231</td>\n",
       "      <td>0.254902</td>\n",
       "      <td>0.240964</td>\n",
       "      <td>0.115031</td>\n",
       "      <td>0.109792</td>\n",
       "      <td>0.229965</td>\n",
       "      <td>0.373550</td>\n",
       "      <td>0.150954</td>\n",
       "      <td>0.390909</td>\n",
       "      <td>0.325581</td>\n",
       "      <td>0.247312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4880</th>\n",
       "      <td>0.269231</td>\n",
       "      <td>0.254902</td>\n",
       "      <td>0.240964</td>\n",
       "      <td>0.115031</td>\n",
       "      <td>0.109792</td>\n",
       "      <td>0.229965</td>\n",
       "      <td>0.373550</td>\n",
       "      <td>0.150954</td>\n",
       "      <td>0.390909</td>\n",
       "      <td>0.325581</td>\n",
       "      <td>0.247312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4881</th>\n",
       "      <td>0.115385</td>\n",
       "      <td>0.151961</td>\n",
       "      <td>0.162651</td>\n",
       "      <td>0.171012</td>\n",
       "      <td>0.062315</td>\n",
       "      <td>0.111498</td>\n",
       "      <td>0.252900</td>\n",
       "      <td>0.159823</td>\n",
       "      <td>0.318182</td>\n",
       "      <td>0.325581</td>\n",
       "      <td>0.225806</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4882</th>\n",
       "      <td>0.163462</td>\n",
       "      <td>0.235294</td>\n",
       "      <td>0.078313</td>\n",
       "      <td>0.010736</td>\n",
       "      <td>0.083086</td>\n",
       "      <td>0.149826</td>\n",
       "      <td>0.341067</td>\n",
       "      <td>0.091190</td>\n",
       "      <td>0.490909</td>\n",
       "      <td>0.186047</td>\n",
       "      <td>0.435484</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4883</th>\n",
       "      <td>0.105769</td>\n",
       "      <td>0.382353</td>\n",
       "      <td>0.102410</td>\n",
       "      <td>0.019939</td>\n",
       "      <td>0.077151</td>\n",
       "      <td>0.202091</td>\n",
       "      <td>0.322506</td>\n",
       "      <td>0.048776</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.151163</td>\n",
       "      <td>0.564516</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4884</th>\n",
       "      <td>0.259615</td>\n",
       "      <td>0.245098</td>\n",
       "      <td>0.228916</td>\n",
       "      <td>0.118098</td>\n",
       "      <td>0.115727</td>\n",
       "      <td>0.229965</td>\n",
       "      <td>0.382831</td>\n",
       "      <td>0.150569</td>\n",
       "      <td>0.381818</td>\n",
       "      <td>0.325581</td>\n",
       "      <td>0.258065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4885</th>\n",
       "      <td>0.269231</td>\n",
       "      <td>0.254902</td>\n",
       "      <td>0.240964</td>\n",
       "      <td>0.115031</td>\n",
       "      <td>0.109792</td>\n",
       "      <td>0.229965</td>\n",
       "      <td>0.373550</td>\n",
       "      <td>0.150954</td>\n",
       "      <td>0.390909</td>\n",
       "      <td>0.325581</td>\n",
       "      <td>0.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4886</th>\n",
       "      <td>0.230769</td>\n",
       "      <td>0.127451</td>\n",
       "      <td>0.168675</td>\n",
       "      <td>0.078221</td>\n",
       "      <td>0.056380</td>\n",
       "      <td>0.149826</td>\n",
       "      <td>0.259861</td>\n",
       "      <td>0.088105</td>\n",
       "      <td>0.445455</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.669355</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4887</th>\n",
       "      <td>0.230769</td>\n",
       "      <td>0.323529</td>\n",
       "      <td>0.132530</td>\n",
       "      <td>0.019939</td>\n",
       "      <td>0.041543</td>\n",
       "      <td>0.010453</td>\n",
       "      <td>0.109049</td>\n",
       "      <td>0.041835</td>\n",
       "      <td>0.290909</td>\n",
       "      <td>0.662791</td>\n",
       "      <td>0.806452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4888</th>\n",
       "      <td>0.288462</td>\n",
       "      <td>0.137255</td>\n",
       "      <td>0.216867</td>\n",
       "      <td>0.009202</td>\n",
       "      <td>0.127596</td>\n",
       "      <td>0.125436</td>\n",
       "      <td>0.273782</td>\n",
       "      <td>0.119337</td>\n",
       "      <td>0.290909</td>\n",
       "      <td>0.372093</td>\n",
       "      <td>0.193548</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4889</th>\n",
       "      <td>0.105769</td>\n",
       "      <td>0.151961</td>\n",
       "      <td>0.162651</td>\n",
       "      <td>0.171012</td>\n",
       "      <td>0.062315</td>\n",
       "      <td>0.111498</td>\n",
       "      <td>0.252900</td>\n",
       "      <td>0.159823</td>\n",
       "      <td>0.318182</td>\n",
       "      <td>0.325581</td>\n",
       "      <td>0.225806</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4890</th>\n",
       "      <td>0.221154</td>\n",
       "      <td>0.254902</td>\n",
       "      <td>0.174699</td>\n",
       "      <td>0.024540</td>\n",
       "      <td>0.080119</td>\n",
       "      <td>0.080139</td>\n",
       "      <td>0.211137</td>\n",
       "      <td>0.043763</td>\n",
       "      <td>0.309091</td>\n",
       "      <td>0.255814</td>\n",
       "      <td>0.612903</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4891</th>\n",
       "      <td>0.182692</td>\n",
       "      <td>0.127451</td>\n",
       "      <td>0.192771</td>\n",
       "      <td>0.004601</td>\n",
       "      <td>0.086053</td>\n",
       "      <td>0.125436</td>\n",
       "      <td>0.259861</td>\n",
       "      <td>0.069983</td>\n",
       "      <td>0.472727</td>\n",
       "      <td>0.279070</td>\n",
       "      <td>0.419355</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4892</th>\n",
       "      <td>0.259615</td>\n",
       "      <td>0.147059</td>\n",
       "      <td>0.228916</td>\n",
       "      <td>0.010736</td>\n",
       "      <td>0.068249</td>\n",
       "      <td>0.094077</td>\n",
       "      <td>0.238979</td>\n",
       "      <td>0.113168</td>\n",
       "      <td>0.518182</td>\n",
       "      <td>0.372093</td>\n",
       "      <td>0.274194</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4893</th>\n",
       "      <td>0.230769</td>\n",
       "      <td>0.127451</td>\n",
       "      <td>0.174699</td>\n",
       "      <td>0.015337</td>\n",
       "      <td>0.089021</td>\n",
       "      <td>0.076655</td>\n",
       "      <td>0.192575</td>\n",
       "      <td>0.077694</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.325581</td>\n",
       "      <td>0.516129</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4894</th>\n",
       "      <td>0.269231</td>\n",
       "      <td>0.235294</td>\n",
       "      <td>0.216867</td>\n",
       "      <td>0.113497</td>\n",
       "      <td>0.112760</td>\n",
       "      <td>0.191638</td>\n",
       "      <td>0.368910</td>\n",
       "      <td>0.150183</td>\n",
       "      <td>0.390909</td>\n",
       "      <td>0.279070</td>\n",
       "      <td>0.258065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4895</th>\n",
       "      <td>0.259615</td>\n",
       "      <td>0.156863</td>\n",
       "      <td>0.114458</td>\n",
       "      <td>0.009202</td>\n",
       "      <td>0.094955</td>\n",
       "      <td>0.097561</td>\n",
       "      <td>0.236659</td>\n",
       "      <td>0.104685</td>\n",
       "      <td>0.245455</td>\n",
       "      <td>0.279070</td>\n",
       "      <td>0.225806</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4896</th>\n",
       "      <td>0.163462</td>\n",
       "      <td>0.205882</td>\n",
       "      <td>0.180723</td>\n",
       "      <td>0.007669</td>\n",
       "      <td>0.038576</td>\n",
       "      <td>0.062718</td>\n",
       "      <td>0.234339</td>\n",
       "      <td>0.030461</td>\n",
       "      <td>0.563636</td>\n",
       "      <td>0.186047</td>\n",
       "      <td>0.774194</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4897</th>\n",
       "      <td>0.211538</td>\n",
       "      <td>0.127451</td>\n",
       "      <td>0.228916</td>\n",
       "      <td>0.003067</td>\n",
       "      <td>0.032641</td>\n",
       "      <td>0.069686</td>\n",
       "      <td>0.206497</td>\n",
       "      <td>0.044342</td>\n",
       "      <td>0.490909</td>\n",
       "      <td>0.116279</td>\n",
       "      <td>0.612903</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4898 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n",
       "0          0.307692          0.186275     0.216867        0.308282   0.106825   \n",
       "1          0.240385          0.215686     0.204819        0.015337   0.118694   \n",
       "2          0.413462          0.196078     0.240964        0.096626   0.121662   \n",
       "3          0.326923          0.147059     0.192771        0.121166   0.145401   \n",
       "4          0.326923          0.147059     0.192771        0.121166   0.145401   \n",
       "5          0.413462          0.196078     0.240964        0.096626   0.121662   \n",
       "6          0.230769          0.235294     0.096386        0.098160   0.106825   \n",
       "7          0.307692          0.186275     0.216867        0.308282   0.106825   \n",
       "8          0.240385          0.215686     0.204819        0.015337   0.118694   \n",
       "9          0.413462          0.137255     0.259036        0.013804   0.103858   \n",
       "10         0.413462          0.186275     0.246988        0.013037   0.071217   \n",
       "11         0.461538          0.147059     0.240964        0.055215   0.077151   \n",
       "12         0.394231          0.098039     0.222892        0.009202   0.091988   \n",
       "13         0.269231          0.078431     0.240964        0.013804   0.103858   \n",
       "14         0.432692          0.333333     0.373494        0.286043   0.091988   \n",
       "15         0.269231          0.088235     0.228916        0.013804   0.068249   \n",
       "16         0.240385          0.392157     0.024096        0.007669   0.109792   \n",
       "17         0.230769          0.568627     0.289157        0.009202   0.059347   \n",
       "18         0.346154          0.254902     0.253012        0.007669   0.071217   \n",
       "19         0.259615          0.225490     0.084337        0.105828   0.103858   \n",
       "20         0.230769          0.568627     0.289157        0.009202   0.059347   \n",
       "21         0.250000          0.225490     0.228916        0.035276   0.086053   \n",
       "22         0.288462          0.176471     0.253012        0.016871   0.118694   \n",
       "23         0.365385          0.578431     0.084337        0.013804   0.192878   \n",
       "24         0.269231          0.186275     0.246988        0.010736   0.127596   \n",
       "25         0.307692          0.166667     0.192771        0.128834   0.109792   \n",
       "26         0.298077          0.156863     0.210843        0.006135   0.127596   \n",
       "27         0.307692          0.196078     0.234940        0.124233   0.124629   \n",
       "28         0.346154          0.186275     0.289157        0.007669   0.112760   \n",
       "29         0.326923          0.235294     0.216867        0.021472   0.071217   \n",
       "...             ...               ...          ...             ...        ...   \n",
       "4868       0.192308          0.147059     0.186747        0.059816   0.109792   \n",
       "4869       0.269231          0.156863     0.198795        0.145706   0.068249   \n",
       "4870       0.221154          0.235294     0.168675        0.092025   0.035608   \n",
       "4871       0.115385          0.117647     0.240964        0.019939   0.017804   \n",
       "4872       0.211538          0.333333     0.246988        0.180982   0.068249   \n",
       "4873       0.182692          0.127451     0.192771        0.015337   0.062315   \n",
       "4874       0.173077          0.117647     0.216867        0.029141   0.115727   \n",
       "4875       0.346154          0.137255     0.156627        0.009202   0.077151   \n",
       "4876       0.230769          0.294118     0.253012        0.029141   0.086053   \n",
       "4877       0.201923          0.450980     0.000000        0.003067   0.068249   \n",
       "4878       0.230769          0.441176     0.012048        0.004601   0.077151   \n",
       "4879       0.269231          0.254902     0.240964        0.115031   0.109792   \n",
       "4880       0.269231          0.254902     0.240964        0.115031   0.109792   \n",
       "4881       0.115385          0.151961     0.162651        0.171012   0.062315   \n",
       "4882       0.163462          0.235294     0.078313        0.010736   0.083086   \n",
       "4883       0.105769          0.382353     0.102410        0.019939   0.077151   \n",
       "4884       0.259615          0.245098     0.228916        0.118098   0.115727   \n",
       "4885       0.269231          0.254902     0.240964        0.115031   0.109792   \n",
       "4886       0.230769          0.127451     0.168675        0.078221   0.056380   \n",
       "4887       0.230769          0.323529     0.132530        0.019939   0.041543   \n",
       "4888       0.288462          0.137255     0.216867        0.009202   0.127596   \n",
       "4889       0.105769          0.151961     0.162651        0.171012   0.062315   \n",
       "4890       0.221154          0.254902     0.174699        0.024540   0.080119   \n",
       "4891       0.182692          0.127451     0.192771        0.004601   0.086053   \n",
       "4892       0.259615          0.147059     0.228916        0.010736   0.068249   \n",
       "4893       0.230769          0.127451     0.174699        0.015337   0.089021   \n",
       "4894       0.269231          0.235294     0.216867        0.113497   0.112760   \n",
       "4895       0.259615          0.156863     0.114458        0.009202   0.094955   \n",
       "4896       0.163462          0.205882     0.180723        0.007669   0.038576   \n",
       "4897       0.211538          0.127451     0.228916        0.003067   0.032641   \n",
       "\n",
       "      free sulfur dioxide  total sulfur dioxide   density        pH  \\\n",
       "0                0.149826              0.373550  0.267785  0.254545   \n",
       "1                0.041812              0.285383  0.132832  0.527273   \n",
       "2                0.097561              0.204176  0.154039  0.490909   \n",
       "3                0.156794              0.410673  0.163678  0.427273   \n",
       "4                0.156794              0.410673  0.163678  0.427273   \n",
       "5                0.097561              0.204176  0.154039  0.490909   \n",
       "6                0.097561              0.294664  0.150183  0.418182   \n",
       "7                0.149826              0.373550  0.267785  0.254545   \n",
       "8                0.041812              0.285383  0.132832  0.527273   \n",
       "9                0.090592              0.278422  0.128976  0.454545   \n",
       "10               0.031359              0.125290  0.071139  0.245455   \n",
       "11               0.052265              0.232019  0.146327  0.381818   \n",
       "12               0.048780              0.153132  0.094274  0.418182   \n",
       "13               0.160279              0.310905  0.078851  0.745455   \n",
       "14               0.135889              0.378190  0.252362  0.236364   \n",
       "15               0.090592              0.238979  0.082707  0.481818   \n",
       "16               0.097561              0.208817  0.109697  0.472727   \n",
       "17               0.094077              0.153132  0.040293  0.554545   \n",
       "18               0.052265              0.375870  0.088490  0.363636   \n",
       "19               0.111498              0.287703  0.161751  0.454545   \n",
       "20               0.094077              0.153132  0.040293  0.554545   \n",
       "21               0.059233              0.215777  0.078851  0.409091   \n",
       "22               0.135889              0.262181  0.113553  0.681818   \n",
       "23               0.080139              0.368910  0.127048  0.300000   \n",
       "24               0.048780              0.308585  0.154039  0.636364   \n",
       "25               0.188153              0.547564  0.161751  0.481818   \n",
       "26               0.114983              0.317865  0.113553  0.663636   \n",
       "27               0.104530              0.306265  0.173318  0.600000   \n",
       "28               0.052265              0.285383  0.082707  0.427273   \n",
       "29               0.121951              0.243619  0.067284  0.345455   \n",
       "...                   ...                   ...       ...       ...   \n",
       "4868             0.139373              0.266821  0.118180  0.536364   \n",
       "4869             0.020906              0.167053  0.176403  0.427273   \n",
       "4870             0.094077              0.285383  0.091961  0.390909   \n",
       "4871             0.062718              0.206497  0.049933  0.590909   \n",
       "4872             0.167247              0.394432  0.175631  0.381818   \n",
       "4873             0.108014              0.262181  0.064199  0.554545   \n",
       "4874             0.048780              0.269142  0.110083  0.700000   \n",
       "4875             0.055749              0.204176  0.102950  0.363636   \n",
       "4876             0.111498              0.250580  0.081164  0.581818   \n",
       "4877             0.034843              0.169374  0.110854  0.481818   \n",
       "4878             0.013937              0.167053  0.100829  0.472727   \n",
       "4879             0.229965              0.373550  0.150954  0.390909   \n",
       "4880             0.229965              0.373550  0.150954  0.390909   \n",
       "4881             0.111498              0.252900  0.159823  0.318182   \n",
       "4882             0.149826              0.341067  0.091190  0.490909   \n",
       "4883             0.202091              0.322506  0.048776  0.500000   \n",
       "4884             0.229965              0.382831  0.150569  0.381818   \n",
       "4885             0.229965              0.373550  0.150954  0.390909   \n",
       "4886             0.149826              0.259861  0.088105  0.445455   \n",
       "4887             0.010453              0.109049  0.041835  0.290909   \n",
       "4888             0.125436              0.273782  0.119337  0.290909   \n",
       "4889             0.111498              0.252900  0.159823  0.318182   \n",
       "4890             0.080139              0.211137  0.043763  0.309091   \n",
       "4891             0.125436              0.259861  0.069983  0.472727   \n",
       "4892             0.094077              0.238979  0.113168  0.518182   \n",
       "4893             0.076655              0.192575  0.077694  0.500000   \n",
       "4894             0.191638              0.368910  0.150183  0.390909   \n",
       "4895             0.097561              0.236659  0.104685  0.245455   \n",
       "4896             0.062718              0.234339  0.030461  0.563636   \n",
       "4897             0.069686              0.206497  0.044342  0.490909   \n",
       "\n",
       "      sulphates   alcohol  \n",
       "0      0.267442  0.129032  \n",
       "1      0.313953  0.241935  \n",
       "2      0.255814  0.338710  \n",
       "3      0.209302  0.306452  \n",
       "4      0.209302  0.306452  \n",
       "5      0.255814  0.338710  \n",
       "6      0.290698  0.258065  \n",
       "7      0.267442  0.129032  \n",
       "8      0.313953  0.241935  \n",
       "9      0.267442  0.483871  \n",
       "10     0.395349  0.645161  \n",
       "11     0.360465  0.274194  \n",
       "12     0.476744  0.451613  \n",
       "13     0.348837  0.709677  \n",
       "14     0.523256  0.274194  \n",
       "15     0.383721  0.548387  \n",
       "16     0.162791  0.258065  \n",
       "17     0.197674  0.774194  \n",
       "18     0.360465  0.532258  \n",
       "19     0.325581  0.241935  \n",
       "20     0.197674  0.774194  \n",
       "21     0.151163  0.483871  \n",
       "22     0.302326  0.403226  \n",
       "23     0.337209  0.209677  \n",
       "24     0.290698  0.322581  \n",
       "25     0.325581  0.387097  \n",
       "26     0.255814  0.322581  \n",
       "27     0.360465  0.403226  \n",
       "28     0.313953  0.580645  \n",
       "29     0.569767  0.693548  \n",
       "...         ...       ...  \n",
       "4868   0.488372  0.451613  \n",
       "4869   0.337209  0.290323  \n",
       "4870   0.162791  0.556452  \n",
       "4871   0.383721  0.653226  \n",
       "4872   0.441860  0.274194  \n",
       "4873   0.348837  0.629032  \n",
       "4874   0.313953  0.322581  \n",
       "4875   0.220930  0.274194  \n",
       "4876   0.430233  0.580645  \n",
       "4877   0.162791  0.129032  \n",
       "4878   0.151163  0.241935  \n",
       "4879   0.325581  0.247312  \n",
       "4880   0.325581  0.247312  \n",
       "4881   0.325581  0.225806  \n",
       "4882   0.186047  0.435484  \n",
       "4883   0.151163  0.564516  \n",
       "4884   0.325581  0.258065  \n",
       "4885   0.325581  0.250000  \n",
       "4886   1.000000  0.669355  \n",
       "4887   0.662791  0.806452  \n",
       "4888   0.372093  0.193548  \n",
       "4889   0.325581  0.225806  \n",
       "4890   0.255814  0.612903  \n",
       "4891   0.279070  0.419355  \n",
       "4892   0.372093  0.274194  \n",
       "4893   0.325581  0.516129  \n",
       "4894   0.279070  0.258065  \n",
       "4895   0.279070  0.225806  \n",
       "4896   0.186047  0.774194  \n",
       "4897   0.116279  0.612903  \n",
       "\n",
       "[4898 rows x 11 columns]"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scaler = MinMaxScaler()\n",
    "#scaler.fit_transform(wine.drop('quality',axis=1))\n",
    "pd.DataFrame(scaler.fit_transform(wine.drop('quality',axis=1)), columns=wine.drop('quality',axis=1).columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Veja que o resultado da transformação é um vetor de numpy. Podemos criar um novo data frame a partir desse vetor e assinalar as colunas do data frame original, ou podemos substituir os valores no data frame original."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n",
      "0            7.0              0.27         0.36            20.7      0.045   \n",
      "1            6.3              0.30         0.34             1.6      0.049   \n",
      "2            8.1              0.28         0.40             6.9      0.050   \n",
      "3            7.2              0.23         0.32             8.5      0.058   \n",
      "4            7.2              0.23         0.32             8.5      0.058   \n",
      "\n",
      "   free sulfur dioxide  total sulfur dioxide  density    pH  sulphates  \\\n",
      "0                 45.0                 170.0   1.0010  3.00       0.45   \n",
      "1                 14.0                 132.0   0.9940  3.30       0.49   \n",
      "2                 30.0                  97.0   0.9951  3.26       0.44   \n",
      "3                 47.0                 186.0   0.9956  3.19       0.40   \n",
      "4                 47.0                 186.0   0.9956  3.19       0.40   \n",
      "\n",
      "   alcohol  \n",
      "0      8.8  \n",
      "1      9.5  \n",
      "2     10.1  \n",
      "3      9.9  \n",
      "4      9.9  \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.307692</td>\n",
       "      <td>0.186275</td>\n",
       "      <td>0.216867</td>\n",
       "      <td>0.308282</td>\n",
       "      <td>0.106825</td>\n",
       "      <td>0.149826</td>\n",
       "      <td>0.373550</td>\n",
       "      <td>0.267785</td>\n",
       "      <td>0.254545</td>\n",
       "      <td>0.267442</td>\n",
       "      <td>0.129032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.240385</td>\n",
       "      <td>0.215686</td>\n",
       "      <td>0.204819</td>\n",
       "      <td>0.015337</td>\n",
       "      <td>0.118694</td>\n",
       "      <td>0.041812</td>\n",
       "      <td>0.285383</td>\n",
       "      <td>0.132832</td>\n",
       "      <td>0.527273</td>\n",
       "      <td>0.313953</td>\n",
       "      <td>0.241935</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.413462</td>\n",
       "      <td>0.196078</td>\n",
       "      <td>0.240964</td>\n",
       "      <td>0.096626</td>\n",
       "      <td>0.121662</td>\n",
       "      <td>0.097561</td>\n",
       "      <td>0.204176</td>\n",
       "      <td>0.154039</td>\n",
       "      <td>0.490909</td>\n",
       "      <td>0.255814</td>\n",
       "      <td>0.338710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.326923</td>\n",
       "      <td>0.147059</td>\n",
       "      <td>0.192771</td>\n",
       "      <td>0.121166</td>\n",
       "      <td>0.145401</td>\n",
       "      <td>0.156794</td>\n",
       "      <td>0.410673</td>\n",
       "      <td>0.163678</td>\n",
       "      <td>0.427273</td>\n",
       "      <td>0.209302</td>\n",
       "      <td>0.306452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.326923</td>\n",
       "      <td>0.147059</td>\n",
       "      <td>0.192771</td>\n",
       "      <td>0.121166</td>\n",
       "      <td>0.145401</td>\n",
       "      <td>0.156794</td>\n",
       "      <td>0.410673</td>\n",
       "      <td>0.163678</td>\n",
       "      <td>0.427273</td>\n",
       "      <td>0.209302</td>\n",
       "      <td>0.306452</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n",
       "0       0.307692          0.186275     0.216867        0.308282   0.106825   \n",
       "1       0.240385          0.215686     0.204819        0.015337   0.118694   \n",
       "2       0.413462          0.196078     0.240964        0.096626   0.121662   \n",
       "3       0.326923          0.147059     0.192771        0.121166   0.145401   \n",
       "4       0.326923          0.147059     0.192771        0.121166   0.145401   \n",
       "\n",
       "   free sulfur dioxide  total sulfur dioxide   density        pH  sulphates  \\\n",
       "0             0.149826              0.373550  0.267785  0.254545   0.267442   \n",
       "1             0.041812              0.285383  0.132832  0.527273   0.313953   \n",
       "2             0.097561              0.204176  0.154039  0.490909   0.255814   \n",
       "3             0.156794              0.410673  0.163678  0.427273   0.209302   \n",
       "4             0.156794              0.410673  0.163678  0.427273   0.209302   \n",
       "\n",
       "    alcohol  \n",
       "0  0.129032  \n",
       "1  0.241935  \n",
       "2  0.338710  \n",
       "3  0.306452  \n",
       "4  0.306452  "
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wineCopy = wine.copy().drop('quality',axis=1)\n",
    "print(wineCopy.head())\n",
    "wineCopy.iloc[:,:] = scaler.fit_transform(wine.drop('quality',axis=1))\n",
    "wineCopy.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "#### Normalização por *z-score*\n",
    "\n",
    "A normalização por *z-score* consiste em centralizar os dados em torno da média e reescalá-los para que o desvio-padrão seja unitário. A fórmula para o ajuste é:\n",
    "\n",
    "$$\n",
    "v[i] = \\frac{v[i]-\\bar{v}}{\\sigma_v}\n",
    "$$\n",
    "\n",
    "Novamente, podemos fazê-lo aplicando a fórmula diretamente com Pandas, ou usando [StandardScaler](http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html#sklearn.preprocessing.StandardScaler) de `sklearn`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>2.653755e-14</td>\n",
       "      <td>-1.053431e-14</td>\n",
       "      <td>5.344610e-14</td>\n",
       "      <td>-2.538326e-15</td>\n",
       "      <td>-1.419036e-15</td>\n",
       "      <td>6.210721e-18</td>\n",
       "      <td>-1.387439e-16</td>\n",
       "      <td>2.148461e-12</td>\n",
       "      <td>1.316599e-14</td>\n",
       "      <td>-1.280696e-14</td>\n",
       "      <td>-2.846868e-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-3.619982e+00</td>\n",
       "      <td>-1.966784e+00</td>\n",
       "      <td>-2.761461e+00</td>\n",
       "      <td>-1.141827e+00</td>\n",
       "      <td>-1.683102e+00</td>\n",
       "      <td>-1.958477e+00</td>\n",
       "      <td>-3.043919e+00</td>\n",
       "      <td>-2.312802e+00</td>\n",
       "      <td>-3.101091e+00</td>\n",
       "      <td>-2.364468e+00</td>\n",
       "      <td>-2.043089e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-6.574340e-01</td>\n",
       "      <td>-6.770318e-01</td>\n",
       "      <td>-5.304215e-01</td>\n",
       "      <td>-9.249530e-01</td>\n",
       "      <td>-4.472890e-01</td>\n",
       "      <td>-7.237012e-01</td>\n",
       "      <td>-7.144009e-01</td>\n",
       "      <td>-7.706280e-01</td>\n",
       "      <td>-6.507699e-01</td>\n",
       "      <td>-6.996389e-01</td>\n",
       "      <td>-8.241915e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>-6.492444e-02</td>\n",
       "      <td>-1.809733e-01</td>\n",
       "      <td>-1.172660e-01</td>\n",
       "      <td>-2.348977e-01</td>\n",
       "      <td>-1.268931e-01</td>\n",
       "      <td>-7.691388e-02</td>\n",
       "      <td>-1.026084e-01</td>\n",
       "      <td>-9.608339e-02</td>\n",
       "      <td>-5.474574e-02</td>\n",
       "      <td>-1.739035e-01</td>\n",
       "      <td>-9.285319e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>5.275851e-01</td>\n",
       "      <td>4.142970e-01</td>\n",
       "      <td>4.611517e-01</td>\n",
       "      <td>6.917479e-01</td>\n",
       "      <td>1.935028e-01</td>\n",
       "      <td>6.286722e-01</td>\n",
       "      <td>6.738976e-01</td>\n",
       "      <td>6.929749e-01</td>\n",
       "      <td>6.075033e-01</td>\n",
       "      <td>5.270772e-01</td>\n",
       "      <td>7.197450e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>8.704217e+00</td>\n",
       "      <td>8.152811e+00</td>\n",
       "      <td>1.095530e+01</td>\n",
       "      <td>1.171292e+01</td>\n",
       "      <td>1.374167e+01</td>\n",
       "      <td>1.491679e+01</td>\n",
       "      <td>7.097720e+00</td>\n",
       "      <td>1.502976e+01</td>\n",
       "      <td>4.183648e+00</td>\n",
       "      <td>5.171074e+00</td>\n",
       "      <td>2.995020e+00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       fixed acidity  volatile acidity   citric acid  residual sugar  \\\n",
       "count   4.898000e+03      4.898000e+03  4.898000e+03    4.898000e+03   \n",
       "mean    2.653755e-14     -1.053431e-14  5.344610e-14   -2.538326e-15   \n",
       "std     1.000000e+00      1.000000e+00  1.000000e+00    1.000000e+00   \n",
       "min    -3.619982e+00     -1.966784e+00 -2.761461e+00   -1.141827e+00   \n",
       "25%    -6.574340e-01     -6.770318e-01 -5.304215e-01   -9.249530e-01   \n",
       "50%    -6.492444e-02     -1.809733e-01 -1.172660e-01   -2.348977e-01   \n",
       "75%     5.275851e-01      4.142970e-01  4.611517e-01    6.917479e-01   \n",
       "max     8.704217e+00      8.152811e+00  1.095530e+01    1.171292e+01   \n",
       "\n",
       "          chlorides  free sulfur dioxide  total sulfur dioxide       density  \\\n",
       "count  4.898000e+03         4.898000e+03          4.898000e+03  4.898000e+03   \n",
       "mean  -1.419036e-15         6.210721e-18         -1.387439e-16  2.148461e-12   \n",
       "std    1.000000e+00         1.000000e+00          1.000000e+00  1.000000e+00   \n",
       "min   -1.683102e+00        -1.958477e+00         -3.043919e+00 -2.312802e+00   \n",
       "25%   -4.472890e-01        -7.237012e-01         -7.144009e-01 -7.706280e-01   \n",
       "50%   -1.268931e-01        -7.691388e-02         -1.026084e-01 -9.608339e-02   \n",
       "75%    1.935028e-01         6.286722e-01          6.738976e-01  6.929749e-01   \n",
       "max    1.374167e+01         1.491679e+01          7.097720e+00  1.502976e+01   \n",
       "\n",
       "                 pH     sulphates       alcohol  \n",
       "count  4.898000e+03  4.898000e+03  4.898000e+03  \n",
       "mean   1.316599e-14 -1.280696e-14 -2.846868e-14  \n",
       "std    1.000000e+00  1.000000e+00  1.000000e+00  \n",
       "min   -3.101091e+00 -2.364468e+00 -2.043089e+00  \n",
       "25%   -6.507699e-01 -6.996389e-01 -8.241915e-01  \n",
       "50%   -5.474574e-02 -1.739035e-01 -9.285319e-02  \n",
       "75%    6.075033e-01  5.270772e-01  7.197450e-01  \n",
       "max    4.183648e+00  5.171074e+00  2.995020e+00  "
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wine.drop('quality',axis=1).apply(lambda x: (x-x.mean())/x.std()).describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "      <td>4.898000e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>7.584333e-17</td>\n",
       "      <td>5.196832e-16</td>\n",
       "      <td>6.019639e-16</td>\n",
       "      <td>-1.024542e-16</td>\n",
       "      <td>6.482723e-17</td>\n",
       "      <td>-4.671641e-17</td>\n",
       "      <td>1.057636e-16</td>\n",
       "      <td>3.212112e-14</td>\n",
       "      <td>-1.203860e-15</td>\n",
       "      <td>-7.572999e-16</td>\n",
       "      <td>-2.178784e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.000102e+00</td>\n",
       "      <td>1.000102e+00</td>\n",
       "      <td>1.000102e+00</td>\n",
       "      <td>1.000102e+00</td>\n",
       "      <td>1.000102e+00</td>\n",
       "      <td>1.000102e+00</td>\n",
       "      <td>1.000102e+00</td>\n",
       "      <td>1.000102e+00</td>\n",
       "      <td>1.000102e+00</td>\n",
       "      <td>1.000102e+00</td>\n",
       "      <td>1.000102e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-3.620351e+00</td>\n",
       "      <td>-1.966985e+00</td>\n",
       "      <td>-2.761743e+00</td>\n",
       "      <td>-1.141944e+00</td>\n",
       "      <td>-1.683274e+00</td>\n",
       "      <td>-1.958677e+00</td>\n",
       "      <td>-3.044230e+00</td>\n",
       "      <td>-2.313038e+00</td>\n",
       "      <td>-3.101408e+00</td>\n",
       "      <td>-2.364709e+00</td>\n",
       "      <td>-2.043297e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-6.575011e-01</td>\n",
       "      <td>-6.771010e-01</td>\n",
       "      <td>-5.304757e-01</td>\n",
       "      <td>-9.250474e-01</td>\n",
       "      <td>-4.473347e-01</td>\n",
       "      <td>-7.237750e-01</td>\n",
       "      <td>-7.144739e-01</td>\n",
       "      <td>-7.707066e-01</td>\n",
       "      <td>-6.508363e-01</td>\n",
       "      <td>-6.997104e-01</td>\n",
       "      <td>-8.242757e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>-6.493106e-02</td>\n",
       "      <td>-1.809917e-01</td>\n",
       "      <td>-1.172780e-01</td>\n",
       "      <td>-2.349217e-01</td>\n",
       "      <td>-1.269060e-01</td>\n",
       "      <td>-7.692173e-02</td>\n",
       "      <td>-1.026189e-01</td>\n",
       "      <td>-9.609320e-02</td>\n",
       "      <td>-5.475133e-02</td>\n",
       "      <td>-1.739212e-01</td>\n",
       "      <td>-9.286267e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>5.276390e-01</td>\n",
       "      <td>4.143393e-01</td>\n",
       "      <td>4.611988e-01</td>\n",
       "      <td>6.918185e-01</td>\n",
       "      <td>1.935226e-01</td>\n",
       "      <td>6.287364e-01</td>\n",
       "      <td>6.739664e-01</td>\n",
       "      <td>6.930457e-01</td>\n",
       "      <td>6.075653e-01</td>\n",
       "      <td>5.271310e-01</td>\n",
       "      <td>7.198184e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>8.705106e+00</td>\n",
       "      <td>8.153643e+00</td>\n",
       "      <td>1.095642e+01</td>\n",
       "      <td>1.171411e+01</td>\n",
       "      <td>1.374308e+01</td>\n",
       "      <td>1.491831e+01</td>\n",
       "      <td>7.098444e+00</td>\n",
       "      <td>1.503130e+01</td>\n",
       "      <td>4.184075e+00</td>\n",
       "      <td>5.171602e+00</td>\n",
       "      <td>2.995326e+00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       fixed acidity  volatile acidity   citric acid  residual sugar  \\\n",
       "count   4.898000e+03      4.898000e+03  4.898000e+03    4.898000e+03   \n",
       "mean    7.584333e-17      5.196832e-16  6.019639e-16   -1.024542e-16   \n",
       "std     1.000102e+00      1.000102e+00  1.000102e+00    1.000102e+00   \n",
       "min    -3.620351e+00     -1.966985e+00 -2.761743e+00   -1.141944e+00   \n",
       "25%    -6.575011e-01     -6.771010e-01 -5.304757e-01   -9.250474e-01   \n",
       "50%    -6.493106e-02     -1.809917e-01 -1.172780e-01   -2.349217e-01   \n",
       "75%     5.276390e-01      4.143393e-01  4.611988e-01    6.918185e-01   \n",
       "max     8.705106e+00      8.153643e+00  1.095642e+01    1.171411e+01   \n",
       "\n",
       "          chlorides  free sulfur dioxide  total sulfur dioxide       density  \\\n",
       "count  4.898000e+03         4.898000e+03          4.898000e+03  4.898000e+03   \n",
       "mean   6.482723e-17        -4.671641e-17          1.057636e-16  3.212112e-14   \n",
       "std    1.000102e+00         1.000102e+00          1.000102e+00  1.000102e+00   \n",
       "min   -1.683274e+00        -1.958677e+00         -3.044230e+00 -2.313038e+00   \n",
       "25%   -4.473347e-01        -7.237750e-01         -7.144739e-01 -7.707066e-01   \n",
       "50%   -1.269060e-01        -7.692173e-02         -1.026189e-01 -9.609320e-02   \n",
       "75%    1.935226e-01         6.287364e-01          6.739664e-01  6.930457e-01   \n",
       "max    1.374308e+01         1.491831e+01          7.098444e+00  1.503130e+01   \n",
       "\n",
       "                 pH     sulphates       alcohol  \n",
       "count  4.898000e+03  4.898000e+03  4.898000e+03  \n",
       "mean  -1.203860e-15 -7.572999e-16 -2.178784e-15  \n",
       "std    1.000102e+00  1.000102e+00  1.000102e+00  \n",
       "min   -3.101408e+00 -2.364709e+00 -2.043297e+00  \n",
       "25%   -6.508363e-01 -6.997104e-01 -8.242757e-01  \n",
       "50%   -5.475133e-02 -1.739212e-01 -9.286267e-02  \n",
       "75%    6.075653e-01  5.271310e-01  7.198184e-01  \n",
       "max    4.184075e+00  5.171602e+00  2.995326e+00  "
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wineCopy = wine.copy().drop('quality',axis=1)\n",
    "scaler = StandardScaler().fit(wineCopy)\n",
    "wineCopy.iloc[:,:] = scaler.fit_transform(wineCopy)\n",
    "wineCopy.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}