cap13/CH13EJ3-Capacidad-predictiva-del-modelo-generado.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"width: 100%; clear: both;\">\n",
    "<div style=\"float: left; width: 50%;\">\n",
    "<img src=\"http://www.uoc.edu/portal/_resources/common/imatges/marca_UOC/UOC_Masterbrand.jpg\" align=\"left\">\n",
    "</div>\n",
    "<div style=\"float: right; width: 50%;\">\n",
    "<p style=\"margin: 0; padding-top: 22px; text-align:right;\"><a href=\"http://www.editorialuoc.cat/mineria-de-datos\" target=\"_blank\">Minería de datos: Modelos y algoritmos</a></p>\n",
    "<p style=\"margin: 0; text-align:right;\">Estudios de Informática, Multimedia y Telecomunicación</p>\n",
    "<p style=\"margin: 0; text-align:right; padding-button: 100px;\">Universitat Oberta de Catalunya (<b>UOC</b>)</p>\n",
    "</div>\n",
    "</div>\n",
    "<div style=\"width:100%;\">&nbsp;</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbpresent": {
     "id": "ea6cb8e6-4d3d-45c8-abc9-44f65205068f"
    }
   },
   "source": [
    "# CH13EJ3 Capacidad predictiva del modelo generado"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "En este ejemplo se muestra como generar un árbol C5.0 con *K-fold cross validation*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Instalar los packages necesarios"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Loading required package: C50\n",
      "\n",
      "Loading required package: plyr\n",
      "\n",
      "Warning message:\n",
      "“package ‘plyr’ was built under R version 3.5.2”\n"
     ]
    }
   ],
   "source": [
    "if(!require(C50)){\n",
    "    install.packages('C50',repos='http://cran.es.r-project.org')\n",
    "    require(C50)\n",
    "}\n",
    "if(!require(plyr)){\n",
    "    install.packages('plyr',repos='http://cran.es.r-project.org')\n",
    "    require(plyr)\n",
    "}\n",
    "set.seed(5813)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbpresent": {
     "id": "fafb1f54-b4ab-4c2e-bc0d-c723f2c6814d"
    }
   },
   "source": [
    "### Carga de los datos. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "nbpresent": {
     "id": "352d9955-18b6-49b7-a343-102b723bf3cd"
    }
   },
   "outputs": [],
   "source": [
    "# Carga del dataset\n",
    "Alumnos_usos_sociales <-read.csv(\"../data/B2.332_Students.csv\",comment.char = \"#\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preparamos los datos para crear el modelo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "X <- Alumnos_usos_sociales[,c(\"sex\",\"age\",\"studytime\",\"failures\",\"schoolsup\",\"famsup\",\"absences\",\"Supera\")]\n",
    "form <- \"Supera ~ sex + age + studytime + failures + schoolsup\"\n",
    "folds <- split(X, cut(sample(1:nrow(X)),10))\n",
    "errs.c50 <- rep(NA, length(folds))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 2. Creamos *n* modelos de árbol con los diferentes *k-folds*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "c50 code called exit with value 1\n",
      "c50 code called exit with value 1\n"
     ]
    },
    {
     "ename": "ERROR",
     "evalue": "Error in predict.C5.0(tmp.model, newdata = test): either a tree or rules must be provided\n",
     "output_type": "error",
     "traceback": [
      "Error in predict.C5.0(tmp.model, newdata = test): either a tree or rules must be provided\nTraceback:\n",
      "1. predict(tmp.model, newdata = test)",
      "2. predict.C5.0(tmp.model, newdata = test)"
     ]
    }
   ],
   "source": [
    "for (i in 1:length(folds)) {\n",
    "  test <- ldply(folds[i], data.frame)\n",
    "  train <- ldply(folds[-i], data.frame)\n",
    "  tmp.model <- C5.0(as.formula(form), train)\n",
    "  tmp.model <- C5.0(as.formula(form), train)\n",
    "  tmp.predict <- predict(tmp.model, newdata=test)\n",
    "  conf.mat <- table(test$Supera, tmp.predict)\n",
    "  errs.c50[i] <- 1 - sum(diag(conf.mat))/sum(conf.mat)\n",
    "  }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculamos la media de error de clasificación para todas las iteraciones."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1] \"Media de error con k-fold cross validation: NA percent\"\n"
     ]
    }
   ],
   "source": [
    "print(sprintf(\"Media de error con k-fold cross validation: %.3f percent\", 100*mean(errs.c50)))   "
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}