Exercise-5-problem-2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "89b1e6f867ddc7945dd6d006c8f4a10a",
     "grade": false,
     "grade_id": "cell-0e5a9bb9ec9e7ac9",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "## Problem 2 - Data manipulation and selection (*4 points*)\n",
    "\n",
    "In this problem you will clean the data from our data file by removing no-data values, convert temperature values in Fahrenheit to Celsius, and split the data into separate datasets using the weather station identification code. We will start this problem by cleaning and converting our temperature data. Please perform the tasks below by writing your code into the codeblocks in each section.\n",
    "\n",
    "**Notice**: Closely follow the instructions! For example, you should be sure to use **exactly** the same variable names mentioned in the instructions because your answers will be automatically graded, and the tests that grade your answers rely on following the same formatting or variable naming as in the instructions.\n",
    "\n",
    "**Your score on this problem will be based on following criteria:**\n",
    "\n",
    "- Creating a new dataframe called `selected` that contains select columns from the data file\n",
    "- Cleaning the new dataframe by removing no-data values\n",
    "- Creating a new column for temperatures converted from Fahrenheit to Celsius\n",
    "- Dividing the data into separate dataframes for the Helsinki Kumpula and Rovaniemi stations\n",
    "- Saving the new dataframes to CSV files\n",
    "- Including comments that explain what most lines in the code do\n",
    "- Answering a couple questions at the end of the problem\n",
    "- Uploading your notebook and data files to your GitHub repository for this week's exercise  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "365bc253c9ab0da39efc238710f09479",
     "grade": false,
     "grade_id": "cell-a49b67e59c4796cb",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### Part 1 (*0 points*)\n",
    "\n",
    "The first step for this problem is to read the data file `6153237444115dat.csv` again into a variable `data` using Pandas. Remember to specify the no-data values (you can copy your code from Problem 1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "077376761a2fce1bff0be961d70a03a6",
     "grade": false,
     "grade_id": "problem_2_ag_p1_read_csv",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "data = None\n",
    "\n",
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "a52fc694c1559248e144f53dd7b0b5b9",
     "grade": false,
     "grade_id": "cell-1d46fa7bb8225c54",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Check that the first rows of the DataFrame look ok: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "2ef690a87ff17a6d8eecf293b4a26a4b",
     "grade": false,
     "grade_id": "cell-b95a02f5f889a56a",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Check the number of rows in the DataFrame:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "e1ffc4bdcc888168bb04aac22f96d2ff",
     "grade": false,
     "grade_id": "cell-9880f219a5fc3f36",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### Part 2 (*1 point*)\n",
    "\n",
    "Next, your task is to subset the data and remove rows with missing temperature values.\n",
    "\n",
    " - Select the columns `USAF`, `YR--MODAHRMN`, `TEMP`, `MAX`, and `MIN` from the `data` dataframe and assign them to the variable `selected`\n",
    " - Remove all rows from `selected` that have NoData in the column `TEMP` using the `dropna()` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "df0349c15e29b97f6dc16a454d7baab8",
     "grade": false,
     "grade_id": "problem_2_ag_p2_dataframe_subset",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "selected = None\n",
    "\n",
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "6bb3c4b113bf2726bea454d00823914c",
     "grade": false,
     "grade_id": "cell-ac41cccbd18488db",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Check that you selected the correct column names:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "1c2a497ced94c32984fcda7f4f959b97",
     "grade": true,
     "grade_id": "problem_2_ag_p2_dataframe_subset_test1",
     "locked": true,
     "points": 0.5,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "selected.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "187bc4df32010378e35cf51a6c3017d8",
     "grade": false,
     "grade_id": "cell-7d0260b3b7512daf",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Check how many rows you have after removing the no-data values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "2ee08ead760618beddcab94c96a56188",
     "grade": true,
     "grade_id": "problem_2_ag_p2_dataframe_subset_test2",
     "locked": true,
     "points": 0.5,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "len(selected)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "54a9086685cf2aa0b1cc86f71b72bb4f",
     "grade": false,
     "grade_id": "cell-afd940c0f8c0e2c7",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### Part 3 (*1 point*)\n",
    "\n",
    "Next, you can convert the temperature values in Fahrenheit to Celsius.\n",
    "\n",
    "- Create a new column in `selected` called `Celsius`\n",
    "- Convert the Fahrenheit temperatures from `TEMP` using the conversion formula (below) and store the results in the new `Celsius`column.\n",
    "\n",
    "$$\n",
    "\\Large\n",
    "\\begin{equation}\n",
    "  T_{\\mathrm{Celsius}} = (T_{\\mathrm{Fahrenheit}} - 32)~/~1.8\n",
    "\\end{equation}\n",
    "$$\n",
    "\n",
    "- Round the values in the `Celsius` column to have 0 decimals (**do not** create a new column, update the current one)\n",
    "- Convert the `Celsius` values into integers (**do not** create a new column, update the current one)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "3b95208426bde59f422229e3e213ae2c",
     "grade": false,
     "grade_id": "problem_2_ag_p3_temperature_calculation",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "7db43e546f42eeca940a62e39f7e07fa",
     "grade": true,
     "grade_id": "problem_2_ag_p3_temperature_calculation_test1",
     "locked": true,
     "points": 0.5,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Check your dataframe\n",
    "selected.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "b6c99a78d2dd5861897c801af1835c18",
     "grade": true,
     "grade_id": "problem_2_ag_p3_temperature_calculation_test2",
     "locked": true,
     "points": 0.5,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Check data types\n",
    "print(selected.dtypes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "df93fa4064287c4b5253c6774f7417ed",
     "grade": false,
     "grade_id": "cell-068cfca2d756fb80",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### Part 4 (*1 point*)\n",
    "\n",
    "Your next task is to divide `selected` into two separate dataframes. Please use the given variable names and write your answer to the codeblock below.\n",
    "\n",
    "- Select all rows from the `selected` DataFrame with the `USAF` code `29980` into a variable called `kumpula`\n",
    "- Select all rows from the `selected` DataFrame with the `USAF` code `28450` into a variable called `rovaniemi`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "f08df66c0eeeff8a6ab17771b32cb244",
     "grade": false,
     "grade_id": "problem_2_ag_p4_subsets",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "kumpula = None\n",
    "rovaniemi = None\n",
    "\n",
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "705067db27a87295f0e33782aad90eda",
     "grade": true,
     "grade_id": "problem_2_ag_p4_subsets_test1",
     "locked": true,
     "points": 0.5,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Check the dataframe\n",
    "print(\"Kumpula: \\n\", kumpula.head(), \"\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "b1137772390645a1557f1e584437771b",
     "grade": true,
     "grade_id": "problem_2_ag_p4_subsets_test2",
     "locked": true,
     "points": 0.5,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Check the dataframe\n",
    "print(\"Rovaniemi: \\n\", rovaniemi.head(), \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "3a1e0af73188db71902a07829c7ed911",
     "grade": false,
     "grade_id": "cell-e7473ef24ff78a17",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### Part 5 (*1 point*)\n",
    "\n",
    "Now you can save your selections to csv files.\n",
    "\n",
    "- Save the `kumpula` DataFrame into the file `Kumpula_temps_May_Aug_2017.csv` (CSV format) \n",
    "    - Separate the columns with commas (`,`)\n",
    "    - Use only 2 decimals for the floating point numbers\n",
    "- Save the `rovaniemi` DataFrame into the file `Rovaniemi_temps_May_Aug_2017.csv` (CSV format) \n",
    "    - Separate the columns with commas (`,`)\n",
    "    - Use only 2 decimals for the floating point numbers\n",
    "- Remember to commit both data files to your Exercise 5 repository"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "6ccdbd0010d5b32f3b5b8f9095f33099",
     "grade": true,
     "grade_id": "problem_2_mg_p5_to_csv",
     "locked": false,
     "points": 0,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "ebf2afdcc573b039bbc9df5981779962",
     "grade": true,
     "grade_id": "problem_2_ag_p5_to_csv_test",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "#Read-only cell for hidden tests :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "74bad1c95e4f7fc6e7587ab889da4287",
     "grade": false,
     "grade_id": "cell-3d8b5609d0cea2a9",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "### Problem 2 summary\n",
    "\n",
    "- Was anything unclear to you in Problem 2?\n",
    "- Did you encounter any problems with decimal formatting?\n",
    "\n",
    "Also, make sure you:\n",
    "\n",
    "- Check that your code includes informative comments explaining what your code does\n",
    "- Commit and push your changes to your GitHub repository for Exercise 5 (including your 2 new data files)\n",
    "\n",
    "Please write your answers and comments below (**activate the cell by double clicking**):"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "dcd4c45cfbeb46cb7947778d1cd06f44",
     "grade": true,
     "grade_id": "problem_2_mg_p6_problem_2_questions",
     "locked": false,
     "points": 0,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "source": [
    "YOUR ANSWER HERE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "96a69edaaaab214acf38f25aaec0336b",
     "grade": false,
     "grade_id": "cell-2aac3add6dc14b23",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "### On to Problem 3\n",
    "\n",
    "Now you can continue to [Problem 3: Data analysis](Exercise-5-problem-3.ipynb)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}