5-GalaxyC.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8095f449-a104-435d-b1ba-807d21df3073",
   "metadata": {},
   "source": [
    "# Tutorial 5: Classification and clustering analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb5b0a79-ef75-4a5c-ac92-2ac6fbc97faf",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d58f0eaa-00f4-4e2f-84a1-1c272d86e202",
   "metadata": {},
   "source": [
    "### Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbf23a70-dacb-4d5b-b900-f9f97c5ad1c7",
   "metadata": {},
   "source": [
    "In this notebook, we will examine the galaxy data in two different ways:\n",
    "   1. *Clustering*: Clustering uses unsupervised learning (with unlabeled data) to group together similar images. Note however that the groups may not correspond to the actual classes that we're interested in.\n",
    "   2. *Classifcation*: classification uses supervised learning (with labeled data) to develop an algorithm for determining the class of new images.\n",
    "   \n",
    "\n",
    "We begin with the usual imports."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ea84abd5-e8e5-4e21-a4a0-f34ad3de4c21",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "# For evaluating classification accuracy\n",
    "from sklearn.metrics import precision_recall_fscore_support"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "935b7600-5d9e-4439-95c9-b9a4f0f82d99",
   "metadata": {},
   "source": [
    "Next let's load the data. Remember that this data has already been preprocessed using the shallow encoder (see Tutorial 2).  For clustering we don't need the labeled data, but we'll need it later for classification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "560601af-8372-48b3-9b42-8d2e3ef88db9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Preprocessed input data for train and test\n",
    "%store -r auto_df_train \n",
    "%store -r auto_df_test\n",
    "# Labels for train and test\n",
    "%store -r labels_train\n",
    "%store -r labels_test\n",
    "# 'classes' contains class names\n",
    "%store -r classes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e239782-9664-4a7d-a6c1-908deabdb279",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7abf5d7e-18f4-4043-ae1d-6ae891c2f0b4",
   "metadata": {},
   "source": [
    "### Clustering "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f6a3a0a-d55e-4012-884c-f37edeb2e678",
   "metadata": {},
   "source": [
    "We will use `kmeans` for clustering the galaxy images. `kmeans` is a popular unsupervised learning method. Other unsupervised learning methods can be found [here](https://scikit-learn.org/stable/modules/clustering.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a3a4ddf1-de57-4688-b398-cf514be85b5b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.cluster import KMeans"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "68c492cf-1908-4e36-9d7f-232b675e8ac4",
   "metadata": {},
   "outputs": [],
   "source": [
    "kmeans = KMeans(init=\"k-means++\", n_clusters=4)\n",
    "kmeans.fit(auto_df_train)\n",
    "# 'pred_cluster' is a 1-d array that gives the cluster assigned to each data point (0,1,2, or 3)\n",
    "pred_cluster = kmeans.predict(auto_df_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7545a5e-5902-4b82-aca2-008a41041ecb",
   "metadata": {},
   "source": [
    "Now let's see if the clustering agrees with the actual galaxy labels. The function `plot_confusion_matrix` (in the `source` directory) graphically represents the accuracy using a *confusion matrix*. The rows of the confusion matrix correspond to the actual classes, while the columns correspond to the different clusters with their assigned labels.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1eeb1bc3-cfed-4260-8129-d8038b11e754",
   "metadata": {},
   "outputs": [],
   "source": [
    "from source.clustering import plot_confusion_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7260b2b5",
   "metadata": {},
   "source": [
    "The question is, which cluster should correspond to which label?  Initially, we simply assign clusters 0,1,2,3 to labels 0,1,2,3. (This is just a random choice--there's no reason to suppose it's going to be the best assignment)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e1305ab-83cc-45a2-b81d-2d69dab93fb4",
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_confusion_matrix(pred_cluster, auto_df_test, labels_test, classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92ab8d3e",
   "metadata": {},
   "source": [
    "The confusion matrix shows that clustering does not do a very good job of distinguishing the actual classes. For example, 51\\% of unbarred spiral galaxies are located in the cluster that is assigned to 'smooth round'. Also, only 5\\% of the actual 'edge_on_disk' galaxies are correctly identified by the cluster assigned by the algorithm.\n",
    "\n",
    "To get the precision, recall, and f-score  for the above classification, we may use the function from `scikit-learn`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04a7bbc7",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(precision_recall_fscore_support(y_pred=pred_cluster, y_true=labels_test, average='weighted'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3396d2b7-d5ff-48ef-9ee7-cccc1bf450b6",
   "metadata": {},
   "source": [
    "The f-score is only 0.328, Let's try to improve this.  The function `labelMap` (in the `source` directory) finds the assignment of clusters to labels that maximizes accuracy. \n",
    "\n",
    "(For another example (using different data) where clusters are optimally assigned to actual labels, see the following [reference](https://smorbieu.gitlab.io/accuracy-from-classification-to-clustering-evaluation/).)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "491bd90c-07d5-474e-a876-d9ed4776d647",
   "metadata": {},
   "outputs": [],
   "source": [
    "from source.clustering import labelMap, convertLabels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c2138d1-abfb-4223-a1d2-7b0b3b61b852",
   "metadata": {},
   "outputs": [],
   "source": [
    "lmap = labelMap(np.array(labels_test), pred_cluster)\n",
    "print(lmap)\n",
    "# Reorder the cluster labels according to the results of lmap\n",
    "conv_preds_cluster = lmap[pred_cluster]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "569aad24",
   "metadata": {},
   "source": [
    "Let's replot the confusion matrix with the new cluster assignment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "482e7917-2c25-498b-b56d-1d84e42b7dc6",
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_confusion_matrix(conv_preds_cluster, auto_df_test, labels_test, classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d0e13a6-f7ed-4d6d-8ffb-f15ad635f4dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(precision_recall_fscore_support(y_pred=conv_preds_cluster, y_true=labels_test, average='weighted'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b188ca33",
   "metadata": {},
   "source": [
    "Even with an optimzed cluster assignment, the f-score is only 0.34. Simple clustering is not sufficient to distinguish between the classes we're interested in.  So let's try supervised learning instead."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f909f6b-0127-49e2-a336-bb2e786eeae2",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63b88b29-77de-46b9-915e-ed4da2d582b1",
   "metadata": {},
   "source": [
    "**Exercise :** from [this list](https://scikit-learn.org/stable/modules/clustering.html)\n",
    "choose another clustering method and report the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "48059dd1-44ba-4a30-bc81-7141fc34b209",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "2894dce7-029d-46f6-94a0-9f1ed7b61409",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc2b0085-99be-4187-a9f8-ead81dd0030e",
   "metadata": {},
   "source": [
    "### Supervised learning"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a490ced",
   "metadata": {},
   "source": [
    "The `scikit-learn`package implements many different  methods for classification using supervised learning. Here we'll try *stochastic gradient descent* (SDG).  You may try others, to see if you can get a better result. They all follow pretty much the same procedure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "46ed4f82-c729-4aa2-a10f-762dcc8576f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import SGDClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e46237e7-8012-4655-a0ee-bdb84481f545",
   "metadata": {},
   "outputs": [],
   "source": [
    "sgd = SGDClassifier()\n",
    "\n",
    "sgd.fit(auto_df_train, labels_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9610477a-75a6-4fd4-872a-16559b5c0023",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_predict = sgd.predict(auto_df_test)\n",
    "\n",
    "plot_confusion_matrix(y_predict, auto_df_test, labels_test, classes)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c8660b5-8d39-4829-b610-f66a0a98b509",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(precision_recall_fscore_support(y_pred=y_predict, y_true=labels_test, average='weighted'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "649f7b39-8b38-49e3-b381-d966c223c9e8",
   "metadata": {},
   "source": [
    "We do get slightly better results than the unsupervised learning. But it seems very difficult to distinguish between smooth round, smooth cigar and edge on disk."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25b90edd-f743-4d43-a3cf-d9cc401718b9",
   "metadata": {},
   "source": [
    "**Exercise :** Use Google and try to search other supervise learning tools and try them out, and show the results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a047dea-babc-413c-9c27-4e0fafc68d96",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a2f9b90-13bc-484e-93e6-5f17bbb180a5",
   "metadata": {},
   "source": [
    "**Notice :** For both the clustering and the classification approach, instead of the shallow autoencoder you can try other methods like manual feature engineering and/or feature reduction from PCA."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6e13291-b1a9-4481-b302-0ba7a646766c",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85cc449e-845b-4def-a4ec-dc212fcc41f7",
   "metadata": {},
   "source": [
    "#### Final task"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fdf435a-2c41-4c2c-8c80-3db44ceec6e5",
   "metadata": {},
   "source": [
    "Now the fun starts. How about if we get a totally new data set and try some of the tools that we have learned from the past few tutorials. You can go and explore data in any subject area, as long as it's image data. The data doesn't have to be labeled, as we already taught you how to do unsupervised learning (clustering). it also has to have fewer  than 10,000 images for the pipeline to run in a reasonable amount of time. Please  approach an instructor if you need some advice. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89731220-3f48-4c66-aebf-e396a5461557",
   "metadata": {},
   "source": [
    "here are some links that we can use to donwload new data sets:\n",
    "1. https://www.kaggle.com/datasets\n",
    "2. https://github.com/awesomedata/awesome-public-datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8cb3229f-afc2-48af-beeb-60537f85adaa",
   "metadata": {},
   "outputs": [],
   "source": [
    "#"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "gal_ker",
   "language": "python",
   "name": "gal_ker"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}