CSN_notebook.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "SeU25AnsCQYG"
   },
   "source": [
    "<font size=8>  Setting up the Collection Space Navigator\n",
    "\n",
    "In this How-To guide you will produce all necessary files to create a custom version of the Collection Space Navigator (CSN).\n",
    "\n",
    "Project link: https://collection-space-navigator.github.io/ \n",
    "\n",
    "> Note: We highly recommend to first get familiar with the example collection before trying your with your own data."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "BcWeuOxUCQYI"
   },
   "source": [
    "<font size=6> 1) Prepare Collection Data\n",
    "\n",
    "Loads the dataset and sets up the metadata.\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clone repository and import libraries\n",
    "Downloads the offical CSN repository from https://github.com/Collection-Space-Navigator/CSN and loads all necessary libraries.\n",
    ">Note: Optional libraries will be installed only if needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "cellView": "form",
    "id": "w25_J6m4CQYJ"
   },
   "outputs": [],
   "source": [
    "# !git clone https://github.com/Collection-Space-Navigator/CSN\n",
    "# %cd CSN\n",
    "\n",
    "import json, math, os, io\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import ipywidgets as widgets\n",
    "from ipywidgets import interactive,HBox,VBox,Label\n",
    "from IPython.display import display"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define INPUT\n",
    ">Note: Running this cell opens a dialog in which the input data can be defined. Use the provided example data (recommended first) or your own.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "r6jZRVHhCQYK",
    "tags": []
   },
   "outputs": [],
   "source": [
    "style = {'description_width': '250px'}\n",
    "layout = {'width': '600px', 'justify-content': 'lex-satrt'}\n",
    "useExample = widgets.Checkbox(value=True,description='use example data',indent=True)\n",
    "datasetTitle = widgets.Text(placeholder='title of the dataset', description='Title:', style=style, layout=layout, value = \"Testset\")\n",
    "description = widgets.Textarea(placeholder='Short description of the dataset and method(s)', description='Description (optional):', style=style, layout=layout, value=\"\")\n",
    "embeddingsLocation = widgets.Text(placeholder='path to embeddings file (.csv)', description='Embeddings Filepath (optional):', style=style, layout=layout, value = \"CSN/example_data/embeddings_testset.csv\")\n",
    "metadataLocation = widgets.Text(placeholder='path to metadata file (.csv)', description='Metadata Filepath:', style=style, layout=layout, value = \"CSN/example_data/metadata_testset.csv\")\n",
    "imageLocation = widgets.Text(placeholder='path to image collection folder', description='Image Folder:', style=style, layout=layout, value = \"CSN/example_data/testset_images/\")\n",
    "imageWebLocation = widgets.Text(placeholder='URL prefix to public image directory', description='Image URL prefix:',style=style, layout=layout, value =  \"https://github.com/Collection-Space-Navigator/CSN/raw/main/example_data/testset_images/\")\n",
    "\n",
    "if os.path.exists(\"/content/gdrive/MyDrive/\"):\n",
    "    imageLocation.value = \"/content/gdrive/MyDrive/CSN/example_data/testset_images/\"\n",
    "    embeddingsLocation.value = \"/content/gdrive/MyDrive/CSN/example_data/embeddings_testset.csv\"\n",
    "    metadataLocation.value = \"/content/gdrive/MyDrive/CSN/example_data/metadata_testset.csv\"  \n",
    "else:\n",
    "    imageLocation.value = \"example_data/testset_images/\"\n",
    "    embeddingsLocation.value = \"example_data/embeddings_testset.csv\"\n",
    "    metadataLocation.value = \"example_data/metadata_testset.csv\"  \n",
    "\n",
    "imageWebLocation.value =  \"https://github.com/Collection-Space-Navigator/CSN/raw/main/example_data/testset_images/\"\n",
    "\n",
    "\n",
    "subset = widgets.Checkbox(value=False,description='make subset',indent=True)\n",
    "subsetSize = widgets.BoundedIntText(value=2048,min=10,max=9999999, step=1,description='Subset size:')\n",
    "def makeSubset(SUBSET):\n",
    "    if SUBSET:\n",
    "        display(subsetSize)\n",
    "    else:\n",
    "        subsetSize.value == None\n",
    "i = interactive(makeSubset, SUBSET = subset)\n",
    "left = VBox([datasetTitle, description, embeddingsLocation, metadataLocation, imageLocation, imageWebLocation])\n",
    "right = VBox([useExample, i])\n",
    "display(HBox([left,right]))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load INPUT files\n",
    "Loads and checks all files.\n",
    "\n",
    ">Note: Example data will be downloaded only if needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "tHhLCEWOykFu"
   },
   "outputs": [],
   "source": [
    "imagNumb = len(os.listdir(imageLocation.value))\n",
    "print(f'found {imagNumb} files in {imageLocation.value}')\n",
    "mappings = []\n",
    "\n",
    "metadata = pd.read_csv(metadataLocation.value, skipinitialspace=True)\n",
    "if subset.value:\n",
    "    metadata = metadata[:subsetSize.value]\n",
    "metaNumb = len(metadata)\n",
    "print(f'found {metaNumb} entries in {metadataLocation.value}')\n",
    "\n",
    "if embeddingsLocation.value != \"\":\n",
    "  embeddings = pd.read_csv(embeddingsLocation.value, skipinitialspace=True)\n",
    "  embeddings = embeddings.loc[:, embeddings.columns!='id']\n",
    "  embeddings = embeddings.loc[:, embeddings.columns!='ID']\n",
    "  if subset.value:\n",
    "    embeddings = embeddings[:subsetSize.value]\n",
    "  vecNumb = len(embeddings)\n",
    "  print(f'found {vecNumb} entries in {embeddingsLocation.value}')\n",
    "\n",
    "  if metaNumb == vecNumb:\n",
    "    if vecNumb <= imagNumb:\n",
    "      print(\"Looks ok.\")\n",
    "      print()\n",
    "      print(f'Embedding file contains {vecNumb} vectors in {len(embeddings.columns)} dimensions.')\n",
    "      print(\"Metadata Head:\")\n",
    "      print(metadata.head())\n",
    "    else:\n",
    "      print()\n",
    "      print(\"ERROR: number of images is smaller than number of vectors\")\n",
    "\n",
    "if metaNumb <= imagNumb:\n",
    "  print(\"Looks ok.\")\n",
    "  print(\"Metadata Head:\")\n",
    "  print(metadata.head())\n",
    "else:\n",
    "  print()\n",
    "  print(\"ERROR: number of images and metadata elements don't match!\")\n",
    "foldername = datasetTitle.value.lower().replace(\" \",\"_\")\n",
    "print()\n",
    "print(f'Creating new dataset directory: build/datasets/{foldername}...')\n",
    "if not os.path.exists(f\"build/datasets/{foldername}\"):\n",
    "    os.makedirs(f\"build/datasets/{foldername}\")\n",
    "    print(\"... success\")\n",
    "else:\n",
    "    print(\"... folder already exists (might overwrite existing files)\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Assign metadata fields\n",
    "Choose which field names in the metadata file should be used.   \n",
    "\n",
    ">Note: Select multiple values using ctrl+click, command+mouseclick, or shift+arrow keys.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "AtOUyvwapOfx"
   },
   "outputs": [],
   "source": [
    "filenameColumn = widgets.Dropdown(description=\"Image filenames (JPG or PNG):\",options=[mf for mf in metadata.columns if pd.api.types.is_string_dtype(metadata[mf]) and metadata[mf].str.endswith((\".jpg\",\".JPEG\",\"JPG\",\".jpeg\",\".png\",\".PNG\")).all()], style=style, layout=layout)\n",
    "classColumns = widgets.SelectMultiple(options=[mf for mf in metadata.columns if mf != \"index\"],description='optional: Cluster data:', style=style, layout=layout)\n",
    "infoColumns = widgets.SelectMultiple(options=[mf for mf in metadata.columns if mf != \"index\"],description='Info fields (display in preview):', style=style, layout=layout)\n",
    "sliderColumns = widgets.SelectMultiple(options=[mf for mf in metadata.columns if pd.api.types.is_numeric_dtype(metadata[mf]) and mf != \"index\"],description='Slider data (floats or integers):', style=style, layout=layout)\n",
    "filterColumns = widgets.SelectMultiple(options=[mf for mf in metadata.columns if pd.api.types.is_string_dtype(metadata[mf]) and mf != 'URL'],description='optional: Filter & Search fields (string):', style=style, layout=layout)\n",
    "if useExample.value == True:\n",
    "  infoColumns.value = (\"Prompt\", \"Colors\", \"Contrast\", \"File Size\")\n",
    "  sliderColumns.value = (\"Colorfulness\", \"Colors\", \"Contrast\", \"File Size\")\n",
    "  filterColumns.value = (\"Prompt\",)\n",
    "  classColumns.value = (\"Class\",)\n",
    "left = VBox([filenameColumn, infoColumns, sliderColumns])\n",
    "right = VBox([filterColumns, classColumns])\n",
    "display(HBox([left,right]))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "IJ29FLXmpIKT"
   },
   "source": [
    "----------------"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "4OHuwVEJlZFj"
   },
   "source": [
    "<font size=6> 2) Prepare Image Data\n",
    "\n",
    "To handle large amounts of images efficiently, the CSN uses sprite sheets with multiple thumbnails behind the scenes. These sprite-sheets need to be generated."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate sprite sheets\n",
    ">Note: only needed for new datasets or to update existing sprites (skip this part if you already generated them)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "pBSPbVtXCQYU"
   },
   "outputs": [],
   "source": [
    "from CSN_utils import ImageSpriteGenerator\n",
    "\n",
    "# generate sprite sheets\n",
    "sprite_generator = ImageSpriteGenerator(foldername, spriteSize=2048, spriteRows=32, imageFolder=imageLocation.value, files=metadata[filenameColumn.value]).generate()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=6> 2b) Feature Extraction (optional)\n",
    " \n",
    "Run simple image feature extraction method using a pretrained Convolutionial Neural Network (CNN).\n",
    ">Note: This can take a while depending on your hardware setup and size of image dataset..."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try: \n",
    "    from tensorflow.keras import applications, models, Model\n",
    "    from tensorflow.keras.applications.resnet50 import preprocess_input\n",
    "    from tensorflow.keras.preprocessing import image\n",
    "except:\n",
    "    print(\"Installing tensorflow via Pip\")\n",
    "    %pip install flask tensorflow\n",
    "    from tensorflow.keras import applications, models, Model\n",
    "    from tensorflow.keras.applications.resnet50 import preprocess_input\n",
    "    from tensorflow.keras.preprocessing import image\n",
    "try:\n",
    "    from PIL import ImageFile\n",
    "except:\n",
    "    print(\"Installing Pillow via Pip\")\n",
    "    %pip install pillow\n",
    "\n",
    "from feature_extraction import extract_features\n",
    "\n",
    "ImageFile.LOAD_TRUNCATED_IMAGES = True"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Choose model\n",
    "Learn more about the pretrained models: https://keras.io/api/applications/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = 'ResNet50' #'ResNet50' # 'Xception', 'VGG16', 'VGG19', 'InceptionV3', 'MobileNet'"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Perform feature extraction \n",
    "Runs the extraction method with the chosen model and saves the embeddings as a CSV file\n",
    ">Note: This will overwrite the Embeddings Filepath value from earlier (see 1. 'Define Input')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "files = [f'{imageLocation.value}/{file}' for file in metadata[filenameColumn.value]]\n",
    "features = extract_features(files, model=model)\n",
    "df = pd.DataFrame(features[0])\n",
    "embeddings = df # overwrites the Input"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save embeddings in csv file\n",
    "df_2 = pd.DataFrame(features[0])\n",
    "df_2[\"id\"] = features[1]\n",
    "em_filename = f\"subset_embeddings_{model}.csv\"\n",
    "df_2.to_csv(em_filename, index=False)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "qUD79KzapGlx"
   },
   "source": [
    "----------------"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "qvsa4czdCQYV"
   },
   "source": [
    "<font size=6> 3) Generate Mappings\n",
    "\n",
    "Mappings are plots containing 2D coordinates (x,y) of the image objects. \n",
    "\n",
    "Here are several methods you can run. The Collection Space Navigator can handle many mappings but needs at least one to work."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "LsI4HATziR69"
   },
   "source": [
    "<font size=5> 3.1 From metadata (optional)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (optional) Create 2D plots\n",
    "Choose 2 metadadata fields (float or integer) and click \"make plot\". Repeat for every combination you want to add.\n",
    ">Note: Running this step opens a dialog in which you can chose X and Y dimensions from the available data and create additional 2D plots\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "9sg3a8k_CQYY"
   },
   "outputs": [],
   "source": [
    "from CSN_utils import SimplePlot\n",
    "\n",
    "def plot(v):\n",
    "    result = SimplePlot(foldername,A=AColumn.value, B=BColumn.value, metadata=metadata)\n",
    "    filename = (AColumn.value + \"_\" + BColumn.value).replace(\" \",\"\")\n",
    "    mappings.append({\"name\": filename, \"file\": f\"{filename}.json\"})\n",
    "    \n",
    "AColumn = widgets.Dropdown(description=\"x-axis:\",options=[mf for mf in metadata.columns if pd.api.types.is_numeric_dtype(metadata[mf]) and mf != \"index\"], style=style, layout=layout)\n",
    "BColumn = widgets.Dropdown(description=\"y-axis:\",options=[mf for mf in metadata.columns if pd.api.types.is_numeric_dtype(metadata[mf]) and mf != \"index\"], style=style, layout=layout)\n",
    "button2DPlot = widgets.Button(description='make plot',icon='check')\n",
    "button2DPlot.on_click(plot)\n",
    "left = VBox([AColumn,BColumn])\n",
    "right = VBox([button2DPlot])\n",
    "HBox([left,right])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "TdAkDgquhU73"
   },
   "source": [
    "<font size=5> 3.2 From embeddings (optional)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (optional) Run Principal Component Analysis (PCA)\n",
    ">See PCA documentation: https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.eA.html\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "IBwRz1RYnpUr"
   },
   "outputs": [],
   "source": [
    "components = 2\n",
    "add_slider = True\n",
    "\n",
    "from CSN_utils import PCAGenerator\n",
    "\n",
    "PCAEembedding = PCAGenerator(foldername, scale=True, data=embeddings.values, components=components).generate()\n",
    "mappings.append({\"name\": \"PCA\", \"file\": \"PCA.json\"})\n",
    "\n",
    "# add columns to metadata for each component\n",
    "for i in range(components):\n",
    "  metadata[f\"PC{i+1}\"] = PCAEembedding[:,i]\n",
    "  print(f\"... added PC{i+1} to metadata\")\n",
    "\n",
    "# add slider for each component\n",
    "if add_slider:\n",
    "  sliderCols = list(sliderColumns.value)\n",
    "  for i in range(components):\n",
    "    sliderCols.append(f\"PC{i+1}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (optional) Run UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction\n",
    ">See UMAP documentation: https://umap-learn.readthedocs.io/en/latest/\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "GrJqZPjYCQYW"
   },
   "outputs": [],
   "source": [
    "n_neighbors=15 \n",
    "min_dist=0.18 \n",
    "metric=\"correlation\"\n",
    "verbose=True\n",
    "\n",
    "\n",
    "from CSN_utils import UMAPGenerator\n",
    "\n",
    "fullEmbeddings = UMAPGenerator(foldername, data=embeddings.values, n_neighbors=n_neighbors, min_dist=min_dist, metric=metric, verbose=verbose).generate()\n",
    "mappings.append({\"name\": \"UMAP\", \"file\": \"UMAP.json\"})\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (optional) Run t-SNE: t-distributed Stochastic Neighbor Embedding\n",
    ">See t-SNE documentation: https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "0NlqxmzVCQYX"
   },
   "outputs": [],
   "source": [
    "n_components = 2\n",
    "verbose = 1\n",
    "random_state = 123\n",
    "\n",
    "from CSN_utils import TSNEGenerator\n",
    "\n",
    "tsneEembedding = TSNEGenerator(foldername, data=embeddings.values, n_components=n_components, verbose=verbose, random_state=random_state).generate()\n",
    "mappings.append({\"name\": \"t-SNE\", \"file\": \"TSNE.json\"})\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (optional) Create 2D plots\n",
    "Choose 2 metadadata fields (float or integer) and click \"make plot\". Repeat for every combination you want to add.\n",
    ">Note: Running this step opens a dialog in which you can chose X and Y dimensions from the available data and create additional 2D plots\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "f5_ucbG2i_te"
   },
   "outputs": [],
   "source": [
    "from CSN_utils import SimplePlot\n",
    "\n",
    "def plot(v):\n",
    "    result = SimplePlot(foldername,A=AColumn.value, B=BColumn.value, metadata=metadata)\n",
    "    filename = (AColumn.value + \"_\" + BColumn.value).replace(\" \",\"\")\n",
    "    mappings.append({\"name\": filename, \"file\": f\"{filename}.json\"})\n",
    "    \n",
    "AColumn = widgets.Dropdown(description=\"x-axis:\",options=[mf for mf in metadata.columns if pd.api.types.is_numeric_dtype(metadata[mf]) and mf != \"index\"], style=style, layout=layout)\n",
    "BColumn = widgets.Dropdown(description=\"y-axis:\",options=[mf for mf in metadata.columns if pd.api.types.is_numeric_dtype(metadata[mf]) and mf != \"index\"], style=style, layout=layout)\n",
    "button2DPlot = widgets.Button(description='make plot',icon='check')\n",
    "button2DPlot.on_click(plot)\n",
    "left = VBox([AColumn,BColumn])\n",
    "right = VBox([button2DPlot])\n",
    "HBox([left,right])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "MedpdOTQpEoA"
   },
   "source": [
    "----------------"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "e7r2I7QzCQYa"
   },
   "source": [
    "<font size=6> 4) Create Config Files\n",
    "\n",
    "All customization and component settings are defined in the config files."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set Sliders\n",
    "Set the appearance of the range slider elements and histograms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "YoSHLl1gCT8H"
   },
   "outputs": [],
   "source": [
    "try:\n",
    "  import distinctipy\n",
    "except:\n",
    "  print(\"Installing distinctipy via Pip\")\n",
    "  !pip install distinctipy --quiet\n",
    "  import distinctipy\n",
    "  \n",
    "# check if sliderCols exists\n",
    "try:\n",
    "  sliderCols\n",
    "except NameError:\n",
    "  sliderCols = list(sliderColumns.value)\n",
    "  \n",
    "if len(sliderCols) > 0:    \n",
    "  layoutCol = {'width': '110px'}\n",
    "  sliderColorDict = {}\n",
    "  left = [Label('display name')]\n",
    "  middle = [Label('description text')]\n",
    "  right = [Label('histogram color')]\n",
    "  colors = distinctipy.get_colors(len(sliderCols),pastel_factor=1)\n",
    "  for i, sliderName in enumerate(sliderCols):\n",
    "    sliderColorDict[sliderName] = widgets.ColorPicker(concise=False,value=distinctipy.get_hex(colors[i]),layout=layoutCol)\n",
    "    right.append(sliderColorDict[sliderName])\n",
    "  sliderInfoDict = {}\n",
    "  for sliderName in sliderCols:\n",
    "    sliderInfoDict[sliderName] = widgets.Text(placeholder=\"info text for slider\",layout=layout)\n",
    "    middle.append(sliderInfoDict[sliderName])\n",
    "  sliderNameDict = {}\n",
    "  for sliderName in sliderCols:\n",
    "    sliderNameDict[sliderName] = widgets.Text(placeholder=\"name of slider\",value=sliderName)\n",
    "    left.append(sliderNameDict[sliderName])\n",
    "  print(\"\\nSlider Settings:\\n\") \n",
    "  idx = VBox([Label('')]+[Label(f\"{n}:\") for n in sliderCols])\n",
    "  left_box = VBox([l for l in left])\n",
    "  middle_box = VBox([m for m in middle])\n",
    "  right_box = VBox([r for r in right])\n",
    "  display(HBox([idx,left_box,middle_box,right_box]))\n",
    "else:\n",
    "  print(\"No Cluster fields selected!\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (optional) Set Cluster colors\n",
    ">Note: only necessary if categorical data was assigned for clusters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "lyqTIu-wLjDD",
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "if len(classColumns.value) > 0:\n",
    "  classColorDict = {}\n",
    "  amount = len(classColumns.value)\n",
    "  styleCol = {'description_width': '25px'}\n",
    "  layoutCl = {'width': '135px'}\n",
    "  allClasses = {}\n",
    "  for className in classColumns.value:\n",
    "    clusters = metadata[className].unique()\n",
    "    allClasses[className] = len(clusters)\n",
    "  l = sorted(allClasses.items(), key=lambda item: item[1])[0]\n",
    "  length = max(allClasses.values())\n",
    "  allColors = {}\n",
    "  colors = distinctipy.get_colors(length)\n",
    "  col = 5\n",
    "  row = math.ceil(length/col)\n",
    "  i=0\n",
    "  rows = []\n",
    "  for r in range(0,col):\n",
    "    newRow = []\n",
    "    for c in range(0,row):\n",
    "      # classColorDict[className] = widgets.ColorPicker(concise=True, value=distinctipy.get_hex(colors[i]))\n",
    "      if i < len(colors):\n",
    "        allColors[i] = widgets.ColorPicker(concise=False, description=str(i), value=distinctipy.get_hex(colors[i]),layout=layoutCl,style=styleCol)\n",
    "        newRow.append(allColors[i])\n",
    "        i+=1\n",
    "    rows.append(VBox([nr for nr in newRow]))\n",
    "  display(HBox(rows))\n",
    "else:\n",
    "  allColors = False\n",
    "  print(\"No cluster was selected.\")\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create metadata.json\n",
    "Creates and saves the metadata.json file\n",
    ">Note: This step is necessary!\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "eDcpx2pwCQYT"
   },
   "outputs": [],
   "source": [
    "from CSN_utils import Utils\n",
    "\n",
    "try:\n",
    "  sliderCols\n",
    "except NameError:\n",
    "  sliderCols = list(sliderColumns.value)\n",
    "\n",
    "imageFolder = f'public/datasets/{foldername}/images/'\n",
    "if useExample.value == True:\n",
    "  metadata[\"URL\"] = metadata[filenameColumn.value]\n",
    "else:\n",
    "  metadata[\"URL\"] = f\"{imageFolder}/{metadata[filenameColumn.value]}\"\n",
    "metadataColumns = set(list(infoColumns.value) + sliderCols + list(filterColumns.value) + list(classColumns.value))\n",
    "metadataColumns.add(filenameColumn.value)\n",
    "metadata = metadata[metadataColumns]\n",
    "Utils.write_metadata(foldername, metadata, filenameColumn.value)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate Histograms and create config files\n",
    "The CSN features Range Sliders with interactive histograms. This step calculates the necessary bins and prepares the data to display the histograms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "nu88VZMUR8Km"
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    sliderCols\n",
    "except NameError:\n",
    "    sliderCols = list(sliderColumns.value)\n",
    "\n",
    "\n",
    "from CSN_utils import HistogramGenerator, Utils\n",
    "\n",
    "BarChartData = HistogramGenerator(foldername, data=metadata, selection=sliderCols, bucketCount = 50).generate()\n",
    "\n",
    "def save_datasetsJSON():\n",
    "  with open(f'build/datasets/datasets_config.json', \"w\") as fd:\n",
    "    json.dump(datasetsJSON , fd)\n",
    "  print(\"saved datasets_config.json\")\n",
    "\n",
    "def make_default(DEFAULT):\n",
    "  datasetsJSON[\"default\"] = DEFAULT\n",
    "  print(f\"changed default dataset to {datasetsJSON['data'][DEFAULT]['name']}\")\n",
    "  save_datasetsJSON()\n",
    "  \n",
    "\n",
    "sliderSetting = []\n",
    "\n",
    "for k in sliderCols:\n",
    "  dtype = 'float'\n",
    "  if pd.api.types.is_integer_dtype(metadata[k]):\n",
    "    dtype = 'int'\n",
    "  slider = {\"id\":k,\"title\":sliderNameDict[k].value,\"info\":sliderInfoDict[k].value,\"typeNumber\":dtype,\"color\":sliderColorDict[k].value, \"min\":metadata[k].min(),\"max\":metadata[k].max()}\n",
    "  sliderSetting.append(slider)\n",
    "searchFields = []\n",
    "for k in filterColumns.value:\n",
    "  filter = {\"columnField\":k,\"type\":\"selection\"}\n",
    "  searchFields.append(filter)\n",
    "try:\n",
    "    allColors\n",
    "except NameError:\n",
    "    clusters = {\"clusterList\":[],\"clusterColors\":[]}\n",
    "else:\n",
    "    clusters = {\"clusterList\":list(classColumns.value),\"clusterColors\":[allColors[g].value for g in allColors]}\n",
    "\n",
    "    \n",
    "configData = Utils.write_config(directory=foldername, title=datasetTitle.value, description=description.value, mappings=mappings, clusters=clusters, total=len(metadata), sliderSetting=sliderSetting, infoColumns=infoColumns.value, searchFields=searchFields, imageWebLocation=imageWebLocation.value, spriteRows=32, squareSize=2048, spriteSize=64, spriteDir=None)\n",
    "newDataset = {'name': datasetTitle.value, 'directory': foldername}\n",
    "\n",
    "datasetsJSON = {\"default\": 0, \"data\": [newDataset]}\n",
    "save_datasetsJSON()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "GcRP9nmafoc7"
   },
   "source": [
    "----------------"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "K5sixPIZr16X"
   },
   "source": [
    "<font size=6> 5) Test and use your custom Collection Space Navigator"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "imqkuAZRiRIB"
   },
   "source": [
    "## (optional) Run in localhost\n",
    "\n",
    ">Note: only works locally (not within Colab)\n",
    "\n",
    "To run your CSN version on localhost, unzip your downloaded file, open a terminal, navigate to your CSN directory and run `serve -s build`\n",
    "\n",
    "The CSN should be then accessible at http://localhost:3000 in your browser.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from flask import Flask, render_template,send_from_directory\n",
    "\n",
    "app = Flask(__name__, static_folder='build/static',template_folder='build/')\n",
    "\n",
    "@app.route('/')\n",
    "def home():\n",
    "    return render_template('index.html')\n",
    "\n",
    "@app.route('/<path:path>')\n",
    "def send_report(path):\n",
    "  # remove the replace in next to lines later later <-- important !!!!!!!!\n",
    "  print(\"files_report:\",path)\n",
    "  if path==\"manifest.json\":\n",
    "      path=\"manifest.json\"\n",
    "  return send_from_directory('build/', str(path))\n",
    "\n",
    "@app.route('/static/<path:path>')\n",
    "def send_report2(path):\n",
    "  # remove the replace in next to lines later later <-- important !!!!!!!!\n",
    "  print(\"files_report2:\",path)\n",
    "  return send_from_directory('build/static/', str(path))\n",
    "\n",
    "@app.route('/datasets/<path:path>')\n",
    "def send_report3(path):\n",
    "  # remove the replace in next to lines later later <-- important !!!!!!!!\n",
    "  print(\"files_report3:\",path)\n",
    "  return send_from_directory('build/datasets/', str(path))\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "  #app.run(debug=False, port=port)\n",
    "  app.run(debug=False, port=8000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "id": "XgEk2EG9jXyy"
   },
   "outputs": [],
   "source": [
    "#@title (optional) Run ngrok server\n",
    "ngrok_Authtoken = 'YOUR_NGROK_TOKEN' #@param {type:\"string\"}\n",
    "#@markdown >Note: **This will run until stopped!** Requieres an ngrok account. See: https://ngrok.com/\n",
    "\n",
    "try:\n",
    "  from pyngrok import ngrok\n",
    "  from flask_ngrok import run_with_ngrok\n",
    "  from flask import Flask, render_template,send_from_directory\n",
    "except:\n",
    "  print(\"Installing flask, flask_ngrok and pyngrok via Pip\")\n",
    "  %pip install flask flask_ngrok pyngrok\n",
    "  from flask import Flask, render_template,send_from_directory\n",
    "  from pyngrok import ngrok\n",
    "  from flask_ngrok import run_with_ngrok\n",
    "  \n",
    "app = Flask(__name__,static_folder='/content/CSN/build/',template_folder='/content/CSN/build/')\n",
    "\n",
    "ngrok.set_auth_token(ngrok_Authtoken)\n",
    "run_with_ngrok(app)\n",
    "@app.route('/<path:path>')\n",
    "def send_report(path):\n",
    "  # remove the replace in next to lines later later <-- important !!!!!!!!\n",
    "  print(\"files\",path)\n",
    "  return send_from_directory('/content/CSN/build/', str(path))\n",
    "\n",
    "@app.route('/CSN/static/<path:path>')\n",
    "def send_report2(path):\n",
    "  # remove the replace in next to lines later later <-- important !!!!!!!!\n",
    "  print(\"files\",path)\n",
    "  return send_from_directory('/content/CSN/build/static/', str(path))\n",
    "  \n",
    "@app.route('/CSN/datasets/<path:path>')\n",
    "def send_report3(path):\n",
    "  # remove the replace in next to lines later later <-- important !!!!!!!!\n",
    "  print(\"files\",path)\n",
    "  return send_from_directory('/content/CSN/build/datasets/', str(path))\n",
    "\n",
    "@app.route(\"/\")\n",
    "def home():\n",
    "    return render_template('index.html')\n",
    "    \n",
    "if __name__ == \"__main__\":\n",
    "  app.run()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "1zyht2IFkpWT"
   },
   "source": [
    "### (optional) Use as production web tool\n",
    "\n",
    "We recommend to use GitHub for hosting your custom CSN version and an external server for hosting your image collection. Note that GitHub limits any dataset to 1000 files.\n",
    "\n",
    "To deploy your version as a web tool in GitHub:\n",
    "\n",
    ">Note: Make sure your GitHub branch is called `gh-pages` and has the GitHub Pages option set. See more about GitHub Pages here: https://pages.github.com/\n",
    "\n",
    "1. clone the official CSN repository: https://github.com/Collection-Space-Navigator/CSN  \n",
    "\n",
    "2. install NVM: https://github.com/nvm-sh/nvm    \n",
    "\n",
    "`curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash\n",
    "export NVM_DIR=\"$([ -z \"${XDG_CONFIG_HOME-}\" ] && printf %s \"${HOME}/.nvm\" || printf %s \"${XDG_CONFIG_HOME}/nvm\")\"\n",
    "[ -s \"$NVM_DIR/nvm.sh\" ] && \\. \"$NVM_DIR/nvm.sh\" # This loads nvm`  \n",
    "\n",
    "3. install node 16.16.0: by running `nvm install v16.16.0`  \n",
    "\n",
    "4. replace the `CSN/build` folder with your own  \n",
    "\n",
    "5. in `package.json,` change `\"homepage\": \"\"` to your GitHub pages URL (e.g. `\"homepage\": \"https://collection-space-navigator.github.io/CSN\"`)  \n",
    "\n",
    "6. deploy the build folder to your GitHub pages by running `npm run deploy`  \n",
    "\n",
    "\n",
    "For more information and other deployment options, see https://create-react-app.dev/docs/deployment/\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  },
  "vscode": {
   "interpreter": {
    "hash": "894f073ee313a387d8ab0b54a40d2928fe73692ba23550c72584c9df3304ca06"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}