Skip to content

Jupyter Notebooks Tips & Tricks

Jump to bottom
Bill Hereth edited this page Mar 6, 2023 · 73 revisions

Table of Contents

Jupyter Widgets

Widgets for use in jupyter notebooks: https://ipywidgets.readthedocs.io/

from ipywidgets import *

Pandas

import pandas as pd

Data Frames

Show all columns in a table:

pd.options.display.max_columns = None

Aggregating based on select columns:

import numpy as np

#group by Institution, Site, and ZIP5 and aggregate BaseDistribution to number of records (np.size) and sum (np.sum)
df_BD_rows_IS_ZC.groupby(['Institution','Site','ZIP5']).agg({'BaseDistribution': [np.size, np.sum]})

#use as_index=False to remove indexing by group by fields and have them as columns in the data frame
df_BD_rows_IS_ZC.groupby(['Institution','Site','ZIP5'], as_index=False).agg({'BaseDistribution': [np.size, np.sum]})

Spatially Enabled Data Frames

Spatially enabled data frames allow for manipulation of data within shapefiles and feature classes as if they were pandas data frames while still retaining geographic information.

ESRI guides:

Show simple basemap centered on Salt Lake:

#import arcgis libraries
from arcgis.gis import *
gis = GIS()

#create map1 that centers on Salt Lake (can replace 'Salt Lake' with any place name or leave blank for entire world)
map1 = gis.map('Salt Lake')

#show map1
map1

Connecting to a local shapefile and visualizing:

import os

#create spatially-enabled data frame from shapefile
working_directory = os.getcwd()
shp = os.path.join(working_directory, r"ZIPCode\ZIPCode.shp")
sedf = pd.DataFrame.spatial.from_featureclass(shp)

#plot sdf using class breaks and blue colors for 'columnname' field
#layer will be added to 'map1' above
sedf.spatial.plot(map_widget = map1,
                  renderer_type='c', # specify the classes renderer using its notation 'c'
                  method='esriClassifyNaturalBreaks',  # classification algorithm
                  class_count=5,  # choose the number of classes            
                  col='columnname',  # numeric column to classify
                  cmap='Blues',  # color map to pick colors from for each class
                  alpha=0.7  # specify opacity
                  )

Display Custom Classes using Arcade Expression:

arcade_expression = ("var public = $feature.Public;"
                     "if (public<250) {"
                     "  return 'class1';"
                     "} else if (public<1000) {"
                     "  return 'class2';"
                     "} else if (public<2000) {"
                     "  return 'class3';"
                     "} else if (public<3000) {"
                     "  return 'class4';"
                     "} else {"
                     "  return 'class5';"
                     "}"
                    )

uv = [{"value":"class1", "label":"Less than 250","symbol":{"type":"esriSFS","color":[130,165,217,168],
                                                           "outline":{"type":"esriSLS","color":[255,255,255,51],
                                                                      "width":1.5,"style":"esriSLSSolid"},
                                                           "style":"esriSFSSolid"}},
      {"value":"class2", "label":"250 to 1,000","symbol":{"type":"esriSFS","color":[140,125,164,168],
                                                          "outline":{"type":"esriSLS","color":[255,255,255,51],
                                                                     "width":1.5,"style":"esriSLSSolid"},
                                                          "style":"esriSFSSolid"}},
      {"value":"class3", "label":"1,000 to 2,000","symbol":{"type":"esriSFS","color":[149,85,111,168],
                                                                   "outline":{"type":"esriSLS","color":[255,255,255,51],
                                                                              "width":1.5,"style":"esriSLSSolid"},
                                                                   "style":"esriSFSSolid"}},
      {"value":"class4", "label":"2,000 to 3,000","symbol":{"type":"esriSFS","color":[159,44,57,168],
                                                            "outline":{"type":"esriSLS","color":[255,255,255,51],
                                                                       "width":1.5,"style":"esriSLSSolid"},
                                                            "style":"esriSFSSolid"}},
      {"value":"class5", "label":"More than 3,000","symbol":{"type":"esriSFS","color":[168,4,4,168],
                                                             "outline":{"type":"esriSLS","color":[255,255,255,51],
                                                                        "width":1.5,"style":"esriSLSSolid"},
                                                             "style":"esriSFSSolid"}}]
map2 = gis.map('Salt Lake')

sedf_tract_college.spatial.plot(map_widget = map2,
                                renderer_type='u-a', #'u-a' stands for uniqe value with arcade expression
                                unique_values=uv,
                                arcade_expression=arcade_expression,
                                default_symbol="" #don't include an 'other' category
                                )

map2

Adjust map height:

#adjust the map widget height
map1.layout.height='600px'

Add legend to map:

#turn on legend
map1.legend=True

Add title to map:

#add a markdown map title before map
from IPython.display import display, Markdown
display(Markdown('# Map Title'))

Change projections of SDF:
SEDFs must be in same spatial reference in order to perform spatial joins.

#project data frame to WGS84 (4326)
sedf.spatial.project(4326)
#project data frame to NAD83 / UTM zone 12N (26912)
sedf.spatial.project(26912)

Change data frame table into spatially enabled data frame using x,y columns, default projection code is 4326 (WGS84)

sedf = pd.DataFrame.spatial.from_xy(sdf,x_column,y_column,projection_code)

Seaborn

Seaborn library is supposed to be easier to use than base matplotlib and makes nicer looking (non-spatial) plots
https://seaborn.pydata.org/examples/index.html

Clone this wiki locally