PlanCatalyst Data Dashboard

We're building an interactive dashboard for PlanCatalyst’s redesigned website that forecasts country-level development progress on:

• UN SDGs (United Nations Sustainable Development Goals) – Human Rights & Gender Equity
• ND-GAIN (Notre Dame Global Adaptation Index) – Climate Change Resilience
• World Bank Data – Financial Capacity

We are exploring ML regression techniques using scikit-learn to forecast country-level development and NumPy to construct composite indexes. AWS automates data fetching and data storage, while Microsoft Power BI delivers the interactive frontend.

📌 References/Resources

UN SDGs

• UN SDG API V1
• UN SDG API V5
• UN Statistics Division – Gateway to UN SDG data
  • Provides lots of background on the goals, indicators, methodology, statistics, etc.
• UN SDG Data Commons – Resource that shows SDG progress by goal, indicator, and country
  • Can be used to preview API request for a given query
  • Features interactive maps (good reference)
• UN SDG Indicators Home – Gateway to UN SDG data resources
  • Provides background on lots of SDG information as well
• UN SDG Data Portal – Database of all indicator data

ND-GAIN Index

• ND-GAIN CSV – Official University of Notre Dame source for downloading the latest ND-GAIN Country Index in CSV format
• ND-GAIN Technical Report
• ND-GAIN Indicators

World Bank Group

• World Bank Open Data – Repo of global development & economic indicators
  • Features an interactive map (good reference)
• World Bank DataBank – Browser tool; helps define API parameters to use before making API calls
• API V2 Documentation – API Guide
• World Bank API Documentation – API guide

☁️ AWS Architecture

The AWS architecture of this project is still being designed & developed.

🏙️ Code Structure

The structure of this project is still being designed.

PC-DATA-DASH/
├── .vscode/
│
├── data/                             # LOCAL ONLY - for development/testing
│   ├── external/
│   ├── interim/
│   │   ├── ndgain/
│   │   ├── unsdg/
│   │   └── worldbank/
│   ├── processed/
│   └── raw/
│
├── notebooks/                        # For analysis
│   ├── EDA_un_sdg.ipynb
│   └── EDA_world_bank.ipynb
│
├── src/                              # DEPLOYABLE CODE
│   ├── __init__.py
│   │
│   ├── config/
│   │   ├── __init__.py
│   │   ├── settings.yaml
│   │   └── config.py                 # Config loader class ?
│   │
│   ├── fetch/                        # Keep your existing structure
│   │   ├── __init__.py
│   │   ├── base_fetch.py             # DataClient Template (interface)
│   │   ├── client_factory.py         # DataClient Factory
│   │   ├── un_sdg_fetch.py
│   │   ├── nd_gain_fetch.py
│   │   ├── world_bank_fetch.py
│   │   ├── data_fetch.py             # Main fetching script
│   │   └── README.md
│   │
│   ├── transform/                    # Data Processing
│   │   ├── __init__.py
│   │   ├── processor.py              # Cleaning logic
│   │   ├── clean_un_sdg.py 
│   │   ├── clean_nd_gain.py
│   │   └── clean_world_bank.py
│   │
│   ├── pipeline/
│   │   ├── __init__.py
│   │   ├── orchestrator.py           # Main pipeline controller
│   │   ├── run_pipeline.py           # Orchestrator
│   │   └── utils.py
│   │
│   ├── aws/                          # AWS Plans
│   │   ├── __init__.py
│   │   ├── s3.py                     # S3 upload operations
│   │   └── logger.py                 # CloudWatch logging
│   │
│   └── models/                       # Keep for future ML
│       ├── __init__.py
│       ├── regression.py
│       └── forecasting.py
│
├── lambda/                           # NEW: Lambda deployment package
│   ├── handler.py                    # Lambda entry point
│   ├── requirements.txt              # Runtime dependencies
│   └── README.md                     # Deployment notes
│
├── infrastructure/                  # IaC templates
│   ├── cloudformation/  
│   │   ├── s3-buckets.yaml          # S3 bucket definitions
│   │   ├── lambda.yaml              # Lambda + IAM roles
│   │   ├── eventbridge.yaml         # Scheduling
│   │   └── glue-database.yaml       # Glue catalog (for Athena)
│   │
│   ├── terraform/                    # Future migration
│   │   └── (empty for now)
│   │
│   └── athena-schemas/               # SQL DDL for Athena tables
│       ├── sdg_indicators.sql        # Run once in Athena console
│       ├── nd_gain_scores.sql
│       └── world_bank_data.sql
│
├── deployment/                      # Deployment automation
│   ├── build-lambda.sh              # Package Lambda code
│   ├── deploy-lambda.sh             # Deploy to AWS
│   └── setup-athena.sh              # Run Athena DDL scripts
│
├── powerbi/                         # Power BI connection info
│   ├── athena-connection.md         # How to connect to Athena
│   └── example-queries.sql          # Sample queries for testing
│
├── tests/                            # Tests
│   ├── test_fetch.py
│   ├── test_transform.py
│   └── test_pipeline.py
│
├── .env.example                      # Environment variables template ?
├── .gitignore
├── AWS-arch.png
├── requirements.txt
├── README.md
└── LICENSE

🌐 Team

This dashboard is made by Tech for Social Impact (TSI).

• Project Managers: Thomas Llamzon, Anthony Lam
• Developers: Adeline Lue Sang, Caroline Shen, Christina Wong, Kayden Jaffer, Tyler Asai