An exploratory data analysis of Philippine poverty data. Data includes 1991-2015 data, appended FIES 2018 & 2021 data, and 2024 & 2027 poverty estimates personally calculated using ARIMA.
| Filename | Description |
|---|---|
| .gitignore | Utility file to avoid pushing temporary local files |
| 2018_FIES_processed.csv | 2018 FIES data manually formatted in Google Sheets to match the main dataset |
| 2021_FIES_processed.csv | 2021 FIES data manually formatted in Google Sheets to match the main dataset |
| README.md | This file you're reading! :) |
| Slide Deck.pdf | Presentation slides of key insights found from EDA |
| ph-poverty-statistics.ipynb | Jupyter notebook containing all code for data preparation and EDA |
| pov_incidence_change.csv | Data for poverty incidence change between 1991 & 2021, generated from ph-poverty-statistics.ipynb |
| povstat_analysis.twb | All Tableau data visualizations and dashboards |
| povstat_processed.csv | 1991-2015 data provided by Thinking Machines |
| povstat_until_2021.csv | 1991-2021 data from povstat_processed.csv + 2018_FIES_processed.csv + 2021_FIES_processed.csv, generated from ph-poverty-statistics.ipynb |
| povstat_until_2027.csv | 1991-2027 data from povstat_until_2021.csv + 2024 & 2027 forecasts, generated from ph-poverty-statistics.ipynb |
All data visualizations created of the dataset can be found here. Please click the links for the interactive versions, as the ones shown below are screenshots only.
- Philippine Poverty Data Explorer - For exploring the dataset
- Philippine Poverty Statistics - Key insights from EDA
- 1991-2015 Data - Provided by Thinking Machines
- FIES 2018 & 2021 Data - "2021 Full Year Official Poverty Statistics Tables" from https://psa.gov.ph/poverty-press-releases/nid/167972 Unfortunately, the 2021 data here is from the preliminary 2021 FIES results, as I could not find a full and official release of the finalized 2021 data with these variables. I then created the .csv files manually to match the formatting of this dataset. For easy viewing, I copied the original attachment data here. I then cleaned and reformatted the 2018 & 2021 data here. The final processed .csv files are in this GitHub repo (2018, 2021).
- 2024 & 2027 Data - Personally forecasted using Autoregressive Integrated Moving Average (ARIMA). Process and full details seen in the .ipynb file. Caveats: Forecasts are likely not very accurate due to very few data points for the model to train on and data may be slightly skewed due to the 2021 pandemic. Overall, however, the forecast can at least give us at least some general idea of developments in poverty for the coming years.