Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 catalog: Create tables method to export excel with metadata #4003

Merged
merged 5 commits into from
Feb 20, 2025

Conversation

pabloarosado
Copy link
Contributor

@pabloarosado pabloarosado commented Feb 19, 2025

Inspired by this comment, I thought it would be useful to have a .to_excel() method on our Table that by default exports a table with a "data" and a "metadata" sheet.
The metadata sheet contains for now very limited info (colum name, title, short description, and producers with urls). I haven't put much thought on what exactly this metadata should look like, but I suppose there's no harm in adding it as-is for now, and we can improve it in the future (unless .to_excel() is being used elsewhere).

@owidbot
Copy link
Contributor

owidbot commented Feb 19, 2025

Quick links (staging server):

Site Dev Site Preview Admin Wizard Docs

Login: ssh owid@staging-site-feature-export-excel-metadat

chart-diff: ✅ No charts for review.
data-diff: ❌ Found differences
= Dataset garden/oecd/2025-02-19/official_development_assistance
  = Table official_development_assistance
    ~ Column capital_subscriptions_deposits (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column capital_subscriptions_deposits_recipient (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column development_food_aid_recipient (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column development_food_aid_recipient_per_capita (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column grants (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column grants_recipient (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column grants_recipient_per_capita (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column humanitarian_aid_recipient (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column humanitarian_aid_recipient_per_capita (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
-   - Column i_a_10_other_loans_repayments_net_disbursements
-   - Column i_a_11_private_sector_instruments_grant_equivalents
-   - Column i_a_12_other_oda_not_assigned_to_the_above_categories_historical_series_net_disbursements
-   - Column i_a_1_budget_support_grant_equivalents
-   - Column i_a_1_budget_support_net_disbursements
-   - Column i_a_2_bilateral_core_contributions_pooled_programmes_funds_grant_equivalents
-   - Column i_a_2_bilateral_core_contributions_pooled_programmes_funds_net_disbursements
-   - Column i_a_3_project_type_interventions_grant_equivalents
-   - Column i_a_3_project_type_interventions_net_disbursements
-   - Column i_a_4_experts_other_technical_assistance_grant_equivalents
-   - Column i_a_4_experts_other_technical_assistance_net_disbursements
-   - Column i_a_5_scholarships_student_costs_donor_countries_grant_equivalents
-   - Column i_a_5_scholarships_student_costs_donor_countries_net_disbursements
-   - Column i_a_6_debt_relief_grant_equivalents
-   - Column i_a_6_debt_relief_net_disbursements
-   - Column i_a_7_administrative_costs_not_included_elsewhere_grant_equivalents
-   - Column i_a_7_administrative_costs_not_included_elsewhere_net_disbursements
-   - Column i_a_8_1_development_awareness_grant_equivalents
-   - Column i_a_8_1_development_awareness_net_disbursements
-   - Column i_a_8_2_refugees_in_donor_countries_grant_equivalents
-   - Column i_a_8_2_refugees_in_donor_countries_net_disbursements
-   - Column i_a_8_other_in_donor_expenditures_grant_equivalents
-   - Column i_a_8_other_in_donor_expenditures_net_disbursements
-   - Column i_a_9_recoveries_bilateral_oda_grants_negative_commitments_grant_equivalents
-   - Column i_a_9_recoveries_bilateral_oda_grants_negative_commitments_net_disbursements
    ~ Column i_oda_net_disbursements_multilaterals_private_grants (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column loans (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column loans_recipient (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column loans_recipient_per_capita (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column oda_by_sector (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column oda_by_sector_per_capita (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column oda_recipient (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column oda_recipient_per_capita (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column oda_share_gni_recipient (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column recoveries (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column recoveries_recipient (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column technical_cooperation_recipient (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column technical_cooperation_recipient_per_capita (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
  = Table dac2a
    ~ Column capital_subscriptions_deposits (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column development_food_aid (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column grants (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column humanitarian_aid (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column loans (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column oda (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column oda_share_gni (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column recoveries (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^
    ~ Column technical_cooperation (changed metadata)
-       -     date_published: '2025-02-17'
        ?                         ^
+       +     date_published: '2024-02-17'
        ?                         ^


Legend: +New  ~Modified  -Removed  =Identical  Details
Hint: Run this locally with etl diff REMOTE data/ --include yourdataset --verbose --snippet

Automatically updated datasets matching weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk are not included

Edited: 2025-02-19 18:16:59 UTC
Execution time: 47.75 seconds

@pabloarosado pabloarosado marked this pull request as ready for review February 19, 2025 18:14
Copy link
Collaborator

@Marigold Marigold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, tried it on cherry blossom, and it looked like a good start.

@pabloarosado pabloarosado merged commit 1fd0c12 into master Feb 20, 2025
9 checks passed
@pabloarosado pabloarosado deleted the feature-export-excel-metadata branch February 20, 2025 08:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants