@wagnerlmichael This one is yours now. Let's use it to pilot use of Spark models within dbt, since we may want to convert sales val, source-of-truth, etc to Spark. Let's also take this opportunity to clean up the ratio_stats table a little bit (get the dtypes corrected, drop extraneous columns, etc.).

ratio_stats is used in production for our public-facing ratio study dashboards, which are published for mailed stage each reassessed township when it mails.

This is one dashboard serving all townships, with an extract of the ratio_stats table that is refreshed with each 2024 township mailing. Because of that I'd very strongly prefer to not make any changes to the production table until after we have mailed the last tri town this year.

If changes must be made now because it's blocking other work, please sequence with me on schedule so that changes aren't pushed close to a town mail date.

If it helps, the current structure of the reporting depends on no changes (data type, etc.) to the following columns in the production table:

• geography_id
• property_group
• assessment_stage
• sale_year
• sale_n
• detect_chasing
• med_ratio, cod, prb, prd
• ratio_met, cod_met, prb_met, prd_met

This table is filtered to geography_type = "Town", so if other types are added, it should be robust to those changes.

Which extraneous columns are you thinking of getting rid of?

Got it. @wagnerlmichael don't mess with any of the column dtypes. We'll move the cleanup stuff to a separate issue.