-
Notifications
You must be signed in to change notification settings - Fork 26
Notes on UAL code merger
This document is a description of the merger of the UAL code into merger. The work was done between Feb-April of 2017 by @fscottfoti. Is is divided into sections of the type of changes that were made.
Orca test is a very useful tool to ensure certain orca attributes are available, that they're the right types, and that the values match certain ranges. In short, it's a schema checker. Unfortunately it's very slow right now, I think this is because it does multi-level joins to test foreign keys.
Currently the orca_test code is commented out of bayarea_urbansim and an issue has been opened on orca_test. So far there's been little movement on that issue. If the speed improvements don't happen in orca_test, it would be good to implement a mode where the code can be turned on and off depending on the user's needs.
A second performance bug was found and fixed in UrbanSim, which was uncovered by the UAL changes. The system now runs in about 7 minutes longer because of the UAL changes, for a total of about 80 minutes for 5 year iterations.
Does assigning tenure using the higher of rent and price really work? I mapped prices and rents over the 30 year simulation and the results seem quite good. I do think the one major problem is that the tenure of a building goes to the price/rent that is the highest in an area. This is not incorrect, but it seems like the results should be probabilistic and now be all rent or all own, but be dependent on the relative ratio of rents / prices in an area.
Given that rent/price are used to drive the tenure of new units and buildings, we also needed a mechanism to drive the relative magnitude of rents and prices. We do that be looking at the utilization of rental and owner units using this code. If rental units have a lower vacancy rate, rental rates go up, and vice versa with ownership and prices. This change is affected through the cap rate.
It also seems that using this kind of logic could leave us with vacancies at the end of the simulation in one of the rent/own segments, while at the same time we have unplaced households. Since unplaced households are not allowed in the MTC planning context, we run special HLCMs at the end which do not respect tenure in order to place any unplaced households. Luckily, at this point we have tuned the rent/price balance such that we these HLCMs do not currently place any households.
ual.py line 291 and various places use 1 and 2 to indicate whether a household or unit are rental or owner. This has been changed to the strings "rent" and "own" so that the modeler doesn't have to remember which is which. Along those lines, the hownrent attribute on both households and units has been changed to the name "tenure."
In most places, the use of the ual_ prefix has been removed. For instance, ual_owner_hlcm is just owner_hlcm. The non-UAL versions of these models have been removed. ual.py is still named the same, and most of the code is kept together in that module as a way to justify the slightly different coding styles and to give UAL credit for their excellent work.
Along those lines, the separate ual settings file has been moved to attributes in the main settings.yaml file.
This is entirely unrelated to the UAL code, but careful thinking about the rent and price sides of the model lead me to realize we should rename the attribute non_residential_price on buildings to non_residential_rent.
In a few places, there was a proliferation of modeling steps. It's a matter of taste, but I feel like orca.step should only be called a couple dozen times in a modeling run. In this case there was a model step which updated the building price from simulated unit prices and rents. I turned this into an aggregation variable rather than a separate step. In all fairness, it was likely a step in order to keep separate modeling workflows between UAL and MTC, which we decided to merge together.
Initialization has been moved into the "preprocessing" step. This ended up being harder than it sounds because of dependencies on other tables that were also being preprocessed, but has been accomplished.
The old building-level hedonics/LCMs have been removed and the building-level workflow in baus.py has been removed as well. UAL models are used everywhere.
The UAL code did not create deed restricted units, so this attribute was added to the unit tables after the developer model is run and the attribute is used in the HLCMs. New HLCMs were added for low income people to be placed into deed restricted units (filters high income households so that can't place in deed restricted units). A lot of code was removed to compute vacant affordable and market rate unit using the old building attributes.
reconcile_placed_households is used to keep building_id and unit_id in sync. This allows being able to do joins from households to buildings to parcels and so forth without having to join to units first. This is nice because it increases performance significantly.
This was also used to that the building code would not be affected by the UAL changes, and so could be removed when removing the building workflow, but I decided to leave it.
I wondered if we should be adding parcel_id to this step as well so that the join could be done directly to parcels. So far I haven't made that change, but it's an option in the future.
There were three cap rates in play - one is in ual_settings, one in the sqft proforma, and one implicit in the building price calculation. Now there's only one cap rate which is read from settings and is also used to balance the rental and owner hedonics.
The core urbansim would crash when doing an lcm with no alternatives, which is a problem when running the unplaced hlcm on the renter side (no unplaced renters). This couldn't be fixed outside of UrbanSim, since filters are applied by UrbanSim and it's therefore not known how many choosers there are until the filters run. I fixed the bug here.