Testing disabled garbage collection

[jbusecke]

No description provided.

[jbusecke]

Whoooooa. This worked for the first time! https://console.cloud.google.com/dataflow/jobs/us-central1/2024-03-28_14_50_18-1319302957846820623?authuser=1&project=leap-pangeo&supportedpurview=project

[jbusecke]

YOOOOOOOOO this runs!

[jbusecke]

Now running recipes from https://github.com/pangeo-forge/pangeo-forge-recipes/tree/jb/xarray-hack which does not disable the fsspec caching. Lets see if this performs even better.

[jbusecke]

As per discussion in pangeo community meeting, trying out if this is with copy_to_local=False: https://pangeo-forge.readthedocs.io/en/latest/api_reference.html#pangeo_forge_recipes.openers.open_with_xarray

[jbusecke]

Just tested pangeo-forge/pangeo-forge-runner#183 here and got the desired increase in HDD. Lets see what this does now...running here

[jbusecke]

Well cool, now fsspec is hanging on the copy to local step 🙈.
Lets try one last time with load=True.