update to xarray 2025.03.0

[rwegener2]

Would Cryocloud be interested in bumping xarray to the latest version? Official datatree support came out in 2024.10.0, as well as number of bugfixes and improvements since then. We are currently using 2024.7.0.

What was done & how

• xarray version was increased to 2025.03.0 in environment.yml

[github-actions]

👈 Test this PR on Binder

[rwegener2]

/condalock

[rwegener2]

Update: I've been poking around and I have to say I'm a bit confused by this error. The error appears to be that libmamba couldn't solve the environment (critical libmamba Could not solve for environment specs).

When I build the environment with the updated environment.yml manually from the hub, though (using conda env create -f environment.yml) I don't get any conflicts.

Failure step from the logs:

Step 40/55 : RUN TIMEFORMAT='time: %3R' bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_PREFIX} --file "environment.yml" && time ${MAMBA_EXE} clean --all -f -y && ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} '

[weiji14]

Hmm, the error might be something to do with mamba=2? I tried using mamba=1 and it managed to solve the environment.yml file, but when I upgraded to mamba=2.0.6, I got this error locally:

error    libmamba Could not solve for environment specs
    The following package could not be installed
    └─ py-xgboost >=2.1.1 ^cpu.*,2.1.*$ does not exist (perhaps a typo or a missing channel).
critical libmamba Could not solve for environment specs

Looks like a bug with mamba=2 not being able to parse this line

hub-image/environment.yml

Line 86 in a42bab4

- py-xgboost~=2.1.1=cpu*

The cpu* build string glob pattern might be the problematic part. I found this semi-related issue - mamba-org/mamba#3783 that hints at mamba=2 only supporting trailing globs in the version string (i.e. the 2.1.1 part), maybe it doesn't support the build string part (e.g. cpu_pyh1ce2f49_3). See also mamba-org/mamba#3601

[jjerphan]

The problem was related to ~= (see mamba-org/mamba#3647) which just has been fixed, the next version of mamba should resolve the problem.

In the meantime, using py-xgboost >=2.1.1,2.1.* *cpu* works.

[weiji14]

/condalock

[weiji14]

The problem was related to ~= (see mamba-org/mamba#3647) which just has been fixed, the next version of mamba should resolve the problem.

Thanks! Using mamba=2.0.7 (released yesterday) with the relevant bugfix at mamba-org/mamba#3818 works now. @rwegener2, this looks good to merge 🚀

[rwegener2]

Woo, thanks so much for digging into this and getting it to work Weiji!

It looks like there is 1 failing check -- should I still merge?

[weiji14]

Let's try re-running the job. Btw, you tagged someone else 😅

[rwegener2]

Btw, you tagged someone else

Oh yikes, sorry about that. 😅 indeed

[weiji14]

Actually, might as well update to the latest version released a couple of days ago - https://github.com/pydata/xarray/releases/tag/v2025.03.0

Suggested change

	- xarray>=2025.01.0
	- xarray>=2025.03.0

[weiji14]

/condalock

[weiji14]

Hmm, CI is still showing this error at https://github.com/CryoInTheCloud/hub-image/actions/runs/14002309174/job/39210936883#step:5:4820:

  #21 [16/22] RUN TIMEFORMAT='time: %3R' bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_PREFIX} --file "environment.yml" && time ${MAMBA_EXE} clean --all -f -y && ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} '
  #21 8.256 DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
  #21 8.856 
  #21 522.3 critical libmamba Could not solve for environment specs
  #21 522.3 time: 522.173
  #21 ERROR: process "/bin/sh -c TIMEFORMAT='time: %3R' bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_PREFIX} --file \"environment.yml\" && time ${MAMBA_EXE} clean --all -f -y && ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} '" did not complete successfully: exit code: 1
  ------
   > importing cache manifest from quay.io/cryointhecloud/cryo-hub-image:
  ------
  ------
   > [16/22] RUN TIMEFORMAT='time: %3R' bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_PREFIX} --file "environment.yml" && time ${MAMBA_EXE} clean --all -f -y && ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} ':
  8.256 DEPRECATION: --no-python-version-warning is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to remove the flag as it's a no-op. Discussion can be found at https://github.com/pypa/pip/issues/13154
  8.856 
  522.3 critical libmamba Could not solve for environment specs
  522.3 time: 522.173
  ------
  Dockerfile:124
  --------------------
   123 |     USER ${NB_USER}
   124 | >>> RUN TIMEFORMAT='time: %3R' \
   125 | >>> bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_PREFIX} --file "environment.yml" && \
   126 | >>> time ${MAMBA_EXE} clean --all -f -y && \
   127 | >>> ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} \
   128 | >>> '
   129 |     
  --------------------
  ERROR: failed to solve: process "/bin/sh -c TIMEFORMAT='time: %3R' bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_PREFIX} --file \"environment.yml\" && time ${MAMBA_EXE} clean --all -f -y && ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} '" did not complete successfully: exit code: 1
  Traceback (most recent call last):
    File "/opt/venv/bin/jupyter-repo2docker", line 8, in <module>
      sys.exit(main())
               ^^^^^^
    File "/opt/venv/lib/python3.12/site-packages/repo2docker/__main__.py", line 476, in main
      r2d.start()
    File "/opt/venv/lib/python3.12/site-packages/repo2docker/app.py", line 843, in start
      self.build()
    File "/opt/venv/lib/python3.12/site-packages/repo2docker/app.py", line 806, in build
      for l in picked_buildpack.build(
               ^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/venv/lib/python3.12/site-packages/repo2docker/buildpacks/base.py", line 671, in build
      yield from client.build(**build_kwargs)
    File "/opt/venv/lib/python3.12/site-packages/repo2docker/docker.py", line 155, in build
      yield from execute_cmd(args, True)
    File "/opt/venv/lib/python3.12/site-packages/repo2docker/utils.py", line 76, in execute_cmd
      raise subprocess.CalledProcessError(ret, cmd)
  subprocess.CalledProcessError: Command '['docker', 'buildx', 'build', '--progress', 'plain', '--build-arg', 'NB_USER=jovyan', '--build-arg', 'NB_UID=1000', '--build-arg', 'REPO_DIR=/srv/repo', '--cache-from', 'quay.io/cryointhecloud/cryo-hub-image', '--tag', 'quay.io/cryointhecloud/cryo-hub-image:84d42d3e4798', '--platform', 'linux/amd64', '/tmp/tmpjky_cgqe']' returned non-zero exit status 1.

I'm starting to think if there's some cache invalidation that needs to happen to get this to work 🤔 Because that .github/workflows/test.yml workflow is still using mamba 2.0.5 instead of 2.0.7

[weiji14]

Ah found it, repo2docker pins to mamba=2.0.5 at https://github.com/jupyterhub/repo2docker/blob/d0ddd2e3c47776c11eaa006c0b3bb2e2d372f89b/repo2docker/buildpacks/conda/install-base-env.bash#L8, we need to update it there.

[rwegener2]

Wow, thanks @weiji14 for putting all the time into this troubleshooting. It's really kind and appreciated.

[tsnow03]

Thanks @weiji14 and @rwegener2 for staying on top of this! Hopefully jupyterhub will merge that soon.

[weiji14]

Builds are passing now so I'll merge this in :)

[rwegener2]

Thanks a million, @weiji14! I'm going to open everything with datatree on Cryocloud now 😎