Create Conda CI test env in one step #144

KyleFromNVIDIA · 2025-02-11T20:19:35Z

dependencies.yaml

bdice · 2025-02-11T20:39:50Z

ci/test_python.sh

Are ARM tests doing anything here? Maybe they should exit earlier (or be skipped in the CI matrix) if not.

There are a few comments of the form # Reactivate the test environment back. I’m not sure if we have a test environment to go back to, with the new single-solve environments.

We can do that in a follow-up PR. I'd like to limit the scope of this one to consolidating the environment creation.

jameslamb

Left some suggestions. Leaving a blocking review, as a couple should definitely be resolved before merging this, and the question about depends_on_ogb needs a little discussion.

ci/test_python.sh

jameslamb · 2025-02-14T22:21:15Z

ci/test_python.sh

+    --matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}"  \
+    --prepend-channel "${CPP_CHANNEL}" \
+    --prepend-channel "${PYTHON_CHANNEL}" \
+    --prepend-channel pytorch \


Not a blocking comment for this PR, but @tingyu66 @alexbarghi-nv are we planning to drop use of the pytorch channel in the 25.04 release?

Linking this related conversation: #99 (comment)

Yes, we should do that.

Thank you! I can get that started in a follow-up PR if you'd like.

dependencies.yaml

jameslamb · 2025-02-14T22:27:29Z

dependencies.yaml

+      - depends_on_cudf
+      - depends_on_dgl
+      - depends_on_pytorch
+      - depends_on_ogb


I don't think this is what you want, to replace conda install ogb.

depends_on_ogb... doesn't seem to contain ogb:

cugraph-gnn/dependencies.yaml

Lines 438 to 464 in 2f8afde

# Will remove this after snap-stanford/ogb#497 is resolved.

# Temporarily sets the max pytorch version to 2.5 for compatibility

# with ogb.

depends_on_ogb:

common:

- output_types: [conda]

packages:

- pytorch>=2.3,<2.6a0

specific:

- output_types: [requirements]

matrices:

- matrix: {cuda: "12.*"}

packages:

- --extra-index-url=https://download.pytorch.org/whl/cu121

- matrix: {cuda: "11.*"}

packages:

- --extra-index-url=https://download.pytorch.org/whl/cu118

- {matrix: null, packages: null}

- output_types: [requirements, pyproject]

matrices:

- matrix: {cuda: "12.*"}

packages:

- torch>=2.3,<2.6a0

- matrix: {cuda: "11.*"}

packages:

- torch>=2.3,<2.6a0

- {matrix: null, packages: [*pytorch_pip]}

That looks like a mistake, not sure where it happened. @tingyu66 could you please take a look? Maybe that was the result of a bad merge conflict resolution or something. depends_on_ogb seems to just contain torch (which we already have depends_on_pytorch for)

It's not a mistake

This is a temporary constraint of PyTorch that only applies to test environments where ogb is being used.

Bradley and I discussed this here: #104

And on Slack

A better name for this might be ogb_pytorch_constraint.

We typically use depends_on_ to indicate an actual dependency on just that package.

Got it, thanks! The comment directly above this, about this being there for the benefit of ogb, now makes sense to me.

It still would be better to name it ogb_pytorch_constraint, I think, but that doesn't have to hold up this PR.

Ok, also devcontainer builds will fail until #148 gets merged. I expect that to pass CI now that the cudf fix is merged.

Now that that's merged, I've merged in latest branch-25.04 here to re-run CI.

jameslamb

Approving, my comments have been addressed.

I'll do the renaming suggested in #144 (comment) + dropping the pytorch channel in a follow-up PR.

KyleFromNVIDIA · 2025-02-24T19:14:41Z

/merge

Create Conda CI test env in one step

5bc3762

Issue: rapidsai/build-planning#22

KyleFromNVIDIA requested a review from msarahan February 11, 2025 20:19

KyleFromNVIDIA requested review from a team as code owners February 11, 2025 20:19

KyleFromNVIDIA added improvement Improves an existing functionality non-breaking Introduces a non-breaking change labels Feb 11, 2025

msarahan reviewed Feb 11, 2025

View reviewed changes

dependencies.yaml Outdated Show resolved Hide resolved

KyleFromNVIDIA added 3 commits February 11, 2025 15:23

Oops

477da45

Put scipy back

cbf1e4a

Copyright

93ce71e

bdice reviewed Feb 11, 2025

View reviewed changes

KyleFromNVIDIA mentioned this pull request Feb 11, 2025

Consider consolidating conda solves in CI tests rapidsai/build-planning#22

Closed

19 tasks

KyleFromNVIDIA added 2 commits February 13, 2025 11:54

Merge branch 'branch-25.04' into one-pass-conda-test-env

1cbccef

Consolidate build_docs.sh and test_cpp.sh

f90fcd7

jameslamb requested changes Feb 14, 2025

View reviewed changes

KyleFromNVIDIA and others added 2 commits February 14, 2025 17:34

Review feedback

9f40411

Merge branch 'branch-25.04' into one-pass-conda-test-env

3cfed66

jameslamb approved these changes Feb 20, 2025

View reviewed changes

rapids-bot bot merged commit a11b148 into rapidsai:branch-25.04 Feb 24, 2025
82 checks passed

jameslamb mentioned this pull request Feb 25, 2025

Use conda-build instead of conda-mambabuild #150

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create Conda CI test env in one step #144

Create Conda CI test env in one step #144

KyleFromNVIDIA commented Feb 11, 2025

bdice Feb 11, 2025 •

edited

Loading

KyleFromNVIDIA Feb 11, 2025

jameslamb left a comment

jameslamb Feb 14, 2025

alexbarghi-nv Feb 19, 2025

jameslamb Feb 19, 2025

jameslamb Feb 14, 2025

alexbarghi-nv Feb 19, 2025

alexbarghi-nv Feb 19, 2025

alexbarghi-nv Feb 19, 2025

alexbarghi-nv Feb 19, 2025

bdice Feb 19, 2025 •

edited

Loading

jameslamb Feb 19, 2025 •

edited

Loading

alexbarghi-nv Feb 19, 2025

jameslamb Feb 20, 2025

jameslamb left a comment

KyleFromNVIDIA commented Feb 24, 2025

	# Will remove this after snap-stanford/ogb#497 is resolved.
	# Temporarily sets the max pytorch version to 2.5 for compatibility
	# with ogb.
	depends_on_ogb:
	common:
	- output_types: [conda]
	packages:
	- pytorch>=2.3,<2.6a0
	specific:
	- output_types: [requirements]
	matrices:
	- matrix: {cuda: "12.*"}
	packages:
	- --extra-index-url=https://download.pytorch.org/whl/cu121
	- matrix: {cuda: "11.*"}
	packages:
	- --extra-index-url=https://download.pytorch.org/whl/cu118
	- {matrix: null, packages: null}
	- output_types: [requirements, pyproject]
	matrices:
	- matrix: {cuda: "12.*"}
	packages:
	- torch>=2.3,<2.6a0
	- matrix: {cuda: "11.*"}
	packages:
	- torch>=2.3,<2.6a0
	- {matrix: null, packages: [*pytorch_pip]}

Create Conda CI test env in one step #144

Create Conda CI test env in one step #144

Conversation

KyleFromNVIDIA commented Feb 11, 2025

bdice Feb 11, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jameslamb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bdice Feb 19, 2025 • edited Loading

Choose a reason for hiding this comment

jameslamb Feb 19, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jameslamb left a comment

Choose a reason for hiding this comment

KyleFromNVIDIA commented Feb 24, 2025

bdice Feb 11, 2025 •

edited

Loading

bdice Feb 19, 2025 •

edited

Loading

jameslamb Feb 19, 2025 •

edited

Loading