Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

increase parallelism in nightly builds #17792

Open
wants to merge 1 commit into
base: branch-25.02
Choose a base branch
from

Conversation

jameslamb
Copy link
Member

Description

Contributes to rapidsai/build-planning#136

For nightly builds, some wheel-build-{project} jobs currently wait to start until some other wheel-publish-{dependency} jobs complete. This is unnecessary... wheel-build-{dependency} jobs will upload packages to S3, which is where wheel-build-{project} jobs will download them from.

This proposes changing that such that all nightly wheel-build-* jobs depend only other wheel-build-* jobs. This should decrease the end-to-end time it takes for all wheels to be built and published on nightly / branch builds.

Also updates pre-commit config to the latest rapids-dependency-file-generator version.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@jameslamb jameslamb added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Jan 23, 2025
@jameslamb jameslamb requested a review from vyasr January 23, 2025 16:05
@jameslamb jameslamb requested a review from a team as a code owner January 23, 2025 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant