Skip to content

Use Target.get_job_attrs() and propagate to Batch.new_job() calls#17

Merged
jmarshall merged 2 commits intomainfrom
propagate-job-attrs
Aug 14, 2025
Merged

Use Target.get_job_attrs() and propagate to Batch.new_job() calls#17
jmarshall merged 2 commits intomainfrom
propagate-job-attrs

Conversation

@jmarshall
Copy link
Contributor

Production-pipelines code generally uses b.new_job('name_of_job', job_attrs | {'tool': 'used_tool'}) when setting up Hail jobs, where job_attrs is passed in from the stage via self.get_job_attrs(sequencing_group/cohort/whatever).

This PR changes test_workflows_shared to do the same thing. Previously the resulting Hail batch jobs have not had (most of) their job attributes set up.

@jmarshall jmarshall requested a review from a team as a code owner August 12, 2025 23:49

# Merge the no evens lists for all sequencing groups into a single file
job = b.new_job(name=title)
job = b.new_job(name=title, attributes=job_attrs | {'tool': 'cat'})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tool cat? 😂

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep 😸

The command for this job is just job.command(f'cat {inputs} >> {job.pyramid}'). But really this is just an example of unioning in some more attributes entries.

@jmarshall jmarshall merged commit cce7b80 into main Aug 14, 2025
1 of 2 checks passed
@jmarshall jmarshall deleted the propagate-job-attrs branch August 14, 2025 00:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants