Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Recursively inherit configs #195

Merged
merged 12 commits into from
Sep 20, 2023
Merged

Recursively inherit configs #195

merged 12 commits into from
Sep 20, 2023

Conversation

adamrtalbot
Copy link
Contributor

@adamrtalbot adamrtalbot commented Sep 18, 2023 •
edited
Loading

Changes:

  • Remove public_aws_ecr profile
  • Replace large config file with individual configs per module
  • Each module has it's own config file
  • Each workflow/subworkflow that uses it uses includeConfig to read the relevant config in
  • Use if statements to read only read relevant configs

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/fetchngs branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@github-actions
Copy link

github-actions bot commented Sep 18, 2023 •
edited
Loading

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 8284cc4

+| ✅ 134 tests passed       |+
#| ❔  13 tests were ignored |#
!| ❗   7 tests had warnings |!

❗ Test warnings:

  • files_exist - File not found: conf/igenomes.config
  • files_exist - File not found: .github/workflows/awstest.yml
  • files_exist - File not found: .github/workflows/awsfulltest.yml
  • files_exist - File not found: lib/WorkflowFetchngs.groovy
  • nextflow_config - Config manifest.version should end in dev: 1.10.1
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your prefered methods description, e.g. add publication citation for this pipeline
  • schema_description - No description provided in schema for parameter: dbgap_key

❔ Tests ignored:

✅ Tests passed:

Run details

  • nf-core/tools version 2.9
  • Run at 2023-09-19 09:31:17

Midnighter
Midnighter reviewed Sep 19, 2023
View reviewed changes
Copy link
Contributor

@Midnighter Midnighter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general these changes look fine to me. I think, I probably missed some discussion on Slack. What is the big advantage with this approach? To me, it seems like a lot of maintenance to ensure that all configs have been included correctly when making changes for the small gain of having smaller individual config files. It also doesn't seem ideal to add config files in modules installed from nf-core where the directories might be changed by upgrades. Do you have a place where I can read up on the motivation?

@adamrtalbot adamrtalbot merged commit acd2dd2 into nf-core:refactor Sep 20, 2023
22 of 35 checks passed
@adamrtalbot adamrtalbot deleted the inherit_configs_recursively branch September 20, 2023 08:51
@adamrtalbot
Copy link
Contributor Author

In general these changes look fine to me. I think, I probably missed some discussion on Slack. What is the big advantage with this approach? To me, it seems like a lot of maintenance to ensure that all configs have been included correctly when making changes for the small gain of having smaller individual config files. It also doesn't seem ideal to add config files in modules installed from nf-core where the directories might be changed by upgrades. Do you have a place where I can read up on the motivation?

From @drpatelh:

  • Checking for changes only for components that have changed rather than re-running everything, all of the time.
  • Re-running tests when config as well as tests have changed (we don't do this at the moment)
  • Ability to share and re-use config's easier rather than having a massive modules.config
  • Ability to patch configs like nf-core modules patch to track differences with standard configs stored in nf-core/modules
  • Ability to share workflows by making them more self-enclosed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Midnighter Midnighter Midnighter left review comments

@maxulysse maxulysse maxulysse approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@adamrtalbot @Midnighter @maxulysse