Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ucd sonic #489

Merged
merged 26 commits into from
Aug 15, 2023
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
e28f018
Create ucd_sonic.config
brucemoran Apr 17, 2023
2e2f008
Create ucd_sonic.md
brucemoran Apr 17, 2023
6d88c33
Update nfcore_custom.config
brucemoran Apr 17, 2023
7d8f9ab
Update README.md
brucemoran Apr 17, 2023
028e5c0
Update main.yml
brucemoran Apr 17, 2023
b2ab61e
Update ucd_sonic.config
brucemoran Apr 17, 2023
671e616
Update ucd_sonic.config
brucemoran Apr 17, 2023
08ea9f6
Update ucd_sonic.config
brucemoran Apr 17, 2023
d25044d
Update ucd_sonic.config
brucemoran Apr 17, 2023
bcd3010
Update ucd_sonic.config
brucemoran Apr 17, 2023
dac9f38
Update ucd_sonic.config
brucemoran Apr 17, 2023
3ad8cd1
Update ucd_sonic.config
brucemoran Apr 17, 2023
995aaa6
Update ucd_sonic.config
brucemoran Apr 17, 2023
0fed766
Update ucd_sonic.md
brucemoran Apr 17, 2023
7f62129
Update ucd_sonic.config
brucemoran Apr 17, 2023
cb41ae0
Merge branch 'master' into UCD_SONIC
brucemoran Apr 25, 2023
2070790
Merge branch 'master' into UCD_SONIC
jfy133 Jul 3, 2023
b1d15d3
[automated] Fix linting with Prettier
nf-core-bot Jul 3, 2023
30f0c11
Update ucd_sonic.md
brucemoran Jul 4, 2023
7d2bdd4
Merge branch 'master' into UCD_SONIC
brucemoran Jul 4, 2023
509b407
Update conf/ucd_sonic.config
brucemoran Jul 4, 2023
c02a5a7
Merge branch 'master' into UCD_SONIC
maxulysse Jul 8, 2023
208515d
Update conf/ucd_sonic.config
brucemoran Aug 15, 2023
01ae9c9
Update docs/ucd_sonic.md
brucemoran Aug 15, 2023
7e113e2
Remove bmoran profile
brucemoran Aug 15, 2023
2f45ca0
Merge branch 'master' into UCD_SONIC
brucemoran Aug 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ jobs:
- "tubingen_apg"
- "ucl_myriad"
- "uct_hpc"
- "ucd_sonic"
- "uge"
- "unibe_ibu"
- "uppmax"
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,7 @@ Currently documentation is available for the following systems:
- [Super Computing Wales](docs/scw.md)
- [TIGEM](docs/tigem.md)
- [TUBINGEN_APG](docs/tubingen_apg.md)
- [UCD_SONIC](docs/ucd_sonic.md)
- [UCL_MYRIAD](docs/ucl_myriad.md)
- [UCT_HPC](docs/uct_hpc.md)
- [UGE](docs/uge.md)
Expand Down
37 changes: 37 additions & 0 deletions conf/ucd_sonic.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
params {
config_profile_name = 'UCD_SONIC'
config_profile_description = 'University College Dublin Sonic HPC profile provided by nf-core/configs.'
config_profile_contact = 'Bruce Moran (@brucemoran)'
config_profile_url = 'https://www.ucd.ie/itservices/ourservices/researchit/researchcomputing/sonichpc/'
//max_memory = 1.MB //yes really, allocation made based on other factors and cannot be done via params.max_memory...
brucemoran marked this conversation as resolved.
Show resolved Hide resolved
max_cpus = 40
max_time = 12.h
}

process {
executor = 'slurm'
queue = 'shared'
queueSize = 50
submitRateLimit = '10 sec'
maxRetries = 2
beforeScript = 'export NXF_OPTS="-Xms2G -Xmx40G"; module load nextflow/22.04.5.5708 singularity/3.5.2'
clusterOptions = { "--mem 1M" }
brucemoran marked this conversation as resolved.
Show resolved Hide resolved
cache = 'lenient'
memory = 1.MB
brucemoran marked this conversation as resolved.
Show resolved Hide resolved
}

cleanup = true

profiles {
moranb {
brucemoran marked this conversation as resolved.
Show resolved Hide resolved
singularity {
enabled = true
autoMounts = true
cacheDir = '/home/people/moranb/singularity/nextflow'
runOptions = '--bind /home/people/moranb,/scratch/moranb'
}
env {
NXF_OPTS="-Xms2G -Xmx40G"
}
}
}
17 changes: 17 additions & 0 deletions docs/ucd_sonic.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# nf-core/configs: UCD Sonic Configuration

The nf-core pipeline [sarek](https://nf-co.re/sarek) has been successfully tested on [Sonic HPC] (https://www.ucd.ie/itservices/ourservices/researchit/researchcomputing/sonichpc) using this config.

This Sonic config offers very minimal options to users but should allow the use of all nf-core pipelines.

## sbatch_nxf_creator

To run a pipeline, you can use the [sbatch_nxf_creator](https://github.com/brucemoran/sbatch_nxf_creator) method.
brucemoran marked this conversation as resolved.
Show resolved Hide resolved

This allows you to write a YAML file including all of your sample names that you want to process individually, as well as your basic Nextflow command and where the output should be written.

There are some additional requirements on Sonic based on its use of SLURM which are resolved using this write this method.

The `sbatch_nxf_creator.sh` parses your YAML and makes an `sbatch` file that can be submitted to the scheduler via a simple `sbatch my_file.sbatch` command.

Practically, it means appending `srun` to the command and appending `-with-mpi`, as well as a few other vaguaries like specifying output and loading `Nextflow` and `Singularity`.
1 change: 1 addition & 0 deletions nfcore_custom.config
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ profiles {
seg_globe { includeConfig "${params.custom_config_base}/conf/seg_globe.config"}
tigem { includeConfig "${params.custom_config_base}/conf/tigem.config"}
tubingen_apg { includeConfig "${params.custom_config_base}/conf/tubingen_apg.config" }
ucd_sonic { includeConfig "${params.custom_config_base}/conf/ucd_sonic.config"}
ucl_myriad { includeConfig "${params.custom_config_base}/conf/ucl_myriad.config"}
uct_hpc { includeConfig "${params.custom_config_base}/conf/uct_hpc.config" }
uge { includeConfig "${params.custom_config_base}/conf/uge.config" }
Expand Down