-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME.qmd
135 lines (106 loc) · 5.12 KB
/
README.qmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
---
title: mspipelines
format: gfm
---
<!-- README.md is generated by running 'quarto render README.qmd' -->
```{r, echo = FALSE, message = FALSE, error = FALSE, warning = FALSE}
library(tidyverse)
```
## Requirements
To use this repository, please install the following dependencies:
* Bash
* Java (Java 11 or higher)
* [Docker](https://docs.docker.com/get-docker/)
* [Nextflow](https://www.nextflow.io/index.html#GetStarted)
* [Viash](https://viash.io/installation/)
Assuming you've already got Bash, Java, Docker installed. To install Nextflow and Viash,
create a `bin/` directory in your home directory and make sure it's on your PATH variable.
```bash
mkdir "$HOME/bin"
curl -fsSL get.viash.io | bash -s -- --bin $HOME/bin --tools false
curl -s https://get.nextflow.io | bash; mv nextflow $HOME/bin
```
Make sure that Viash and Nextflow are on the $PATH by checking whether the following commands work:
```bash
viash -v
nextflow -v
```
## Step 1: Build all the components
The `src/` folder contains Viash components for processing Mass Spec data. With the command below, you can turn your script into a containerized executable and a containerized Nextflow module. Use the `-q 'xxx'` parameter to build a subset of components in the repository.
```bash
viash ns build --query 'maxquant' --parallel --setup cachedbuild
```
Exporting maxquant (maxquant) =nextflow=> mspipelines/target/nextflow/maxquant/maxquant
Exporting maxquant (maxquant) =docker=> mspipelines/target/docker/maxquant/maxquant
Exporting maxquant_to_h5ad (convert) =docker=> mspipelines/target/docker/convert/maxquant_to_h5ad
Exporting maxquant_to_h5ad (convert) =nextflow=> mspipelines/target/nextflow/convert/maxquant_to_h5ad
[notice] Building container 'ghcr.io/czbiohub/mspipelines/maxquant_maxquant:dev' with Dockerfile
[notice] Building container 'ghcr.io/czbiohub/mspipelines/convert_maxquant_to_h5ad:dev' with Dockerfile
Not all configs built successfully
6/10 configs were disabled
4/10 configs built successfully
Viash will build a whole namespace (`ns`) into executables and Nextflow pipelines into the `target/docker` and `target/nextflow` folders respectively.
* By adding the `-q/--query` flag, you can filter which components to build using a regex.
* By adding the `--parallel` flag, these components are built in parallel (otherwise it will take a really long time).
* The flag `--setup cachedbuild` will automatically start building Docker containers for each of these methods.
The command might take a while to run, since it is building a docker container for each of the components.
## Step 2: Download the test resources
Use the `sync_test_resources` component to download the test resources.
```bash
viash run src/download/sync_test_resources/config.vsh.yaml
```
Completed 256.0 KiB/7.2 MiB (302.6 KiB/s) with 6 file(s) remaining
Completed 512.0 KiB/7.2 MiB (595.8 KiB/s) with 6 file(s) remaining
Completed 768.0 KiB/7.2 MiB (880.3 KiB/s) with 6 file(s) remaining
Completed 1.0 MiB/7.2 MiB (1.1 MiB/s) with 6 file(s) remaining
Completed 1.2 MiB/7.2 MiB (1.3 MiB/s) with 6 file(s) remaining
...
## That's it!
You're all set to start developing new components. Below are a few more examples to help you get started.
## Example 1: Unit test a component
```bash
viash test src/maxquant/maxquant/config.vsh.yaml
```
Running tests in temporary directory: '/home/rcannood/workspace/viash_temp/viash_test_maxquant750761290439648338'
====================================================================
+/home/rcannood/workspace/viash_temp/viash_test_maxquant750761290439648338/build_executable/maxquant ---verbosity 6 ---setup cachedbuild
[notice] Building container 'ghcr.io/czbiohub/mspipelines/maxquant_maxquant:test_jrHF3N' with Dockerfile
...
Successfully built d502546463de
Successfully tagged ghcr.io/czbiohub/mspipelines/maxquant_maxquant:test_jrHF3N
====================================================================
+/home/rcannood/workspace/viash_temp/viash_test_maxquant750761290439648338/test_test/test_executable
>> Running maxquant
Configuring
Assemble run info
Finish run info
Testing fasta files
Testing raw files
...
Writing tables
Finish writing tables
>> Checking whether output files can be found
>> All tests succeeded!
====================================================================
SUCCESS! All 1 out of 1 test scripts succeeded!
Cleaning up temporary directory
## Example 2: Run a pipeline using the local code
```bash
dir=resources_test/msdial_demo_files/raw/LCMS_DDA
nextflow run . \
-main-script target/nextflow/msdial/msdial_lcms/main.nf \
-with-docker \
--input "$dir/Nega_Ida_QC_1_1.mzML" \
--input "$dir/Nega_Ida_QC_1_9.mzML" \
--class_id foo \
--class_id bar \
--ms1_data_type Profile \
--ms2_data_type Profile \
--ion_mode Negative \
--minimum_peak_height 1 \
--adduct_list "[M-H]-,[M+FA-H]-,[M-H2O-H]-" \
--retention_time_tolerance_for_identification 4 \
--accurate_ms1_tolerance_for_identification 0.025 \
--accurate_ms2_tolerance_for_identification 0.25 \
--publish_dir output
```