Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AWS full testing ignores staging of s3 files #4

Closed
matq007 opened this issue Jun 22, 2023 · 1 comment
Closed

AWS full testing ignores staging of s3 files #4

matq007 opened this issue Jun 22, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@matq007
Copy link
Member

matq007 commented Jun 22, 2023 •
edited
Loading

Description of the bug

Currently, when testing pipeline on AWS, only reference building is executed. The issues is that in the execution, the PREPARE_PIPELINE module can't access the s3 bucket files, they are not staged (dowloaded as FASTQ files, not sure why). Here is a link to tower execution.

You can see that as the docs state, the s3 link is converted to /nf-core/test-datasets/marsseq/testdata/SB26/, however, the files are not staged and therefore the pipeline fails.

I've checked that the files are present in the bucket.

$ aws --no-sign-request s3 ls s3://ngi-igenomes/test-data/marsseq/

2023-06-06 16:10:32  312138696 Undetermined_S0_L001_R1_001.fastq.gz
2023-06-06 16:10:32  133363364 Undetermined_S0_L001_R2_001.fastq.gz
2023-06-06 16:10:32       9516 amp_batches.xlsx
2023-06-06 16:10:32        567 design.csv
2023-06-06 16:10:32       8737 seq_batches.xlsx
2023-06-06 16:10:32     180347 wells_cells.xlsx

I can't test this with internal dangpu cluster because I don't have AWS credentials and whenever I try to execute the pipeline it will timeout, probably because of the missing credentials. I think this could be associated with how the bucket is public/not public, see nextflow/issues/3281.

Possible solution

Check if the files are coming from s3 bucket, if yes, then convert the s3 to normal url format and download the files with WGET module.

http://test-bucket.s3.amazonaws.com/test-folder/test-file.txt
http://ngi-igenomes.s3.amazonaws.com/test-data/marsseq/wells_cells.xlsx

The question still remains, will the reference folder be staged correctly, or will it fail as well?

Command used and terminal output

$ nextflow run . -profile ku_sund_dangpu --genome mm10 --outdir ./results --input /home/fdb589/Brickman/projects/proks_et_al_2022/pipeline/SB26/design_remote.csv


```csv
batch,fastq_1,fastq_2,amp_batches,seq_batches,well_cells
SB26,s3://ngi-igenomes/test-data/marsseq/Undetermined_S0_L001_R1_001.fastq.gz,s3://ngi-igenomes/test-data/marsseq/Undetermined_S0_L001_R2_001.fastq.gz,s3://ngi-igenomes/test-data/marsseq/amp_batches.xlsx,s3://ngi-igenomes/test-data/marsseq/seq_batches.xlsx,s3://ngi-igenomes/test-data/marsseq/wells_cells.xlsx


### Relevant files

```log
Jun-21 22:00:52.235 [main] DEBUG nextflow.cli.Launcher - $> nextflow run . -profile ku_sund_dangpu --genome mm10 --outdir ./results --input /home/fdb589/Brickman/projects/proks_et_al_2022/pipeline/SB26/design_remote.csv
Jun-21 22:00:52.448 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 23.04.1
Jun-21 22:00:52.467 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/scratch/Brickman/pipelines/nfcore/plugins; core-plugins: nf-amazon@1.16.2,nf-azure@1.0.1,nf-codecommit@0.1.4,nf-console@1.0.5,nf-ga4gh@1.0.5,nf-google@1.7.3,nf-tower@1.5.12,nf-wave@0.8.2
Jun-21 22:00:52.477 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Jun-21 22:00:52.478 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Jun-21 22:00:52.481 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Jun-21 22:00:52.490 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Jun-21 22:00:53.367 [main] DEBUG nextflow.config.ConfigBuilder - Found config local: /scratch/Brickman/pipelines/nf-core-marsseq/nextflow.config
Jun-21 22:00:53.367 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /scratch/Brickman/pipelines/nf-core-marsseq/nextflow.config
Jun-21 22:00:53.377 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `ku_sund_dangpu`
Jun-21 22:00:54.362 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [cfc_dev, ifb_core, denbi_qbic, alice, mjolnir_globe, uppmax, uge, rosalind_uge, lugh, unibe_ibu, vai, czbiohub_aws, jax, ccga_med, scw, tigem, tubingen_apg, google, ipop_up, googlels, eddie, medair, apptainer, bi, bigpurple, sbc_sharc, adcra, cedars, vsc_kul_uhasselt, pawsey_nimbus, ucl_myriad, utd_ganymede, charliecloud, icr_davros, ceres, munin, arm, rosalind, hasta, cfc, uzh, ebc, ku_sund_dangpu, ccga_dx, crick, marvin, biohpc_gen, shifter, mana, mamba, wehi, awsbatch, imperial, maestro, genotoul, abims, janelia, nihbiowulf, nu_genomics, googlebatch, oist, sahmri, mpcdf, leicester, vsc_ugent, sage, cambridge, podman, cheaha, xanadu, test, computerome, seg_globe, sanger, dkfz, pasteur, test_full, azurebatch, hki, crukmi, docker, engaging, gis, psmn, eva, fgcz, conda, crg, singularity, prince, utd_sysbio, debug, genouest, cbe, phoenix, gitpod, seawulf, uct_hpc, aws_tower, binac]
Jun-21 22:00:54.395 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
Jun-21 22:00:54.411 [main] INFO  nextflow.cli.CmdRun - Launching `./main.nf` [deadly_elion] DSL2 - revision: 691bff9081
Jun-21 22:00:54.412 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Jun-21 22:00:54.412 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Jun-21 22:00:54.417 [main] DEBUG nextflow.secret.LocalSecretsProvider - Secrets store: /scratch/Brickman/pipelines/nfcore/secrets/store.json
Jun-21 22:00:54.420 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@29bd85db] - activable => nextflow.secret.LocalSecretsProvider@29bd85db
Jun-21 22:00:54.477 [main] DEBUG nextflow.Session - Session UUID: 9eeae9d5-2f33-4bcc-bc65-9a05757e3280
Jun-21 22:00:54.477 [main] DEBUG nextflow.Session - Run name: deadly_elion
Jun-21 22:00:54.478 [main] DEBUG nextflow.Session - Executor pool size: 224
Jun-21 22:00:54.488 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=672; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Jun-21 22:00:54.514 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 23.04.1 build 5866
  Created: 15-04-2023 06:51 UTC (08:51 CEST)
  System: Linux 4.18.0-425.3.1.el8.x86_64
  Runtime: Groovy 3.0.16 on Java HotSpot(TM) 64-Bit Server VM 11.0.15+8-LTS-149
  Encoding: UTF-8 (UTF-8)
  Process: 1904246@dangpu01fl.unicph.domain [10.84.1.4]
  CPUs: 224 - Mem: 3.9 TB (181 GB) - Swap: 4 GB (0)
Jun-21 22:00:54.539 [main] DEBUG nextflow.Session - Work-dir: /scratch/Brickman/pipelines/nf-core-marsseq/work [xfs]
Jun-21 22:00:54.558 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Jun-21 22:00:54.568 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Jun-21 22:00:54.618 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Jun-21 22:00:54.629 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 225; maxThreads: 1000
Jun-21 22:00:54.693 [main] DEBUG nextflow.Session - Session start
Jun-21 22:00:54.698 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /scratch/Brickman/pipelines/nf-core-marsseq/results/pipeline_info/execution_trace_2023-06-21_22-00-54.txt
Jun-21 22:00:54.707 [main] DEBUG nextflow.Session - Using default localLib path: /scratch/Brickman/pipelines/nf-core-marsseq/lib
Jun-21 22:00:54.710 [main] DEBUG nextflow.Session - Adding to the classpath library: /scratch/Brickman/pipelines/nf-core-marsseq/lib
Jun-21 22:00:54.710 [main] DEBUG nextflow.Session - Adding to the classpath library: /scratch/Brickman/pipelines/nf-core-marsseq/lib/nfcore_external_java_deps.jar
Jun-21 22:00:55.465 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Jun-21 22:00:55.512 [main] INFO  nextflow.Nextflow - 

-�[2m----------------------------------------------------�[0m-
                                        �[0;32m,--.�[0;30m/�[0;32m,-.�[0m
�[0;34m        ___     __   __   __   ___     �[0;32m/,-._.--~'�[0m
�[0;34m  |\ | |__  __ /  ` /  \ |__) |__         �[0;33m}  {�[0m
�[0;34m  | \| |       \__, \__/ |  \ |___     �[0;32m\`-._,-`-,�[0m
                                        �[0;32m`._,._,'�[0m
�[0;35m  nf-core/marsseq v1.0.0dev�[0m
-�[2m----------------------------------------------------�[0m-
�[1mCore Nextflow options�[0m
  �[0;34mrunName                   : �[0;32mdeadly_elion�[0m
  �[0;34mcontainerEngine           : �[0;32msingularity�[0m
  �[0;34mlaunchDir                 : �[0;32m/scratch/Brickman/pipelines/nf-core-marsseq�[0m
  �[0;34mworkDir                   : �[0;32m/scratch/Brickman/pipelines/nf-core-marsseq/work�[0m
  �[0;34mprojectDir                : �[0;32m/scratch/Brickman/pipelines/nf-core-marsseq�[0m
  �[0;34muserName                  : �[0;32mfdb589�[0m
  �[0;34mprofile                   : �[0;32mku_sund_dangpu�[0m
  �[0;34mconfigFiles               : �[0;32m/scratch/Brickman/pipelines/nf-core-marsseq/nextflow.config�[0m

�[1mInput/output options�[0m
  �[0;34minput                     : �[0;32m/home/fdb589/Brickman/projects/proks_et_al_2022/pipeline/SB26/design_remote.csv�[0m
  �[0;34moutdir                    : �[0;32m./results�[0m
  �[0;34mgenomes_base              : �[0;32m./results/references�[0m

�[1mReference genome options�[0m
  �[0;34mfasta                     : �[0;32m./results/references/mm10/GRCm39.fa�[0m
  �[0;34mgtf                       : �[0;32m./results/references/mm10/GRCm39.gtf�[0m
  �[0;34mbowtie2_index             : �[0;32m./results/references/mm10/bowtie2/�[0m
  �[0;34mstar_index                : �[0;32m./results/references/mm10/star/�[0m

�[1mRNA velocity�[0m
  �[0;34mread_length               : �[0;32m75�[0m

�[1mInstitutional config options�[0m
  �[0;34mconfig_profile_description: �[0;32mdangpufl01 configuration�[0m
  �[0;34mconfig_profile_contact    : �[0;32mAdrija Kalvisa <adrija.kalvisa@sund.ku.dk>�[0m

�[1mMax job request options�[0m
  �[0;34mmax_cpus                  : �[0;32m30�[0m
  �[0;34mmax_memory                : �[0;32m200 GB�[0m
  �[0;34mmax_time                  : �[0;32m3d�[0m

!! Only displaying parameters that differ from the pipeline defaults !!
-�[2m----------------------------------------------------�[0m-
If you use nf-core/marsseq for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/marsseq/blob/master/CITATIONS.md
-�[2m----------------------------------------------------�[0m-
Jun-21 22:00:57.764 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_single` matches labels `process_single` for process with name NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK
Jun-21 22:00:57.765 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:SAMPLESHEET_CHECK` matches process NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK
Jun-21 22:00:57.770 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm
Jun-21 22:00:57.770 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm'
Jun-21 22:00:57.779 [main] DEBUG nextflow.executor.Executor - [warm up] executor > slurm
Jun-21 22:00:57.783 [main] DEBUG n.processor.TaskPollingMonitor - Creating task monitor for executor 'slurm' > capacity: 5; pollInterval: 5s; dumpInterval: 5m 
Jun-21 22:00:57.787 [main] DEBUG n.executor.AbstractGridExecutor - Creating executor 'slurm' > queue-stat-interval: 1m
Jun-21 22:00:57.891 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: FASTQC, ERCC_CREATE, BOWTIE2_BUILD, MULTIQC, CUTADAPT, VELOCITY_WHITELIST, DOWNLOAD_GTF, VELOCITY_STARSOLO, GUNZIP_GTF, MERGE_READS, DOWNLOAD_FASTA, CAT_CAT, QC_ALIGNED, WGET, DEMULTIPLEX, GUNZIP, SAMPLESHEET_CHECK, VELOCITY_TRIM, QC_REPORT, GUNZIP_FASTA, VELOCITY_CONVERT, FASTP_SPLIT, CAT_FASTA, NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK, QC_BATCH, PREPARE, STAR_GENOMEGENERATE, CUSTOM_DUMPSOFTWAREVERSIONS, BOWTIE2_ALIGN, CUT_SAM, STAR_ALIGN, EXTRACT_LABELS
Jun-21 22:00:57.896 [main] DEBUG nextflow.Session - Igniting dataflow network (11)
Jun-21 22:00:57.901 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK
Jun-21 22:00:57.902 [main] DEBUG nextflow.script.ScriptRunner - > Awaiting termination 
Jun-21 22:00:57.902 [main] DEBUG nextflow.Session - Session await
Jun-21 22:00:57.906 [PathVisitor-3] DEBUG nextflow.plugin.PluginUpdater - Installing plugin nf-amazon version: 1.16.2
Jun-21 22:00:57.914 [PathVisitor-3] INFO  org.pf4j.AbstractPluginManager - Plugin 'nf-amazon@1.16.2' resolved
Jun-21 22:00:57.914 [PathVisitor-3] INFO  org.pf4j.AbstractPluginManager - Start plugin 'nf-amazon@1.16.2'
Jun-21 22:00:57.932 [PathVisitor-3] DEBUG nextflow.plugin.BasePlugin - Plugin started nf-amazon@1.16.2
Jun-21 22:00:57.947 [PathVisitor-3] DEBUG nextflow.file.FileHelper - > Added 'S3FileSystemProvider' to list of installed providers [s3]
Jun-21 22:00:57.947 [PathVisitor-3] DEBUG nextflow.file.FileHelper - Started plugin 'nf-amazon' required to handle file: s3://ngi-igenomes/test-data/marsseq/wells_cells.xlsx
Jun-21 22:00:57.954 [PathVisitor-3] DEBUG nextflow.file.FileHelper - Creating a file system instance for provider: S3FileSystemProvider
Jun-21 22:00:57.960 [PathVisitor-3] DEBUG nextflow.file.FileHelper - AWS S3 config details: {max_error_retry=5}
Jun-21 22:00:57.977 [Actor Thread 14] DEBUG nextflow.container.SingularityCache - Singularity found local store for image=https://depot.galaxyproject.org/singularity/python:3.8.3; path=/scratch/Brickman/singularity-images/depot.galaxyproject.org-singularity-python-3.8.3.img
Jun-21 22:00:58.119 [Task submitter] DEBUG nextflow.executor.GridTaskHandler - [SLURM] submitted process NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv) > jobId: 61809; workDir: /scratch/Brickman/pipelines/nf-core-marsseq/work/e1/168e23842e46da72bb193df6d24e09
Jun-21 22:00:58.120 [Task submitter] INFO  nextflow.Session - [e1/168e23] Submitted process > NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)
Jun-21 22:00:58.297 [PathVisitor-3] DEBUG com.upplication.s3fs.AmazonS3Client - Setting S3 glacierRetrievalTier=null
Jun-21 22:01:02.797 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[jobId: 61809; id: 1; name: NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv); status: COMPLETED; exit: 0; error: -; workDir: /scratch/Brickman/pipelines/nf-core-marsseq/work/e1/168e23842e46da72bb193df6d24e09 started: 1687377662794; exited: 2023-06-21T20:01:02.132786Z; ]
Jun-21 22:01:02.816 [Task monitor] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'PublishDir' minSize=10; maxSize=672; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Jun-21 22:01:02.820 [main] DEBUG nextflow.Session - Session await > all processes finished
Jun-21 22:01:02.835 [main] DEBUG nextflow.Session - Session await > all barriers passed
Jun-21 22:01:08.136 [Actor Thread 19] WARN  c.a.i.InstanceMetadataServiceResourceFetcher - Fail to retrieve token 
com.amazonaws.SdkClientException: Failed to connect to service endpoint: 
	at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100)
	at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.getToken(InstanceMetadataServiceResourceFetcher.java:91)
	at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:69)
	at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:66)
	at com.amazonaws.auth.InstanceMetadataServiceCredentialsFetcher.getCredentialsEndpoint(InstanceMetadataServiceCredentialsFetcher.java:60)
	at com.amazonaws.auth.InstanceMetadataServiceCredentialsFetcher.getCredentialsResponse(InstanceMetadataServiceCredentialsFetcher.java:48)
	at com.amazonaws.auth.BaseCredentialsFetcher.fetchCredentials(BaseCredentialsFetcher.java:124)
	at com.amazonaws.auth.BaseCredentialsFetcher.getCredentials(BaseCredentialsFetcher.java:80)
	at com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials(InstanceProfileCredentialsProvider.java:166)
	at com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper.getCredentials(EC2ContainerCredentialsProviderWrapper.java:75)
	at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:118)
	at com.amazonaws.services.s3.S3CredentialsProviderChain.getCredentials(S3CredentialsProviderChain.java:36)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1269)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1290)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5397)
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:928)
	at com.upplication.s3fs.AmazonS3Client.listObjects(AmazonS3Client.java:167)
	at com.upplication.s3fs.util.S3ObjectSummaryLookup.lookup(S3ObjectSummaryLookup.java:112)
	at com.upplication.s3fs.S3FileSystemProvider.checkAccess(S3FileSystemProvider.java:668)
	at java.base/java.nio.file.Files.exists(Files.java:2440)
	at java.base/jdk.internal.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
	at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.invoke(StaticMetaMethodSite.java:44)
	at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.call(StaticMetaMethodSite.java:89)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.call(StaticMetaMethodSite.java:94)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
	at nextflow.extension.FilesEx.exists(FilesEx.groovy:456)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.codehaus.groovy.runtime.metaclass.ReflectionMetaMethod.invoke(ReflectionMetaMethod.java:54)
	at org.codehaus.groovy.runtime.metaclass.NewInstanceMetaMethod.invoke(NewInstanceMetaMethod.java:54)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1258)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
	at groovy.runtime.metaclass.NextflowDelegatingMetaClass.invokeMethod(NextflowDelegatingMetaClass.java:64)
	at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:44)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:130)
	at Script_80cf5f94.create_fastq_channel(Script_80cf5f94:31)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:362)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:61)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:51)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:171)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:185)
	at Script_80cf5f94$_runScript_closure1$_closure2$_closure3.doCall(Script_80cf5f94:15)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:38)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
	at nextflow.extension.MapOp$_apply_closure1.doCall(MapOp.groovy:56)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
	at groovy.lang.Closure.call(Closure.java:412)
	at groovyx.gpars.dataflow.operator.DataflowOperatorActor.startTask(DataflowOperatorActor.java:120)
	at groovyx.gpars.dataflow.operator.DataflowOperatorActor.onMessage(DataflowOperatorActor.java:108)
	at groovyx.gpars.actor.impl.SDAClosure$1.call(SDAClosure.java:43)
	at groovyx.gpars.actor.AbstractLoopingActor.runEnhancedWithoutRepliesOnMessages(AbstractLoopingActor.java:293)
	at groovyx.gpars.actor.AbstractLoopingActor.access$400(AbstractLoopingActor.java:30)
	at groovyx.gpars.actor.AbstractLoopingActor$1.handleMessage(AbstractLoopingActor.java:93)
	at groovyx.gpars.util.AsyncMessagingCore.run(AsyncMessagingCore.java:132)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.net.SocketException: Network is unreachable (connect failed)
	at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
	at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
	at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
	at java.base/java.net.Socket.connect(Socket.java:608)
	at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
	at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:341)
	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:362)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1253)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1232)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1015)
	at com.amazonaws.internal.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:95)
	at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:80)
	... 99 common frames omitted
Jun-21 22:01:24.646 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'PublishDir' shutdown completed (hard=false)
Jun-21 22:01:24.649 [main] INFO  nextflow.Nextflow - -�[0;35m[nf-core/marsseq]�[0;32m Pipeline completed successfully�[0m-
Jun-21 22:01:24.649 [main] INFO  nextflow.Nextflow - -�[0;35m[nf-core/marsseq]�[0;32m Pipeline completed successfully�[0m-
Jun-21 22:01:24.655 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=1; failedCount=0; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=1s; failedDuration=0ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=1; peakCpus=1; peakMemory=6 GB; ]
Jun-21 22:01:24.655 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file
Jun-21 22:01:24.657 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report
Jun-21 22:01:24.682 [main] DEBUG nextflow.trace.ReportObserver - Execution report summary data:
  [{"cpuUsage":{"mean":65.4,"min":65.4,"q1":65.4,"q2":65.4,"q3":65.4,"max":65.4,"minLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","maxLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q1Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q2Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q3Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)"},"process":"SAMPLESHEET_CHECK","mem":{"mean":4808704,"min":4808704,"q1":4808704,"q2":4808704,"q3":4808704,"max":4808704,"minLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","maxLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q1Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q2Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q3Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)"},"memUsage":{"mean":0.07,"min":0.07,"q1":0.07,"q2":0.07,"q3":0.07,"max":0.07,"minLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","maxLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q1Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q2Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q3Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)"},"timeUsage":{"mean":0.01,"min":0.01,"q1":0.01,"q2":0.01,"q3":0.01,"max":0.01,"minLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","maxLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q1Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q2Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q3Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)"},"vmem":{"mean":31240192,"min":31240192,"q1":31240192,"q2":31240192,"q3":31240192,"max":31240192,"minLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","maxLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q1Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q2Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q3Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)"},"reads":{"mean":2072025,"min":2072025,"q1":2072025,"q2":2072025,"q3":2072025,"max":2072025,"minLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","maxLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q1Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q2Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q3Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)"},"cpu":{"mean":65.4,"min":65.4,"q1":65.4,"q2":65.4,"q3":65.4,"max":65.4,"minLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","maxLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q1Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q2Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q3Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)"},"time":{"mean":1000,"min":1000,"q1":1000,"q2":1000,"q3":1000,"max":1000,"minLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","maxLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q1Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q2Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q3Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)"},"writes":{"mean":774,"min":774,"q1":774,"q2":774,"q3":774,"max":774,"minLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","maxLabel":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q1Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q2Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)","q3Label":"NFCORE_MARSSEQ:MARSSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (design_remote.csv)"}}]
Jun-21 22:01:25.214 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline
Jun-21 22:01:25.318 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Jun-21 22:01:25.318 [main] INFO  org.pf4j.AbstractPluginManager - Stop plugin 'nf-amazon@1.16.2'
Jun-21 22:01:25.318 [main] DEBUG nextflow.plugin.BasePlugin - Plugin stopped nf-amazon
Jun-21 22:01:25.350 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'FileTransfer' shutdown completed (hard=false)
Jun-21 22:01:25.351 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

System information

DanGPU server
@matq007 matq007 added the bug Something isn't working label Jun 22, 2023
matq007 added a commit that referenced this issue Jun 22, 2023
@matq007
fix: support s3, issue #4
83d54a8
@matq007 matq007 closed this as completed Jun 26, 2023
@matq007 matq007 reopened this Jun 26, 2023
@matq007
Copy link
Member Author

matq007 commented Jun 26, 2023

One more remaining problem with velocity: https://tower.nf/orgs/nf-core/workspaces/AWSmegatests/watch/1NJxfbjgZqUonC

@matq007 matq007 closed this as completed Jun 26, 2023
@matq007 matq007 reopened this Jun 26, 2023
@matq007 matq007 mentioned this issue Jul 10, 2023
Merged
10 tasks
@matq007 matq007 closed this as completed Jul 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@matq007