sort: cannot create temporary file in ''

[orionzhou]

in this code:

  genomeCoverageBed -ibam SPT5_INPUT_R2.mLb.clN.sorted.bam -bg -scale $SCALE_FACTOR -pc  | sort -k1,1 -k2,2n >  SPT5_INPUT_R2.mLb.clN
.bedGraph

the temporary directory location is not specified for the command "sort", which results in:

Command error:
sort: cannot create temporary file in '': No such file or directory

Is it possible to insert the TMP location after sort: sort -T /tmp/?

[drpatelh]

Hi @orionzhou . Good idea 👍 Although it would have to be ./tmp/ so it's created in the relevant Nextflow work/ directory. Mainly because all platforms/systems may not have a /tmp/ directory. There may be other ways to do this too e.g. using the Nextflow env scope but that needs some testing.

The easy fix for now should just be to export that variable before you run the command for the pipeline:
export TMPDIR=/tmp/

Let me know if that works.

[orionzhou]

Thanks for the reply, indeed by setting TMPDIR=/tmp/ solves the problem!

However, I also tried to set TMPDIR to my scratch space but it gave this error:
sort: cannot create temporary file in '/scratch.global/zhoux379/nf/tmp': No such file or directory

I can make sure that this directory "/scratch.global/zhoux379/nf/tmp" exists and is writable:

(nextflow) zhoux379@ln0003:nf $ ll /scratch.global/zhoux379/nf/
total 192
drwxr-sr-x   2 zhoux379 springer  4096 Nov 22 10:13 singularity
drwxrwxrwx  11 zhoux379 springer 12288 Nov 22 10:22 tmp
drwxr-sr-x 148 zhoux379 springer 20480 Nov 22 10:13 work

Any clue how I can get this to work? I'm using singularity with "autoMounts = true", could it be because singularity didn't mount the scratch space?

[drpatelh]

Hi @orionzhou . Yes, its probably because you need to add a custom bind path for Singularity to mount that directory. Can you send me the full command you are using please?

One option is to create a file called custom.config with the options below and add an additional parameter to the command i.e. -c custom.config:

singularity {
  enabled = true
  autoMounts = true
  runOptions = '-B /scratch.global/zhoux379/nf/tmp/'
}

env {
  TMPDIR=/scratch.global/zhoux379/nf/tmp/
}

With these settings, you wont have to manually export TMPDIR before you run the command but I think you will have to create /scratch.global/zhoux379/nf/tmp/ first though.

[orionzhou]

Hi @drpatelh : wow that worked like a charm! Thanks very much for the useful instruction!

[alirizaaribas-ibg]

Hello,
I have a problem about TMP but in R compiled app: "No space left on device". My archive has 77TB space. Could you please help me? Also I did set these parameters:

export _JAVA_OPTIONS="-Djava.io.tmpdir=/archive/tmp"

TMPDIR=/archive/tmp
TEMP=/archive/tmp
TMP=/archive/tmp
export TMPDIR TMP TEMP

#in .Renviron .Rprofile files
TMP=/archive/tmp
TMPDIR=/archive/tmp
TEMP=/archive/tmp
TEMPDIR=/archive/tmp

Error in log file:

Caused by:
  Process `PHANTOMPEAKQUALTOOLS (SV-HUC-1_MLL3_IP_R1)` terminated with an error exit status (1)

Command executed:

  RUN_SPP=`which run_spp.R`
  Rscript -e "library(caTools); source(\"$RUN_SPP\")" -c="SV-HUC-1_MLL3_IP_R1.mLb.clN.sorted.bam" -savp="SV-HUC-1_MLL3_IP_R1.spp.pdf" -savd="SV-HUC-1_MLL3_IP_R1.spp.Rdata" -out="SV-HUC-1_MLL3_IP_R1.spp.out" -p=6
  cp spp_correlation_header.txt SV-HUC-1_MLL3_IP_R1_spp_correlation_mqc.tsv
  Rscript -e "load('SV-HUC-1_MLL3_IP_R1.spp.Rdata'); write.table(crosscorr\$cross.correlation, file=\"SV-HUC-1_MLL3_IP_R1_spp_correlation_mqc.tsv\", sep=",", quote=FALSE, row.names=FALSE, col.names=FALSE,append=TRUE)"
  
  awk -v OFS='	' '{print "SV-HUC-1_MLL3_IP_R1", $9}' SV-HUC-1_MLL3_IP_R1.spp.out | cat spp_nsc_header.txt - > SV-HUC-1_MLL3_IP_R1_spp_nsc_mqc.tsv
  awk -v OFS='	' '{print "SV-HUC-1_MLL3_IP_R1", $10}' SV-HUC-1_MLL3_IP_R1.spp.out | cat spp_rsc_header.txt - > SV-HUC-1_MLL3_IP_R1_spp_rsc_mqc.tsv

Command exit status:
  1

Command output:
  ################
  ChIP data: SV-HUC-1_MLL3_IP_R1.mLb.clN.sorted.bam 
  Control data: NA 
  strandshift(min): -500 
  strandshift(step): 5 
  strandshift(max) 1500 
  user-defined peak shift NA 
  exclusion(min): 10 
  exclusion(max): NaN 
  num parallel nodes: 6 
  FDR threshold: 0.01 
  NumPeaks Threshold: NA 
  Output Directory: . 
  narrowPeak output file name: NA 
  regionPeak output file name: NA 
  Rdata filename: SV-HUC-1_MLL3_IP_R1.spp.Rdata 
  plot pdf filename: SV-HUC-1_MLL3_IP_R1.spp.pdf 
  result filename: SV-HUC-1_MLL3_IP_R1.spp.out 
  Overwrite files?: FALSE
  
  Reading ChIP tagAlign/BAM file SV-HUC-1_MLL3_IP_R1.mLb.clN.sorted.bam 

Command error:
  Warning message:
  package ‘caTools’ was built under R version 3.6.3 
  Loading required package: Rcpp
  sh: 1: cannot create /tmp/RtmpJAK3bO/SV-HUC-1_MLL3_IP_R1.mLb.clN.sorted.tagAlign1f1c9ff676: No space left on device
  [W::bam_hdr_read] EOF marker is absent. The input is probably truncated
  Error converting BAM to tagalign file: /tmp/RtmpJAK3bO/SV-HUC-1_MLL3_IP_R1.mLb.clN.sorted.bam1f1c9ff676
  Warning messages:
  1: package ‘Rcpp’ was built under R version 3.6.3 
  2: In file.append(to[okay], from[okay]) : write error during file append

Work dir:
  /fastqfiles/work/87/151f746e841abdcd2f889d97be09cb