conda-env create is using /tmp

[katsikora]

snakePipes 3.0.0 - snakemake is calling conda create env and pointing it to /tmp rather than to condaEnvDir defined in the defaults.yaml

[katsikora]

snakePipes config function needs to be modified such that it modifies conda-prefix in the slurm profile file rather than in defaults.yaml

[WardDeb]

As of 3.0.0 condaEnvDir should not be set in defaults.yaml anymore, but should be set in the appropriate profile (wherever that lives).
defaults.yaml points to the local profile by default, and the conda-prefix there is set to /tmp. This is needed for the gh actions to work, and is a reasonable default (I think).

As for the snakePipes config function, this would work, but do we still need a function that makes changes to these profiles to begin with ?

[katsikora]

Hi Ward,

from my point of view, the user should be able to edit the config shipped with snakePipes. This is what the snakePipes config function is for. I'd just update it so that all the necessary variables can be edited in the snakemake profile without the user having to look for it in some custom location. Or do you have another solution for this?

The documentation would also have to be updated, e.g. in the quick start section.

Best,
Katarzyna

[katsikora]

Btw, snakePipes config has a setting that works for gha. To be honest, I don't like /tmp as default, I'd prefer at least /scratch/local .

[WardDeb]

from my point of view, the user should be able to edit the config shipped with snakePipes. This is what the snakePipes config function is for. I'd just update it so that all the necessary variables can be edited in the snakemake profile without the user having to look for it in some custom location. Or do you have another solution for this?

Agreed for defaults.yaml, though note that profiles are allowed to live outside of the snakePipes directory (in a specific directory, or in the default locations that snakemake expects them to be, /etc/xdg/snakemake/ & ~/.config/snakemake/), and I think it's not so nice to change these files. If you insist of keeping that perhaps a comment line (# changed by snakePipes config) or something similar would be appropriate..

[katsikora]

Ok to comment line after changing configuration.
I'd only change the conda-prefix in the snakemake profile file, if it was provided by the user via snakePipes config --condaEnvDir. I'm assuming that they are using the config that is shipped with snakePipes and has /tmp as conda-prefix.
If condaEnvDir is not provided by the user, because they have everything they need in their private snakemake profile, then I wouldn't change it at all.
Would you agree?

[WardDeb]

Makes sense yea, thanks ! 👍🏼