Skip to content
This repository has been archived by the owner on Mar 23, 2023. It is now read-only.

CreateCondaEnvironmentException upon invoking snakemake in pore-c-snakemake environment #35

Open
gchchung opened this issue Jun 29, 2022 · 2 comments

Comments

@gchchung
Copy link

gchchung commented Jun 29, 2022

Used the instructions provided in Readme to create an environment called 'pore-c-snakemake':

$ git clone https://github.com/nanoporetech/Pore-C-Snakemake.git
$ cd Pore-C-Snakemake
$ conda env create

A cup of coffee later,

$ conda activate pore-c-snakemake

Note the - instead of _.

Dry run works. However, both test run (with the provided test data) and own data (by indicating filepaths in basecalls.tsv and references.tsv) returned what appears to be the same CreateCondaEnvironmentException. Using a SLURM interactive srun:

$ srun --nodes=1 --cpus-per-task=8 --time=02:00:00 -i --mem-per-cpu=32GB snakemake --use-conda --cores 8 salsa
srun: job 21704971 queued and waiting for resources
srun: job 21704971 has been allocated resources
Building DAG of jobs...
Removing incomplete Conda environment envs/pore_c.yml...
Creating conda environment envs/pore_c.yml...
Downloading and installing remote packages.
CreateCondaEnvironmentException:
Could not create conda environment from /scratch/gc95/Pore-C-Snakemake/rules/../envs/pore_c.yml:
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working...
  File "/scratch/gc95/miniconda3/envs/pore-c-snakemake/lib/python3.8/site-packages/snakemake/deployment/conda.py", line 340, in create
slurmstepd: error: Detected 1 oom-kill event(s) in StepId=21704971.0. Some of your processes may have been killed by the cgroup out-of-memory handler.
srun: error: gv005: task 0: Out Of Memory
srun: launch/slurm: _step_signal: Terminating StepId=21704971.0

Wondering which package in the yml might be causing an incompatibility?

@gchchung
Copy link
Author

gchchung commented Jun 29, 2022

The pore_c.yml file associated with the CreateCondaEnvironmentException contains the following.

channels:
- conda-forge
- bioconda
- defaults
dependencies:
- pore-c==0.4.0
- python==3.8

Probably not the fault of python 3.8? So attempt to install pore-c using conda install.

$ conda install -c bioconda pore-c
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: \
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versionsThe following specifications were found to be incompatible with your system:

  - feature:/linux-64::__glibc==2.28=0
  - feature:|@/linux-64::__glibc==2.28=0

Your installed version is: 2.28

So currently stuck here.

@gchchung
Copy link
Author

gchchung commented Jun 30, 2022

Attempt to rule out low memory (indicated by the oom-kill) by assigning up to 512 GB to the srun.

$ srun --nodes=1 --cpus-per-task=4 --time=02:00:00 -i --mem=512GB snakemake --use-conda  test -j 4 --config=output_dir=results.test
srun: job 21712655 queued and waiting for resources
srun: job 21712655 has been allocated resources
Building DAG of jobs...
Creating conda environment envs/juicer_tools.yml...
Downloading and installing remote packages.
CreateCondaEnvironmentException:
Could not create conda environment from /scratch/gc95/Pore-C-Snakemake/rules/../envs/juicer_tools.yml:
Collecting package metadata (repodata.json): ...working...
  File "/scratch/gc95/miniconda3/envs/pore-c-snakemake/lib/python3.8/site-packages/snakemake/deployment/conda.py", line 340, in create
slurmstepd: error: Detected 1 oom-kill event(s) in StepId=21712655.0. Some of your processes may have been killed by the cgroup out-of-memory handler.
srun: error: gr062: task 0: Out Of Memory
srun: launch/slurm: _step_signal: Terminating StepId=21712655.0

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant