Additional arguments for slurm launcher

[nviets]

I use a standard tmpl file with tools like clustermq to set the partition and R_HOME variables. How can I do the same with crew.cluster?

#!/bin/sh
#SBATCH --job-name={{ job_name }}
#SBATCH --partition={{ partition | MyBatchPartition }}
#SBATCH --output={{ log_file | /dev/null }} # you can add .%a for array index
#SBATCH --error={{ log_file | /dev/null }}
#SBATCH --mem-per-cpu={{ memory | 1024 }}
#SBATCH --array=1-{{ n_jobs }}
#SBATCH --cpus-per-task={{ n_cores | 2 }}
ulimit -v $(( 1024 * {{ memory | 1024 }} ))
CMQ_AUTH={{ auth }} ${MY_CUSTOM_R_HOME}/bin/R --no-save --no-restore -e 'clustermq:::worker("{{ master }}")'

There are already arguments some of the SBATCH setting (e.g. slurm_cpus_per_task), but I'm not sure how to set the partition and a customer R path for remote workers. Thank you for any tips!

[nviets]

Maybe the answer is here, but I'm not sure. Sorry I wasn't keeping up with the thread where I was tagged.

[nviets]

From the earlier discussion, i see how to use:

controller <- crew_controller_slurm(
    ...
    slurm_args = list(
        account = "project_00001",
        `gres:nvme` = 100,
        M = "large",
        partition = "MyBatchPartition"
    )
)

to get:

#SBATCH --account="project_00001"
#SBATCH --gres:nvme=100
#SBATCH -M="large"
#SBATCH --partition="MyBatchPartition"

But I'm still not sure how to set a specific R path. In the traditional tmpl interface, I use the MY_CUSTOM_R_HOME variable to control it.

[nviets]

If there's another way to designate the child R path, I suppose that would work just as well. For some background, I use nix to manage my HPC environments, and the parent and child session R_HOME aren't necessarily the same. Moreover, there are typically many, non-standard paths to R on the machines, which I need to set precisely.

[wlandau]

You can add arbitrary lines to the job script with the script_lines argument of crew_controller_slurm(). The development version of crew.clusteralso now has a separateslurm_partitionargument for the partition. See https://wlandau.github.io/crew.cluster/reference/crew_controller_slurm.html.crew.clusterdoes not support aslurm_args` list.