Skip to content

Rename pod directive to podOptions #2747

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/config.rst
Original file line number Diff line number Diff line change
@@ -396,7 +396,7 @@ serviceAccount Defines the Kubernetes `service account name <https://kubern
launchDir Defines the path where the workflow is launched and the user data is stored. This must be a path in a shared K8s persistent volume (default: ``<volume-claim-mount-path>/<user-name>``.
workDir Defines the path where the workflow temporary data is stored. This must be a path in a shared K8s persistent volume (default:``<user-dir>/work``).
projectDir Defines the path where Nextflow projects are downloaded. This must be a path in a shared K8s persistent volume (default: ``<volume-claim-mount-path>/projects``).
pod Allows the definition of one or more pod configuration options such as environment variables, config maps, secrets, etc. It allows the same settings as the :ref:`process-pod` process directive.
podOptions Defines one or more pod configuration options such as environment variables, config maps, secrets, etc. It allows the same options as the :ref:`process-podOptions` process directive.
pullPolicy Defines the strategy to be used to pull the container image e.g. ``pullPolicy: 'Always'``.
runAsUser Defines the user ID to be used to run the containers. Shortcut for the ``securityContext`` option.
securityContext Defines the `security context <https://kubernetes.io/docs/tasks/configure-pod-container/security-context/>`_ for all pods.
10 changes: 5 additions & 5 deletions docs/kubernetes.rst
Original file line number Diff line number Diff line change
@@ -118,13 +118,13 @@ with the directory in the volume to be mounted (default: ``/``).
one specified in your Nextflow configuration file.
Note also that the ``run`` command does not support the ``-v`` option.

.. tip:: It is also possible to mount multiple volumes using the ``pod`` directive, setting such as ``k8s.pod = [ [volumeClaim: "other-pvc", mountPath: "/other" ]]``
.. tip:: It is also possible to mount multiple volumes using the ``podOptions`` directive, setting such as ``k8s.podOptions = [ [volumeClaim: "other-pvc", mountPath: "/other" ]]``

Pod settings
============
Pod options
===========

The process :ref:`process-pod` directive allows the definition of pods specific settings, such as environment variables,
secrets and config maps when using the :ref:`k8s-executor` executor. See the :ref:`process-pod` directive for more details.
The process :ref:`process-podOptions` directive allows the definition of pod-specific options, such as environment variables,
secrets and config maps when using the :ref:`k8s-executor` executor. See the :ref:`process-podOptions` directive for more details.

Limitation
==========
25 changes: 14 additions & 11 deletions docs/process.rst
Original file line number Diff line number Diff line change
@@ -1352,7 +1352,7 @@ The directives are:
* `memory`_
* `module`_
* `penv`_
* `pod`_
* `podOptions`_
* `publishDir`_
* `queue`_
* `scratch`_
@@ -1981,18 +1981,18 @@ cluster documentation or contact your admin to learn more about this.
See also: `cpus`_, `memory`_, `time`_


.. _process-pod:
.. _process-podOptions:

pod
---
podOptions
----------

The ``pod`` directive allows the definition of pods specific settings, such as environment variables, secrets
The ``podOptions`` directive allows the definition of pod-specific options, such as environment variables, secrets
and config maps when using the :ref:`k8s-executor` executor.

For example::

process your_task {
pod env: 'FOO', value: 'bar'
podOptions env: 'FOO', value: 'bar'

'''
echo $FOO
@@ -2001,7 +2001,7 @@ For example::

The above snippet defines an environment variable named ``FOO`` which value is ``bar``.

The ``pod`` directive allows the definition of the following options:
The ``podOptions`` directive supports the following options:

================================================= =================================================
``label: <K>, value: <V>`` Defines a pod label with key ``K`` and value ``V``.
@@ -2023,17 +2023,20 @@ The ``pod`` directive allows the definition of the following options:
``priorityClassName: <V>`` Specifies the `priority class name <https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/>`_ for pods.
================================================= =================================================

When defined in the Nextflow configuration file, a pod setting can be defined using the canonical
When defined in the Nextflow configuration file, a pod option can be defined using the canonical
associative array syntax. For example::

process {
pod = [env: 'FOO', value: 'bar']
podOptions = [env: 'FOO', value: 'bar']
}

When more than one setting needs to be provides they must be enclosed in a list definition as shown below::
When more than one option needs to be provided they must be enclosed in a list as shown below::

process {
pod = [ [env: 'FOO', value: 'bar'], [secret: 'my-secret/key1', mountPath: '/etc/file.txt'] ]
podOptions = [
[env: 'FOO', value: 'bar'],
[secret: 'my-secret/key1', mountPath: '/etc/file.txt']
]
}


11 changes: 10 additions & 1 deletion modules/nextflow/src/main/groovy/nextflow/k8s/K8sConfig.groovy
Original file line number Diff line number Diff line change
@@ -47,7 +47,16 @@ class K8sConfig implements Map<String,Object> {
K8sConfig(Map<String,Object> config) {
target = config ?: Collections.<String,Object>emptyMap()

this.podOptions = createPodOptions(target.pod)
// -- backward compatibility
if( target.pod ) {
log.warn "Process directive 'pod' has been renamed to 'podOptions'"
target.put('podOptions', target.pod)
}

// -- initialize pod options
this.podOptions = createPodOptions(target.podOptions)

// -- add pod volume claim
if( getStorageClaimName() ) {
final name = getStorageClaimName()
final mount = getStorageMountPath()
Original file line number Diff line number Diff line change
@@ -279,17 +279,22 @@ class K8sDriverLauncher {
}

def config = loadConfig(pipelineName)

// normalize pod entries
def k8s = config.k8s

if( !k8s.isSet('pod') )
k8s.pod = []
else if( k8s.pod instanceof Map ) {
k8s.pod = [ k8s.pod ]
// -- backward compatibility
if( k8s.pod ) {
log.warn "Config setting k8s.pod has been renamed to k8s.podOptions"
k8s.put('podOptions', k8s.pod)
k8s.remove('pod')
}
else if( !(k8s.pod instanceof List) )
throw new IllegalArgumentException("Illegal k8s.pod configuratun value: ${k8s.pod}")

// -- normalize pod options
if( !k8s.podOptions )
k8s.podOptions = []
else if( k8s.podOptions instanceof Map )
k8s.podOptions = [ k8s.podOptions ]
else if( !(k8s.podOptions instanceof List) )
throw new IllegalArgumentException("Illegal k8s.podOptions configuratun value: ${k8s.podOptions}")

// -- use the volume claims specified in the command line
// to populate the pod config
@@ -303,7 +308,7 @@ class K8sDriverLauncher {
k8s.storageMountPath = path
}
else {
k8s.pod.add( [volumeClaim: name, mountPath: path] )
k8s.podOptions.add( [volumeClaim: name, mountPath: path] )
}
}

@@ -318,7 +323,7 @@ class K8sDriverLauncher {
k8s.storageMountPath = path
}
else if( !cmd.volMounts ) {
k8s.pod.add( [volumeClaim: name, mountPath: path] )
k8s.podOptions.add( [volumeClaim: name, mountPath: path] )
}
}
// remove it
@@ -339,8 +344,8 @@ class K8sDriverLauncher {
k8s.workDir = config.workDir

// -- some cleanup
if( !k8s.pod )
k8s.remove('pod')
if( !k8s.podOptions )
k8s.remove('podOptions')

if( !k8s.storageClaimName )
k8s.remove('storageClaimName')
Original file line number Diff line number Diff line change
@@ -380,7 +380,7 @@ class TaskConfig extends LazyMap implements Cloneable {
}

PodOptions getPodOptions() {
new PodOptions((List)get('pod'))
new PodOptions((List)get('podOptions'))
}

AcceleratorResource getAccelerator() {
Original file line number Diff line number Diff line change
@@ -60,17 +60,18 @@ class ProcessConfig implements Map<String,Object>, Cloneable {
'errorStrategy',
'executor',
'ext',
'machineType',
'queue',
'label',
'machineType',
'maxErrors',
'maxForks',
'maxRetries',
'memory',
'module',
'penv',
'pod',
'podOptions',
'publishDir',
'queue',
'scratch',
'shell',
'storeDir',
@@ -93,7 +94,13 @@ class ProcessConfig implements Map<String,Object>, Cloneable {
* Names of directives that can be used more than once in the process definition
*/
@PackageScope
static final List<String> repeatableDirectives = ['label','module','pod','publishDir']
static final List<String> repeatableDirectives = [
'label',
'module',
'pod',
'podOptions',
'publishDir'
]

/**
* Default directives values
@@ -825,22 +832,22 @@ class ProcessConfig implements Map<String,Object>, Cloneable {
}

/**
* Allow use to specify K8s `pod` options
* Allow the user to specify K8s pod options
*
* @param entry
* A map object representing pod config options
* @return
* The {@link ProcessConfig} instance itself
*/
ProcessConfig pod( Map entry ) {
ProcessConfig podOptions( Map entry ) {

if( !entry )
return this

def allOptions = (List)configProperties.get('pod')
def allOptions = (List)configProperties.get('podOptions')
if( !allOptions ) {
allOptions = new ConfigList()
configProperties.put('pod', allOptions)
configProperties.put('podOptions', allOptions)
}

allOptions.add(entry)
25 changes: 14 additions & 11 deletions modules/nextflow/src/test/groovy/nextflow/k8s/K8sConfigTest.groovy
Original file line number Diff line number Diff line change
@@ -89,17 +89,20 @@ class K8sConfigTest extends Specification {

when:
cfg = new K8sConfig([
storageClaimName: 'pvc-2',
storageMountPath: '/data',
pod: [ [volumeClaim:'foo', mountPath: '/here'],
[volumeClaim: 'bar', mountPath: '/there']] ])
storageClaimName: 'pvc-2',
storageMountPath: '/data',
podOptions: [
[volumeClaim:'foo', mountPath: '/here'],
[volumeClaim: 'bar', mountPath: '/there']
]
])
then:
cfg.getStorageClaimName() == 'pvc-2'
cfg.getStorageMountPath() == '/data'
cfg.getPodOptions().getVolumeClaims() == [
new PodVolumeClaim('pvc-2', '/data'),
new PodVolumeClaim('foo', '/here'),
new PodVolumeClaim('bar', '/there')
new PodVolumeClaim('pvc-2', '/data'),
new PodVolumeClaim('foo', '/here'),
new PodVolumeClaim('bar', '/there')
] as Set


@@ -185,7 +188,7 @@ class K8sConfigTest extends Specification {


when:
opts = new K8sConfig(pod: [ [pullPolicy: 'Always'], [env: 'HELLO', value: 'WORLD'] ]).getPodOptions()
opts = new K8sConfig(podOptions: [ [pullPolicy: 'Always'], [env: 'HELLO', value: 'WORLD'] ]).getPodOptions()
then:
opts.getImagePullPolicy() == 'Always'
opts.getEnvVars() == [ PodEnv.value('HELLO','WORLD') ] as Set
@@ -285,20 +288,20 @@ class K8sConfigTest extends Specification {
def 'should create k8s config with one volume claim' () {

when:
def cfg = new K8sConfig( pod: [runAsUser: 1000] )
def cfg = new K8sConfig( podOptions: [runAsUser: 1000] )
then:
cfg.getPodOptions().getSecurityContext() == new PodSecurityContext(1000)
cfg.getPodOptions().getVolumeClaims().size() == 0

when:
cfg = new K8sConfig( pod: [volumeClaim: 'nf-0001', mountPath: '/workspace'] )
cfg = new K8sConfig( podOptions: [volumeClaim: 'nf-0001', mountPath: '/workspace'] )
then:
cfg.getPodOptions().getSecurityContext() == null
cfg.getPodOptions().getVolumeClaims() == [new PodVolumeClaim('nf-0001', '/workspace')] as Set


when:
cfg = new K8sConfig( pod: [
cfg = new K8sConfig( podOptions: [
[runAsUser: 1000],
[volumeClaim: 'nf-0001', mountPath: '/workspace'],
[volumeClaim: 'nf-0002', mountPath: '/data', subPath: '/home']
Original file line number Diff line number Diff line change
@@ -136,17 +136,17 @@ class K8sDriverLauncherTest extends Specification {
def 'should create launcher spec' () {

given:
def pod = Mock(PodOptions)
pod.getVolumeClaims() >> [ new PodVolumeClaim('pvc-1', '/mnt/path/data') ]
pod.getMountConfigMaps() >> [ new PodMountConfig('cfg-2', '/mnt/path/cfg') ]
pod.getAutomountServiceAccountToken() >> true
def podOptions = Mock(PodOptions)
podOptions.getVolumeClaims() >> [ new PodVolumeClaim('pvc-1', '/mnt/path/data') ]
podOptions.getMountConfigMaps() >> [ new PodMountConfig('cfg-2', '/mnt/path/cfg') ]
podOptions.getAutomountServiceAccountToken() >> true

def k8s = Mock(K8sConfig)
k8s.getNextflowImageName() >> 'the-image'
k8s.getLaunchDir() >> '/the/user/dir'
k8s.getWorkDir() >> '/the/work/dir'
k8s.getProjectDir() >> '/the/project/dir'
k8s.getPodOptions() >> pod
k8s.getPodOptions() >> podOptions

def driver = Spy(K8sDriverLauncher)
driver.runName = 'foo-boo'
@@ -161,7 +161,11 @@ class K8sDriverLauncherTest extends Specification {
spec == [
apiVersion: 'v1',
kind: 'Pod',
metadata: [name:'foo-boo', namespace:'foo', labels:[app:'nextflow', runName:'foo-boo']],
metadata: [
name:'foo-boo',
namespace:'foo',
labels: [app:'nextflow', runName:'foo-boo']
],
spec: [
restartPolicy: 'Never',
containers: [
@@ -194,16 +198,16 @@ class K8sDriverLauncherTest extends Specification {
def 'should use user provided pod image' () {

given:
def pod = Mock(PodOptions)
pod.getVolumeClaims() >> [ new PodVolumeClaim('pvc-1', '/mnt/path/data') ]
pod.getMountConfigMaps() >> [ new PodMountConfig('cfg-2', '/mnt/path/cfg') ]
pod.getAutomountServiceAccountToken() >> true
def podOptions = Mock(PodOptions)
podOptions.getVolumeClaims() >> [ new PodVolumeClaim('pvc-1', '/mnt/path/data') ]
podOptions.getMountConfigMaps() >> [ new PodMountConfig('cfg-2', '/mnt/path/cfg') ]
podOptions.getAutomountServiceAccountToken() >> true

def k8s = Mock(K8sConfig)
k8s.getLaunchDir() >> '/the/user/dir'
k8s.getWorkDir() >> '/the/work/dir'
k8s.getProjectDir() >> '/the/project/dir'
k8s.getPodOptions() >> pod
k8s.getPodOptions() >> podOptions

def driver = Spy(K8sDriverLauncher)
driver.runName = 'foo-boo'
@@ -219,7 +223,11 @@ class K8sDriverLauncherTest extends Specification {
spec == [
apiVersion: 'v1',
kind: 'Pod',
metadata: [name:'foo-boo', namespace:'foo', labels:[app:'nextflow', runName:'foo-boo']],
metadata: [
name:'foo-boo',
namespace:'foo',
labels: [app:'nextflow', runName:'foo-boo']
],
spec: [
restartPolicy: 'Never',
containers: [
@@ -311,7 +319,7 @@ class K8sDriverLauncherTest extends Specification {
then:
1 * driver.loadConfig(NAME) >> CFG_EMPTY
config.process.executor == 'k8s'
config.k8s.pod == null
config.k8s.podOptions == null
config.k8s.storageMountPath == null
config.k8s.storageClaimName == null

@@ -336,16 +344,15 @@ class K8sDriverLauncherTest extends Specification {
config.process.executor == 'k8s'
config.k8s.storageClaimName == 'pvc-1'
config.k8s.storageMountPath == '/this'
config.k8s.pod == [ [volumeClaim: 'pvc-2', mountPath: '/that'] ]
config.k8s.podOptions == [ [volumeClaim: 'pvc-2', mountPath: '/that'] ]
and:
new K8sConfig(config.k8s).getStorageClaimName() == 'pvc-1'
new K8sConfig(config.k8s).getStorageMountPath() == '/this'
new K8sConfig(config.k8s).getPodOptions() == new PodOptions([
[volumeClaim:'pvc-1', mountPath: '/this'],
[volumeClaim:'pvc-2', mountPath: '/that']
[volumeClaim:'pvc-1', mountPath: '/this'],
[volumeClaim:'pvc-2', mountPath: '/that']
])


when:
driver.cmd = new CmdKubeRun(volMounts: ['xyz:/this'] )
config = driver.makeConfig(NAME).toMap()
@@ -354,16 +361,15 @@ class K8sDriverLauncherTest extends Specification {
config.process.executor == 'k8s'
config.k8s.storageClaimName == 'xyz'
config.k8s.storageMountPath == '/this'
config.k8s.pod == null
config.k8s.podOptions == null
and:
and:
new K8sConfig(config.k8s).getStorageClaimName() == 'xyz'
new K8sConfig(config.k8s).getStorageMountPath() == '/this'
new K8sConfig(config.k8s).getPodOptions() == new PodOptions([
[volumeClaim:'xyz', mountPath: '/this']
[volumeClaim:'xyz', mountPath: '/this']
])


when:
driver.cmd = new CmdKubeRun(volMounts: ['xyz', 'bar:/mnt/bar'] )
config = driver.makeConfig(NAME).toMap()
@@ -372,13 +378,13 @@ class K8sDriverLauncherTest extends Specification {
config.process.executor == 'k8s'
config.k8s.storageClaimName == 'xyz'
config.k8s.storageMountPath == null
config.k8s.pod == [ [volumeClaim: 'bar', mountPath: '/mnt/bar'] ]
config.k8s.podOptions == [ [volumeClaim: 'bar', mountPath: '/mnt/bar'] ]
and:
new K8sConfig(config.k8s).getStorageClaimName() == 'xyz'
new K8sConfig(config.k8s).getStorageMountPath() == '/workspace'
new K8sConfig(config.k8s).getPodOptions() == new PodOptions([
[volumeClaim:'xyz', mountPath: '/workspace'],
[volumeClaim:'bar', mountPath: '/mnt/bar']
[volumeClaim:'xyz', mountPath: '/workspace'],
[volumeClaim:'bar', mountPath: '/mnt/bar']
])

}
@@ -417,8 +423,7 @@ class K8sDriverLauncherTest extends Specification {
config.process.executor == 'k8s'
config.k8s.storageClaimName == 'pvc-1'
config.k8s.storageMountPath == '/this'
config.k8s.pod == [ [volumeClaim: 'pvc-2', mountPath: '/that'] ]

config.k8s.podOptions == [ [volumeClaim: 'pvc-2', mountPath: '/that'] ]

when:
driver.cmd = new CmdKubeRun(volMounts: ['xyz:/this'] )
@@ -428,12 +433,12 @@ class K8sDriverLauncherTest extends Specification {
config.process.executor == 'k8s'
config.k8s.storageClaimName == 'xyz'
config.k8s.storageMountPath == '/this'
config.k8s.pod == null
config.k8s.podOptions == null
and:
new K8sConfig(config.k8s).getStorageClaimName() == 'xyz'
new K8sConfig(config.k8s).getStorageMountPath() == '/this'
new K8sConfig(config.k8s).getPodOptions() == new PodOptions([
[volumeClaim:'xyz', mountPath: '/this']
[volumeClaim:'xyz', mountPath: '/this']
])

}
Original file line number Diff line number Diff line change
@@ -620,11 +620,13 @@ class K8sTaskHandlerTest extends Specification {
proc.getName() >> 'hello-proc'
exec.getSession() >> sess
sess.getUniqueId() >> uuid
exec.getK8sConfig() >> [pod: [
exec.getK8sConfig() >> [
podOptions: [
[label: 'foo', value: 'bar'],
[label: 'app', value: 'nextflow'],
[label: 'x', value: 'hello_world']
]]
]
]

labels.app == 'nextflow'
labels.foo == 'bar'
Original file line number Diff line number Diff line change
@@ -536,17 +536,19 @@ class TaskConfigTest extends Specification {

when:
def process = new ProcessConfig(script)
process.pod secret: 'foo', mountPath: '/this'
process.pod secret: 'bar', env: 'BAR_XXX'
process.podOptions secret: 'foo', mountPath: '/this'
process.podOptions secret: 'bar', env: 'BAR_XXX'

then:
process.get('pod') == [
[secret: 'foo', mountPath: '/this'],
[secret: 'bar', env: 'BAR_XXX'] ]
process.get('podOptions') == [
[secret: 'foo', mountPath: '/this'],
[secret: 'bar', env: 'BAR_XXX']
]

process.createTaskConfig().getPodOptions() == new PodOptions([
[secret: 'foo', mountPath: '/this'],
[secret: 'bar', env: 'BAR_XXX'] ])
[secret: 'foo', mountPath: '/this'],
[secret: 'bar', env: 'BAR_XXX']
])

}

Original file line number Diff line number Diff line change
@@ -544,27 +544,27 @@ class ProcessConfigTest extends Specification {
}


def 'should apply pod configs' () {
def 'should apply pod options' () {

when:
def process = new ProcessConfig([:])
process.applyConfigDefaults( pod: [secret: 'foo', mountPath: '/there'] )
process.applyConfigDefaults( podOptions: [secret: 'foo', mountPath: '/there'] )
then:
process.pod == [
[secret: 'foo', mountPath: '/there']
process.podOptions == [
[secret: 'foo', mountPath: '/there']
]

when:
process = new ProcessConfig([:])
process.applyConfigDefaults( pod: [
[secret: 'foo', mountPath: '/here'],
[secret: 'bar', mountPath: '/there']
process.applyConfigDefaults( podOptions: [
[secret: 'foo', mountPath: '/here'],
[secret: 'bar', mountPath: '/there']
] )

then:
process.pod == [
[secret: 'foo', mountPath: '/here'],
[secret: 'bar', mountPath: '/there']
process.podOptions == [
[secret: 'foo', mountPath: '/here'],
[secret: 'bar', mountPath: '/there']
]

}