Design input for the IMP exec subcommand

[grondo]

Format and contents of the input for the IMP exec subcommand still remains to be fully designed.

Because of the IMP's privsep interface, we can define the input in terms of required and optional keys and values, since the format that the "privileged parent" (pIMP) takes is already in the well-defined struct kv format. This allows the "unprivilged child of IMP" (chIMP) to be more flexible in the exact format of its input. In fact it may make sense to have multiple exec input "readers" that can be loaded by the chIMP, as long as the data can be formatted into the struct kv that the parent expects. (For instance, initially a "testing" format could be used. Future implementations may be able to load a "reader" from their enclosing instance).

[grondo]

From RFC 15, required input to the IMP includes

• Input from instance owner:
  • R_local - Table identifying the locally assigned resources
  • options - Table of key,value pairs identifying optional behavior of the IMP requested by the instance
• Signed input from guest user (J):
  • security envelope - security implementation dependent security/signature/credential information (should be transparent to the IMP, decoded as part of verification)
  • jobspec - jobspec per RFC14
  • options - table of key,value pairs identifying optional behavior of the IMP requested by the user
  • receipient - Intended receipient of job request (should equal instance owner, uid running the IMP)
  • guest userid - may be part of security envelope
  • uuid - unique identifier for this job request
  • job shell path - may be part of jobspec(?)

Strategy for J at this time may be JWT or similar dotted base64 as implemented in the signer interface from the flux-security project, and this data will be passed to the privileged parent under a single key, likely just J = x.y.z.

This leaves just the design of the "instance owner input to the IMP".

For the purposes of starting simple, I propose that R_local initially take the form of a table of resource types (e.g. core/cpu, memory, gpu, etc), with each table entry comprising a list of logical ids granted to the job's resource set. This will hopefully keep initial implementation very simple.

E.g., in JSON a job using cores and memory might have a R_local that looks something like:

{ 
  "cpu": [  0,1 ],
  "memory":  [ "2G" ],
}

While this representation may not actually reflect the topological hierarchy, any topological resource selection would be preserved via use of the logical ids of resources. For memory, it is not likely at this time that a hierarchical memory container works on Linux anyway (AFAIK, the memcg just limits RAM use, and we would rely on Linux memory allocator to grab local pages anyway)

In the long term, it probably makes sense to switch R_local to using the hwloc XML format, but at this point I think that approach would take too long.