[pitch] Renku 2.0 Compute Sessions #35
Conversation
lokijuhy
changed the title
| - There are a whole bunch of restrictions for what must be in the image for it to work on RenkuLab. We will have to document these limitations for users. | ||
| - We *think* that any image coming from the Jupyter stack should work | ||
| - ... Or at least Renku base images | ||
| - Only publicly accessible images are supported |
There was a problem hiding this comment.
This should either say "... will be supported as a part of this build, and authentication options added later." or add a "nice to have" to deal with authentication.
| ##### Priority 4: [for the future/out of scope] Search & select from preexisting RenkuLab environments | ||
|
|
||
| #### User Flow: Start Session | ||
|
|
There was a problem hiding this comment.
there is no provision here for selecting the resource pool / class?
| - When a user creates a new project, the project has no sessions until a user adds one. | ||
| - [nice to have] Provide shortcuts for adding a session from the RenkuLab default environments | ||
| - A user can create as many sessions as they want in a project. | ||
| - A user can run 1 session per environment per project (similar to our restriction now of 1 session per user per project) |
There was a problem hiding this comment.
So we need to actually prevent the user from creating multiple sessions from the same environment - should this be mentioned in the user flow?
| - → Solved by offering RenkuLab built-in session “templates” (name TBD) as the fastest route to adding a session to a project | ||
| - When I create a new Renku project, I want to get started right away with a pre-built environment- **I don’t want to have to wait for the environment image to be built** to get started. | ||
| - → Same as above | ||
| - When I start a session, I want to **clearly understand what environment image the session will use and what is installed in it**, so I am not confused where the compute environment comes from. |
There was a problem hiding this comment.
imho this doesn't really address the "clearly understand what is installed" part?
| - date of creation | ||
| - [optional] description (e.g. why it is useful) | ||
| - [optional] keywords/tags to aid in searching (for example, I might tag an environment with `#python` `#pytorch` to give a quick impression of what is in the image) | ||
|
|
There was a problem hiding this comment.
In a very general sense, I feel that these changes are centered on the actual Docker images Renku is using. This is very helpful, however there is something that we cannot do with docker images: "mixing" the environments.
Let's put a concrete case:
Let's say I have an app for "paraview" GUI viewer and an app for a finite element software and another and finally one for Solidipes. The user would wish to create a single session with /paraview, /solidipes, /fe and /lab automatically. In the current Renku, I would do with python plugins and jupyter proxy: this is conditioned by the currently installed python packages. But what is proposed here, do not allow that, I think.
How am I supposed to mix environments ? Can I create as session with two environments ? From your proposal, I think that someone has to make a Dockerfile (somewhere) and describe the installation for the mix.
However, it would make sense to split the process of installation (and not requiring the users to write the Dockerfile for any combination).
I think the metadata attached to an environment should also include possible (compatible) requirements. That would allow the dependencies to be somehow selected while creating the session (and select them in a catalog).
Another possibility could be to have two types of "environments": one for the Dockerfile level and one for the requirements. (Sadly, this is orthogonal to the request that users do not want to wait installation times when spawning a new session....).
To summarize: I think it is important to think of a way/procedure, which allows mixing environment/apps easily.
There was a problem hiding this comment.
yikes, we left this unanswered for a little too long. You are totally right @anciaux that mixing environments should be possible, but I think that has to be done after this initial work (btw, this is already implemented). There are two possible options for what you are proposing:
- pluggable front-ends (the user provides a docker image with their runtime, but another independent front-end is added to their session)
- multi-container sessions
I think both of these have use-cases and I would like to see us work out how to support them. In the case of front-ends (option 1), they would need to be configured and maintained by the admins of the platform. The disadvantage is that this would be less flexible for the user, but probably lead to better UX because it would be more controlled and carefully managed. Option 2 gives more flexibility but the containers would not share the runtime so in the case of e.g. python or other packages needed across containers, it would not be straightforward to share them.
lokijuhy
changed the title
This pitch introduces how we plan to build Compute Sessions in the next generation of Renku ("Renku 2.0").