nebari-dev
Is there a way to disable conda-store? #1150
-
|
Hello. When deployed to AWS, the conda-store deployment uses a PVC backed by gp2 (EBS) as storage. However, the gp2 storage class only supports RWO mode, meaning it can only be mounted to one node. When creating a user notebook server, we are also mounting the same storage (conda-store) to the pod. This prevents the user notebook server to be scheduled to other nodes. If the node which has the PVC attached to it does not have enough resource, then the user notebook server pod will hang and failed to init. Is there any way to resolve this issue? Shall we use NFS for conda-store storage as well? Or the component can be disabled maybe when deploying a user notebook server? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments
-
|
@costrouc ☝🏽 |
Beta Was this translation helpful? Give feedback.
-
|
Hey @yuhuishi-convect I would say at the moment conda-store is difficult to disable since it is the central part to so many components since it managed the environments used within dask, dashboards, and jupyter notebooks. We are working on better ways of serving environments. Yes NFS is not a great solution and preferable would be to use docker environments or tarballs of the environment. NFS just doesn't scale well. Conda-Store is capable of serving these enviornments via docker and tarballs (already) but it is more about how this integrated within nebari which needs to be figured out. Do we upload the docker images to container storage? |
Beta Was this translation helpful? Give feedback.
Hey @yuhuishi-convect I would say at the moment conda-store is difficult to disable since it is the central part to so many components since it managed the environments used within dask, dashboards, and jupyter notebooks.
We are working on better ways of serving environments. Yes NFS is not a great solution and preferable would be to use docker environments or tarballs of the environment. NFS just doesn't scale well.
Conda-Store is capable of serving these enviornments via docker and tarballs (already) but it is more about how this integrated within nebari which needs to be figured out. Do we upload the docker images to container storage?