Culling feature can not save in-memory variables

[Jeffwan]

I am using jupyter on Kubernetes and I have a question on the culling feature. When I enable gateway culling feature, notebook would be killed once a notebook doesn't have activities after a period. The problem is all the state get lost. Using following code snippet as an example. Once the pod is terminated, I have to rerun cell 1 to get result of a.

# cell 1
a = 1+1

# cell 2
print(a)

Is there a way to persist previous state or variables without reruning all previous cells? Does jupyter have any extension to customize this part like ContentManager? https://jupyter-notebook.readthedocs.io/en/stable/extending/contents.html

[kevin-bates]

Hi @Jeffwan. As you have found, the culling functionality (inherited from Jupyter Server via its MappingKernelManager) terminates a kernel based on its time at the idle state and whether or not there are active connections and/or its busy. When those criteria are met, the equivalent functionality to a kernel shutdown request occurs.

Is there a way to persist previous state or variables without reruning all previous cells?

No, not today. The Real-time Collaboration (RTC) effort will likely be able to capture output for replay following disconnected scenarios, but even in those cases, the same kernel remains running. I am not aware of anything that can capture a kernel's state. I suppose it would need to capture the kernels namespace but really have no idea. But, even having captured its state, there is nothing that can re-hydrate a new kernel with that captured state.

EG does have a kernel session persistence feature, but even that feature assumes the remote kernel will remain running while its launching server is terminated and another instance started, at which time the captured kernel session can be used to re-hyrate a connection to the same, previously running, kernel.

So, I think we (the Jupyter community) are a number of steps away from achieving anything like this.

Does jupyter have any extension to customize this part like ContentManager?

I believe you're referring to the ability to use your own MappingKernelManager. Yes, provided your MappingKernelManager derives from EG's RemoteMappingKernelManager, EG can be configured to use your implementation. That said, there's a lot of ground to cover to persist and, more importantly, properly restore a kernel's state.

Is there a reason why you're configuring culling at all? It sounds like this is something you don't really want. What is the idle timeout you're using and do you have a use case for enabling it?

[Jeffwan]

Thanks a lot for your quick reply. @kevin-bates. One motivation is considering GPU resource is super expensive, if we can make jupyter command like granular resource request per query. Then the utilization would be super great. I know there're other problems like GPU cold start or kernel warm up issues but this is an interesting area we are exploring. I will further check the code reference you mentioned and see if there're acceptable workaround for us. I will come back and report my progress later