API design

[c-cube]

We need to discuss the design of this library, how it can be used for various kernels, what capabilities does it expose, etc.
cc @andrewray .

[c-cube]

So far, we expose Client.Kernel.t as the main structure to be provided by implementation. Should we use a functor instead? Also need to list what is missing.

[andrewray]

I think implementing the ocaml kernel on top of this will help expose all the required functionality.

stdout/stderr (and potentially stdin) might need some thought.

[c-cube]

Yes indeed. Would you be willing to migrate iocaml here, and try and use the new API in it? I will migrate my own project (easy, since it already uses this API).

We might also want to bring people from Owl (@ryanrhymes) and other ML/numerical libs in there, it would help design a good API for mime data (pictures, mostly). Sorry if it's already working automatically in the old iocaml.

[andrewray]

Indeed, I definitely intend to port the ocaml kernel across and make it live here.

(Not before the new year though...)

Regarding MIME stuff it should just work.....98% of that is actually handled in the javascript on the web front end. I will look into it.

[c-cube]

If you have time to look at the current API, it would be great! :-)

[andrewray]

Will do. I will ensure that everything that used to work in iocaml still does when I port it across, including mime, stdio stuff and whatever else I find (but also clean up some stuff that didn't work out too).

[c-cube]

I've been using mime types myself (can at least display a .png in the notebook after base64 encoding) but not the stdio part. However, it should not have changed much, except it should use lwt-zmq :-)

[andrewray]

Yeah - the stdio stuff has specific messages I seem to remember. In iocaml there's a bit of dup2 magic followed by some ocaml threads that drain the channels and send them across. This might be why I didn't use lwt...

[c-cube]

I think that reading stdin/printing on stdout should be done on the frontend side (jupyter-console, etc.), so not a concern to the kernel itself. Of course if some executed code contains input_line I have no idea what should happen ^^.

But draining a channel concurrently to the main loop is very easy with lwt; I'd say it easier than with blocking IO + threads, actually. If it's not, we can use lwt.preemptive.

[c-cube]

https://jupyter-client.readthedocs.io/en/latest/messaging.html using this as a ref, btw. Will look into the "stream" part tomorrow.

[c-cube]

I added a Client_main module that makes it easy to write a kernel server. Now writing a kernel is almost only writing the relevant functions (execute, complete, etc.)

[c-cube]

I now have two (basic) kernels on top of that library.
@andrewray any news from the iocaml side? I suppose you've been busy with other things, but it'd be nice to move forward with this :)

[c-cube]

Note that completion and inspection work (and they are compliant with later versions of the protocole, by using offsets expressed in number of codepoints).