Enable streaming of build logs

[munishchouhan]

This PR will enalbe log streaming to get build log endpoint, while build is in process

[munishchouhan]

@pditommaso is this required or should it refresh automatically?

wavebuildpodlogs1.mov

[pditommaso]

If only build box content is refreshed while the status is PENDING, it would be cool

[munishchouhan]

If only build box content is refreshed while the status is PENDING, it would be cool

I will give it a try

[munishchouhan]

@pditommaso build logs with refresh now

refreshwavebuildlogs.mov

[pditommaso]

It should be considered to use a streaming approach, see

https://github.com/kubernetes-client/java/blob/master/examples/examples-release-latest/src/main/java/io/kubernetes/client/examples/LogsExample.java#L36-L49

[munishchouhan]

I did tried it, but streamNamespacedPodLog() waits till the pod execution is completed. I will dig more

[munishchouhan]

It happens because podlogs set the follow true and under the hood use readNamespacedPodLog

https://github.com/kubernetes-client/java/blob/f20788272291c0e79a8c831d8d5a7dd94d96d2de/util/src/main/java/io/kubernetes/client/PodLogs.java#L87

kubernetes-client/java#571 (comment)

[munishchouhan]

I will move the logic to convert string to inputstream to this function so that getLogs return input stream

[pditommaso]

I don't know if this is a limitation of java client or a bug, but I'm quite sure it's possible to stream the logs of a running pod.

I've made a simple test running a pod like this

kubectl run counter --image=busybox --command -- /bin/sh -c 'n=0; while true; do echo $((n++)); sleep 1;

Then I've used the K8s client implemented in Nextflow, that's a bare simple http request over URLConnection.

This snippet just stream the pod logs until the pod completes

    def 'should get pod log' () {
        given:
        def settings = ['context': 'docker-desktop', name: 'default']
        def config = new K8sConfig(settings)
        def client = new K8sClient( config.getClient() )

        when:
        def logs = client.podLog([follow:'true'], 'counter')
        then:
        logs.text == 'hello\n'
    }

[munishchouhan]

@pditommaso its ready for review

[munishchouhan]

I have moved the docs changes to #464
it will easier to apply once the docs repo is in sync with wave

[munishchouhan]

Suggestion from @ewels
Was thinking about what you were saying about ASCII codes in the logs yesterday
. I wonder if we can keep them (maybe via a flag in the API?). I would love to be able to render the colours if possible as it makes the logs much easier to read.
The back-end used by Hub is written Python, and my favourite library Rich is able to convert ascii codes to html. So we might be able to do that on the round trip and preserve colour. On the Wave side we just have to not throw them out. What do you think?

[pditommaso]

Please keep colour outside of this PR. We need fist makes the logs stream to work

[munishchouhan]

Please keep colour outside of this PR. We need fist makes the logs stream to work

I will create another issue