Get process output from Malt, instead of capturing it ourselves

[giordano]

julia> using Malt, ParallelTestRunner

julia> testsuite = Dict(
           "abort" => quote
               @info("I'm about to crash")
               abort() = ccall(:abort, Nothing, ())
               abort()
           end,
           "works" => quote
               println("This will work")
               @test true
           end,
           "silent" => quote
               @test true
           end,
       );

julia> runtests(ParallelTestRunner, ["--jobs=1"]; testsuite);
Running 1 tests in parallel. If this is too many, specify the `--jobs=N` argument to the tests, or set the `JULIA_CPU_THREADS` environment variable.
               │          │ ──────────────── CPU ──────────────── │
Test  (Worker) │ Time (s) │ GC (s) │ GC % │ Alloc (MB) │ RSS (MB) │
abort     (13) |         crashed at 2026-01-19T15:58:30.948
works     (14) │     0.07 │   0.00 │  0.0 │       5.40 │  1017.21 │
silent    (14) │     0.00 │   0.00 │  0.0 │       0.00 │  1017.21 │

Output generated during execution of 'abort':
┌ [ Info: I'm about to crash
│ 
│ [61478] signal 6 (-6): Aborted
│ in expression starting at none:1
│ unknown function (ip: 0x7f70552973dc) at /usr/lib/x86_64-linux-gnu/libc.so.6
│ gsignal at /usr/lib/x86_64-linux-gnu/libc.so.6 (unknown line)
│ abort at /usr/lib/x86_64-linux-gnu/libc.so.6 (unknown line)
│ abort at ./REPL[15]:4
│ unknown function (ip: 0x7f704df313cf) at (unknown file)
│ macro expansion at ./REPL[15]:5 [inlined]
│ macro expansion at /home/mose/.julia/dev/ParallelTestRunner/src/ParallelTestRunner.jl:280 [inlined]
│ macro expansion at /cache/build/tester-amdci4-14/julialang/julia-release-1-dot-12/usr/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined]
│ macro expansion at /home/mose/.julia/dev/ParallelTestRunner/src/ParallelTestRunner.jl:280 [inlined]
│ macro expansion at /cache/build/tester-amdci4-14/julialang/julia-release-1-dot-12/usr/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined]
│ macro expansion at /home/mose/.julia/dev/ParallelTestRunner/src/ParallelTestRunner.jl:279 [inlined]
│ macro expansion at ./timing.jl:689 [inlined]
│ top-level scope at /home/mose/.julia/dev/ParallelTestRunner/src/ParallelTestRunner.jl:278
│ jl_toplevel_eval_flex at /cache/build/tester-amdci4-14/julialang/julia-release-1-dot-12/src/toplevel.c:1024
│ ijl_toplevel_eval at /cache/build/tester-amdci4-14/julialang/julia-release-1-dot-12/src/toplevel.c:1047
│ ijl_toplevel_eval_in at /cache/build/tester-amdci4-14/julialang/julia-release-1-dot-12/src/toplevel.c:1092
│ eval at ./boot.jl:489
│ inner at /home/mose/.julia/dev/ParallelTestRunner/src/ParallelTestRunner.jl:271
│ runtest at /home/mose/.julia/dev/ParallelTestRunner/src/ParallelTestRunner.jl:302
│ unknown function (ip: 0x7f704df23460) at (unknown file)
│ jl_apply at /cache/build/tester-amdci4-14/julialang/julia-release-1-dot-12/src/julia.h:2391 [inlined]
│ jl_f_invokelatest at /cache/build/tester-amdci4-14/julialang/julia-release-1-dot-12/src/builtins.c:881
│ #handle##0 at /home/mose/.julia/dev/Malt/src/worker.jl:120
│ unknown function (ip: 0x7f704df2323f) at (unknown file)
│ jl_apply at /cache/build/tester-amdci4-14/julialang/julia-release-1-dot-12/src/julia.h:2391 [inlined]
│ start_task at /cache/build/tester-amdci4-14/julialang/julia-release-1-dot-12/src/task.c:1249
└ Allocations: 2975744 (Pool: 2975695; Big: 49); GC: 4

Output generated during execution of 'works':
[ This will work

Test Summary: | Pass  Error  Total  Time
  Overall     |    2      1      3  7.0s
    abort     |           1      1  4.6s
    works     |    1             1  0.0s
    silent    |    1             1  0.0s
    FAILURE

Error in testset abort:
Error During Test at none:1
  Got exception outside of a @test
  Malt.TerminatedWorkerException()
ERROR: Test run finished with errors

Note: still needs tests (can adapt the example above). Tests added

Requires JuliaPluto/Malt.jl#103. Fix #83.

[giordano]

So, the remaining failures now are only due to buffering: the tests at

ParallelTestRunner.jl/test/runtests.jl

Lines 236 to 238 in 608f947

	"color" => quote
	printstyled("Roses Are Red"; color=:red)
	end

ParallelTestRunner.jl/test/runtests.jl

Lines 248 to 251 in 608f947

	"no color" => quote
	print("Violets are ")
	printstyled("blue"; color=:blue)
	end

rely on the fact that the previous IO capturing mechanism was able to capture also from print-like function, without a newline which would flush the stream.

Unless someone knows a magic way to run julia unbuffered like python -u without using external tools like unbuffer/stdbuf (but decade-old issues like JuliaLang/julia#13050 don't inspire much hope), only thing I can think of is to append a println to those tests and live with the fact that lone prints without newlines/flushing won't be captured anymore. But that's hopefully a non-super-common situation, and I'd argue the ability of getting the output of crashed workers overcomes that limitation.

[giordano]

@vchuravy @maleadt with the caveat that we need to wait for the upstream PR JuliaPluto/Malt.jl#103, I'd appreciate some feedback about the design of this PR and its tradeoffs.

[vchuravy]

Something, something about isreadable/isopen

Julia base uses the pattern

buf = Base.BufferStream()
t = @async try
  while !eof(stdout_read)
        l = readavailable(stdout_read)
        write(buf, l)
  end
finally
  closewrite(buf)
end
Base.errormonitor(t)
return buf

See the tee function in stdlib/REPL/test/repl.jlor an even more advanced fake_repl

[giordano]

That seems to work, I replaced isopen -> !eof, thanks!