Restrict support to Julia 1.10.9+ by omus · Pull Request #6 · beacon-biosignals/julia-container-scripts

omus · 2025-07-08T19:04:31Z

In #5 I've noticed some tests failures for the "concurrent shared access" tests with this context. I'm investigating this separately from the other changes.

Update: Issue only is with Julia 1.10.8 so we could still support Julia 1.10.0 - 1.10.7. However, as removing support for a Julia version is breaking (in some scenarios 1.10.8 does work) we are better of encouraging users to use 1.10.9+. Uses who need older versions can stick to the v0.1 tag.

omus · 2025-07-08T19:14:01Z

Symptoms of the issue:

#16 [julia 11/11] RUN --mount=type=cache,id=julia-concurrent-LGsMfXEh,sharing=shared,target=/mnt/julia-depot     JULIA_DEBUG=pkg-precompile pkg-precompile.jl "/mnt/julia-depot" &&     find "/mnt/julia-depot" -name "*.ji" -type f | sort
#16 0.836 [ Info: Precompile packages...
#16 0.794 [ Info: Precompile packages...
#16 3.176 Precompiling#16 3.121 Precompiling packages...
 packages...
#16 3.535     MultilineStrings Being precompiled by an async task in this process (pidfile: /usr/local/share/julia-depot/compiled/v1.10/MultilineStrings/Rjnab_QQGN9.ji.pidfile)
#16 3.657     Preferences Being precompiled by an async task in this process (pidfile: /usr/local/share/julia-depot/compiled/v1.10/Preferences/pWSk8_QQGN9.ji.pidfile)
#16 3.976     475.5 ms  ✓ MultilineStrings
#16 4.061 ┌ Debug: Precompile files to transfer (new additions 0/0):
#16 4.061 └ @ Main /usr/local/bin/pkg-precompile.jl:271
#16 4.068 [ Info: Copy precompilation files into image...
#16 4.092 [ Info: Initialize dependencies...
#16 5.030 Unhandled Task ERROR: MethodError: no method matching isprecompiled(::PkgId; ignore_loaded::Bool, flags::Base.CacheFlags)
#16 5.030 
#16 5.030 Closest candidates are:
#16 5.030   isprecompiled(::PkgId; ignore_loaded, stale_cache, cachepaths, sourcepath) got unsupported keyword argument "flags"
#16 5.030    @ Base loading.jl:1459
#16 5.030 
#16 5.030 Stacktrace:
#16 5.030  [1] kwerr(::@NamedTuple{ignore_loaded::Bool, flags::Base.CacheFlags}, ::Function, ::PkgId)
#16 5.030    @ Base ./error.jl:165
#16 5.030  [2] (::Pkg.API.Precompilation.var"#77#79"{Pkg.API.Precompilation.var"#27#67"{Pipe, Dict{PkgId, PkgId}, Dict{PkgId, Vector{PkgId}}, String, PkgId}, Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, String}, Base.CacheFlags, PkgId})()
#16 5.030    @ Pkg.API.Precompilation /usr/local/julia/share/julia/stdlib/v1.10/Pkg/src/precompilation.jl:1073
#16 5.030  [3] mkpidlock(f::Pkg.API.Precompilation.var"#77#79"{Pkg.API.Precompilation.var"#27#67"{Pipe, Dict{PkgId, PkgId}, Dict{PkgId, Vector{PkgId}}, String, PkgId}, Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, String}, Base.CacheFlags, PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64})
#16 5.030    @ FileWatching.Pidfile /usr/local/julia/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93
#16 5.030  [4] #mkpidlock#6
#16 5.030    @ /usr/local/julia/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined]
#16 5.030  [5] precompile_pkgs_maybe_cachefile_lock(f::Pkg.API.Precompilation.var"#27#67"{Pipe, Dict{PkgId, PkgId}, Dict{PkgId, Vector{PkgId}}, String, PkgId}, io::IOContext{IO}, print_lock::ReentrantLock, fancyprint::Bool, pkg_config::Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, pkgspidlocked::Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, String}, hascolor::Bool)
#16 5.030    @ Pkg.API.Precompilation /usr/local/julia/share/julia/stdlib/v1.10/Pkg/src/precompilation.jl:1071
#16 5.030  [6] macro expansion
#16 5.030    @ ./timing.jl:395 [inlined]
#16 5.030  [7] (::Pkg.API.Precompilation.var"#24#64"{Bool, Bool, Vector{Task}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, String}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, IOBuffer}, Base.Event, Base.Event, ReentrantLock, Vector{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, String}, Vector{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}}, Vector{PkgId}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, Bool}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, Base.Event}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, Bool}, Vector{PkgId}, Pkg.API.Precompilation.var"#describe_pkg#37"{Dict{PkgId, PkgId}, Pkg.API.Precompilation.var"#color_string#36"{Bool}, Int64}, Dict{PkgId, PkgId}, Dict{PkgId, Vector{PkgId}}, Dict{Tuple{PkgId, UInt128, String, String}, Bool}, Pkg.API.Precompilation.var"#color_string#36"{Bool}, Bool, Base.Semaphore, Bool, String, Vector{String}, Vector{PkgId}, PkgId, Base.CacheFlags, Cmd, Pair{Cmd, Base.CacheFlags}, Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}})()
#16 5.030    @ Pkg.API.Precompilation /usr/local/julia/share/julia/stdlib/v1.10/Pkg/src/precompilation.jl:864
#16 5.358 ERROR: lld: error: cannot open /usr/local/share/julia-depot/compiled/v1.10/Preferences/jl_7u8DJa: No such file or directory
#16 5.389 LoadError: Precompilation incomplete for Parsers#16 6.275 

#16 6.161 Stacktrace:
#16 6.170  #16 6.412 ┌ Debug: Precompile files to transfer (new additions 1/1):
#16 6.412 │ /usr/local/share/julia-depot/compiled/v1.10/MultilineStrings/Rjnab_Eb8c5.ji (new)
#16 6.412 └ @ Main /usr/local/bin/pkg-precompile.jl:271
Unhandled Task ERROR: failed process: Process(setenv(`/usr/local/julia/libexec/julia/lld -flavor gnu '' -shared -o /usr/local/share/julia-depot/compiled/v1.10/Preferences/jl_zmLJrC --whole-archive /usr/local/share/julia-depot/compiled/v1.10/Preferences/jl_7u8DJa --no-whole-archive -L/usr/local/julia/lib/julia -L/usr/local/julia/lib -ljulia -ljulia-internal`,["JULIA_PROJECT=/julia-project", "PATH=/usr/local/julia/libexec/julia:/usr/local/julia/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "***", "FIXED_PACKAGE_TIMESTAMPS=false", "JULIA_PATH=/usr/local/julia", "JULIA_GPG=3673DF529D9049477F76B37566E3C7DC03D6E495", "INVALIDATE_INSTANTIATE=", "LD_LIBRARY_PATH=/usr/local/julia/lib/julia:/usr/local/julia/lib", "JULIA_VERSION=1.10.8", "JULIA_DEPOT_CACHE_ID=julia-concurrent-LGsMfXEh"  …  "GID=1000", "JULIA_CPU_TARGET=generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1);x86-64-v4,-rdrnd,base(1)", "JULIA_DEBUG=pkg-precompile", "JULIA_HISTORY=/dev/null", "HOME=/root", "JULIA_PKG_PRECOMPILE_AUTO=0", "GEN_PKG_SRC=false", "JULIA_DEPOT_PATH=/usr/local/share/julia-depot:", "OPENBLAS_MAIN_FREE=1", "GROUP=docker"]), ProcessExited(1)) [1]
#16 6.406 
#16 6.406 Stacktrace:
#16 6.406   [1] pipeline_error
#16 6.406     @ ./process.jl:565 [inlined]
#16 6.406   [2] run(::Cmd, ::Base.DevNull, ::Vararg{Any}; wait::Bool)
#16 6.406     @ Base ./process.jl:480
#16 6.406   [3] run
#16 6.406     @ ./process.jl:477 [inlined]
#16 6.406   [4] link_image (repeats 2 times)
#16 6.406     @ ./linking.jl:166 [inlined]
#16 6.406   [5] compilecache(pkg::PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
#16 6.406     @ Base ./loading.jl:2450
#16 6.406   [6] (::Pkg.API.Precompilation.var"#28#68"{Pipe, Dict{PkgId, PkgId}, Dict{PkgId, Vector{PkgId}}, String, PkgId})()
#16 6.406     @ Pkg.API.Precompilation /usr/local/julia/share/julia/stdlib/v1.10/Pkg/src/precompilation.jl:871
#16 6.406   [7] with_logstate(f::Function, logstate::Any)
#16 6.406     @ Base.CoreLogging ./logging.jl:515
#16 6.406   [8] with_logger
#16 6.406     @ ./logging.jl:627 [inlined]
#16 6.406   [9] #27
#16 6.406     @ /usr/local/julia/share/julia/stdlib/v1.10/Pkg/src/precompilation.jl:865 [inlined]
#16 6.406  [10] mkpidlock(f::Pkg.API.Precompilation.var"#27#67"{Pipe, Dict{PkgId, PkgId}, Dict{PkgId, Vector{PkgId}}, String, PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
#16 6.406     @ FileWatching.Pidfile /usr/local/julia/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93
#16 6.406  [11] #mkpidlock#6
#16 6.406     @ /usr/local/julia/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined]
#16 6.406  [12] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
#16 6.406     @ FileWatching.Pidfile /usr/local/julia/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:111
#16 6.406  [13] trymkpidlock
#16 6.406     @ /usr/local/julia/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:109 [inlined]
#16 6.406  [14] precompile_pkgs_maybe_cachefile_lock(f::Pkg.API.Precompilation.var"#27#67"{Pipe, Dict{PkgId, PkgId}, Dict{PkgId, Vector{PkgId}}, String, PkgId}, io::IOContext{IO}, print_lock::ReentrantLock, fancyprint::Bool, pkg_config::Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, pkgspidlocked::Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, String}, hascolor::Bool)
#16 6.406     @ Pkg.API.Precompilation /usr/local/julia/share/julia/stdlib/v1.10/Pkg/src/precompilation.jl:1055
#16 6.406  [15] macro expansion
#16 6.406     @ ./timing.jl:395 [inlined]
#16 6.406  [16] (::Pkg.API.Precompilation.var"#24#64"{Bool, Bool, Vector{Task}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, String}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, IOBuffer}, Base.Event, Base.Event, ReentrantLock, Vector{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, String}, Vector{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}}, Vector{PkgId}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, Bool}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, Base.Event}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, Bool}, Vector{PkgId}, Pkg.API.Precompilation.var"#describe_pkg#37"{Dict{PkgId, PkgId}, Pkg.API.Precompilation.var"#color_string#36"{Bool}, Int64}, Dict{PkgId, PkgId}, Dict{PkgId, Vector{PkgId}}, Dict{Tuple{PkgId, UInt128, String, String}, Bool}, Pkg.API.Precompilation.var"#color_string#36"{Bool}, Bool, Base.Semaphore, Bool, String, Vector{String}, Vector{PkgId}, PkgId, Base.CacheFlags, Cmd, Pair{Cmd, Base.CacheFlags}, Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}})()
#16 6.406     @ Pkg.API.Precompilation /usr/local/julia/share/julia/stdlib/v1.10/Pkg/src/precompilation.jl:864
#16 6.406 [1] error(s::String)
#16 6.406    @ Base ./error.jl:35
#16 6.406  [2] top-level scope
#16 6.406    @ /usr/local/bin/pkg-precompile.jl:328
#16 6.406 in expression starting at /usr/local/bin/pkg-precompile.jl:319
#16 ERROR: process "/bin/sh -c JULIA_DEBUG=pkg-precompile pkg-precompile.jl \"${JULIA_DEPOT_CACHE_TARGET}\" &&     find \"${JULIA_DEPOT_CACHE_TARGET}\" -name \"*.ji\" -type f | sort" did not complete successfully: exit code: 1
------
 > [julia 11/11] RUN --mount=type=cache,id=julia-concurrent-LGsMfXEh,sharing=shared,target=/mnt/julia-depot     JULIA_DEBUG=pkg-precompile pkg-precompile.jl "/mnt/julia-depot" &&     find "/mnt/julia-depot" -name "*.ji" -type f | sort:
6.406     @ Pkg.API.Precompilation /usr/local/julia/share/julia/stdlib/v1.10/Pkg/src/precompilation.jl:1055
6.406  [15] macro expansion
6.406     @ ./timing.jl:395 [inlined]
6.406  [16] (::Pkg.API.Precompilation.var"#24#64"{Bool, Bool, Vector{Task}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, String}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, IOBuffer}, Base.Event, Base.Event, ReentrantLock, Vector{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, String}, Vector{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}}, Vector{PkgId}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, Bool}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, Base.Event}, Dict{Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}, Bool}, Vector{PkgId}, Pkg.API.Precompilation.var"#describe_pkg#37"{Dict{PkgId, PkgId}, Pkg.API.Precompilation.var"#color_string#36"{Bool}, Int64}, Dict{PkgId, PkgId}, Dict{PkgId, Vector{PkgId}}, Dict{Tuple{PkgId, UInt128, String, String}, Bool}, Pkg.API.Precompilation.var"#color_string#36"{Bool}, Bool, Base.Semaphore, Bool, String, Vector{String}, Vector{PkgId}, PkgId, Base.CacheFlags, Cmd, Pair{Cmd, Base.CacheFlags}, Tuple{PkgId, Pair{Cmd, Base.CacheFlags}}})()
6.406     @ Pkg.API.Precompilation /usr/local/julia/share/julia/stdlib/v1.10/Pkg/src/precompilation.jl:864
6.406 [1] error(s::String)
6.406    @ Base ./error.jl:35
6.406  [2] top-level scope
6.406    @ /usr/local/bin/pkg-precompile.jl:328
6.406 in expression starting at /usr/local/bin/pkg-precompile.jl:319
------
Dockerfile:87
--------------------
  86 |     COPY pkg-precompile.jl /usr/local/bin
  87 | >>> RUN --mount=type=cache,id=${JULIA_DEPOT_CACHE_ID},sharing=shared,target=${JULIA_DEPOT_CACHE_TARGET} \
  88 | >>>     # curl -fsSL https://raw.githubusercontent.com/beacon-biosignals/julia-container-scripts/refs/tags/v1/pkg-precompile.jl | JULIA_DEBUG=pkg-precompile julia
  89 | >>>     JULIA_DEBUG=pkg-precompile pkg-precompile.jl "${JULIA_DEPOT_CACHE_TARGET}" && \
  90 | >>>     find "${JULIA_DEPOT_CACHE_TARGET}" -name "*.ji" -type f | sort
  91 |     
--------------------
ERROR: failed to build: failed to solve: process "/bin/sh -c JULIA_DEBUG=pkg-precompile pkg-precompile.jl \"${JULIA_DEPOT_CACHE_TARGET}\" &&     find \"${JULIA_DEPOT_CACHE_TARGET}\" -name \"*.ji\" -type f | sort" did not complete successfully: exit code: 1

– https://github.com/beacon-biosignals/julia-container-scripts/actions/runs/16152107908/job/45585680596

The issue is from a Pkg.jl call to Base.isprecompiled which passes in the flags keyword. We could support 1.10.8 by overriding the Base.isprecompiled method but I think we should encourage users to use 1.10.9+ instead.

Note that 1.10.8 doesn't always fail so we alternatively could just document this but I think that could end up wasting user's time.

kleinschmidt

let's just make the minimum supported version 1.10.9 then, and not mess with the older ones. I think that's fine.

omus · 2025-07-10T16:09:52Z

let's just make the minimum supported version 1.10.9 then, and not mess with the older ones. I think that's fine.

I'm a little concerned about backwards compatibility here as we expect users to pull this script based upon a tag such as v0.1. Dropping older support for Julia versions would possible break Dockerfile currently using v0.1.

Now that I've said that just dropping Julia 1.10.8 should also be considered breaking as Julia 1.10.8 can successfully work under the right conditions. So I think the right thing to do is to make this a v0.2 release which means we can use 1.10.9 as the minimum supported version. I'll call out I'd like to keep LTS support.

omus · 2025-07-10T19:35:17Z

Julia 1.11.6 is now out but the image doesn't yet exist:

ERROR: failed to build: failed to solve: julia:1.11.6-bookworm: failed to resolve source metadata for docker.io/library/julia:1.11.6-bookworm: docker.io/library/julia:1.11.6-bookworm: not found

I'll just wait for the new image to be available before continuing here

omus added 2 commits July 8, 2025 13:56

Test additionally against 1.10.8

101221a

Avoid supporting Julia 1.10.8

0180a52

omus changed the title ~~Address issue with Julia 1.10.8~~ Disable support for Julia 1.10.8 Jul 8, 2025

omus marked this pull request as ready for review July 8, 2025 19:17

omus requested a review from kleinschmidt July 8, 2025 19:18

omus mentioned this pull request Jul 8, 2025

Support dependencies using tracked paths #5

Merged

kleinschmidt approved these changes Jul 9, 2025

View reviewed changes

omus added the breaking label Jul 10, 2025

omus added 2 commits July 10, 2025 11:33

Limit support to Julia 1.10.9+

bf4c4a2

Set minimum Julia version in test/Project.toml

d3010a8

Merge branch 'main' into cv/julia-1.10.8

b50e819

omus changed the title ~~Disable support for Julia 1.10.8~~ Restrict support to Julia 1.10.9+ Jul 11, 2025

omus merged commit aef7848 into main Jul 14, 2025
11 of 12 checks passed

omus deleted the cv/julia-1.10.8 branch July 14, 2025 12:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Restrict support to Julia 1.10.9+#6

Restrict support to Julia 1.10.9+#6
omus merged 5 commits intomainfrom
cv/julia-1.10.8

omus commented Jul 8, 2025 •

edited

Loading

Uh oh!

omus commented Jul 8, 2025 •

edited

Loading

Uh oh!

kleinschmidt left a comment

Uh oh!

omus commented Jul 10, 2025

Uh oh!

omus commented Jul 10, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

omus commented Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

omus commented Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kleinschmidt left a comment

Choose a reason for hiding this comment

Uh oh!

omus commented Jul 10, 2025

Uh oh!

omus commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

omus commented Jul 8, 2025 •

edited

Loading

omus commented Jul 8, 2025 •

edited

Loading

omus commented Jul 10, 2025 •

edited

Loading