Skip to content

Conversation

@vishal2005025
Copy link

Title: Add compile-time _trim_enabled preference during --trim precompilation

Summary

When building with --trim, some packages need to know at precompile time that trimming is enabled so they can register / specialize code accordingly (see SciML/LinearSolve.jl#778). This PR ensures a compile-time preference _trim_enabled is visible to packages during the Pkg.instantiate(); Pkg.precompile() step invoked by JuliaC when trimming is requested.

What I changed

  • Implemented temporary depot-based LocalPreferences injection for precompilation in:
    • src/compiling.jl

Behavior

  • If an ImageRecipe has trimming enabled (i.e. recipe.trim_mode !== nothing && recipe.trim_mode != "no"), the precompile subprocess will:
    1. Create a temporary depot directory.
    2. Write config/LocalPreferences.toml containing:
      [Preferences]
      _trim_enabled = true
      
    3. Invoke Pkg.instantiate(); Pkg.precompile() with JULIA_DEPOT_PATH pointed to the temporary depot so Preferences.load_preference(Preferences, "_trim_enabled") will return true for packages during precompilation.
    4. Clean up the temporary depot after the precompile subprocess finishes.

Why this approach

  • The repo discussion suggested using a LocalPreferences.toml hack so packages can call:
    Preferences.load_preference(Preferences, "_trim_enabled")
    and see a compile-time flag during precompilation.
  • Using a temporary depot avoids touching the user's global depot or permanently modifying configuration.
  • This design is minimal, low-risk, and limited only to the precompile subprocess invoked by JuliaC.

Files edited

  • src/compiling.jl — add logic to create temporary depot, write LocalPreferences.toml, set JULIA_DEPOT_PATH for the precompile subprocess, and cleanup afterward.

Testing & verification

  • Ran the package test suite: julia --project -e "using Pkg; Pkg.test()" — all tests passed locally on Windows.
  • The existing test suite includes trimming-related tests (CLI and programmatic flows) and completed successfully.
  • Suggested additional test (optional): add a small integration test that asserts a package reading the preference during precompile returns true, e.g. a tiny test project that calls Preferences.load_preference(Preferences, "_trim_enabled") in a top-level file and checks the behavior during the Pkg.precompile() invoked by the compile pipeline.

Caveats & notes

  • This relies on packages using Preferences.jl and reading LocalPreferences.toml via Preferences.load_preference. Packages that obtain preferences from other sources will not be affected.
  • The temporary depot is only used during Pkg.instantiate(); Pkg.precompile() subprocess and is removed afterwards; user's depot is not modified.
  • The change affects only the precompile subprocess for trimming-enabled runs. Runtime behavior is unchanged.
  • If you want to expose the preference differently (e.g., via an exported Base function or environment variable), we can iterate — this approach follows the repo discussion and avoids invalidating the global pkgimage cache.

How to try locally

Run the existing test-suite (this was used to validate the change):

julia --project -e "using Pkg; Pkg.test()"

@vishal2005025
Copy link
Author

please review!

@vishal2005025
Copy link
Author

@topolarity please take a look!

src/compiling.jl Outdated
# read a compile-time preference (e.g. Preferences.load_preference(Preferences, "_trim_enabled")).
env_overrides = Dict{String,Any}("JULIA_LOAD_PATH"=>nothing)
tmp_depot = nothing
if is_trim_enabled(recipe)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this cause Julia to re-precompile the whole environment for every run? Or just the packages that have read that preference

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this is working around JuliaPackaging/Preferences.jl#86

We should probably fix that bug instead

Copilot AI review requested due to automatic review settings December 4, 2025 19:37
@gbaraldi
Copy link
Member

gbaraldi commented Dec 4, 2025

Sorry to have hijacked your PR slightly

Copilot finished reviewing on behalf of gbaraldi December 4, 2025 19:40
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds compile-time preference support for trim mode, enabling packages to detect during precompilation whether trimming is enabled. The implementation creates a temporary Julia environment with a LocalPreferences.toml file that sets trim_enabled = true, making this preference available to packages during the precompilation subprocess.

Key Changes:

  • Modified precompilation flow to inject trim preferences via a temporary environment in JULIA_LOAD_PATH
  • Added test project TrimPrefsProject that reads the preference and validates propagation
  • Added integration test to verify the preference affects compiled executable behavior

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
src/compiling.jl Adds temporary environment creation with LocalPreferences.toml for trim mode preference injection during precompilation
test/programatic.jl Adds integration test verifying trim preference propagation to compiled executables
test/runtests.jl Adds constant for new test project path
test/TrimPrefsProject/src/TrimPrefsProject.jl New test module that reads trim_enabled preference at compile time
test/TrimPrefsProject/Project.toml Project definition for trim preference test
.gitignore Updates pattern to ignore additional Manifest file variants

Critical Issues Found:

  1. Undefined variables in cleanup code - The cleanup block (lines 148-160) references local_prefs_file and local_prefs_backup which are never defined, causing this code to fail at runtime
  2. Windows compatibility issue - JULIA_LOAD_PATH uses Unix path separator : which won't work on Windows (needs ;)
  3. Misleading test assertion - Test checks for cleanup of a file that's never created in that location
  4. Naming inconsistency - PR description mentions _trim_enabled but implementation uses trim_enabled

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@vishal2005025
Copy link
Author

Sorry to have hijacked your PR slightly

@gbaraldi No problem at all - thanks for taking a look! I’m happy to update anything if needed.

src/compiling.jl Outdated
end
# Prepend the temp env to JULIA_LOAD_PATH

env_overrides["JULIA_LOAD_PATH"] = join([tmp_prefs_env, "@", "@stdlib"], load_path_sep)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
env_overrides["JULIA_LOAD_PATH"] = join([tmp_prefs_env, "@", "@stdlib"], load_path_sep)
env_overrides["JULIA_LOAD_PATH"] = tmp_prefs_env * load_path_sep

since JULIA_LOAD_PATH=":foo" expands to: ["@", "@v#.#", "@stdlib", "foo"]

src/compiling.jl Outdated
spinner_done[] = true
wait(spinner_task)
# Cleanup temporary preferences environment if created
if tmp_prefs_env !== nothing
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mktempdepot(f, trim_mode) = trim_mode ? mktempdir(f) : f(nothing)
mktempdepot(trim_mode) do tmp_depot
   ...
end

might be a more standard pattern for this

@vishal2005025
Copy link
Author

@topolarity @gbaraldi i have address all the reviewed changes.
please take a look and let me know if changes are required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants