Consider dune-project environment variables

[eureka-cpu]

Hi there, I opened this thread on discourse where my main inquiry was how to properly get the version information from my dune-project into my code so I can use it with cmdliner.

The discourse thread is pretty short, but to summarize I'm coming from using Rust and I really enjoy how clap handles Cargo package/workspace information. It's able to do this because Cargo makes environment variables available at compile time which can then be picked by clap, or any other Rust code for that matter using the env! macro. This particular part of the thread shows how if dune also made similarly named environment variables available, it could achieve the same ease of use for OCaml programs, in particular command line applications and build orchestration tools, like nix.

I am new to OCaml, and also not familiar with the dune repository, but the following could be a short route to achieving this:

1. dune parses dune-project and makes environment variables available for use at compile time
2. compile time getters could be created using ppx_get_env
3. ppx_get_env could be used in a deriver which could populate Cmdliner.Cmd.Info

The theoretical deriver might look like:

module Info = struct
  type t =
  { name : string
  ; version : string
  ; synopsis : string
  ; description : string
  }
  [@@deriving cmdliner_info]

  let info = t_cmdliner_info (* generated by the deriver *)
end

let cmd entrypoint = Cmd.v Info.info Term.(const entrypoint $ target_dir)

This could be a nice addition, which avoids generating files via action and uses the dune-project file as the singular source of truth for package/workspace data.

[eureka-cpu]

I was just made aware of https://ocaml.org/p/dune-build-info/latest/doc/index.html which seems to do this without environment variables through some library magic, albeit with a very limited api. Regardless I'll keep this discussion open on the off chance its well received, though feel free to close if not. 🙂

[eureka-cpu]

Here's a nix implementation I wrote today which just parses dune-project into a nix expression which can then be used to populate the derivation attrs: https://github.com/eureka-cpu/dunix. This solves my issue on the nix side, but it would still be great to have more than just the version available in the dune-build-info library, or envvars as I originally suggested.

[eureka-cpu]

Am I wrong thinking the .opam is the authority file, not dune-project?

I think for just getting project information either is fine. My line of thinking was to use it as you might a Cargo.toml file. Maybe an interesting addition could be to not include the opam file but generate it in the derivation and then have an assertion that they are the same like in buildRustPackage.

[toastal]

I would be hesitant since, many projects just leave it at

(lang dune 2.0)
(name soupault)

See Soupault: https://codeberg.org/PataphysicalSociety/soupault/src/branch/main/dune-project

I actually had issues packaging up a Rust project last week since the project was trying to generate Cargo.toml+Cargo.lock files with tooling instead of having something easy/static to read. Even still it has been a good practice to check your *.opam file into the VCS which makes using the generate-opam-file (?) pretty redundant.

[toastal]

In the future when more projects test out Alice or b0, the Opam file will still be the common thread to pull on.

[eureka-cpu]

Can you build a dune project without a dune-project file? I am able to build one without the opam file.

This check works for my use case and generally feels more comfortable to me, but I will check out Alice/b0 as well.

        doCheck = true;
        checkPhase = '' # Check that the generated opam file matches source
          if ! diff "${finalAttrs.src}/${dune-project.name}.opam" "./${dune-project.name}.opam"; then
            echo "Error: Generated opam file does not match source opam file"
            exit 1
          fi
        '';

[eureka-cpu]

Alice is missing some features I'd like to use, and b0 seems like there is yet another learning curve to use it.

WARNING this package is unstable and work in progress, do not depend on it.

[rgrinberg]

How does cargo handle recompilation when these environment variables change? For example, if I update my "crate description" which is supposedly exposed via an environment variable, how does cargo know which recompilation actions need to re-run?

[rgrinberg]

I think this bit is what's stopping dune from implementing a similar feature. Seems like there's special support for the compiler to tell cargo which environment variables were read rust-lang/rust#71858

If we had such a feature in ocamldep, we could surely implement something similar in dune. One simple alternative would be for you to write in your dune file that an action intends to read a particular environment variable and dune will remember that the action depends on the variable. Of course, this isn't a whole lot better than just writing the value that you want to pass to the action in the dune file as a command line flag.

[eureka-cpu]

I'm not familiar with ocamldep, but I'm assuming for your wording that it's unlikely to add support for this 😅 but that's okay. The library I wrote in nix and the dune-build-info library are enough for now. It would be very nice to have in the future, and I'm glad to have opened this discussion if it eventually makes something like this possible.

[rgrinberg]

Actually, I do want to add support for this, but there are technical challenges to make this happen. In short, actions need a way to tell dune what information they're reading for dune to know when to-run them. We are investigating a more dynamic approach to do this with the "dynamic action plugin". It is not production ready at the moment.

What we did for dune-build-info is a pile of hacks where we rewrite the binary to insert the version number.

[eureka-cpu]

Ah that's fantastic to hear. I'm not very experienced with OCaml, but if you happen to have any tasks to do with this feature I'm happy to take a few of them.

[rgrinberg]

I'll let you know once we have something to test. In all honesty, this isn't a great feature to get acquainted with the dune code base. But we have plenty of other issues in the repo that are far more suitable for that though. Feel free to pick up one of those if you'd like to help out.