WIP: implement a secret vars store in nixpkgs

[Lassulus]

This allows to create secrets (and public files) outside or inside the nix store in a more delclarative way. This is shipped with an example (working) implementation of an on-machine storage. The vars options can easily be used to implement custom backends and extend the behaviour to integrate with already existing solutions, like sops-nix or agenix.

I wanted to upstream this, since we use code similiar to this in https://clan.lol now for a while. Happy to discuss it a bit more. I'm not very fond of the name vars anymore, so maybe a first change could be to come up with a better name :)

I'm not sure if this code is working yet, I will try to develop it a bit more in the next coming days, but I wanted to get this out earlier so other people can throw in some feedback already :)

[lucasew]

How this compares to sops-nix?

[Lassulus]

How this compares to sops-nix?

This enhances sops-nix, It is more of an upstream evolution of agenix-rekey.

The gist is: This supports declaring how to generate the secrets you store in sops-nix in nixpkgs in the service module.

I will add a declaration to a service soon, so it is easier to understand :)

[Lassulus]

alright, removed some features to make the first implementation easier.
Some things are still missing in the on-machine backend like dependencies and prompts. but the rough shape should be more clear now :)

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2025-01-08-clan-weekly-changelog/58501/1

[oddlama]

This looks awesome, I'll have to play around with it a bit more to understand all the details.

Have you already thought about how this would work when someone wants to use vars/secrets in the initrd in stage1? I imagine there could be some kind of hen-egg problem since the initrd will be generated pretty early on when a new generation is activated. The configured vars/secrets may probably not exist at that time, so this might require activating twice (?)

[lucasew]

I am curious about how it would work in stage 1 because AFAIK / is not mounted yet in this phase, so how it will read the host key to decrypt the secrets.

[Lassulus]

Have you already thought about how this would work when someone wants to use vars/secrets in the initrd in stage1? I imagine there could be some kind of hen-egg problem since the initrd will be generated pretty early on when a new generation is activated.

The current code here doesn't run during activation but creates a script that can be run before activation. in clan.lol we solved that issue differently by having a requiredBy option which can take "activation". We would create the files necessary for activation beforehand in that case, but I think this adds too much complexity to this initial PR.
But this can be something we can add later on top of it.
For now just having all the files created in one phase would be easier and since the creation script can be run at any time (even before installation) they should be available for all phases.

Another problem is also the prompts which some vars can ask for. Having the generation run in the activation would make prompting for input very hard. So I think a better way would be to prompt in nixos-install/nixos-rebuild, so before running activation. But this is also something I will add in a later iteration :)

[Lassulus]

I am curious about how it would work in stage 1 because AFAIK / is not mounted yet in this phase, so how it will read the host key to decrypt the secrets.

This is not really in scope for this PR. The current on-machine implementation I added here, doesn't encrypt the secrets (yet?) so there should be no issue with stage-1. secrets can still be added to initrd if they are required to early boot and they should be available at that time.