Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

typst: add initial support for typst packages #369283

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

cherrypiejam
Copy link

@cherrypiejam cherrypiejam commented Dec 30, 2024

This PR adds initial support to build Typst documents with Typst packages. In particular, it adds a Typst package builder and exposes it at the top level to ease the process of defining new Typst packages. It further adds a new function to the Typst derivation for creating a new Typst environment in which the Typst compiler understands a specific set of packages, namely typst.withPackages. Moreover, three Typst packages are defined and collected as a single attribution set as an initial demonstration.

Only a few Typst packages are included in this PR for a reason. The Typst Universe (the official site for Typst packages) maintains a single git repo that contains all Typst packages. Each package entry does not necessarily map to the same version specified in their respective git repo published by their authors (if there is). This is because the package authors need to manually move their package and copy the source, instead of a pointer, to the Typst Universe repo. Breaking the Typst Universe repo down to per package requires separate places to store each of them, and due to the aforementioned reason, the exact source may disagree with the original git repo 1. This is why, in my opinion, we might want to collect our own set of Typst packages instead of directly pulling from the Typst Universe repo. For gradual adaption, typst.withPackages also provides a method to override the predefined set of Typst packages, namely typst.withPackages.override, making it easier for end users overriding the existing set of Typst packages and integrating new packages into their project.

All packages from the Typst Universe are included in this PR, along with a maintenance script. The script simply generates a set of nix expressions for all packages from the Typst Universe. Each individual package is now fetched as a tarball from the Typst Universe instead of their individual repository.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

Footnotes

  1. Assuming the author(s) also publish their package in another repo outside of the Typst Universe repo.

@nix-owners nix-owners bot requested a review from philiptaron December 30, 2024 02:28
@NixOSInfra NixOSInfra added the 12. first-time contribution This PR is the author's first one; please be gentle! label Dec 30, 2024
@github-actions github-actions bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Dec 30, 2024
@cherrypiejam cherrypiejam force-pushed the typst branch 3 times, most recently from b5ab425 to 0bcf7c9 Compare December 30, 2024 12:20
Copy link
Contributor

@SigmaSquadron SigmaSquadron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome to Nixpkgs! This is truly amazing; I've wanted to build Typst packages for a very long time but never got around to it.

Note that the commits should be split as much as possible. Each new typstPackage should be added in a separate commit. The changes to the base typst package to wrap it should also be a separate commit.

pkgs/by-name/ty/typst/with-packages.nix Outdated Show resolved Hide resolved
pkgs/build-support/build-typst-package.nix Outdated Show resolved Hide resolved
pkgs/development/typst-packages/cetz.nix Outdated Show resolved Hide resolved
Copy link
Contributor

@SigmaSquadron SigmaSquadron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the actual package set, even though we can't import from Universe directly, it might still be a good idea to scrape it for information and build packages automatically. typst.toml gives us all of the necessary information to build each package.

pkgs/build-support/build-typst-package.nix Outdated Show resolved Hide resolved
pkgs/build-support/build-typst-package.nix Outdated Show resolved Hide resolved
pkgs/build-support/build-typst-package.nix Outdated Show resolved Hide resolved
pkgs/build-support/build-typst-package.nix Outdated Show resolved Hide resolved
pkgs/development/typst-packages/polylux.nix Outdated Show resolved Hide resolved
pkgs/development/typst-packages/polylux.nix Outdated Show resolved Hide resolved
pkgs/development/typst-packages/oxifmt.nix Outdated Show resolved Hide resolved
pkgs/development/typst-packages/cetz.nix Outdated Show resolved Hide resolved
pkgs/development/typst-packages/cetz.nix Outdated Show resolved Hide resolved
pkgs/by-name/ty/typst/with-packages.nix Outdated Show resolved Hide resolved
Copy link
Contributor

@drupol drupol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello,

Thanks for tackling this, I have a few comments, let me know if you have questions.

@jtojnar : I know you're busy these days, but if you could have a look at this, it would be great.

Thanks!

pkgs/build-support/build-typst-package.nix Outdated Show resolved Hide resolved
pkgs/build-support/build-typst-package.nix Outdated Show resolved Hide resolved
pkgs/by-name/ty/typst/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/ty/typst/with-packages.nix Outdated Show resolved Hide resolved
@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 4, 2025
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 5, 2025
@cherrypiejam
Copy link
Author

Regarding the actual package set, even though we can't import from Universe directly, it might still be a good idea to scrape it for information and build packages automatically. typst.toml gives us all of the necessary information to build each package.

Ah that is great. Didn't know they already have that information included. I'll work on the script when I got time, but it will have to be waited for another week or so.

Copy link
Contributor

@SigmaSquadron SigmaSquadron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, just one final open question for anyone who knows about packagesFromDirectoryRecursive.

pkgs/top-level/all-packages.nix Outdated Show resolved Hide resolved
@cherrypiejam cherrypiejam force-pushed the typst branch 2 times, most recently from e753a45 to 7560346 Compare January 5, 2025 19:52
pkgs/development/typst-packages/cetz.nix Outdated Show resolved Hide resolved
pkgs/top-level/typst-packages.nix Outdated Show resolved Hide resolved
@drupol
Copy link
Contributor

drupol commented Jan 6, 2025

Would it be a good idea to have some documentation for this ?

@cherrypiejam
Copy link
Author

Would it be a good idea to have some documentation for this ?

I can write it, but where should I put it?

(Module updates) Added a release notes entry if the change is significant

Not sure if this change should be considered as significant

@cherrypiejam cherrypiejam force-pushed the typst branch 2 times, most recently from edf13ad to a19af78 Compare January 12, 2025 19:48
@cherrypiejam cherrypiejam requested a review from drupol January 12, 2025 19:50
pkgs/build-support/build-typst-package.nix Outdated Show resolved Hide resolved
pkgs/development/typst-packages/cetz.nix Outdated Show resolved Hide resolved
pkgs/build-support/build-typst-package.nix Outdated Show resolved Hide resolved
pkgs/build-support/build-typst-package.nix Outdated Show resolved Hide resolved
pkgs/by-name/ty/typst/with-packages.nix Outdated Show resolved Hide resolved
pkgs/build-support/build-typst-package.nix Show resolved Hide resolved
pkgs/development/typst-packages/cetz.nix Outdated Show resolved Hide resolved
pkgs/development/typst-packages/cetz.nix Outdated Show resolved Hide resolved
pkgs/development/typst-packages/cetz.nix Outdated Show resolved Hide resolved
@cherrypiejam cherrypiejam force-pushed the typst branch 3 times, most recently from 5e2199a to 7089bbd Compare January 15, 2025 20:13
@github-actions github-actions bot added 6.topic: python 8.has: maintainer-list (update) This PR changes `maintainers/maintainer-list.nix` labels Jan 15, 2025
Copy link
Contributor

@philiptaron philiptaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to see the widely scoped with removed and the function documented.

In the event that #234651 lands in the next few days, basing buildTypstPackage on top of that seems like a wise move. As I see it, this function is adapting stdenvNoCC.mkDerivation to provide a build environment for typst.

I haven't taken a look at the changes in this PR other than in pkgs/build-support.

pkgs/build-support/build-typst-package.nix Outdated Show resolved Hide resolved
pkgs/build-support/build-typst-package.nix Show resolved Hide resolved
pkgs/build-support/build-typst-package.nix Show resolved Hide resolved
@cherrypiejam
Copy link
Author

All packages from the Typst Universe is here, and I added a maintenance script to generate the attribution set for them. I guess right it's just the style and comments --- e.g., do I need to add some comments at the beginning of the generated list? I am up for suggestions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.has: maintainer-list (update) This PR changes `maintainers/maintainer-list.nix` 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux 12. first-time contribution This PR is the author's first one; please be gentle!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants