Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

haskellPackages: update hackage #339272

Merged
merged 85 commits into from
Oct 5, 2024
Merged

haskellPackages: update hackage #339272

merged 85 commits into from
Oct 5, 2024

Conversation

sternenseemann
Copy link
Member

This Merge

This PR is the regular merge of the haskell-updates branch into master.

This branch is being continually built and tested by hydra at https://hydra.nixos.org/jobset/nixpkgs/haskell-updates. You may be able to find an up-to-date Hydra build report at cdepillabout/nix-haskell-updates-status.

We roughly aim to merge these haskell-updates PRs at least once every two weeks. See the @NixOS/haskell team calendar for who is currently in charge of this branch.

haskellPackages Workflow Summary

Our workflow is currently described in pkgs/development/haskell-modules/HACKING.md.

The short version is this:

  • We regularly update the Stackage and Hackage pins on haskell-updates (normally at the beginning of a merge window).
  • The community fixes builds of Haskell packages on that branch.
  • We aim at at least one merge of haskell-updates into master every two weeks.
  • We only do the merge if the mergeable job is succeeding on hydra.
  • If a maintained package is still broken at the time of merge, we will only merge if the maintainer has been pinged 7 days in advance. (If you care about a Haskell package, become a maintainer!)

More information about Haskell packages in nixpkgs can be found in the nixpkgs manual.


This is the follow-up to #335932. Come to #haskell:nixos.org if you have any questions.

No new version of stackage at this time (2024-09-03)!

This commit has been generated by maintainers/scripts/haskell/update-hackage.sh
and maintainers/scripts/haskell/regenerate-hackage-packages.sh.
marcin-serwin and others added 8 commits September 3, 2024 19:47
fixed auto-update, http2, network-control, network-run and time-manager
by choosing different version.
fixed dejafu and tasty-coverage by jailbreaking.
fixed bsb-http-chunked, hinotify and warp by disabling test suite.
This commit has been generated by maintainers/scripts/haskell/regenerate-hackage-packages.sh
@collinarnett
Copy link

One thing I noticed with one of the new packages tiktoken is that it requires base64_1_0 instead of base64. Not sure how to contribute but I thought I would give a heads up while I'm testing using it before this pr gets merged.

@maralorn
Copy link
Member

maralorn commented Sep 7, 2024

@collinarnett are you sure this is not fixable by using doJailbreak? (you can Ctrl+F this in the nixpkgs manual or in our configuration-common.nix file in nixpkgs.) The first release of tiktoken is 7 months after the release of base64, so I assume they just never tried to compile it with an earlier version.

@collinarnett
Copy link

collinarnett commented Sep 7, 2024

I just tried it with doJailbreak and it breaks.

[1/0/4 built, 0.0 MiB DL] building tiktoken-1.0.3direnv: ([/nix/store/1mrvvx2ygfyfz8sn3gg5f3qidvd9s8j1-direnv-2.34.0/bin/direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.
error: builder for '/nix/store/mk0j2969fp6rsszzi4vfqxmqkbsqi9gx-tiktoken-1.0.3.drv' failed with exit code 1;
       last 25 log lines:
       > No pkg-config found
       > Using runghc version 9.6.5 found on system at:
       > /nix/store/hzdv2lb4sr83cbnx4c3fi6idpimvwab7-ghc-9.6.5/bin/runghc-9.6.5
       > Using strip version 2.41 found on system at:
       > /nix/store/md6hh4rrcrf99nssvcam3qaqs3skj086-gcc-wrapper-13.2.0/bin/strip
       > Using tar found on system at:
       > /nix/store/4m4vnjdy44s4sb8hlv4qsph492zz3rly-gnutar-1.35/bin/tar
       > No uhc found
       > Running phase: buildPhase
       > Preprocessing library for tiktoken-1.0.3..
       > Building library for tiktoken-1.0.3..
       > [1 of 2] Compiling Paths_tiktoken   ( dist/build/autogen/Paths_tiktoken.hs, dist/build/Paths_tiktoken.o, dist/build/Paths_tiktoken.dyn_o )
       > [2 of 2] Compiling Tiktoken         ( src/Tiktoken.hs, dist/build/Tiktoken.o, dist/build/Tiktoken.dyn_o )
       >
       > src/Tiktoken.hs:109:19: error: [GHC-76037]
       >     Not in scope: ‘Base64.Encoding.decodeBase64Untyped’
       >     NB: the module ‘Data.ByteString.Base64’ does not export ‘decodeBase64Untyped’.
       >     Suggested fix:
       >       Perhaps use one of these:
       >         ‘Base64.Encoding.decodeBase64’ (imported from Data.ByteString.Base64),
       >         ‘Base64.Encoding.decodeBase64Lenient’ (imported from Data.ByteString.Base64),
       >         ‘Base64.Encoding.encodeBase64’ (imported from Data.ByteString.Base64)
       >     |
       > 109 |     token <- case Base64.Encoding.decodeBase64Untyped base64Bytes of
       >     |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       For full logs, run 'nix log /nix/store/mk0j2969fp6rsszzi4vfqxmqkbsqi9gx-tiktoken-1.0.3.drv'.
error: 1 dependencies of derivation '/nix/store/iild3js51yv77mp29rk2zmdrklai7akw-ghc-9.6.5-with-packages.drv' failed to build
error: 1 dependencies of derivation '/nix/store/r21kjqdmf6cg91pcz07008zrlaab4jpl-ghc-shell-for-packages-0-env.drv' failed to build

In the repo's flake they explicitly override base64 to version 1.0 as well here. https://github.com/Gabriella439/tiktoken/blob/2e66d1e29acf66f08959f86080d53b07bccb5ae4/flake.nix#L18C1-L22C23

@maralorn
Copy link
Member

maralorn commented Sep 7, 2024

Thank you. If you want you can open a PR which does this override in configuration-common.nix with a comment explaining that at least base64 1.0 is required, but LTS 22 contains an older version. Then you can drop tiktoken from a file called broken.yaml and run maintainers/scripts/haskell/regenerate-hackage-packages.sh.

maralorn and others added 5 commits September 7, 2024 22:44
This is easy in comparison since these tools won't end up in GHC's
settings nor need to be available at runtime, so we can use
the *_FOR_BUILD environment variables.

It is important to add buildCC to depsBuildBuild to engage the
stdenv/wrapper script machinery properly.

Co-authored-by: sternenseemann <sternenseemann@systemli.org>
1. Explicitly set WITH_TERMINFO. We usually match GHC's behavior well,
   but it is better to tie the Nix option to make explicitly.
   Unfortunately, the same is very complicated to achieve with
   hadrian (iirc).
2. Disable enableTerminfo if we are cross-compiling. This matches
   the behavior of GHC's build system, so we'll have to match it now.
   It also reduces the ncurses-related headache a bit.
3. Stop passing --with-curses* flags. Unfortunately, GHC does not
   account for the fact that different platforms need different ncurses
   libraries. This is somewhat migitated by the fact that ncurses is
   only ever needed for the build platform if we are cross compiling,
   but I seem to remember it leaking into the final GHC somehow.
   A more reliable alternative is relying on the cc/ld wrapper scripts,
   as they'll always pull out the correct ncurses out of the environment
   when GHC's build system passes -lcurses.
4. Unconditionally add ncurses to depsBuildBuild. Stage0 unconditionally
   builds terminfo (maybe the stage1 compiler needs it?), so we need to
   make sure that ncurses for the build platform is available.

Co-authored-by: sternenseemann <sternenseemann@systemli.org>
@sternenseemann
Copy link
Member Author

9.10.1 failures we see sometimes are due to: https://gitlab.haskell.org/ghc/ghc/-/issues/25153

sternenseemann and others added 4 commits September 23, 2024 18:44
Some GHC bindists have a normal `$out/lib` directory which contains
symlinks to all core libs. Because it is a normal lib directory, the
bintools setup hook will pick up on it and cause ld to pass the
appropriate -L and -rpath flags. We do not want this to happen,
especially in the case of the stage2 compiler. Not only will the final
ghc have an unnecessary reference (and thus increased closure size) to
the binary ghc, but the extra libraries in the rpath mess with the rts
and cause e.g. segfaults in GHCi.

Unfortunately, there is no way to prevent this. It is a fundamental flaw
in the cc and bintools wrappers that they do not actually distinguish
between the roles of dependencies (build, host, target). Instead
the mangleVar* function will translate the dependencies split up by
roles into platforms. This means that the wrappers can't distinguish
between depsBuildBuild and depsHostTarget (== buildInputs) when natively
compiling. As long as we are natively compiling the wrappers will put
the stage0 ghc (be it in depsBuildBuild, nativeBuildInputs etc.) into
the linker flags of the final ghc.

The solution is to sidestep the issue. We just had ghc in depsBuildBuild
to have it added to PATH. GHC itself will pass the appropriate linker
flags if necessary. To avoid the setup hooks picking up on the GHC
libraries we just don't put it into depsBuildBuild or any other
dependency list. Since the GHC build system accepts the GHC binary via
an absolute path, we don't even need to add the stage0 GHC to PATH.
This seems to be a build time only dependency as there is no
corresponding settings entry.
Ideally we don't want to use bintools.bintools and also not really
encode knowledge of what is wrapped and what not in our GHC derivation.
Unfortunately, not all tools are part of the wrapper derivation as well.
This should be gradually improved (e.g. in the case of the darwin tools
and strip).
github-actions bot and others added 13 commits September 28, 2024 00:15
This commit has been generated by maintainers/scripts/haskell/mark-broken.sh based on
*evaluation [1809145](https://hydra.nixos.org/eval/1809145) of nixpkgs commit [3c833cf](https://github.com/NixOS/nixpkgs/commits/3c833cfcd12780ede91248757e0b4ab758d80a37) as of 2024-10-01 18:51 UTC*
from the haskell-updates jobset on hydra under https://hydra.nixos.org/jobset/nixpkgs/haskell-updates
Curiously when building a cross compiled riscv64 GHC, the libraries (for
the target) turn out way bigger than when building a riscv64 cross
compiler. Profiling libraries are not necessary for bootstrapping GHC,
so we can disable them for now. We may want to revisit this condition,
though, once we have a native bootstrapping path for riscv64-linux set
up.
@sternenseemann sternenseemann merged commit c95b3e3 into master Oct 5, 2024
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update request: haskellPackages.xmonad-contrib 0.18.0 → 0.18.1