Not working with image >2GB

[vaibhavsagar]

Hi, I can't seem to get this working when I try to convert a container image built by Nix to WASM. Here's what my Nix expression looks like:

# default.nix
let
  nixpkgs-src = builtins.fetchTarball {
    url = "https://github.com/NixOS/nixpkgs/tarball/7731670498f0a22c361c9d68f73d382bce05d7dc";
    sha256 = "sha256-M59UCWsadEFO+DMdZFMtTlxpFyT3RQIpBLuEX8oziVc=";
  };
  nixpkgs = import nixpkgs-src {};
  haskellEnv = nixpkgs.haskellPackages.ghcWithPackages (p: [ ]);
  image = nixpkgs.dockerTools.buildLayeredImage {
    name = "small-haskell";
    tag = "latest";
    contents =  [
      haskellEnv
      nixpkgs.bashInteractive
    ];
    config = {
      Cmd = ["${haskellEnv}/bin/ghci"];
    };
  };
in image

And here are the commands I'm running:

$ nix-build default.nix
$ docker load < result
$ c2w --version
c2w version v0.6.2
$ c2w small-haskell:latest out.wasm
<...>
#120 [bochs-dev-packed 1/2] RUN /tools/wasi-vfs/wasi-vfs pack /Bochs/bochs/bochs --mapdir /pack::/minpack -o packed && mkdir /out
#120 128.3 panicked at 'capacity overflow', library/alloc/src/raw_vec.rs:517:5
#120 128.4 thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: the `wasi_vfs_pack_fs` function trapped
#120 128.4 
#120 128.4 Caused by:
#120 128.4     wasm trap: wasm `unreachable` instruction executed
#120 128.4     wasm backtrace:
#120 128.4         0: 0x57515 - <unknown>!<wasm function 540>
#120 128.4         1: 0x5aa96 - <unknown>!<wasm function 567>
#120 128.4         2: 0x5880b - <unknown>!<wasm function 549>
#120 128.4         3: 0x2e180 - <unknown>!<wasm function 229>
#120 128.4         4: 0x3688d - <unknown>!<wasm function 293>
#120 128.4         5: 0x3ce1a - <unknown>!<wasm function 330>
#120 128.4         6: 0x513aa - <unknown>!<wasm function 497>
#120 128.4     ', crates/wasi-vfs-cli/src/bin/wasi-vfs.rs:5:32
#120 128.4 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
#120 ERROR: process "/bin/sh -c /tools/wasi-vfs/wasi-vfs pack /Bochs/bochs/bochs --mapdir /pack::/minpack -o packed && mkdir /out" did not complete successfully: exit code: 101
------
 > [bochs-dev-packed 1/2] RUN /tools/wasi-vfs/wasi-vfs pack /Bochs/bochs/bochs --mapdir /pack::/minpack -o packed && mkdir /out:
128.4     wasm backtrace:
128.4         0: 0x57515 - <unknown>!<wasm function 540>
128.4         1: 0x5aa96 - <unknown>!<wasm function 567>
128.4         2: 0x5880b - <unknown>!<wasm function 549>
128.4         3: 0x2e180 - <unknown>!<wasm function 229>
128.4         4: 0x3688d - <unknown>!<wasm function 293>
128.4         5: 0x3ce1a - <unknown>!<wasm function 330>
128.4         6: 0x513aa - <unknown>!<wasm function 497>
128.4     ', crates/wasi-vfs-cli/src/bin/wasi-vfs.rs:5:32
128.4 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
------
container2wasm1665798681:488
--------------------
 486 |     
 487 |     FROM bochs-dev-${OPTIMIZATION_MODE} AS bochs-dev-packed
 488 | >>> RUN /tools/wasi-vfs/wasi-vfs pack /Bochs/bochs/bochs --mapdir /pack::/minpack -o packed && mkdir /out
 489 |     ARG OUTPUT_NAME
 490 |     RUN mv packed /out/$OUTPUT_NAME
--------------------
ERROR: failed to solve: process "/bin/sh -c /tools/wasi-vfs/wasi-vfs pack /Bochs/bochs/bochs --mapdir /pack::/minpack -o packed && mkdir /out" did not complete successfully: exit code: 101
exit status 1

Am I using the tool incorrectly?

[ktock]

@vaibhavsagar Thanks for reporting this.

#120 128.3 panicked at 'capacity overflow', library/alloc/src/raw_vec.rs:517:5

Seems to related to memory? Could you check if it causes out of memory?

[vaibhavsagar]

Thanks for your response! Where do you do think I'm running out of memory, is it the Docker daemon, and how would I check this? The computer I'm running this on has 32GB of RAM installed which I think should be enough?

[vaibhavsagar]

I ran it again while watching htop in a separate window and I got the same error even though total system RAM usage did not go above 3GB. I tried having docker stats open as well but weirdly did not see anything.

[ktock]

It seems that wasi-vfs fails to package large files that are over several GB. I keep on investigating the root cause but, as a workaround, you can avoid the failure by using a smaller image. (On my machine, small-haskell is 2.79GB. Could you create smaller image if possible?)

[vaibhavsagar]

Thanks for investigating! I also tried with haskell:slim-buster and got the same error, even though it's a little smaller (2.28GB). Do you know what the size limit is?

[vaibhavsagar]

I managed to get this working with a <1.5GB container, I'm guessing the limit is 2GB.