Add Justfile and Docker reproducible build#132
Conversation
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. WalkthroughAdds reproducible firmware build infrastructure and developer automation: a multi-stage reproducible Docker build, a comprehensive Justfile with build/test/verify/fuzz targets, reproducibility documentation, .gitignore additions, and SPDX annotations in REUSE.toml. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Dev as Developer
participant Local as Local host / just
participant Docker as Docker builder
participant Git as External repos
participant IDF as ESP-IDF toolchain
participant Out as /out artifacts
Dev->>Local: run `just docker-build` (or `docker build`)
Local->>Docker: start builder stage (Dockerfile.reproducible)
Docker->>Git: clone pinned repos (secp256k1-frost, libnostr-c, noscrypt, libwally-core)
Docker->>IDF: invoke build (idf.py / cmake) with SOURCE_DATE_EPOCH
IDF-->>Docker: produce firmware binaries (keep.bin, bootloader.bin, ...)
Docker->>Docker: merge binaries (esptool) -> keep-merged.bin
Docker->>Docker: compute SHA256 checksums
Docker->>Out: place artifacts and checksums into `/out`
Docker-->>Local: export final image containing `/out`
Local-->>Dev: artifacts available for verify-release / verify-device
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@Justfile`:
- Around line 105-118: The verify-device target uses GNU-specific stat -c%s to
set SIZE which breaks on macOS; change the SIZE computation in the verify-device
recipe to a cross-platform method (e.g., use "wc -c" or a small portable
python/perl one-liner) so SIZE correctly contains the byte length of
output/keep.bin, ensure you trim whitespace/newline from the command output
before using SIZE in the esptool.py call, and keep the rest of the target (TMP,
trap, DEVICE_HASH/BUILT_HASH comparisons) unchanged.
🧹 Nitpick comments (3)
Dockerfile.reproducible (2)
14-21: Consider shallow clones to reduce build time and layer size.Using
--depth=1for git clones would significantly reduce download time and layer size since only the specific commit is needed.♻️ Suggested improvement
-RUN git clone https://github.com/privkeyio/secp256k1-frost.git && \ - cd secp256k1-frost && git checkout 832bac2dbfaf4c35058cd5dadb0f8fe093cb6c65 && cd .. && \ - git clone https://github.com/privkeyio/libnostr-c.git && \ - cd libnostr-c && git checkout 4e8d01b8d20680204429aaf3680d069ec222fdaa && cd .. && \ - git clone https://github.com/privkeyio/noscrypt.git && \ - cd noscrypt && git checkout 45fcbef32f958145d6797f384a225c0d43616886 && cd .. && \ - git clone https://github.com/ElementsProject/libwally-core.git && \ - cd libwally-core && git checkout 242f841af6819b5d5174da5a9593907f56f1bc89 && cd .. +RUN git clone --depth=1 https://github.com/privkeyio/secp256k1-frost.git && \ + cd secp256k1-frost && git fetch --depth=1 origin 832bac2dbfaf4c35058cd5dadb0f8fe093cb6c65 && git checkout FETCH_HEAD && cd .. && \ + git clone --depth=1 https://github.com/privkeyio/libnostr-c.git && \ + cd libnostr-c && git fetch --depth=1 origin 4e8d01b8d20680204429aaf3680d069ec222fdaa && git checkout FETCH_HEAD && cd .. && \ + git clone --depth=1 https://github.com/privkeyio/noscrypt.git && \ + cd noscrypt && git fetch --depth=1 origin 45fcbef32f958145d6797f384a225c0d43616886 && git checkout FETCH_HEAD && cd .. && \ + git clone --depth=1 https://github.com/ElementsProject/libwally-core.git && \ + cd libwally-core && git fetch --depth=1 origin 242f841af6819b5d5174da5a9593907f56f1bc89 && git checkout FETCH_HEAD && cd ..
45-46: Use directory copy instead of wildcard to avoid BuildKit glob expansion issues.The
COPY --from=builder /out/* /syntax with wildcards has documented inconsistencies in BuildKit, including skipped files and non-standard glob expansion behavior. Additionally, wildcards do not copy hidden files (names beginning with.), while copying the directory itself does.♻️ Suggested fix
FROM scratch AS export -COPY --from=builder /out/* / +COPY --from=builder /out/ /This syntax copies all contents of
/out(including hidden files) into the root of the scratch image without relying on glob expansion.Justfile (1)
87-103: Add error handling for failed downloads.If the curl download fails (network error, 404, etc.), the script continues and may produce confusing errors. Consider adding
-fflag to curl or explicit error checking.♻️ Suggested improvement
TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT - curl -sL "https://github.com/privkeyio/keep-esp32/releases/download/{{version}}/keep-esp32-firmware-{{version}}.tar.gz" | tar -xz -C "$TMP" + curl -sfL "https://github.com/privkeyio/keep-esp32/releases/download/{{version}}/keep-esp32-firmware-{{version}}.tar.gz" -o "$TMP/release.tar.gz" || { echo "Failed to download release {{version}}"; exit 1; } + tar -xzf "$TMP/release.tar.gz" -C "$TMP"
kwsantiago
force-pushed
the
Justfile-and-Docker
branch
from
d2d12c0 to
e4592d1
Compare
2 participants
Summary
Test plan
just testto verify native tests passjust docker-buildto build firmwareVerification Results
Native Tests: 11/11 passing (test_storage, test_secure_element, test_secresult, test_hw_entropy, test_psbt_fraud, test_integration, test_self_test, test_hw_entropy_sha256, test_session, test_anti_glitch, test_psbt_fraud_integration)
Docker Build: Successful - produces keep.bin (897KB), keep-merged.bin (962KB)
Reproducibility: Verified - consecutive builds produce identical hashes:
Summary by CodeRabbit
New Features
Documentation
Chores
✏️ Tip: You can customize this high-level summary in your review settings.