Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmake: Regular rebasing of the cmake-staging branch #334

Closed
wants to merge 71 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
a2317e2
cmake: Add root `CMakeLists.txt` file
hebasto Apr 16, 2024
7068388
cmake: Introduce interface libraries to encapsulate common flags
hebasto Aug 16, 2024
fe5cdac
cmake: Print compiler and linker flags in summary
hebasto Jul 3, 2024
27d687f
cmake: Add `config/bitcoin-config.h` support
hebasto Aug 6, 2024
07069e2
cmake: Add introspection module
hebasto Mar 23, 2024
fd72d00
cmake: Add position independent code support
hebasto Jan 9, 2024
35cffc4
cmake: Add POSIX threads support
hebasto Jun 30, 2024
4a0af29
cmake: Add `TryAppendCXXFlags` module
hebasto Jan 9, 2024
f983279
cmake: Add `TryAppendLinkerFlag` module
hebasto Jan 9, 2024
b6b5e73
cmake: Add global compiler and linker flags
hebasto Jul 24, 2024
cedfdf6
cmake: Redefine/adjust per-configuration flags
hebasto Aug 5, 2024
dbb7ed1
cmake: Add `ccache` support
hebasto Jun 30, 2024
db7a198
cmake: Build `secp256k1` subtree
hebasto Jul 3, 2024
51985c5
cmake: Build `crc32c` static library
hebasto Jul 13, 2024
12bfbc8
cmake: Build `leveldb` static library
hebasto Jul 3, 2024
1f0a78e
cmake: Build `minisketch` static library
hebasto Jul 3, 2024
752747f
cmake: Generate `obj/build.h` header
hebasto Aug 16, 2024
958971f
cmake: Build `univalue` static library
hebasto Jan 9, 2024
0a9a521
cmake: Build `bitcoin_crypto` library
hebasto Jul 24, 2024
809a2f1
cmake: Build `bitcoin_util` static library
hebasto Jun 30, 2024
3118e40
cmake: Build `bitcoin_consensus` library
hebasto Jun 30, 2024
97829ce
cmake: Add `FindLibevent` module
hebasto Jul 24, 2024
a9813df
cmake: Build `bitcoind` executable
hebasto Jul 13, 2024
b27bf97
cmake: Build `bitcoin-cli` executable
hebasto May 20, 2024
959370b
cmake: Build `test_bitcoin` executable
hebasto Aug 10, 2024
ab2e99b
cmake: Create test suite for `ctest`
hebasto Jul 24, 2024
d10c5c3
cmake: Add wallet functionality
hebasto Jul 24, 2024
027c6d7
cmake: Build `bitcoin-tx` executable
hebasto Jun 30, 2024
e73e930
cmake: Build `bitcoin-util` executable
hebasto Jan 9, 2024
6480e1d
cmake: Add `libnatpmp` optional package support
hebasto Jun 4, 2024
ae7b39a
cmake: Add `libminiupnpc` optional package support
hebasto Apr 29, 2024
d2fda82
cmake: Add `libzmq` optional package support
hebasto Apr 29, 2024
353e0c9
cmake: Add `systemtap-sdt` optional package support
hebasto Jul 24, 2024
8017351
cmake: Add external signer support
hebasto Jul 24, 2024
8bb0e85
cmake: Build `bench_bitcoin` executable
hebasto Jul 31, 2024
908530e
cmake: Add `SANITIZERS` option
hebasto Jul 3, 2024
3d85379
cmake: Add fuzzing options
hebasto Aug 16, 2024
a8a2e36
cmake: Add Python-based tests
hebasto Aug 6, 2024
a01cb6e
cmake: Add `HARDENING` option
hebasto Jun 30, 2024
c98d4a4
cmake: Add `REDUCE_EXPORTS` option
hebasto Jul 3, 2024
30f6429
cmake: Add `WERROR` option
hebasto Jul 3, 2024
57a6e2e
cmake: Build `bitcoin-qt` executable
hebasto Jul 13, 2024
5bb5a4b
cmake: Add `libqrencode` optional package support
hebasto Jul 24, 2024
10fcc66
cmake: Add `WITH_DBUS` option
hebasto Jul 24, 2024
975d673
cmake: Build `test_bitcoin-qt` executable
hebasto Aug 16, 2024
aed38ea
cmake: Build `bitcoinkernel` library
hebasto Jul 13, 2024
bb1a450
cmake: Build `bitcoin-chainstate` executable
hebasto Jun 30, 2024
90cec4d
cmake: Add `MULTIPROCESS` option
hebasto Aug 6, 2024
6522af6
depends: Amend handling flags environment variables
hebasto Feb 4, 2024
4a5208a
Revert "build, qt: Do not install *.prl files"
hebasto Jan 17, 2024
9b31209
depends: Rename `cmake_system` -> `cmake_system_name`
hebasto May 15, 2024
91a7992
depends: Add host-specific `cmake_system_version` variables
hebasto May 15, 2024
0d01c22
build: Generate `toolchain.cmake` in depends
hebasto Jul 31, 2024
84ac35c
cmake: Add cross-compiling support
hebasto Jul 3, 2024
973a3b0
cmake: Implement `install` build target
hebasto Jul 24, 2024
2b43c45
cmake: Add `AddWindowsResources` module
hebasto Jan 27, 2024
1f60b30
cmake: Add `APPEND_{CPP,C,CXX,LD}FLAGS` cache variables
hebasto Apr 26, 2024
747adb6
cmake: Add `Maintenance` module
hebasto Aug 16, 2024
e821f0a
cmake: Migrate Guix build scripts to CMake
hebasto Jun 4, 2024
fb75ebb
cmake: Add compiler diagnostic flags
hebasto Jun 29, 2024
65bdbc1
cmake: Add `docs` build target
hebasto Apr 23, 2024
8b6f1c4
cmake: Add `Coverage` and `CoverageFuzz` scripts
hebasto Aug 5, 2024
7681746
cmake: Add vcpkg manifest file
hebasto Jun 4, 2024
3885441
cmake: Add presets for native Windows builds
hebasto Jul 31, 2024
c360837
cmake, lint: Adjust `lint_includes_build_config`
hebasto Apr 28, 2024
9730288
ci: Migrate CI scripts to CMake
hebasto Jul 24, 2024
6ce50fd
doc: Update for CMake-based build system
hebasto Jul 24, 2024
4105129
cmake: Ignore build subdirectories within source directory
hebasto Aug 5, 2024
d1ee64e
[STAGING BRANCH ONLY] Add CI-specific presets
hebasto Jul 31, 2024
a2a5805
[STAGING BRANCH ONLY] ci: Test CMake edge cases
hebasto Jun 30, 2024
818a3c0
[STAGING BRANCH ONLY] ci: Skip `miniupnpc` package in depends
hebasto Jul 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 34 additions & 115 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,12 +67,12 @@ jobs:
echo "TEST_BASE=$(git rev-list -n$((${{ env.MAX_COUNT }} + 1)) --reverse HEAD $EXCLUDE_MERGE_BASE_ANCESTORS | head -1)" >> "$GITHUB_ENV"
- run: |
sudo apt-get update
sudo apt-get install clang ccache build-essential libtool autotools-dev automake pkg-config bsdmainutils python3-zmq libevent-dev libboost-dev libsqlite3-dev libdb++-dev systemtap-sdt-dev libminiupnpc-dev libnatpmp-dev qtbase5-dev qttools5-dev qttools5-dev-tools qtwayland5 libqrencode-dev -y
sudo apt-get install clang ccache build-essential cmake pkg-config bsdmainutils python3-zmq libevent-dev libboost-dev libsqlite3-dev libdb++-dev systemtap-sdt-dev libminiupnpc-dev libnatpmp-dev qtbase5-dev qttools5-dev qttools5-dev-tools qtwayland5 libqrencode-dev -y
- name: Compile and run tests
run: |
# Run tests on commits after the last merge commit and before the PR head commit
# Use clang++, because it is a bit faster and uses less memory than g++
git rebase --exec "echo Running test-one-commit on \$( git log -1 ) && ./autogen.sh && CC=clang CXX=clang++ ./configure --with-incompatible-bdb && make clean && make -j $(nproc) check && ./test/functional/test_runner.py -j $(( $(nproc) * 2 ))" ${{ env.TEST_BASE }}
git rebase --exec "echo Running test-one-commit on \$( git log -1 ) && CC=clang CXX=clang++ cmake -B build -DBUILD_GUI=ON -DBUILD_BENCH=ON -DBUILD_FUZZ_BINARY=ON -DWITH_BDB=ON -DWITH_NATPMP=ON -DWITH_MINIUPNPC=ON -DWITH_USDT=ON && cmake --build build -j $(nproc) && ctest --test-dir build -j $(nproc) && ./build/test/functional/test_runner.py -j $(( $(nproc) * 2 ))" ${{ env.TEST_BASE }}

macos-native-x86_64:
name: 'macOS 13 native, x86_64, no depends, sqlite only, gui'
Expand Down Expand Up @@ -139,18 +139,17 @@ jobs:
if: github.repository != 'bitcoin-core/gui' || github.event_name == 'pull_request'

env:
CCACHE_MAXSIZE: '200M'
CI_CCACHE_VERSION: '4.7.5'
CI_QT_CONF: '-release -silent -opensource -confirm-license -opengl desktop -static -static-runtime -mp -qt-zlib -qt-pcre -qt-libpng -nomake examples -nomake tests -nomake tools -no-angle -no-dbus -no-gif -no-gtk -no-ico -no-icu -no-libjpeg -no-libudev -no-sql-sqlite -no-sql-odbc -no-sqlite -no-vulkan -skip qt3d -skip qtactiveqt -skip qtandroidextras -skip qtcharts -skip qtconnectivity -skip qtdatavis3d -skip qtdeclarative -skip doc -skip qtdoc -skip qtgamepad -skip qtgraphicaleffects -skip qtimageformats -skip qtlocation -skip qtlottie -skip qtmacextras -skip qtmultimedia -skip qtnetworkauth -skip qtpurchasing -skip qtquick3d -skip qtquickcontrols -skip qtquickcontrols2 -skip qtquicktimeline -skip qtremoteobjects -skip qtscript -skip qtscxml -skip qtsensors -skip qtserialbus -skip qtserialport -skip qtspeech -skip qtsvg -skip qtvirtualkeyboard -skip qtwayland -skip qtwebchannel -skip qtwebengine -skip qtwebglplugin -skip qtwebsockets -skip qtwebview -skip qtx11extras -skip qtxmlpatterns -no-openssl -no-feature-bearermanagement -no-feature-printdialog -no-feature-printer -no-feature-printpreviewdialog -no-feature-printpreviewwidget -no-feature-sql -no-feature-sqlmodel -no-feature-textbrowser -no-feature-textmarkdownwriter -no-feature-textodfwriter -no-feature-xml'
CI_QT_DIR: 'qt-everywhere-src-5.15.11'
CI_QT_URL: 'https://download.qt.io/official_releases/qt/5.15/5.15.11/single/qt-everywhere-opensource-src-5.15.11.zip'
PYTHONUTF8: 1
TEST_RUNNER_TIMEOUT_FACTOR: 40

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Remove non-MSVC tool installations
run: |
Remove-Item -Path "$env:ProgramFiles/CMake" -Recurse -Force

- name: Configure Developer Command Prompt for Microsoft Visual C++
# Using microsoft/setup-msbuild is not enough.
uses: ilammy/msvc-dev-cmd@v1
Expand All @@ -159,141 +158,60 @@ jobs:

- name: Get tool information
run: |
cmake -version | Tee-Object -FilePath "cmake_version"
Write-Output "---"
msbuild -version | Tee-Object -FilePath "msbuild_version"
$env:VCToolsVersion | Tee-Object -FilePath "toolset_version"
$env:CI_QT_URL | Out-File -FilePath "qt_url"
$env:CI_QT_CONF | Out-File -FilePath "qt_conf"
py -3 --version
Write-Host "PowerShell version $($PSVersionTable.PSVersion.ToString())"

- name: Restore static Qt cache
id: static-qt-cache
uses: actions/cache/restore@v4
with:
path: C:\Qt_static
key: ${{ github.job }}-static-qt-${{ hashFiles('msbuild_version', 'qt_url', 'qt_conf') }}

- name: Build static Qt. Download
if: steps.static-qt-cache.outputs.cache-hit != 'true'
shell: cmd
run: |
curl --location --output C:\qt-src.zip %CI_QT_URL%
choco install --yes --no-progress jom

- name: Build static Qt. Expand source archive
if: steps.static-qt-cache.outputs.cache-hit != 'true'
shell: cmd
run: tar -xf C:\qt-src.zip -C C:\

- name: Build static Qt. Create build directory
if: steps.static-qt-cache.outputs.cache-hit != 'true'
run: |
Rename-Item -Path "C:\$env:CI_QT_DIR" -NewName "C:\qt-src"
New-Item -ItemType Directory -Path "C:\qt-src\build"

- name: Build static Qt. Configure
if: steps.static-qt-cache.outputs.cache-hit != 'true'
working-directory: C:\qt-src\build
shell: cmd
run: ..\configure %CI_QT_CONF% -prefix C:\Qt_static

- name: Build static Qt. Build
if: steps.static-qt-cache.outputs.cache-hit != 'true'
working-directory: C:\qt-src\build
shell: cmd
run: jom

- name: Build static Qt. Install
if: steps.static-qt-cache.outputs.cache-hit != 'true'
working-directory: C:\qt-src\build
shell: cmd
run: jom install

- name: Save static Qt cache
if: steps.static-qt-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v4
with:
path: C:\Qt_static
key: ${{ github.job }}-static-qt-${{ hashFiles('msbuild_version', 'qt_url', 'qt_conf') }}

- name: Ccache installation cache
id: ccache-installation-cache
uses: actions/cache@v4
with:
path: |
C:\ProgramData\chocolatey\lib\ccache
C:\ProgramData\chocolatey\bin\ccache.exe
C:\ccache\cl.exe
key: ${{ github.job }}-ccache-installation-${{ env.CI_CCACHE_VERSION }}

- name: Install Ccache
if: steps.ccache-installation-cache.outputs.cache-hit != 'true'
run: |
choco install --yes --no-progress ccache --version=$env:CI_CCACHE_VERSION
New-Item -ItemType Directory -Path "C:\ccache"
Copy-Item -Path "$env:ChocolateyInstall\lib\ccache\tools\ccache-$env:CI_CCACHE_VERSION-windows-x86_64\ccache.exe" -Destination "C:\ccache\cl.exe"

- name: Restore Ccache cache
id: ccache-cache
uses: actions/cache/restore@v4
with:
path: ~/AppData/Local/ccache
key: ${{ github.job }}-ccache-${{ github.run_id }}
restore-keys: ${{ github.job }}-ccache-

- name: Using vcpkg with MSBuild
run: |
Set-Location "$env:VCPKG_INSTALLATION_ROOT"
Add-Content -Path "triplets\x64-windows.cmake" -Value "set(VCPKG_BUILD_TYPE release)"
Add-Content -Path "triplets\x64-windows-static.cmake" -Value "set(VCPKG_BUILD_TYPE release)"
.\vcpkg.exe --vcpkg-root "$env:VCPKG_INSTALLATION_ROOT" integrate install
git rev-parse HEAD | Tee-Object -FilePath "$env:GITHUB_WORKSPACE\vcpkg_commit"

- name: vcpkg tools cache
uses: actions/cache@v4
with:
path: C:/vcpkg/downloads/tools
key: ${{ github.job }}-vcpkg-tools

- name: vcpkg binary cache
uses: actions/cache@v4
- name: Restore vcpkg binary cache
uses: actions/cache/restore@v4
id: vcpkg-binary-cache
with:
path: ~/AppData/Local/vcpkg/archives
key: ${{ github.job }}-vcpkg-binary-${{ hashFiles('vcpkg_commit', 'msbuild_version', 'toolset_version', 'build_msvc/vcpkg.json') }}

- name: Generate project files
run: py -3 build_msvc\msvc-autogen.py
key: ${{ github.job }}-vcpkg-binary-${{ hashFiles('cmake_version', 'msbuild_version', 'toolset_version', 'vcpkg.json') }}

- name: Build
shell: cmd
- name: Generate build system
run: |
ccache --zero-stats
msbuild build_msvc\bitcoin.sln -property:CLToolPath=C:\ccache;CLToolExe=cl.exe;UseMultiToolTask=true;Configuration=Release -maxCpuCount -verbosity:minimal -noLogo

- name: Ccache stats
run: ccache --show-stats
cmake -B build --preset vs2022-static -DCMAKE_TOOLCHAIN_FILE="$env:VCPKG_INSTALLATION_ROOT\scripts\buildsystems\vcpkg.cmake" -DBUILD_BENCH=ON -DBUILD_FUZZ_BINARY=ON -DWERROR=ON

- name: Save Ccache cache
- name: Save vcpkg binary cache
uses: actions/cache/save@v4
if: github.event_name != 'pull_request' && steps.ccache-cache.outputs.cache-hit != 'true'
if: github.event_name != 'pull_request' && steps.vcpkg-binary-cache.outputs.cache-hit != 'true'
with:
path: ~/AppData/Local/ccache
# https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache
key: ${{ github.job }}-ccache-${{ github.run_id }}

- name: Run unit tests
run: src\test_bitcoin.exe -l test_suite

- name: Run benchmarks
run: src\bench_bitcoin.exe -sanity-check
path: ~/AppData/Local/vcpkg/archives
key: ${{ github.job }}-vcpkg-binary-${{ hashFiles('cmake_version', 'msbuild_version', 'toolset_version', 'vcpkg.json') }}

- name: Run util tests
run: py -3 test\util\test_runner.py
- name: Build
working-directory: build
run: |
cmake --build . -j $env:NUMBER_OF_PROCESSORS --config Release

- name: Run rpcauth test
run: py -3 test\util\rpcauth-test.py
- name: Run test suite
working-directory: build
run: |
ctest -j $env:NUMBER_OF_PROCESSORS -C Release

- name: Run functional tests
working-directory: build
env:
BITCOIND: '${{ github.workspace }}\build\src\Release\bitcoind.exe'
BITCOINCLI: '${{ github.workspace }}\build\src\Release\bitcoin-cli.exe'
BITCOINUTIL: '${{ github.workspace }}\build\src\Release\bitcoin-util.exe'
BITCOINWALLET: '${{ github.workspace }}\build\src\Release\bitcoin-wallet.exe'
TEST_RUNNER_EXTRA: ${{ github.event_name != 'pull_request' && '--extended' || '' }}
shell: cmd
run: py -3 test\functional\test_runner.py --jobs %NUMBER_OF_PROCESSORS% --ci --quiet --tmpdirprefix=%RUNNER_TEMP% --combinedlogslen=99999999 --timeout-factor=%TEST_RUNNER_TIMEOUT_FACTOR% %TEST_RUNNER_EXTRA%
Expand All @@ -306,8 +224,9 @@ jobs:
git log -1

- name: Run fuzz binaries
working-directory: build
env:
BITCOINFUZZ: "${{ github.workspace}}\\src\\fuzz.exe"
BITCOINFUZZ: '${{ github.workspace }}\build\src\test\fuzz\Release\fuzz.exe'
shell: cmd
run: py -3 test\fuzz\test_runner.py --par %NUMBER_OF_PROCESSORS% --loglevel DEBUG %RUNNER_TEMP%\qa-assets\fuzz_seed_corpus

Expand Down
Loading
Loading