cmake: Remove legacy MSVC build system by hebasto · Pull Request #117 · hebasto/bitcoin

hebasto · 2024-03-09T14:46:27Z

At this stage, the new CMake-based build system offers a comprehensive set of features, surpassing those of the legacy build system located in the build_msvc directory.

As part of the transition to the new build system, the legacy MSVC build system has been removed.

Additionally, a new "Windows / MSVC Build Guide" has been introduced, based on https://gist.github.com/hebasto/32c77c0cd592d13685437271ac6b79f6. Notably, the guide lacks information regarding the disabling of base address randomization (see bitcoin#21045). Even with -DHARDENING=OFF, this feature remains enabled by default.

hebasto · 2024-03-09T14:49:12Z

Friendly ping @sipsorcery @pablomartin4btc @m3dwards @TheCharlatan @vasild :)

Also cc @EthanHeilman regarding the base address randomization feature.

EthanHeilman · 2024-03-12T20:07:05Z

--disable-hardening doesn't work anymore? Is this just a windows think or is it broken on linux as well?

I ran into issues with this on OSX:

To disable PIE in bitcoind on OSX run:
make clean
make all LDFLAGS="$LDFLAGS -Wl,-no_pie" 

to check it worked (compare imgs below):
otool -hv src/bitcoind

hebasto · 2024-03-15T09:50:45Z

--disable-hardening doesn't work anymore? Is this just a windows think or is it broken on linux as well?

I ran into issues with this on OSX:
To disable PIE in bitcoind on OSX run:
make clean
make all LDFLAGS="$LDFLAGS -Wl,-no_pie" 

to check it worked (compare imgs below):
otool -hv src/bitcoind  

On the master branch, while --disable-hardening means "do not attempt to harden the resulting executables", which implies it does not force non-hardening preprocessor macros, compiler and linker flags.

The new CMake-based build system preserves that behaviour.

EthanHeilman · 2024-03-15T19:00:04Z

On the master branch, while --disable-hardening means "do not attempt to harden the resulting executables", which implies it does not force non-hardening preprocessor macros, compiler and linker flags.
The new CMake-based build system preserves that behaviour.

So if we wanted the ability to disable windows base address randomization we would need to add it as a CMake flag?

hebasto · 2024-03-16T20:52:22Z

On the master branch, while --disable-hardening means "do not attempt to harden the resulting executables", which implies it does not force non-hardening preprocessor macros, compiler and linker flags.
The new CMake-based build system preserves that behaviour.

So if we wanted the ability to disable windows base address randomization we would need to add it as a CMake flag?

Yes. Something like that:

cmake -B build --preset vs2022 -DHARDENING=OFF -DCMAKE_EXE_LINKER_FLAGS="/DYNAMICBASE:NO"

I'm still curios about the use cases for binaries (1) with disabled ASLR and (2) with disabled PIE. Is it worth considering adding dedicated build configuration options for such cases?

EthanHeilman · 2024-03-17T22:32:20Z

I'm still curios about the use cases for binaries (1) with disabled ASLR and (2) with disabled PIE.

My main usecase is that bitcoin uses process memory as a source for randomness for it's RNG. In measuring and testing the randomnesss provided by the RNG, I turn all this off to make it is easier to isolate randomness problems. This can also be useful when attempting to replicate heisenbugs. That's a very boutique use case though,

Is it worth considering adding dedicated build configuration options for such cases?

I'll probably add them back in in a PR when restarting the RNG measurement project. It would save me some time if someone else readded them, but given that as far as I am aware I am the primary user of this feature it seems a little unfair to ask someone else to do the work.

hebasto · 2024-03-26T10:46:39Z

Rebased.

Amend "displayName" fields.

hebasto · 2024-04-03T16:33:27Z

Rebased.

sipsorcery · 2024-04-03T21:27:16Z

utACK (winx64) 9c0bf9b.

m3dwards · 2024-04-04T12:40:33Z

+cmake -S .. --preset vs2022-static  # It might take a while if the vcpkg binary cache is unpopulated or invalidated
+cmake --build . --config Release
+ctest --build-config Release
+cmake --install . --config Release  # Optional


Is there a reason why install is listed as part of the static linking build and not the dynamic linking build?

For dynamic linking build, dependency DLLs must be located in the same folder as a binary. That is not implemented for now.

m3dwards · 2024-04-04T12:41:03Z

+One can skip vcpkg manifest default features to speedup the configuration step.
+For example, the following invocation will skip all features except for "wallet" and "tests" and their dependencies:
+```
+cmake -S .. --preset vs2022 -DVCPKG_MANIFEST_NO_DEFAULT_FEATURES=ON -DVCPKG_MANIFEST_FEATURES="wallet;tests"


Is there a way to explain or list what the other manifest features are?

The features are listed in the <root>\vcpkg.json file. What wording would you suggest to list them in the docs?

Available features are listed in the <root>\vcpkg.json file.

Thanks! Added.

m3dwards · 2024-04-04T12:47:31Z

Should the guide mention ccache?

What about only building release versions of dependencies with:

Add-Content -Path "$env:VCPKG_ROOT\triplets\x64-windows.cmake" -Value "set(VCPKG_BUILD_TYPE release)"
Add-Content -Path "$env:VCPKG_ROOT\triplets\${{ matrix.conf.triplet }}.cmake" -Value "set(VCPKG_BUILD_TYPE release)"

hebasto · 2024-04-04T12:51:52Z

Should the guide mention ccache?

It is mentioned in https://github.com/bitcoin/bitcoin/blob/master/doc/productivity.md. No other platform-specific build doc mentions it explicitly.

What about only building release versions of dependencies with:

Add-Content -Path "$env:VCPKG_ROOT\triplets\x64-windows.cmake" -Value "set(VCPKG_BUILD_TYPE release)"
Add-Content -Path "$env:VCPKG_ROOT\triplets\${{ matrix.conf.triplet }}.cmake" -Value "set(VCPKG_BUILD_TYPE release)"

We use this optimization for the CI to speed up the build. However, on the user's machine this vcpkg patching could break other software builds.

m3dwards · 2024-04-04T12:55:03Z

I've found that when I change configuration I have to delete the build dir and start again with a fresh one. I assume this is normal and if it is, should this be mentioned?

hebasto · 2024-04-04T12:58:04Z

I've found that when I change configuration I have to delete the build dir and start again with a fresh one. I assume this is normal and if it is, should this be mentioned?

These words confuse me a bit, as the MSVC generator is multi-configuration. Could you be more specific please?

m3dwards · 2024-04-04T13:09:08Z

If I run:

mkdir build
cd build
cmake -S .. --preset vs2022-static
cmake -S .. --preset vs2022 -DVCPKG_MANIFEST_NO_DEFAULT_FEATURES=ON -DVCPKG_MANIFEST_FEATURES="wallet;tests"

I get:

-- Configuring incomplete, errors occurred!

So moving between static and dynamic presets for example.

If we think that's it's perfectly obvious that this would be expected then no probs. I did expect it as it's common with autotools / make when running the configure step again without a distclean.

m3dwards · 2024-04-04T13:25:28Z

ACK 8d3bb2b

Tested with PowerShell on Windows 11 but I want to give the caveat that these steps were not run on a clean install of windows. I do intend to wipe and reinstall my windows box soon and so will go through the steps again.

hebasto · 2024-04-05T17:42:34Z

If I run:
mkdir build
cd build
cmake -S .. --preset vs2022-static
cmake -S .. --preset vs2022 -DVCPKG_MANIFEST_NO_DEFAULT_FEATURES=ON -DVCPKG_MANIFEST_FEATURES="wallet;tests"
I get:
-- Configuring incomplete, errors occurred!
So moving between static and dynamic presets for example.

If we think that's it's perfectly obvious that this would be expected then no probs. I did expect it as it's common with autotools / make when running the configure step again without a distclean.

Right. When using CMake 3.24 and newer, which is usually the case on Windows, the --fresh command-line option is available as an alternative.

67dc752 cmake, test: Disable tests instead of ignoring them (Hennadii Stepanov) bb9157d cmake, refactor: Switch to `Python3::Interpreter` imported target (Hennadii Stepanov) Pull request description: This PR: 1. Switches to a modern CMake approach by using the `Python3::Interpreter` imported target, which is more robust than using variables. 2. Disables the `util_rpcauth_test` test explicitly instead of silently ignoring it. A build and test log for the case when Python is unavailable is provided below: ``` $ cmake -B build $ cmake --build build -j 16 $ ctest --test-dir build -j $(nproc) -R "^util" Internal ctest changing into directory: /bitcoin/build Test project /bitcoin/build Start 115: util_tests Start 117: util_trace_tests Start 114: util_string_tests Start 116: util_threadnames_tests Start 1: util_rpcauth_test 1/5 Test #1: util_rpcauth_test ................***Not Run (Disabled) 0.00 sec 2/5 Test #114: util_string_tests ................ Passed 0.11 sec 3/5 Test #117: util_trace_tests ................. Passed 0.11 sec 4/5 Test #116: util_threadnames_tests ........... Passed 0.11 sec 5/5 Test #115: util_tests ....................... Passed 0.13 sec 100% tests passed, 0 tests failed out of 4 Total Test time (real) = 0.13 sec The following tests did not run: 1 - util_rpcauth_test (Disabled) ``` ACKs for top commit: purpleKarrot: ACK 67dc752 janb84: tACK 67dc752 Tree-SHA512: 5fc7ebe31ac03f4b8a53ecfcfc1cace0f647a1d2c989651988edae96bdfbbe2dee171714e57cb028e65ead1bb40806a82d9821746451dbf005538601fd33ea88

hebasto added the enhancement New feature or request label Mar 9, 2024

hebasto force-pushed the cmake-staging branch from 700eaae to 8078427 Compare March 26, 2024 09:07

hebasto force-pushed the 240309-cmake-BX branch from 458c1e6 to 8eb45ac Compare March 26, 2024 10:46

hebasto force-pushed the cmake-staging branch from a3313e2 to 418b4f4 Compare April 3, 2024 15:40

fixup! cmake: Add preset for MSVC build

1c8ce0f

Amend "displayName" fields.

hebasto force-pushed the 240309-cmake-BX branch from 8eb45ac to 9c0bf9b Compare April 3, 2024 16:33

m3dwards reviewed Apr 4, 2024

View reviewed changes

hebasto added 2 commits April 4, 2024 13:58

cmake, docs: Update MSVC build docs

00b3061

cmake: Remove legacy MSVC build system

8d3bb2b

hebasto force-pushed the 240309-cmake-BX branch from 9c0bf9b to 8d3bb2b Compare April 4, 2024 12:59

hebasto mentioned this pull request Apr 6, 2024

cmake, msvc: Pin liblzma version #140

Merged

hebasto merged commit 8bcdf2a into cmake-staging Apr 6, 2024

Conversation

hebasto commented Mar 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hebasto commented Mar 9, 2024

Uh oh!

EthanHeilman commented Mar 12, 2024

Uh oh!

hebasto commented Mar 15, 2024

Uh oh!

EthanHeilman commented Mar 15, 2024

Uh oh!

hebasto commented Mar 16, 2024

Uh oh!

EthanHeilman commented Mar 17, 2024

Uh oh!

hebasto commented Mar 26, 2024

Uh oh!

hebasto commented Apr 3, 2024

Uh oh!

sipsorcery commented Apr 3, 2024

Uh oh!

m3dwards Apr 4, 2024

Choose a reason for hiding this comment

Uh oh!

hebasto Apr 4, 2024

Choose a reason for hiding this comment

Uh oh!

m3dwards Apr 4, 2024

Choose a reason for hiding this comment

Uh oh!

hebasto Apr 4, 2024

Choose a reason for hiding this comment

Uh oh!

m3dwards Apr 4, 2024

Choose a reason for hiding this comment

Uh oh!

hebasto Apr 4, 2024

Choose a reason for hiding this comment

Uh oh!

m3dwards commented Apr 4, 2024

Uh oh!

hebasto commented Apr 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

m3dwards commented Apr 4, 2024

Uh oh!

hebasto commented Apr 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

m3dwards commented Apr 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

m3dwards commented Apr 4, 2024

Uh oh!

hebasto commented Apr 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hebasto commented Mar 9, 2024 •

edited

Loading

hebasto commented Apr 4, 2024 •

edited

Loading

hebasto commented Apr 4, 2024 •

edited

Loading

m3dwards commented Apr 4, 2024 •

edited

Loading