Multiprocess for non depends

[Sjors]

No description provided.

[Sjors]

See bitcoin-core/libmultiprocess#117 and bitcoin-core/libmultiprocess#68 for most of the failures.

Cap’n Proto can be installed through distros, but multiprocess can only be built from source. Currently this PR just clones the repo and installs it, which is what a non-depends user would do. But we should probably have a better alternative.

Perhaps a good approach is to have a convenient way to install multiprocess using depends.

Or to just build it, with every other package disabled, and point our regular build system towards it. I haven't tried the latter yet.

[ryanofsky]

I responded to bitcoin-core/libmultiprocess#117 but hopefully that should be easy to fix with a small tweak to this PR.

I think fixing bitcoin-core/libmultiprocess#68 will probably be easiest with a change to libmultiprocess, but I'll have to refresh my memory of that issue and see what that might look ilke.

On idea of using depends system to install multiprocess library in a way where it could be used by a non-depends build or a not 100% depends build, that seems a like it could be tricky because I think depends system isn't currently set to set up to mix inside and outside dependencies, so it might need new configuration options to do that. Also the way the depends build currently passes package paths to the bitcoin build is through a toolchain.cmake file, but I'm not sure if requiring the depends toolchain for non-depends build would be too limiting. If so the bitcoin build might need to be changed to find depends packages a different way.

I think maybe a better long term approach might be to drop libmultiprocess package from depends, and make libmultiproces into a git subtree like leveldb and secp256k1, building it basically as part of bitcoin. This idea was suggested by achow a long time ago, before we were using cmake, and now that we are using cmake it seems worth pursuing. But I don't have a clear idea of how it should be implemented or what cmake wizardry is required to stitch together separate cmake projects into the same build without installing them. And I also haven't used git subtree before.

An intermediate approach might just be to make building and installing libmultiprocess from source easier. For example, maybe I could just add a simple top level Makefile that calls cmake --build cmake --install so building and installing from source could just be done with one command make install instead of multiple mkdir, cd, cmake, make commands.

[Sjors]

I think depends system isn't currently set to set up to mix inside and outside dependencies

I vaguely remember someone tried to make that work in the past.

[fanquake]

I vaguely remember someone tried to make that work in the past

It somewhat used to be a thing, but was removed (and won't be re-added), because it defeats the point of depends being a self contained dependency builder.

[Sjors]

I responded to chaincodelabs/libmultiprocess#117 but hopefully that should be easy to fix with a small tweak to this PR.

I still need to try this.

I think fixing chaincodelabs/libmultiprocess#68 will probably be easiest with a change to libmultiprocess

I rebased bitcoin#30975 past the fix for this.

[Sjors]

Fixed macOS build.

CentOS doesn't have a capnproto package, so for now I'm running it without multiprocess.

Some CI machines need sudo for the cmake --install . step, others don't and they don't have the sudo command. Maybe there's a nicer approach than my workaround for this.

Interesting build failure for fuzzer: https://cirrus-ci.com/task/5727621063901184

21:11:10.356] /usr/bin/ld: /usr/lib/llvm-19/lib/clang/19/lib/linux/libclang_rt.fuzzer-x86_64.a(FuzzerMain.cpp.o): in function main': [21:11:10.356] (.text.main+0x16): undefined reference to LLVMFuzzerTestOneInput'

[Sjors]

Some undefined behaviour: https://github.com/Sjors/bitcoin/actions/runs/11768122372/job/32777598746?pr=65

[ryanofsky]

Some undefined behaviour: https://github.com/Sjors/bitcoin/actions/runs/11768122372/job/32777598746?pr=65

This looks like the same bug that was reported yesterday bitcoin#30975 (comment) and should be fixed by bitcoin-core/libmultiprocess#121