Skip to content

guix: Use LTO to build releases#25391

Closed
fanquake wants to merge 2 commits into
bitcoin:masterfrom
fanquake:lto_in_guix
Closed

guix: Use LTO to build releases#25391
fanquake wants to merge 2 commits into
bitcoin:masterfrom
fanquake:lto_in_guix

Conversation

@fanquake
Copy link
Copy Markdown
Member

@fanquake fanquake commented Jun 16, 2022

Changes required to use LTO in Guix (release) builds.

@DrahtBot
Copy link
Copy Markdown
Contributor

DrahtBot commented Jun 16, 2022

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
Concept ACK laanwj, TheCharlatan
Approach ACK hebasto

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #25573 ([POC] guix: produce a fully -static-pie bitcoind by fanquake)
  • #24123 (guix: Pointer Authentication and Branch Target Identification for aarch64 Linux by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@laanwj
Copy link
Copy Markdown
Member

laanwj commented Jun 22, 2022

Concept ACK. This would be great.

@hebasto
Copy link
Copy Markdown
Member

hebasto commented Jul 5, 2022

... build all binaries except for bitcoin-qt

See #25542.

@hebasto
Copy link
Copy Markdown
Member

hebasto commented Jul 10, 2022

Tested 076aa5ee3e9d86cc5f0b4fdfe8926a7f6a7078a2 on Ubuntu 22.04:

$ make -C depends HOST=riscv64-linux-gnu LTO=1 NO_QT=1
$ ./autogen.sh
$ ./configure CONFIG_SITE=depends/riscv64-linux-gnu/share/config.site
$ make clean
$ make
...
  CXXLD    libbitcoinconsensus.la
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/riscv64-linux-gnu/11/../../../../riscv64-linux-gnu/bin/ld: secp256k1/.libs/libsecp256k1.a: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:6729: libbitcoinconsensus.la] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/home/hebasto/git/bitcoin/src'
make[1]: *** [Makefile:18615: all-recursive] Error 1
make[1]: Leaving directory '/home/hebasto/git/bitcoin/src'
make: *** [Makefile:818: all-recursive] Error 1

UPDATE: CONFIG_SITE needs a absolute path. Sorry for the noise.

@fanquake
Copy link
Copy Markdown
Member Author

Tested 076aa5e on Ubuntu 22.04:

I'm not sure what's causing that (maybe a not-completely-clean build dir, and mix of compiled objects), but I don't have any issues with this branch, following the same steps, using Ubuntu 22.04 and riscv64-linux-gnu-g++-11.

Comment thread depends/packages/expat.mk Outdated
fanquake added a commit to bitcoin-core/gui that referenced this pull request Jul 18, 2022
… code on Linux

3442865 build: Use Link Time Optimization for Qt code on Linux (Hennadii Stepanov)
ebce66e build: pass -fno-lto when building expat (fanquake)

Pull request description:

  See: https://www.qt.io/blog/2019/01/02/qt-applications-lto

  `bitcon-qt` unstripped size:
  | host | master (31c6309) | this PR, depends built with `LTO=1` |
  |---|:-:|:-:|
  | x86_64-pc-linux-gnu | 42 MB | 35 MB |
  | arm-linux-gnueabihf | 31 MB | 26 MB |
  | aarch64-linux-gnu | 41 MB | 32 MB |
  | powerpc64-linux-gnu | 51 MB | 41 MB |
  | powerpc64le-linux-gnu | 48 MB | 39 MB |
  | riscv64-linux-gnu | 35 MB | 29 MB |

  Based on the first commit from bitcoin/bitcoin#25391.

  Using LTO for macOS and Windows hosts has some issues which could be addressed in follow ups.

  x86_64 build:
  ![image](https://user-images.githubusercontent.com/32963518/179326902-f91853ca-23c1-4c04-9a6d-161b695f27b5.png)

ACKs for top commit:
  fanquake:
    ACK 3442865

Tree-SHA512: 03eef2568358df9336e24d6c4e12f28b89d649076fb74e7e5303d61e52865c2360c5345a4fb2b1e4bdfdae194f273fc27a5f67e6cf797ed01a154f3da9117247
@fanquake fanquake changed the title [POC] guix: Use LTO to build releases guix: Use LTO to build releases Jul 18, 2022
@fanquake
Copy link
Copy Markdown
Member Author

Rebased for #25542. All binaries for our Linux HOSTS now build.

fanquake added a commit that referenced this pull request Jul 19, 2022
…th GCC)

658685a depends: default to using GCC tool wrappers (with GCC) (fanquake)
6fdc13c build: Fix autoconf variable names for tools found by `AC_PATH_TOOL` (Hennadii Stepanov)

Pull request description:

  This improves support for LTO by using gcc wrappers for `ar`, `nm`, `ranlib`,
  that correctly setup plugin arguments for LTO, when using GCC.

  Other HOSTS are using clang.

  Portion of #25391.

  Guix Build (x86_64):
  ```bash
  ```

  Guix Build (arm64):
  ```bash
  ```

ACKs for top commit:
  dongcarl:
    Code Review ACK 658685a
  hebasto:
    ACK 658685a
  jarolrod:
    ACK 658685a

Tree-SHA512: 28d6127c118f74336c97e2523117f8a0d11b32cd565124cd4052baeb7cc53e71909d3037cb080d996ae4e3ce600326fced37ee36adcc53d839ba7dd7974ebcd2
@DrahtBot
Copy link
Copy Markdown
Contributor

Guix builds

File commit 8371914
(master)
commit ebb8e75b7aabb7aaf79d7d88869e62f2445afede
(master and this pull)
SHA256SUMS.part 2aaca7b584fcf78b... 4931b66af2ee05a5...
*-aarch64-linux-gnu-debug.tar.gz 14d8c43361c4a585... 3361269aca5100c1...
*-aarch64-linux-gnu.tar.gz 1c6585228d374abd... 2054bd63e6d0caca...
*-arm-linux-gnueabihf-debug.tar.gz daefdef2ce4f4059... e6685b78d6ba3e38...
*-arm-linux-gnueabihf.tar.gz 9988c95631f29bd6... 18b37ff67ca03241...
*-arm64-apple-darwin-unsigned.dmg 5e5176595e346a91... 8eb14b90bc118962...
*-arm64-apple-darwin-unsigned.tar.gz 6600a7635d84381d... f74752dfe3f51a30...
*-arm64-apple-darwin.tar.gz c481d4b7c39f03cb... cb6d0940aaec3c28...
*-powerpc64-linux-gnu-debug.tar.gz b9809613a29d4d0c... 42e72259abd48a32...
*-powerpc64-linux-gnu.tar.gz 156c6d606e6be066... e0e9173685f61e99...
*-powerpc64le-linux-gnu-debug.tar.gz 8d084ccb30dbd6ef... 8eb83f87814480a5...
*-powerpc64le-linux-gnu.tar.gz 6232ec189b6f02d1... 739b6ae4dded20b6...
*-riscv64-linux-gnu-debug.tar.gz 63c9592b6520d0ca... 507c6c9fbbb514fc...
*-riscv64-linux-gnu.tar.gz 536b247ba928a95e... 5b38d4257e46407b...
*-x86_64-apple-darwin-unsigned.dmg 8ea328c6e231e601... d9369ba7af768735...
*-x86_64-apple-darwin-unsigned.tar.gz 91d6f1476931fbfe... a6d0502f2d60ae99...
*-x86_64-apple-darwin.tar.gz 85fe98b7a7e4e4c4... d6eff77dfbf6827c...
*-x86_64-linux-gnu-debug.tar.gz d2c6e87bef723538... dff51a2398dff082...
*-x86_64-linux-gnu.tar.gz cad3a7d029500829... d41bbb179118a213...
*.tar.gz 99c39f7b0e324f59... 22f94cdbe110bf5e...
guix_build.log 0dac6e1d0778f5f2... be5b912c626ed36e...
guix_build.log.diff 6055438e12b7cc43...

@theuni
Copy link
Copy Markdown
Member

theuni commented Sep 20, 2023

I have no idea if this will help, but this should keep the dtor in a single compilation unit.

diff -ruN qtbase-everywhere-src-5.15.2/src/widgets/accessible/simplewidgets.cpp qtbase-everywhere-src-5.15.2.new/src/widgets/accessible/simplewidgets.cpp
--- qtbase-everywhere-src-5.15.2/src/widgets/accessible/simplewidgets.cpp	2020-10-27 08:02:11.000000000 +0000
+++ qtbase-everywhere-src-5.15.2.new/src/widgets/accessible/simplewidgets.cpp	2023-09-20 13:58:14.878755026 +0000
@@ -678,6 +678,8 @@
   \ingroup accessibility
 */

+QAccessibleLineEdit::~QAccessibleLineEdit(){};
+
 /*!
   Constructs a QAccessibleLineEdit object for \a w.
   \a name is propagated to the QAccessibleWidget constructor.
diff -ruN qtbase-everywhere-src-5.15.2/src/widgets/accessible/simplewidgets_p.h qtbase-everywhere-src-5.15.2.new/src/widgets/accessible/simplewidgets_p.h
--- qtbase-everywhere-src-5.15.2/src/widgets/accessible/simplewidgets_p.h	2020-10-27 08:02:11.000000000 +0000
+++ qtbase-everywhere-src-5.15.2.new/src/widgets/accessible/simplewidgets_p.h	2023-09-20 13:52:10.467865415 +0000
@@ -154,7 +154,7 @@
 {
 public:
     explicit QAccessibleLineEdit(QWidget *o, const QString &name = QString());
-
+    ~QAccessibleLineEdit();
     QString text(QAccessible::Text t) const override;
     void setText(QAccessible::Text t, const QString &text) override;
     QAccessible::State state() const override;

@fanquake
Copy link
Copy Markdown
Member Author

I have no idea if this will help,

Looks like it's good enough for working Windows builds. Turned the change into a depends patch, and dropped --allow-multiple-definition. I'll PR this separately, as we can fix the LTO=1 case for Windows, before turning this on for releases.

@DrahtBot
Copy link
Copy Markdown
Contributor

Guix builds (on x86_64)

File commit dcfbf3c
(master)
commit 3c2fbf127095274e441e39edaef0694d08649e2b
(master and this pull)
SHA256SUMS.part 3463192d302e5e7d... 111e79646187a18f...
*-aarch64-linux-gnu-debug.tar.gz 8f13fa75951fbb81... e243fc1c66e6b085...
*-aarch64-linux-gnu.tar.gz 19d03941f96ec32e... 03ecbe20dbf342b9...
*-arm-linux-gnueabihf-debug.tar.gz 8fd162ef67ee4a82... 2667153e882200b5...
*-arm-linux-gnueabihf.tar.gz e83f5032b7023e78... c57d21371053b71a...
*-arm64-apple-darwin-unsigned.tar.gz f6043c9708640d0f...
*-arm64-apple-darwin-unsigned.zip d3f9d8f9920f43bc...
*-arm64-apple-darwin.tar.gz 9239c612e938799b...
*-powerpc64-linux-gnu-debug.tar.gz 76a07705a95e8207... a5b0757e4c257160...
*-powerpc64-linux-gnu.tar.gz 56ff6d0c2e6bfed3... 7d420acf58019984...
*-powerpc64le-linux-gnu-debug.tar.gz b3048dd299324b19... 9bdfe0df449f6777...
*-powerpc64le-linux-gnu.tar.gz 686240c06c306afc... 25c85396bf8f7a18...
*-riscv64-linux-gnu-debug.tar.gz 83e360e956ce9a80... cdf4af3356e8cb91...
*-riscv64-linux-gnu.tar.gz ebf0baee66d26688... cad7b8dadcb27154...
*-x86_64-apple-darwin-unsigned.tar.gz c6b0061e9d466aad...
*-x86_64-apple-darwin-unsigned.zip f9cbd11c2b035f15...
*-x86_64-apple-darwin.tar.gz 36f91b5cc6aba2d2...
*-x86_64-linux-gnu-debug.tar.gz 5a88277d2c0ff08d... 4eb303948ccae829...
*-x86_64-linux-gnu.tar.gz ae946eede28d7eb6... 9d7e3ee4298e53d3...
*.tar.gz 32f00918c5b365a0... 25a79b5e85d5a60c...
guix_build.log ddd5298daf43215c... 19c72938b5ffc163...
guix_build.log.diff f2a746ee342064d9...

@maflcko
Copy link
Copy Markdown
Member

maflcko commented Oct 1, 2023

Is this safe to do in light of https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105469 ?

@theuni
Copy link
Copy Markdown
Member

theuni commented Nov 13, 2023

Any idea where in6addr_any is coming from? Seems fine, just curious.

@fanquake
Copy link
Copy Markdown
Member Author

Any idea where in6addr_any is coming from? Seems fine, just curious.

I can't remember exactly, but it looks like this is actually no-longer required. Did a new round of builds with this branch, and it dropped from the script.

@hebasto
Copy link
Copy Markdown
Member

hebasto commented Jan 6, 2024

Related:

@maflcko
Copy link
Copy Markdown
Member

maflcko commented Jun 4, 2024

Is this safe to do in light of https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105469 ?

Also https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113359 (in combination with #29881)

Would be nice to extend the motivation (pull request description) a bit about the benefits and risks.

@DrahtBot
Copy link
Copy Markdown
Contributor

🐙 This pull request conflicts with the target branch and needs rebase.

@fanquake
Copy link
Copy Markdown
Member Author

fanquake commented Aug 9, 2024

Would be nice to extend the motivation (pull request description) a bit about the benefits and risks.

Thanks. Drafted for the moment, but going to come back here soon with much more information.

```bash
bitcoind: export of symbol in6addr_any not allowed!
bitcoind: failed EXPORTED_SYMBOLS
test/test_bitcoin: export of symbol in6addr_any not allowed!
test/test_bitcoin: failed EXPORTED_SYMBOLS
```

Also new exports for bitcoin-qt.
@fanquake
Copy link
Copy Markdown
Member Author

Going to come back to this after static builds.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants