Strip utility does not have LTO support, mangles archives

[InBetweenNames]

This is a bit of a long running issue that unfortunately needs upstream support in order to fix. I'll detail it here for interested parties.

Upstream Gentoo bug: https://bugs.gentoo.org/603594
Upstream binutils bug: https://sourceware.org/bugzilla/show_bug.cgi?id=21479

The strip utility in LTO builds will commonly emit errors like this for static archives:

x86_64-pc-linux-gnu-strip: /dev/shm/portage/sys-libs/zlib-1.2.8-r1/image/usr/lib64/stzjlms5/adler32.o: plugin needed to handle lto object

Originally I didn't think this was a problem since that should effectively make it a no-op. I was wrong, though -- it actually makes the static archive unusable (I believe this has something to do with the index).

The real solution to this will be introducing LTO support into strip. However, a workaround is needed in the meantime. The linked upstream threads suggest a few options

• Add FEATURES=nostrip (will disable stripping entirely globally)
• Run ranlib after strip (requires a Portage patch, but it does seem to fix it... I'm unsure of the effects of this on non-LTO systems)
• Use -ffat-lto-objects or remove -flto(Obviously not ideal)

@pchome has suggested a workaround that we'll be using in the meantime, which is to set the STRIP_MASK variable to disable stripping static archives only (stripping binaries still). This is the best workaround right now until the upstream issues are sorted out. We can close this issue when things are handled properly upstream.

[wolfwood]

looks like there is now an llvm-strip tool, might be an interesting platform if development work is required https://www.phoronix.com/scan.php?page=news_item&px=LLVM-Strip

[InBetweenNames]

Been looking into this again, and it looks like strip works fine on LTO object files as long as they aren't in a static archive. Putting them in a static archive mangles the archive. It's definitely the index. I do have a rudimentary patch to Portage that invokes ranlib as a workaround, but I'm unwilling to commit to it just yet.

[InBetweenNames]

I tried llvm-strip, too, but it doesn't support a number of GNU strip options which causes other problems. It also doesn't appear to handle static archives. However, I'd be willing to switch to it if it addresses these problems eventually.

One more thing: objcopy suffers from the same problem in my test configuration:

> objcopy lto1.a lto1.a.modified
objcopy: struufsL/lto1.o: plugin needed to handle lto object
> diff lto1.a lto1.a.modified
Binary files lto1.a and lto1.a.modified differ
> g++ lto2.cpp lto1.a -o bin
> g++ lto2.cpp lto1.a.modified -o bin                                          
/tmp/ccE7vu4Y.o:lto2.cpp:function main: error: undefined reference to 'func1(int, int)'
collect2: error: ld returned 1 exit status
> ranlib lto1.a.modified
> g++ lto2.cpp lto1.a.modified -o bin                                                                                                                                                                          

[pchome]

One more thing: objcopy suffers from the same problem in my test configuration

AFAIK strip is objcopy , or shares it's code.

[InBetweenNames]

Based on the binutils sources I think you're right. I found this relevant snippet in archive.c:

if (syms[src_count]->name[0] == '_'
			  && syms[src_count]->name[1] == '_'
			  && strcmp (syms[src_count]->name
				     + (syms[src_count]->name[2] == '_'),
				     "__gnu_lto_slim") == 0)
			_bfd_error_handler
			  (_("%pB: plugin needed to handle lto object"),
			   current);
		      namelen = strlen (syms[src_count]->name);
		      amt = sizeof (char *);
		      map[orl_count].name = (char **) bfd_alloc (arch, amt);

It's not easy to read, and I'm not even sure if this is the offending code just yet, but if I understand correctly it is applying the same logic to non-LTO symbols as it does to LTO symbols, hence the warning. This code appears to be what generates the archive index/header. If I had to guess, this is where the header gets mangled.

[InBetweenNames]

Okay, after more poking around, it does appear to be directly relevant. It is indeed the index generation code. The problem is it doesn't understand how to build an index for LTO symbols. Curiously, ranlib does. strip is basically objcopy, and ranlib is basically ar s.

[InBetweenNames]

Need some review: workaround patch in interim for Portage.

From edee2620d3173cd50762a7ebe7b9cb9a2020f0f9 Mon Sep 17 00:00:00 2001
From: Shane Peelar <lookatyouhacker@gmail.com>
Date: Fri, 31 Aug 2018 17:11:21 -0400
Subject: [PATCH] LTO workaround

---
 bin/estrip | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/bin/estrip b/bin/estrip
index 5709b862c..b2216e964 100755
--- a/bin/estrip
+++ b/bin/estrip
@@ -425,6 +425,9 @@ do
                        # retain the debug info in the archive itself.
                        if ! ${FEATURES_splitdebug} || ${RESTRICT_splitdebug} ; then
                                ${STRIP} -g "${x}"
+                               RANLIB=${RANLIB:-"${CHOST}-ranlib"}
+                               ${RANLIB} "${x}"
+                               echo "${RANLIB}: rebuilding index for ${x}"
                        fi
                fi
        elif [[ ${f} == *"SB executable"* || ${f} == *"SB pie executable"* ||
--
2.18.0

[zeule]

Could you apply the patch to sys-apps/portage-mgorny too, please?

[InBetweenNames]

I can see no reason why not! However, I cannot test as I don't use that version of portage. Could you try it yourself and verify it works as described in README.md? If so, I'd be happy to add it in.

[zeule]

What do I need to test exactly?

[InBetweenNames]

Apply the patch to sys-apps/portage-mgorny, try building zlib and ensure that the fixups are applied at the end of the build sequence, for example. Make sure your STRIP_MASK is unset of course.