ThinLTO lit tests fail with bfd linker

[Fatmice]

I built and install llvm 4.0.1 with
cmake -DCMAKE_INSTALL_PREFIX=../llvm-test -DLLVM_TARGETS_TO_BUILD="X86;AArch64;ARM;PowerPC" -DLLVM_BINUTILS_INCDIR=/usr/include -DCMAKE_BUILD_TYPE=Release -DLLVM_LIBDIR_SUFFIX=64 ../llvm
Then built and test ldc origin/master with
cmake -DCMAKE_INSTALL_PREFIX=../ldc-test -DINCLUDE_PATH=import -DCMAKE_BUILD_TYPE=Release -DLIB_SUFFIX=64 -DLDC_INSTALL_LTOPLUGIN=ON -DLDC_INSTALL_LLVM_RUNTIME_LIBS=ON -DLLVM_ROOT_DIR=../llvm-test -DD_COMPILER=../install/ldc-0.17.4-bootstrap/bin/ldmd2 ../ldc

All tests passed except for two of the lit tests.
linking/thinlto_modulecdtors.d (134 of 153)
linking/thinlto_modulecdtors_2.d (144 of 153)
Here's complete log

The relevant portions

FAIL: LDC :: linking/thinlto_modulecdtors.d (134 of 153)
******************** TEST 'LDC :: linking/thinlto_modulecdtors.d' FAILED ********************
Script:
--
/home/quyen/D/ldc-build/bin/ldc2 -flto=thin -O3 -run /home/quyen/D/ldc/tests/linking/thinlto_modulecdtors.d | FileCheck /home/quyen/D/ldc/tests/linking/thinlto_modulecdtors.d
--
Exit Code: 2

Command Output (stdout):
--
$ "/home/quyen/D/ldc-build/bin/ldc2" "-flto=thin" "-O3" "-run" "/home/quyen/D/ldc/tests/linking/thinlto_modulecdtors.d"
# command stderr:
Error: /tmp/thinlto_modulecdtors-93a9e0a-1f491d failed with status: -2
Error: message: Illegal instruction (core dumped)
Error: program received signal 2 (Interrupt)

error: command failed with exit status: 2
$ "FileCheck" "/home/quyen/D/ldc/tests/linking/thinlto_modulecdtors.d"
# command stderr:
FileCheck error: '-' is empty.
FileCheck command line:  FileCheck /home/quyen/D/ldc/tests/linking/thinlto_modulecdtors.d

error: command failed with exit status: 2

--

********************
UNSUPPORTED: LDC :: sanitizers/asan_dynalloc.d (135 of 153)
UNSUPPORTED: LDC :: sanitizers/asan_interface.d (136 of 153)
UNSUPPORTED: LDC :: sanitizers/asan_stackoverflow.d (137 of 153)
PASS: LDC :: sanitizers/fsanitize_address.d (138 of 153)
PASS: LDC :: linking/thinlto_pgo.d (139 of 153)
PASS: LDC :: sanitizers/fsanitize_blacklist.d (140 of 153)
PASS: LDC :: sanitizers/fsanitize_error_diagnostic.d (141 of 153)
PASS: LDC :: sanitizers/fsanitize_coverage.d (142 of 153)
PASS: LDC :: sanitizers/fsanitize_thread.d (143 of 153)
FAIL: LDC :: linking/thinlto_modulecdtors_2.d (144 of 153)
******************** TEST 'LDC :: linking/thinlto_modulecdtors_2.d' FAILED ********************
Script:
--
/home/quyen/D/ldc-build/bin/ldc2 -flto=thin -O3 /home/quyen/D/ldc/tests/linking/inputs/thinlto_ctor.d -run /home/quyen/D/ldc/tests/linking/thinlto_modulecdtors_2.d | FileCheck --check-prefix=EXECUTE /home/quyen/D/ldc/tests/linking/thinlto_modulecdtors_2.d
--
Exit Code: 2

Command Output (stdout):
--
$ "/home/quyen/D/ldc-build/bin/ldc2" "-flto=thin" "-O3" "/home/quyen/D/ldc/tests/linking/inputs/thinlto_ctor.d" "-run" "/home/quyen/D/ldc/tests/linking/thinlto_modulecdtors_2.d"
# command stderr:
Error: /tmp/thinlto_modulecdtors_2-939d98e-2558f3 failed with status: -2
Error: message: Illegal instruction (core dumped)
Error: program received signal 2 (Interrupt)

error: command failed with exit status: 2
$ "FileCheck" "--check-prefix=EXECUTE" "/home/quyen/D/ldc/tests/linking/thinlto_modulecdtors_2.d"
# command stderr:
FileCheck error: '-' is empty.
FileCheck command line:  FileCheck --check-prefix=EXECUTE /home/quyen/D/ldc/tests/linking/thinlto_modulecdtors_2.d

error: command failed with exit status: 2

--

********************
UNSUPPORTED: LDC :: sanitizers/fuzz_basic.d (145 of 153)
UNSUPPORTED: LDC :: sanitizers/fuzz_asan.d (146 of 153)
UNSUPPORTED: LDC :: sanitizers/link_fuzzer.d (147 of 153)
PASS: LDC :: sanitizers/fsanitize_fuzzer.d (148 of 153)
PASS: LDC :: semantic/target_traits_diag.d (149 of 153)
PASS: LDC :: semantic/dcompute.d (150 of 153)
PASS: LDC :: semantic/target_traits.d (151 of 153)
PASS: LDC :: linking/ir2obj_cache_pruning2.d (152 of 153)
PASS: LDC :: tools/ldc_prune_cache_1.d (153 of 153)
Testing Time: 20.31s
********************
Failing Tests (2):
    LDC :: linking/thinlto_modulecdtors.d
    LDC :: linking/thinlto_modulecdtors_2.d

  Expected Passes    : 128
  Unsupported Tests  : 23
  Unexpected Failures: 2
<end of output>
Test time =  20.59 sec
----------------------------------------------------------
Test Failed.
"lit-tests" end time: Aug 16 16:24 CDT
"lit-tests" time elapsed: 00:00:20
----------------------------------------------------------

End testing: Aug 16 16:24 CDT

I then make install the ldc build and try to repeat those two tests
linking/thinlto_modulecdtors.d
linking/thinlto_modulecdtors_2.d
Running the executables from these two always resulted in Illegal instruction (core dumped) when -flto=thin is used but -flto=full will run without problem.
linking/thinlto_modulecdtors.d
linking/thinlto_modulecdtors_2.d

What is the issue here? @JohanEngelen ?

[Fatmice]

What else should I do to help debug this?

[JohanEngelen]

This looks like a ThinLTO bug (in LLVM). Ideas to help debug:

• system details, OS, processor, ...
• reduce to a tiny testcase: I think you are not able to create any executable with -flto=thin, right? (it's not due to something special in the Lit tests)
• Can you test whether things work with clang?
• Try to figure out if it is really an illegal instruction, or is it a jump/call to invalid address? I think it's very unlikely, given that we pass the CPU in LLVM IR (IIRC), but it could be that the LTO plugin defaults to a newer CPU than your system?

[JohanEngelen]

This may be good to try too:

• compile the super tiny program (try C-like main directly, without having to link to druntime/phobos) to LLVM IR (-output-ll)
• Use llvm llc to get a thinlto object file
• Link with lld.
• see if the executable runs.
  If you still get errors with -flto=thin but not without it, then we know for sure it is a bug somewhere in LLVM.

[JohanEngelen]

For some extra ideas: #2280

Thanks for taking the time to find out what the problem is. (I know it takes a lot of time :( )

[Fatmice]

system details, OS, processor, ...
reduce to a tiny testcase: I think you are not able to create any executable with -flto=thin, right? (it's not due to something special in the Lit tests)

LDC - the LLVM D compiler (1.4.0git-1a3391e):
  based on DMD v2.074.1 and LLVM 4.0.1
  built with LDC - the LLVM D compiler (0.17.4)
  Default target: x86_64-unknown-linux-gnu
  Host CPU: haswell
  http://dlang.org - http://wiki.dlang.org/LDC

  Registered Targets:
    aarch64    - AArch64 (little endian)
    aarch64_be - AArch64 (big endian)
    arm        - ARM
    arm64      - ARM64 (little endian)
    armeb      - ARM (big endian)
    ppc32      - PowerPC 32
    ppc64      - PowerPC 64
    ppc64le    - PowerPC 64 LE
    thumb      - Thumb
    thumbeb    - Thumb (big endian)
    x86        - 32-bit X86: Pentium-Pro and above
    x86-64     - 64-bit X86: EM64T and AMD64
[quyen@thinkpad ldc-test]$ cat /proc/cpuinfo 
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 69
model name	: Intel(R) Core(TM) i3-4005U CPU @ 1.70GHz
stepping	: 1
microcode	: 0x20
cpu MHz		: 1000.036
cache size	: 3072 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 2
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm cpuid_fault epb tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt dtherm ida arat pln pts
bugs		:
bogomips	: 3392.30
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:
[quyen@thinkpad ldc-test]$ uname -a
Linux thinkpad 4.12.5-300.fc26.x86_64 #1 SMP Mon Aug 7 15:27:25 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Also I've found that -flto=thin does work but I can not pass -O# of any kind. It will cause either segmentation fault or illegal instruction. -flto=full will work with optimizations of any kind.

I'm currently using this small test case

[quyen@thinkpad lit-test]$ cat app.d 
import std.stdio;

void main() {
    int a = int.max;
    int b;
    writeln("Max Int: ", a);
    writeln("Overflow: ", a+a);
    writeln("Overflow: ", a+a+a);
    writeln("Overflow: ", a+a+a+a);
    writeln("Overflow: ", a+a+a+a+a);
}

compile the super tiny program (try C-like main directly, without having to link to druntime/phobos) to LLVM IR (-output-ll)
Use llvm llc to get a thinlto object file
Link with lld.
see if the executable runs.
If you still get errors with -flto=thin but not without it, then we know for sure it is a bug somewhere in LLVM.

Looks complicated... So compile with llvm-gcc?

[JohanEngelen]

Minimizing the testcase to be as small as possible will help analyzing the IR (if need be). Does int main() { return 0; } also break?

[Fatmice]

Yep, with any kind of optimization. The size of that executable is also rather large for such a simple program that returns "nothing".

[quyen@thinkpad lit-test]$ ../ldc-test/bin/ldc2 -flto=thin -O3 simple.d
[quyen@thinkpad lit-test]$ ls simple
-rwxrwxr-x. 1 quyen 350928 Aug 22 13:59 simple*
[quyen@thinkpad lit-test]$ gdb -batch -ex run -ex 'bt full' -ex disas ./simple
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGILL, Illegal instruction.
0x00000000004078da in _d_dso_registry ()
#0  0x00000000004078da in _d_dso_registry ()
No symbol table info available.
#1  0x0000000000402766 in ldc.register_dso ()
No symbol table info available.
#2  0x000000000041d00d in __libc_csu_init ()
No symbol table info available.
#3  0x00007f10c681a46b in __libc_start_main () from /lib64/libc.so.6
No symbol table info available.
#4  0x000000000040200a in _start ()
No symbol table info available.
Dump of assembler code for function _d_dso_registry:
   0x00000000004075a0 <+0>:	push   %r15
   0x00000000004075a2 <+2>:	push   %r14
   0x00000000004075a4 <+4>:	push   %r12
   0x00000000004075a6 <+6>:	push   %rbx
   0x00000000004075a7 <+7>:	sub    $0xa8,%rsp
   0x00000000004075ae <+14>:	mov    %rdi,%r14
   0x00000000004075b1 <+17>:	cmpq   $0x0,(%r14)
   0x00000000004075b5 <+21>:	je     0x4078d8 <_d_dso_registry+824>
   0x00000000004075bb <+27>:	mov    0x8(%r14),%rax
   0x00000000004075bf <+31>:	mov    (%rax),%r15
   0x00000000004075c2 <+34>:	test   %r15,%r15
   0x00000000004075c5 <+37>:	je     0x407626 <_d_dso_registry+134>
   0x00000000004075c7 <+39>:	movq   $0x0,(%rax)
   0x00000000004075ce <+46>:	lea    0x22c2cb(%rip),%rax        # 0x6338a0 <_D2rt19sections_elf_shared21_isRuntimeInitializedb>
   0x00000000004075d5 <+53>:	cmpb   $0x0,(%rax)
   0x00000000004075d8 <+56>:	je     0x407886 <_d_dso_registry+742>
   0x00000000004075de <+62>:	mov    %r15,%rdi
   0x00000000004075e1 <+65>:	callq  0x406d70 <_D2rt5minfo11ModuleGroup11runTlsDtorsMFZv>
   0x00000000004075e6 <+70>:	mov    %r15,%rdi
   0x00000000004075e9 <+73>:	callq  0x406dc0 <_D2rt5minfo11ModuleGroup8runDtorsMFZv>
   0x00000000004075ee <+78>:	lea    0x30(%r15),%r14
   0x00000000004075f2 <+82>:	mov    %r14,%rdi
   0x00000000004075f5 <+85>:	callq  0x408630 <_D2rt4util9container5array13__T5ArrayTAvZ5Array7opSliceMNgFNaNbNiZANgAv>
   0x00000000004075fa <+90>:	mov    %rax,%r12
   0x00000000004075fd <+93>:	mov    %rdx,%rbx
   0x0000000000407600 <+96>:	test   %r12,%r12
   0x0000000000407603 <+99>:	je     0x40788a <_d_dso_registry+746>
   0x0000000000407609 <+105>:	add    $0x8,%rbx
   0x000000000040760d <+109>:	nopl   (%rax)
   0x0000000000407610 <+112>:	mov    (%rbx),%rdi
   0x0000000000407613 <+115>:	callq  0x40adc0 <_D4core6memory2GC11removeRangeFNbNixPvZv>
   0x0000000000407618 <+120>:	add    $0x10,%rbx
   0x000000000040761c <+124>:	dec    %r12
   0x000000000040761f <+127>:	jne    0x407610 <_d_dso_registry+112>
   0x0000000000407621 <+129>:	jmpq   0x40788a <_d_dso_registry+746>
   0x0000000000407626 <+134>:	lea    0x22c263(%rip),%rdi        # 0x633890 <_D2rt19sections_elf_shared11_loadedDSOsS2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array>
   0x000000000040762d <+141>:	callq  0x408720 <_D2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array5emptyMxFNaNbNdNiNfZb>
   0x0000000000407632 <+146>:	mov    $0x1,%edi
   0x0000000000407637 <+151>:	mov    $0x58,%esi
   0x000000000040763c <+156>:	callq  0x401d30 <calloc@plt>
   0x0000000000407641 <+161>:	mov    %rax,0x8(%rsp)
   0x0000000000407646 <+166>:	mov    0x8(%r14),%rcx
   0x000000000040764a <+170>:	mov    %rcx,0x50(%rax)
   0x000000000040764e <+174>:	mov    %rax,(%rcx)
   0x0000000000407651 <+177>:	mov    0x10(%r14),%rdx
   0x0000000000407655 <+181>:	mov    0x18(%r14),%rax
   0x0000000000407659 <+185>:	cmp    %rax,%rdx
   0x000000000040765c <+188>:	jb     0x407669 <_d_dso_registry+201>
   0x000000000040765e <+190>:	jmp    0x40766f <_d_dso_registry+207>
   0x0000000000407660 <+192>:	add    $0x8,%rdx
   0x0000000000407664 <+196>:	cmp    %rax,%rdx
   0x0000000000407667 <+199>:	jae    0x40766f <_d_dso_registry+207>
   0x0000000000407669 <+201>:	cmpq   $0x0,(%rdx)
   0x000000000040766d <+205>:	je     0x407660 <_d_dso_registry+192>
   0x000000000040766f <+207>:	cmp    %rax,%rdx
   0x0000000000407672 <+210>:	mov    %rdx,%rcx
   0x0000000000407675 <+213>:	jae    0x40768f <_d_dso_registry+239>
   0x0000000000407677 <+215>:	mov    %rdx,%rcx
   0x000000000040767a <+218>:	nopw   0x0(%rax,%rax,1)
   0x0000000000407680 <+224>:	cmpq   $0x0,(%rcx)
   0x0000000000407684 <+228>:	je     0x40768f <_d_dso_registry+239>
   0x0000000000407686 <+230>:	add    $0x8,%rcx
   0x000000000040768a <+234>:	cmp    %rax,%rcx
   0x000000000040768d <+237>:	jb     0x407680 <_d_dso_registry+224>
   0x000000000040768f <+239>:	mov    0x8(%rsp),%r15
   0x0000000000407694 <+244>:	sub    %rdx,%rcx
   0x0000000000407697 <+247>:	mov    %rcx,%rsi
   0x000000000040769a <+250>:	sar    $0x3f,%rsi
   0x000000000040769e <+254>:	shr    $0x3d,%rsi
   0x00000000004076a2 <+258>:	add    %rcx,%rsi
   0x00000000004076a5 <+261>:	sar    $0x3,%rsi
   0x00000000004076a9 <+265>:	xorps  %xmm0,%xmm0
   0x00000000004076ac <+268>:	movaps %xmm0,0x50(%rsp)
   0x00000000004076b1 <+273>:	movaps %xmm0,0x40(%rsp)
   0x00000000004076b6 <+278>:	movaps %xmm0,0x30(%rsp)
   0x00000000004076bb <+283>:	lea    0x30(%rsp),%rdi
   0x00000000004076c0 <+288>:	callq  0x405420 <_D2rt5minfo11ModuleGroup6__ctorMFNbNcNiAyPS6object10ModuleInfoZS2rt5minfo11ModuleGroup>
   0x00000000004076c5 <+293>:	movaps 0x30(%rsp),%xmm0
   0x00000000004076ca <+298>:	movaps 0x40(%rsp),%xmm1
   0x00000000004076cf <+303>:	movaps 0x50(%rsp),%xmm2
   0x00000000004076d4 <+308>:	movups %xmm2,0x20(%r15)
   0x00000000004076d9 <+313>:	movups %xmm1,0x10(%r15)
   0x00000000004076de <+318>:	movups %xmm0,(%r15)
   0x00000000004076e2 <+322>:	mov    0x8(%r14),%rax
   0x00000000004076e6 <+326>:	mov    %rax,0x20(%rsp)
   0x00000000004076eb <+331>:	lea    0x68(%rsp),%rax
   0x00000000004076f0 <+336>:	mov    %rax,0x28(%rsp)
   0x00000000004076f5 <+341>:	lea    0x1e4(%rip),%rdi        # 0x4078e0 <_D2rt19sections_elf_shared22findImageHeaderForAddrFNbNixPvPS4core3sys5linux4link12dl_phdr_infoZ8callbackUNbNiPS4core3sys5linux4link12dl_phdr_infomPvZi>
   0x00000000004076fc <+348>:	lea    0x20(%rsp),%rsi
   0x0000000000407701 <+353>:	callq  0x401b20 <dl_iterate_phdr@plt>
   0x0000000000407706 <+358>:	test   %eax,%eax
   0x0000000000407708 <+360>:	je     0x4078da <_d_dso_registry+826>
   0x000000000040770e <+366>:	movzwl 0x80(%rsp),%r12d
   0x0000000000407717 <+375>:	test   %r12,%r12
   0x000000000040771a <+378>:	je     0x407784 <_d_dso_registry+484>
   0x000000000040771c <+380>:	mov    0x78(%rsp),%rbx
   0x0000000000407721 <+385>:	lea    0x30(%r15),%r14
   0x0000000000407725 <+389>:	add    $0x28,%rbx
   0x0000000000407729 <+393>:	nopl   0x0(%rax)
   0x0000000000407730 <+400>:	mov    -0x28(%rbx),%eax
   0x0000000000407733 <+403>:	cmp    $0x7,%eax
   0x0000000000407736 <+406>:	je     0x407760 <_d_dso_registry+448>
   0x0000000000407738 <+408>:	cmp    $0x1,%eax
   0x000000000040773b <+411>:	jne    0x40777b <_d_dso_registry+475>
   0x000000000040773d <+413>:	testb  $0x2,-0x24(%rbx)
   0x0000000000407741 <+417>:	je     0x40777b <_d_dso_registry+475>
   0x0000000000407743 <+419>:	mov    -0x18(%rbx),%rdx
   0x0000000000407747 <+423>:	mov    (%rbx),%rsi
   0x000000000040774a <+426>:	add    0x68(%rsp),%rdx
   0x000000000040774f <+431>:	mov    %r14,%rdi
   0x0000000000407752 <+434>:	callq  0x408bd0 <_D2rt4util9container5array13__T5ArrayTAvZ5Array16__T10insertBackZ10insertBackMFNbNiAvZv>
   0x0000000000407757 <+439>:	jmp    0x40777b <_d_dso_registry+475>
   0x0000000000407759 <+441>:	nopl   0x0(%rax)
   0x0000000000407760 <+448>:	mov    0x98(%rsp),%rax
   0x0000000000407768 <+456>:	mov    %rax,0x40(%r15)
   0x000000000040776c <+460>:	mov    (%rbx),%rax
   0x000000000040776f <+463>:	add    $0x7,%rax
   0x0000000000407773 <+467>:	and    $0xfffffffffffffff8,%rax
   0x0000000000407777 <+471>:	mov    %rax,0x48(%r15)
   0x000000000040777b <+475>:	add    $0x38,%rbx
   0x000000000040777f <+479>:	dec    %r12
   0x0000000000407782 <+482>:	jne    0x407730 <_d_dso_registry+400>
   0x0000000000407784 <+484>:	lea    0x22c105(%rip),%rdi        # 0x633890 <_D2rt19sections_elf_shared11_loadedDSOsS2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array>
   0x000000000040778b <+491>:	callq  0x408720 <_D2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array5emptyMxFNaNbNdNiNfZb>
   0x0000000000407790 <+496>:	test   %al,%al
   0x0000000000407792 <+498>:	je     0x4078da <_d_dso_registry+826>
   0x0000000000407798 <+504>:	lea    0x22c0f1(%rip),%rbx        # 0x633890 <_D2rt19sections_elf_shared11_loadedDSOsS2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array>
   0x000000000040779f <+511>:	mov    %rbx,%rdi
   0x00000000004077a2 <+514>:	callq  0x408730 <_D2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array7opSliceMNgFNaNbNiZANgPS2rt19sections_elf_shared3DSO>
   0x00000000004077a7 <+519>:	lea    0x8(%rsp),%rsi
   0x00000000004077ac <+524>:	mov    %rbx,%rdi
   0x00000000004077af <+527>:	callq  0x408b10 <_D2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array16__T10insertBackZ10insertBackMFNbNiKPS2rt19sections_elf_shared3DSOZv>
   0x00000000004077b4 <+532>:	mov    0x8(%rsp),%rax
   0x00000000004077b9 <+537>:	mov    0x48(%rax),%rbx
   0x00000000004077bd <+541>:	callq  0x4073c0 <_D2rt19sections_elf_shared11getTLSRangeFNbNimmZ10endOfBlockFNaNbNiZPv>
   0x00000000004077c2 <+546>:	sub    %rbx,%rax
   0x00000000004077c5 <+549>:	mov    %rbx,0x10(%rsp)
   0x00000000004077ca <+554>:	mov    %rax,0x18(%rsp)
   0x00000000004077cf <+559>:	mov    %fs:0x0,%rax
   0x00000000004077d8 <+568>:	lea    -0x310(%rax),%rax
   0x00000000004077df <+575>:	mov    (%rax),%rbx
   0x00000000004077e2 <+578>:	test   %rbx,%rbx
   0x00000000004077e5 <+581>:	jne    0x407815 <_d_dso_registry+629>
   0x00000000004077e7 <+583>:	mov    $0x1,%edi
   0x00000000004077ec <+588>:	mov    $0x10,%esi
   0x00000000004077f1 <+593>:	callq  0x401d30 <calloc@plt>
   0x00000000004077f6 <+598>:	mov    %rax,%rbx
   0x00000000004077f9 <+601>:	mov    %fs:0x0,%rax
   0x0000000000407802 <+610>:	lea    -0x310(%rax),%rax
   0x0000000000407809 <+617>:	mov    %rbx,(%rax)
   0x000000000040780c <+620>:	test   %rbx,%rbx
   0x000000000040780f <+623>:	je     0x4078da <_d_dso_registry+826>
   0x0000000000407815 <+629>:	lea    0x10(%rsp),%rsi
   0x000000000040781a <+634>:	mov    %rbx,%rdi
   0x000000000040781d <+637>:	callq  0x408b70 <_D2rt4util9container5array13__T5ArrayTAvZ5Array16__T10insertBackZ10insertBackMFNbNiKAvZv>
   0x0000000000407822 <+642>:	lea    0x22c077(%rip),%rax        # 0x6338a0 <_D2rt19sections_elf_shared21_isRuntimeInitializedb>
   0x0000000000407829 <+649>:	cmpb   $0x0,(%rax)
   0x000000000040782c <+652>:	je     0x4078c9 <_d_dso_registry+809>
   0x0000000000407832 <+658>:	mov    0x8(%rsp),%rdi
   0x0000000000407837 <+663>:	add    $0x30,%rdi
   0x000000000040783b <+667>:	callq  0x408630 <_D2rt4util9container5array13__T5ArrayTAvZ5Array7opSliceMNgFNaNbNiZANgAv>
   0x0000000000407840 <+672>:	mov    %rax,%r14
   0x0000000000407843 <+675>:	mov    %rdx,%rbx
   0x0000000000407846 <+678>:	test   %r14,%r14
   0x0000000000407849 <+681>:	je     0x407867 <_d_dso_registry+711>
   0x000000000040784b <+683>:	add    $0x8,%rbx
   0x000000000040784f <+687>:	nop
   0x0000000000407850 <+688>:	mov    -0x8(%rbx),%rsi
   0x0000000000407854 <+692>:	mov    (%rbx),%rdx
   0x0000000000407857 <+695>:	xor    %edi,%edi
   0x0000000000407859 <+697>:	callq  0x40adb0 <_D4core6memory2GC8addRangeFNbNixPvmxC8TypeInfoZv>
   0x000000000040785e <+702>:	add    $0x10,%rbx
   0x0000000000407862 <+706>:	dec    %r14
   0x0000000000407865 <+709>:	jne    0x407850 <_d_dso_registry+688>
   0x0000000000407867 <+711>:	mov    0x8(%rsp),%rbx
   0x000000000040786c <+716>:	mov    %rbx,%rdi
   0x000000000040786f <+719>:	callq  0x406910 <_D2rt5minfo11ModuleGroup9sortCtorsMFZv>
   0x0000000000407874 <+724>:	mov    %rbx,%rdi
   0x0000000000407877 <+727>:	callq  0x406cc0 <_D2rt5minfo11ModuleGroup8runCtorsMFZv>
   0x000000000040787c <+732>:	mov    %rbx,%rdi
   0x000000000040787f <+735>:	callq  0x406d30 <_D2rt5minfo11ModuleGroup11runTlsCtorsMFZv>
   0x0000000000407884 <+740>:	jmp    0x4078c9 <_d_dso_registry+809>
   0x0000000000407886 <+742>:	lea    0x30(%r15),%r14
   0x000000000040788a <+746>:	mov    %fs:0x0,%rax
   0x0000000000407893 <+755>:	lea    -0x310(%rax),%rax
   0x000000000040789a <+762>:	mov    (%rax),%rdi
   0x000000000040789d <+765>:	callq  0x408640 <_D2rt4util9container5array13__T5ArrayTAvZ5Array7popBackMFNbNiZv>
   0x00000000004078a2 <+770>:	lea    0x22bfe7(%rip),%rbx        # 0x633890 <_D2rt19sections_elf_shared11_loadedDSOsS2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array>
   0x00000000004078a9 <+777>:	mov    %rbx,%rdi
   0x00000000004078ac <+780>:	callq  0x408740 <_D2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array7popBackMFNbNiZv>
   0x00000000004078b1 <+785>:	mov    %r14,%rdi
   0x00000000004078b4 <+788>:	callq  0x408560 <_D2rt4util9container5array13__T5ArrayTAvZ5Array5resetMFNbNiZv>
   0x00000000004078b9 <+793>:	mov    %r15,%rdi
   0x00000000004078bc <+796>:	callq  0x401b30 <free@plt>
   0x00000000004078c1 <+801>:	mov    %rbx,%rdi
   0x00000000004078c4 <+804>:	callq  0x408720 <_D2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array5emptyMxFNaNbNdNiNfZb>
   0x00000000004078c9 <+809>:	add    $0xa8,%rsp
   0x00000000004078d0 <+816>:	pop    %rbx
   0x00000000004078d1 <+817>:	pop    %r12
   0x00000000004078d3 <+819>:	pop    %r14
   0x00000000004078d5 <+821>:	pop    %r15
   0x00000000004078d7 <+823>:	retq   
   0x00000000004078d8 <+824>:	ud2    
=> 0x00000000004078da <+826>:	ud2    
End of assembler dump.

[Fatmice]

../ldc-test/bin/ldc2 -flto=thin -output-ll simple.d
simple.ll
../ldc-test/bin/ldc2 -flto=thin -output-ll -O3 simple.d
-O3 simple.ll
../llvm-test/bin/llc -O3 -filetype=obj simple.ll
simple.o.zip
Now I'm stuck on the ../lvm-test/bin/ld.lld as that issued way too many symbols errors.

[quyen@thinkpad lit-test]$ ../llvm-test/bin/ld.lld --lto-O3 -L/home/quyen/D/ldc-test/lib64 -ldruntime-ldc simple.o
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _D2rt19sections_elf_shared3DSO11__xopEqualsFKxS2rt19sections_elf_shared3DSOKxS2rt19sections_elf_shared3DSOZb): undefined symbol 'memcmp'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _D2rt19sections_elf_shared3DSO11__xopEqualsFKxS2rt19sections_elf_shared3DSOKxS2rt19sections_elf_shared3DSOZb): undefined symbol 'memcmp'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _D2rt19sections_elf_shared3DSO11__xopEqualsFKxS2rt19sections_elf_shared3DSOKxS2rt19sections_elf_shared3DSOZb): undefined symbol 'memcmp'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _D2rt19sections_elf_shared3DSO11__xopEqualsFKxS2rt19sections_elf_shared3DSOKxS2rt19sections_elf_shared3DSOZb): undefined symbol 'memcmp'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _D2rt19sections_elf_shared13initTLSRangesFNbNiZPS2rt4util9container5array13__T5ArrayTAvZ5Array): undefined symbol 'calloc'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _D2rt19sections_elf_shared13finiTLSRangesFNbNiPS2rt4util9container5array13__T5ArrayTAvZ5ArrayZv): undefined symbol 'free'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _d_dso_registry): undefined symbol 'calloc'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _d_dso_registry): undefined symbol 'dl_iterate_phdr'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _d_dso_registry): undefined symbol 'calloc'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _d_dso_registry): undefined symbol 'free'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _D2rt19sections_elf_shared22findImageHeaderForAddrFNbNixPvPS4core3sys5linux4link12dl_phdr_infoZb): undefined symbol 'dl_iterate_phdr'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _D2rt19sections_elf_shared7freeDSOFNbNiPS2rt19sections_elf_shared3DSOZv): undefined symbol 'free'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _D2rt19sections_elf_shared8prognameFNbNdNiZPxa): undefined symbol 'program_invocation_name'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _D2rt19sections_elf_shared7dsoNameFNbNixPaZAxa): undefined symbol 'program_invocation_name'
../llvm-test/bin/ld.lld: error: rt/sections_elf_shared.d:(function _D2rt19sections_elf_shared7dsoNameFNbNixPaZAxa): undefined symbol 'strlen'
../llvm-test/bin/ld.lld: error: object.d:(function _D6object14TypeInfo_Class8opEqualsMFC6ObjectZb): undefined symbol 'memcmp'
../llvm-test/bin/ld.lld: error: object.d:(function _D6object9Interface11__xopEqualsFKxS6object9InterfaceKxS6object9InterfaceZb): undefined symbol 'memcmp'
../llvm-test/bin/ld.lld: error: object.d:(function _D6object8TypeInfo8opEqualsMFC6ObjectZb): undefined symbol 'memcmp'
../llvm-test/bin/ld.lld: error: object.d:(function _D6object16TypeInfo_Typedef8opEqualsMFC6ObjectZb): undefined symbol 'memcmp'
../llvm-test/bin/ld.lld: error: object.d:(function _D6object20TypeInfo_StaticArray4swapMxFPvPvZv): undefined symbol 'memcpy'
../llvm-test/bin/ld.lld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)

[Fatmice]

Diff between simple.ll and -O3 simple.ll

[Fatmice]

@JohanEngelen any insight? Maybe LDC 1.4 and a newer LLVM will fix this issue?

[JohanEngelen]

Sorry, haven't looked at this any more.
(the linker errors are because you need to link with the C runtime [you can figure out what LDC links with by adding -v to LDC cmdline when linking])

I think when you use -link-debuglib, you can see more debug info (line number hopefully) where the crash happens. I think you are hitting and assert(0) in druntime code.

[Fatmice]

@JohanEngelen
The cmd line had -v in it in the links of in the bottom of the first post.
The binary only crashes if -flto=thin is used with -Ox (with optimization of any kind). Why is that?
If it didn't link with the c runtime libraries then it wouldn't even run with or without optimization...

[Fatmice]

[quyen@thinkpad lit-test]$ cat simple.d 
int main() { return 0; }

[quyen@thinkpad lit-test]$ ../ldc-test/bin/ldc2 -v -flto=thin -link-debuglib -run simple.d 
binary    /home/quyen/D/ldc-test/bin/ldc2
version   1.4.0git-beb1cd5 (DMD v2.074.1, LLVM 4.0.1)
config    /home/quyen/D/ldc-test/etc/ldc2.conf (x86_64-unknown-linux-gnu)
predefs   LDC all D_Version2 assert X86_64 D_InlineAsm_X86_64 D_HardFloat LittleEndian D_LP64 D_PIC linux Posix CRuntime_Glibc LDC_LLVM_400
parse     simple
importall simple
import    object	(/home/quyen/D/ldc-test/include/ldc/object.d)
semantic  simple
entry     main      	simple.d
semantic2 simple
semantic3 simple
code      simple
/usr/bin/gcc simple-8e342f0.o -o /tmp/simple-8e342f0-5202bc -Wl,-plugin,/home/quyen/D/ldc-test/lib64/LLVMgold-ldc.so -Wl,-plugin-opt=thinlto -Wl,-plugin-opt=O0 -Wl,-plugin-opt=-function-sections -Wl,-plugin-opt=-data-sections -L/home/quyen/D/ldc-test/bin/../lib64 -lphobos2-ldc-debug -ldruntime-ldc-debug -Wl,--gc-sections -lrt -ldl -lpthread -lm -m64 
/tmp/simple-8e342f0-5202bc 

[quyen@thinkpad lit-test]$ ../ldc-test/bin/ldc2 -v -flto=thin -link-debuglib simple.d 
binary    /home/quyen/D/ldc-test/bin/ldc2
version   1.4.0git-beb1cd5 (DMD v2.074.1, LLVM 4.0.1)
config    /home/quyen/D/ldc-test/etc/ldc2.conf (x86_64-unknown-linux-gnu)
predefs   LDC all D_Version2 assert X86_64 D_InlineAsm_X86_64 D_HardFloat LittleEndian D_LP64 D_PIC linux Posix CRuntime_Glibc LDC_LLVM_400
parse     simple
importall simple
import    object	(/home/quyen/D/ldc-test/include/ldc/object.d)
semantic  simple
entry     main      	simple.d
semantic2 simple
semantic3 simple
code      simple
/usr/bin/gcc simple.o -o simple -Wl,-plugin,/home/quyen/D/ldc-test/lib64/LLVMgold-ldc.so -Wl,-plugin-opt=thinlto -Wl,-plugin-opt=O0 -Wl,-plugin-opt=-function-sections -Wl,-plugin-opt=-data-sections -L/home/quyen/D/ldc-test/bin/../lib64 -lphobos2-ldc-debug -ldruntime-ldc-debug -Wl,--gc-sections -lrt -ldl -lpthread -lm -m64 
[quyen@thinkpad lit-test]$ gdb -batch -ex run -ex 'bt full' -ex disas ./simple
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[Inferior 1 (process 12058) exited normally]
No stack.
No frame selected.

[quyen@thinkpad lit-test]$ ldd simple
	linux-vdso.so.1 (0x00007fffd872f000)
	librt.so.1 => /lib64/librt.so.1 (0x00007fb844a1d000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fb844819000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fb8445fa000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fb8442e4000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fb8440cd000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fb843cfc000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fb844c25000)

Only core dump when optimization was used.

[quyen@thinkpad lit-test]$ ../ldc-test/bin/ldc2 -v -flto=thin -O3 -link-debuglib -run simple.d 
binary    /home/quyen/D/ldc-test/bin/ldc2
version   1.4.0git-beb1cd5 (DMD v2.074.1, LLVM 4.0.1)
config    /home/quyen/D/ldc-test/etc/ldc2.conf (x86_64-unknown-linux-gnu)
predefs   LDC all D_Version2 assert X86_64 D_InlineAsm_X86_64 D_HardFloat LittleEndian D_LP64 D_PIC linux Posix CRuntime_Glibc LDC_LLVM_400
parse     simple
importall simple
import    object	(/home/quyen/D/ldc-test/include/ldc/object.d)
semantic  simple
entry     main      	simple.d
semantic2 simple
semantic3 simple
code      simple
/usr/bin/gcc simple-0bca230.o -o /tmp/simple-0bca230-e7783f -Wl,-plugin,/home/quyen/D/ldc-test/lib64/LLVMgold-ldc.so -Wl,-plugin-opt=thinlto -Wl,-plugin-opt=O3 -Wl,-plugin-opt=-function-sections -Wl,-plugin-opt=-data-sections -L/home/quyen/D/ldc-test/bin/../lib64 -lphobos2-ldc-debug -ldruntime-ldc-debug -Wl,--gc-sections -lrt -ldl -lpthread -lm -m64 
/tmp/simple-0bca230-e7783f 
Error: /tmp/simple-0bca230-e7783f failed with status: -2
Error: message: Segmentation fault (core dumped)
Error: program received signal 2 (Interrupt)

[quyen@thinkpad lit-test]$ ldd simple
	linux-vdso.so.1 (0x00007ffe86de5000)
	librt.so.1 => /lib64/librt.so.1 (0x00007ff10a016000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007ff109e12000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff109bf3000)
	libm.so.6 => /lib64/libm.so.6 (0x00007ff1098dd000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ff1096c6000)
	libc.so.6 => /lib64/libc.so.6 (0x00007ff1092f5000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ff10a21e000)

[quyen@thinkpad lit-test]$ gdb -batch -ex run -ex 'bt full' -ex disas ./simple
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x0000000000417b9c in gc_malloc (sz=80, ba=0, ti=0x64fad0 <ClassInfo for core.exception.AssertError>) at proxy.d:117
117	        return instance.malloc(sz, ba, ti);
#0  0x0000000000417b9c in gc_malloc (sz=80, ba=0, ti=0x64fad0 <ClassInfo for core.exception.AssertError>) at proxy.d:117
No locals.
#1  0x0000000000413b03 in core.memory.GC.malloc(ulong, uint, const(TypeInfo)) (sz=80, ba=0, ti=0x64fad0 <ClassInfo for core.exception.AssertError>) at memory.d:380
No locals.
#2  0x0000000000402d6e in rt.lifetime._d_newclass!(false)._d_newclass(const(TypeInfo_Class)) (ci=0x64fad0 <ClassInfo for core.exception.AssertError>) at lifetime.d:114
        attr = BlkAttr.NONE
        p = 0x0
#3  _d_allocclass (ci=0x64fad0 <ClassInfo for core.exception.AssertError>) at lifetime.d:157
No locals.
#4  0x0000000000412db3 in onAssertErrorMsg (file=..., line=499, msg=...) at exception.d:455
No locals.
#5  0x00000000004131fe in _d_assert_msg (msg=..., file=..., line=499) at exception.d:632
No locals.
#6  0x000000000040b268 in _d_dso_registry (arg=0x7fffffffc030 "\001") at sections_elf_shared.d:498
        firstDSO = false
        pdso = 0x6530f0
        minfoEnd = 0x652c28 <completed>
        tlsRange = "Hg\201\064&\177\000\000\366u\256\003\003", '\000' <repeats 11 times>, "\006\000\000\000\000\000\000\000\001", '\000' <repeats 15 times>, "_]\245\064&\177\000\000\005\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000s\275[5&\177\000\000\360\233Z5&\177\000"
        minfoBeg = 0x6529d0 <simple.__moduleRef>
        header = {dlpi_addr = 0, dlpi_name = 0x7f26357d38c0 "", dlpi_phdr = 0x400040, dlpi_phnum = 10, dlpi_adds = 9, dlpi_subs = 0, dlpi_tls_modid = 1, dlpi_tls_data = 0x7f26357a5720 ""}
        data = 0x7fffffffc030
#7  0x0000000000402992 in ldc.register_dso ()
No symbol table info available.
#8  0x00000000004029b2 in ldc.dso_ctor.2rt6dmain2 ()
No symbol table info available.
#9  0x00000000004385ad in __libc_csu_init ()
No symbol table info available.
#10 0x00007f26346a249b in __libc_start_main () from /lib64/libc.so.6
No symbol table info available.
#11 0x0000000000401d4a in _start ()
No symbol table info available.
Dump of assembler code for function _d_dso_registry:
   0x000000000040af50 <+0>:	sub    $0x128,%rsp
   0x000000000040af57 <+7>:	mov    %rdi,0x120(%rsp)
   0x000000000040af5f <+15>:	mov    0x120(%rsp),%rdi
   0x000000000040af67 <+23>:	mov    %rdi,0x118(%rsp)
   0x000000000040af6f <+31>:	mov    0x118(%rsp),%rdi
   0x000000000040af77 <+39>:	cmpq   $0x1,(%rdi)
   0x000000000040af7b <+43>:	jae    0x40afad <_d_dso_registry+93>
   0x000000000040af7d <+45>:	xor    %eax,%eax
   0x000000000040af7f <+47>:	mov    %al,%cl
   0x000000000040af81 <+49>:	test   $0x1,%cl
   0x000000000040af84 <+52>:	jne    0x40af88 <_d_dso_registry+56>
   0x000000000040af86 <+54>:	jmp    0x40af8a <_d_dso_registry+58>
   0x000000000040af88 <+56>:	jmp    0x40afad <_d_dso_registry+93>
   0x000000000040af8a <+58>:	lea    0x2db7f(%rip),%rsi        # 0x438b10
   0x000000000040af91 <+65>:	lea    0x2daa8(%rip),%rcx        # 0x438a40
   0x000000000040af98 <+72>:	mov    $0x18,%eax
   0x000000000040af9d <+77>:	mov    %eax,%edx
   0x000000000040af9f <+79>:	mov    $0x1ae,%r8d
   0x000000000040afa5 <+85>:	mov    %rdx,%rdi
   0x000000000040afa8 <+88>:	callq  0x4131c0 <_d_assert_msg>
   0x000000000040afad <+93>:	mov    0x118(%rsp),%rax
   0x000000000040afb5 <+101>:	mov    0x8(%rax),%rax
   0x000000000040afb9 <+105>:	cmpq   $0x0,(%rax)
   0x000000000040afbd <+109>:	jne    0x40b37f <_d_dso_registry+1071>
   0x000000000040afc3 <+115>:	lea    0x247cee(%rip),%rdi        # 0x652cb8 <_D2rt19sections_elf_shared11_loadedDSOsS2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array>
   0x000000000040afca <+122>:	callq  0x40e1d0 <_D2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array5emptyMxFNaNbNdNiNfZb>
   0x000000000040afcf <+127>:	and    $0x1,%al
   0x000000000040afd1 <+129>:	mov    %al,0x117(%rsp)
   0x000000000040afd8 <+136>:	testb  $0x1,0x117(%rsp)
   0x000000000040afe0 <+144>:	je     0x40afe7 <_d_dso_registry+151>
   0x000000000040afe2 <+146>:	callq  0x40b4a0 <_D2rt19sections_elf_shared9initLocksFNbNiZv>
   0x000000000040afe7 <+151>:	mov    $0x1,%eax
   0x000000000040afec <+156>:	mov    %eax,%edi
   0x000000000040afee <+158>:	mov    $0x58,%eax
   0x000000000040aff3 <+163>:	mov    %eax,%esi
   0x000000000040aff5 <+165>:	callq  0x401a80 <calloc@plt>
   0x000000000040affa <+170>:	lea    0x24375f(%rip),%rsi        # 0x64e760 <_D38TypeInfo_S2rt19sections_elf_shared3DSO6__initZ>
   0x000000000040b001 <+177>:	mov    %rsi,%rdi
   0x000000000040b004 <+180>:	mov    %rax,0x108(%rsp)
   0x000000000040b00c <+188>:	mov    (%rsi),%rax
   0x000000000040b00f <+191>:	callq  *0x58(%rax)
   0x000000000040b012 <+194>:	cmp    $0x0,%rdx
   0x000000000040b016 <+198>:	mov    %rax,0x38(%rsp)
   0x000000000040b01b <+203>:	jne    0x40b367 <_d_dso_registry+1047>
   0x000000000040b021 <+209>:	mov    0x108(%rsp),%rax
   0x000000000040b029 <+217>:	mov    0x118(%rsp),%rcx
   0x000000000040b031 <+225>:	mov    0x8(%rcx),%rcx
   0x000000000040b035 <+229>:	mov    %rcx,0x50(%rax)
   0x000000000040b039 <+233>:	mov    0x118(%rsp),%rax
   0x000000000040b041 <+241>:	mov    0x8(%rax),%rax
   0x000000000040b045 <+245>:	mov    0x108(%rsp),%rcx
   0x000000000040b04d <+253>:	mov    %rcx,(%rax)
   0x000000000040b050 <+256>:	mov    0x118(%rsp),%rax
   0x000000000040b058 <+264>:	mov    0x10(%rax),%rax
   0x000000000040b05c <+268>:	mov    %rax,0x100(%rsp)
   0x000000000040b064 <+276>:	xor    %eax,%eax
   0x000000000040b066 <+278>:	mov    %al,%cl
   0x000000000040b068 <+280>:	mov    0x118(%rsp),%rdx
   0x000000000040b070 <+288>:	mov    0x100(%rsp),%rsi
   0x000000000040b078 <+296>:	cmp    0x18(%rdx),%rsi
   0x000000000040b07c <+300>:	mov    %cl,0x37(%rsp)
   0x000000000040b080 <+304>:	jae    0x40b097 <_d_dso_registry+327>
   0x000000000040b082 <+306>:	mov    0x100(%rsp),%rax
   0x000000000040b08a <+314>:	mov    (%rax),%rax
   0x000000000040b08d <+317>:	test   %rax,%rax
   0x000000000040b090 <+320>:	sete   %cl
   0x000000000040b093 <+323>:	mov    %cl,0x37(%rsp)
   0x000000000040b097 <+327>:	mov    0x37(%rsp),%al
   0x000000000040b09b <+331>:	test   $0x1,%al
   0x000000000040b09d <+333>:	jne    0x40b0a1 <_d_dso_registry+337>
   0x000000000040b09f <+335>:	jmp    0x40b0b7 <_d_dso_registry+359>
   0x000000000040b0a1 <+337>:	mov    0x100(%rsp),%rax
   0x000000000040b0a9 <+345>:	add    $0x8,%rax
   0x000000000040b0ad <+349>:	mov    %rax,0x100(%rsp)
   0x000000000040b0b5 <+357>:	jmp    0x40b064 <_d_dso_registry+276>
   0x000000000040b0b7 <+359>:	mov    0x100(%rsp),%rax
   0x000000000040b0bf <+367>:	mov    %rax,0xf8(%rsp)
   0x000000000040b0c7 <+375>:	xor    %eax,%eax
   0x000000000040b0c9 <+377>:	mov    %al,%cl
   0x000000000040b0cb <+379>:	mov    0x118(%rsp),%rdx
   0x000000000040b0d3 <+387>:	mov    0xf8(%rsp),%rsi
   0x000000000040b0db <+395>:	cmp    0x18(%rdx),%rsi
   0x000000000040b0df <+399>:	mov    %cl,0x36(%rsp)
   0x000000000040b0e3 <+403>:	jae    0x40b0f8 <_d_dso_registry+424>
   0x000000000040b0e5 <+405>:	mov    0xf8(%rsp),%rax
   0x000000000040b0ed <+413>:	cmpq   $0x0,(%rax)
   0x000000000040b0f1 <+417>:	setne  %cl
   0x000000000040b0f4 <+420>:	mov    %cl,0x36(%rsp)
   0x000000000040b0f8 <+424>:	mov    0x36(%rsp),%al
   0x000000000040b0fc <+428>:	test   $0x1,%al
   0x000000000040b0fe <+430>:	jne    0x40b102 <_d_dso_registry+434>
   0x000000000040b100 <+432>:	jmp    0x40b118 <_d_dso_registry+456>
   0x000000000040b102 <+434>:	mov    0xf8(%rsp),%rax
   0x000000000040b10a <+442>:	add    $0x8,%rax
   0x000000000040b10e <+446>:	mov    %rax,0xf8(%rsp)
   0x000000000040b116 <+454>:	jmp    0x40b0c7 <_d_dso_registry+375>
   0x000000000040b118 <+456>:	mov    0x108(%rsp),%rax
   0x000000000040b120 <+464>:	movq   $0x0,0xd0(%rsp)
   0x000000000040b12c <+476>:	movq   $0x0,0xc8(%rsp)
   0x000000000040b138 <+488>:	movq   $0x0,0xe0(%rsp)
   0x000000000040b144 <+500>:	movq   $0x0,0xd8(%rsp)
   0x000000000040b150 <+512>:	movq   $0x0,0xf0(%rsp)
   0x000000000040b15c <+524>:	movq   $0x0,0xe8(%rsp)
   0x000000000040b168 <+536>:	mov    0x100(%rsp),%rsi
   0x000000000040b170 <+544>:	mov    0xf8(%rsp),%rdi
   0x000000000040b178 <+552>:	mov    %rax,0x28(%rsp)
   0x000000000040b17d <+557>:	callq  0x40b4b0 <_D2rt19sections_elf_shared35__T7toRangeTyPS6object10ModuleInfoZ7toRangeFNaNbNiPyPS6object10ModuleInfoPyPS6object10ModuleInfoZAyPS6object10ModuleInfo>
   0x000000000040b182 <+562>:	lea    0xc8(%rsp),%rsi
   0x000000000040b18a <+570>:	lea    0xc8(%rsp),%rdi
   0x000000000040b192 <+578>:	mov    %rsi,0x20(%rsp)
   0x000000000040b197 <+583>:	mov    %rax,%rsi
   0x000000000040b19a <+586>:	callq  0x4061b0 <_D2rt5minfo11ModuleGroup6__ctorMFNbNcNiAyPS6object10ModuleInfoZS2rt5minfo11ModuleGroup>
   0x000000000040b19f <+591>:	lea    0x88(%rsp),%rdi
   0x000000000040b1a7 <+599>:	mov    $0x30,%ecx
   0x000000000040b1ac <+604>:	mov    %ecx,%edx
   0x000000000040b1ae <+606>:	lea    0xc8(%rsp),%rsi
   0x000000000040b1b6 <+614>:	mov    0x28(%rsp),%r8
   0x000000000040b1bb <+619>:	mov    %rdi,0x18(%rsp)
   0x000000000040b1c0 <+624>:	mov    %r8,%rdi
   0x000000000040b1c3 <+627>:	mov    %rax,0x10(%rsp)
   0x000000000040b1c8 <+632>:	callq  0x401ab0 <memcpy@plt>
   0x000000000040b1cd <+637>:	mov    0x118(%rsp),%rax
   0x000000000040b1d5 <+645>:	mov    0x8(%rax),%rax
   0x000000000040b1d9 <+649>:	mov    0x18(%rsp),%rdi
   0x000000000040b1de <+654>:	mov    %rax,%rsi
   0x000000000040b1e1 <+657>:	callq  0x40b4e0 <_D2rt19sections_elf_shared22findImageHeaderForAddrFNbNixPvPS4core3sys5linux4link12dl_phdr_infoZb>
   0x000000000040b1e6 <+662>:	test   $0x1,%al
   0x000000000040b1e8 <+664>:	jne    0x40b20f <_d_dso_registry+703>
   0x000000000040b1ea <+666>:	xor    %eax,%eax
   0x000000000040b1ec <+668>:	mov    %al,%cl
   0x000000000040b1ee <+670>:	test   $0x1,%cl
   0x000000000040b1f1 <+673>:	jne    0x40b1f5 <_d_dso_registry+677>
   0x000000000040b1f3 <+675>:	jmp    0x40b1f7 <_d_dso_registry+679>
   0x000000000040b1f5 <+677>:	jmp    0x40b20f <_d_dso_registry+703>
   0x000000000040b1f7 <+679>:	lea    0x2d842(%rip),%rsi        # 0x438a40
   0x000000000040b1fe <+686>:	mov    $0x18,%eax
   0x000000000040b203 <+691>:	mov    %eax,%edi
   0x000000000040b205 <+693>:	mov    $0x1c5,%edx
   0x000000000040b20a <+698>:	callq  0x413210 <_d_assert>
   0x000000000040b20f <+703>:	lea    0x88(%rsp),%rsi
   0x000000000040b217 <+711>:	mov    0x108(%rsp),%rdi
   0x000000000040b21f <+719>:	callq  0x40b530 <_D2rt19sections_elf_shared12scanSegmentsFNbNiKxS4core3sys5linux4link12dl_phdr_infoPS2rt19sections_elf_shared3DSOZv>
   0x000000000040b224 <+724>:	lea    0x247a8d(%rip),%rdi        # 0x652cb8 <_D2rt19sections_elf_shared11_loadedDSOsS2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array>
   0x000000000040b22b <+731>:	callq  0x40e1d0 <_D2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array5emptyMxFNaNbNdNiNfZb>
   0x000000000040b230 <+736>:	test   $0x1,%al
   0x000000000040b232 <+738>:	jne    0x40b268 <_d_dso_registry+792>
   0x000000000040b234 <+740>:	xor    %eax,%eax
   0x000000000040b236 <+742>:	mov    %al,%cl
   0x000000000040b238 <+744>:	test   $0x1,%cl
   0x000000000040b23b <+747>:	jne    0x40b23f <_d_dso_registry+751>
   0x000000000040b23d <+749>:	jmp    0x40b241 <_d_dso_registry+753>
   0x000000000040b23f <+751>:	jmp    0x40b268 <_d_dso_registry+792>
   0x000000000040b241 <+753>:	lea    0x2d8e8(%rip),%rsi        # 0x438b30
   0x000000000040b248 <+760>:	lea    0x2d7f1(%rip),%rcx        # 0x438a40
   0x000000000040b24f <+767>:	mov    $0x33,%eax
   0x000000000040b254 <+772>:	mov    %eax,%edi
   0x000000000040b256 <+774>:	mov    $0x18,%eax
   0x000000000040b25b <+779>:	mov    %eax,%edx
   0x000000000040b25d <+781>:	mov    $0x1f3,%r8d
   0x000000000040b263 <+787>:	callq  0x4131c0 <_d_assert_msg>
=> 0x000000000040b268 <+792>:	lea    0x247a49(%rip),%rdi        # 0x652cb8 <_D2rt19sections_elf_shared11_loadedDSOsS2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array>
   0x000000000040b26f <+799>:	callq  0x40e280 <_D2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array7opSliceMNgFNaNbNiZANgPS2rt19sections_elf_shared3DSO>
   0x000000000040b274 <+804>:	mov    %rax,0x78(%rsp)
   0x000000000040b279 <+809>:	mov    %rdx,0x80(%rsp)
   0x000000000040b281 <+817>:	movq   $0x0,0x70(%rsp)
   0x000000000040b28a <+826>:	mov    0x78(%rsp),%rax
   0x000000000040b28f <+831>:	cmp    %rax,0x70(%rsp)
   0x000000000040b294 <+836>:	jae    0x40b2e5 <_d_dso_registry+917>
   0x000000000040b296 <+838>:	mov    0x70(%rsp),%rax
   0x000000000040b29b <+843>:	mov    0x80(%rsp),%rcx
   0x000000000040b2a3 <+851>:	mov    (%rcx,%rax,8),%rax
   0x000000000040b2a7 <+855>:	mov    %rax,0x68(%rsp)
   0x000000000040b2ac <+860>:	mov    0x68(%rsp),%rax
   0x000000000040b2b1 <+865>:	cmp    0x108(%rsp),%rax
   0x000000000040b2b9 <+873>:	je     0x40b2bd <_d_dso_registry+877>
   0x000000000040b2bb <+875>:	jmp    0x40b2d5 <_d_dso_registry+901>
   0x000000000040b2bd <+877>:	lea    0x2d77c(%rip),%rsi        # 0x438a40
   0x000000000040b2c4 <+884>:	mov    $0x18,%eax
   0x000000000040b2c9 <+889>:	mov    %eax,%edi
   0x000000000040b2cb <+891>:	mov    $0x1f5,%edx
   0x000000000040b2d0 <+896>:	callq  0x413210 <_d_assert>
   0x000000000040b2d5 <+901>:	mov    0x70(%rsp),%rax
   0x000000000040b2da <+906>:	add    $0x1,%rax
   0x000000000040b2de <+910>:	mov    %rax,0x70(%rsp)
   0x000000000040b2e3 <+915>:	jmp    0x40b28a <_d_dso_registry+826>
   0x000000000040b2e5 <+917>:	lea    0x2479cc(%rip),%rdi        # 0x652cb8 <_D2rt19sections_elf_shared11_loadedDSOsS2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array>
   0x000000000040b2ec <+924>:	lea    0x108(%rsp),%rsi
   0x000000000040b2f4 <+932>:	callq  0x40eec0 <_D2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array16__T10insertBackZ10insertBackMFNbNiKPS2rt19sections_elf_shared3DSOZv>
   0x000000000040b2f9 <+937>:	mov    0x108(%rsp),%rsi
   0x000000000040b301 <+945>:	mov    0x108(%rsp),%rdi
   0x000000000040b309 <+953>:	mov    0x40(%rsi),%rsi
   0x000000000040b30d <+957>:	mov    0x48(%rdi),%rdi
   0x000000000040b311 <+961>:	callq  0x40b690 <_D2rt19sections_elf_shared11getTLSRangeFNbNimmZAv>
   0x000000000040b316 <+966>:	mov    %rax,0x58(%rsp)
   0x000000000040b31b <+971>:	mov    %rdx,0x60(%rsp)
   0x000000000040b320 <+976>:	callq  0x40ade0 <_D2rt19sections_elf_shared13initTLSRangesFNbNiZPS2rt4util9container5array13__T5ArrayTAvZ5Array>
   0x000000000040b325 <+981>:	lea    0x58(%rsp),%rsi
   0x000000000040b32a <+986>:	mov    %rax,%rdi
   0x000000000040b32d <+989>:	callq  0x40ef50 <_D2rt4util9container5array13__T5ArrayTAvZ5Array16__T10insertBackZ10insertBackMFNbNiKAvZv>
   0x000000000040b332 <+994>:	lea    0x24798f(%rip),%rax        # 0x652cc8 <_D2rt19sections_elf_shared21_isRuntimeInitializedb>
   0x000000000040b339 <+1001>:	testb  $0x1,(%rax)
   0x000000000040b33c <+1004>:	je     0x40b362 <_d_dso_registry+1042>
   0x000000000040b33e <+1006>:	mov    0x108(%rsp),%rdi
   0x000000000040b346 <+1014>:	callq  0x40b6c0 <_D2rt19sections_elf_shared16registerGCRangesFNbNiPS2rt19sections_elf_shared3DSOZv>
   0x000000000040b34b <+1019>:	mov    $0x1,%edi
   0x000000000040b350 <+1024>:	movb   $0x1,0x57(%rsp)
   0x000000000040b355 <+1029>:	mov    0x108(%rsp),%rsi
   0x000000000040b35d <+1037>:	callq  0x40b740 <_D2rt19sections_elf_shared21runModuleConstructorsFPS2rt19sections_elf_shared3DSObZv>
   0x000000000040b362 <+1042>:	jmpq   0x40b48d <_d_dso_registry+1341>
   0x000000000040b367 <+1047>:	lea    0x2d6d2(%rip),%rsi        # 0x438a40
   0x000000000040b36e <+1054>:	mov    $0x18,%eax
   0x000000000040b373 <+1059>:	mov    %eax,%edi
   0x000000000040b375 <+1061>:	mov    $0x1b7,%edx
   0x000000000040b37a <+1066>:	callq  0x413210 <_d_assert>
   0x000000000040b37f <+1071>:	lea    0x247942(%rip),%rax        # 0x652cc8 <_D2rt19sections_elf_shared21_isRuntimeInitializedb>
   0x000000000040b386 <+1078>:	mov    0x118(%rsp),%rcx
   0x000000000040b38e <+1086>:	mov    0x8(%rcx),%rcx
   0x000000000040b392 <+1090>:	mov    (%rcx),%rcx
   0x000000000040b395 <+1093>:	mov    %rcx,0x48(%rsp)
   0x000000000040b39a <+1098>:	mov    0x118(%rsp),%rcx
   0x000000000040b3a2 <+1106>:	mov    0x8(%rcx),%rcx
   0x000000000040b3a6 <+1110>:	movq   $0x0,(%rcx)
   0x000000000040b3ad <+1117>:	testb  $0x1,(%rax)
   0x000000000040b3b0 <+1120>:	je     0x40b3d0 <_d_dso_registry+1152>
   0x000000000040b3b2 <+1122>:	mov    $0x1,%edi
   0x000000000040b3b7 <+1127>:	movb   $0x1,0x47(%rsp)
   0x000000000040b3bc <+1132>:	mov    0x48(%rsp),%rsi
   0x000000000040b3c1 <+1137>:	callq  0x40b780 <_D2rt19sections_elf_shared20runModuleDestructorsFPS2rt19sections_elf_shared3DSObZv>
   0x000000000040b3c6 <+1142>:	mov    0x48(%rsp),%rdi
   0x000000000040b3cb <+1147>:	callq  0x40b7c0 <_D2rt19sections_elf_shared18unregisterGCRangesFNbNiPS2rt19sections_elf_shared3DSOZv>
   0x000000000040b3d0 <+1152>:	mov    0x48(%rsp),%rax
   0x000000000040b3d5 <+1157>:	mov    %rax,0x8(%rsp)
   0x000000000040b3da <+1162>:	mov    %fs:0x0,%rax
   0x000000000040b3e3 <+1171>:	lea    -0x310(%rax),%rax
   0x000000000040b3ea <+1178>:	mov    (%rax),%rdi
   0x000000000040b3ed <+1181>:	callq  0x40de70 <_D2rt4util9container5array13__T5ArrayTAvZ5Array4backMNgFNaNbNcNdNiZNgAv>
   0x000000000040b3f2 <+1186>:	mov    (%rax),%rax
   0x000000000040b3f5 <+1189>:	mov    0x8(%rsp),%rcx
   0x000000000040b3fa <+1194>:	cmp    %rax,0x48(%rcx)
   0x000000000040b3fe <+1198>:	jne    0x40b475 <_d_dso_registry+1317>
   0x000000000040b400 <+1200>:	mov    %fs:0x0,%rax
   0x000000000040b409 <+1209>:	lea    -0x310(%rax),%rax
   0x000000000040b410 <+1216>:	mov    (%rax),%rdi
   0x000000000040b413 <+1219>:	callq  0x40df20 <_D2rt4util9container5array13__T5ArrayTAvZ5Array7popBackMFNbNiZv>
   0x000000000040b418 <+1224>:	lea    0x247899(%rip),%rdi        # 0x652cb8 <_D2rt19sections_elf_shared11_loadedDSOsS2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array>
   0x000000000040b41f <+1231>:	callq  0x40e210 <_D2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array4backMNgFNaNbNcNdNiZNgPS2rt19sections_elf_shared3DSO>
   0x000000000040b424 <+1236>:	mov    0x48(%rsp),%rcx
   0x000000000040b429 <+1241>:	cmp    (%rax),%rcx
   0x000000000040b42c <+1244>:	jne    0x40b45d <_d_dso_registry+1293>
   0x000000000040b42e <+1246>:	lea    0x247883(%rip),%rdi        # 0x652cb8 <_D2rt19sections_elf_shared11_loadedDSOsS2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array>
   0x000000000040b435 <+1253>:	callq  0x40e2c0 <_D2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array7popBackMFNbNiZv>
   0x000000000040b43a <+1258>:	mov    0x48(%rsp),%rdi
   0x000000000040b43f <+1263>:	callq  0x40b840 <_D2rt19sections_elf_shared7freeDSOFNbNiPS2rt19sections_elf_shared3DSOZv>
   0x000000000040b444 <+1268>:	lea    0x24786d(%rip),%rdi        # 0x652cb8 <_D2rt19sections_elf_shared11_loadedDSOsS2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array>
   0x000000000040b44b <+1275>:	callq  0x40e1d0 <_D2rt4util9container5array41__T5ArrayTPS2rt19sections_elf_shared3DSOZ5Array5emptyMxFNaNbNdNiNfZb>
   0x000000000040b450 <+1280>:	test   $0x1,%al
   0x000000000040b452 <+1282>:	jne    0x40b456 <_d_dso_registry+1286>
   0x000000000040b454 <+1284>:	jmp    0x40b45b <_d_dso_registry+1291>
   0x000000000040b456 <+1286>:	callq  0x40b860 <_D2rt19sections_elf_shared9finiLocksFNbNiZv>
   0x000000000040b45b <+1291>:	jmp    0x40b48d <_d_dso_registry+1341>
   0x000000000040b45d <+1293>:	lea    0x2d5dc(%rip),%rsi        # 0x438a40
   0x000000000040b464 <+1300>:	mov    $0x18,%eax
   0x000000000040b469 <+1305>:	mov    %eax,%edi
   0x000000000040b46b <+1307>:	mov    $0x23d,%edx
   0x000000000040b470 <+1312>:	callq  0x413210 <_d_assert>
   0x000000000040b475 <+1317>:	lea    0x2d5c4(%rip),%rsi        # 0x438a40
   0x000000000040b47c <+1324>:	mov    $0x18,%eax
   0x000000000040b481 <+1329>:	mov    %eax,%edi
   0x000000000040b483 <+1331>:	mov    $0x23b,%edx
   0x000000000040b488 <+1336>:	callq  0x413210 <_d_assert>
   0x000000000040b48d <+1341>:	add    $0x128,%rsp
   0x000000000040b494 <+1348>:	retq   
End of assembler dump.

[kinke]

As Johan mentioned and is now visible in your backtrace, you seem to be hitting this assert here, hidden by a consecutive segfault. We've just had a similar report, #2280 (comment), which isn't LTO-specific though. It appears specific to shared runtime libs though.

[Fatmice]

So what to do? I do not have issues when compiling with flto=full and optimization. Without the flto flag, it also compiles and run just fine.

How are the building nodes that you guys using to build pass a the lto-thin tests? What's different about them?