Initialization overhaul using allocsld?

[stephenrkell]

The initialization of liballocs has always been fragile. It often suffers reentrancy problems, e.g. from early malloc calls in the ld.so. We have ways of dealing with these case-by-case, but it's not very satisfactory. At present, the ld.so always runs before we do, so we are always at its mercy. We already have a certain amount of complexity for bringing ourselves up during normal DSO construction, e.g. allowing for the span where systrapping is not enabled. Recent glibc bug workaround efforts (#81) are an especially nasty case of this.

The addition of allocsld allows us to change that. I am in the process of a basic version of binary instrumentation of allocation functions (#11). Combining these two, we may be able to introspect on the whole of execution much more uniformly, starting from the ld.so or even from allocsld itself, and seeing all mmap calls from the off. (The only code in allocsld that needs to be runnable after it jumps to the real ld.so is the _dl_debug_state function. We could put that on a separate page, even, so that none of the other code retains execute permission.)

Of course there is a meta-completeness issue (#16). Binary instrumentation complicates this because it involves allocating trampolines that currently have no meta-information of their own and break backtraces. This is fixable, however.

I'm vaguely imagining a simplified initialization where

1. all mmaps are caught uniformly (but by what code? it may need to be duplicated in allocsld)
2. liballocs would ideally need no constructor at all (but how is this different from allocsld just being liballocs? I think I considered and rejected that in Dummyweaks library should go away #56)

[stephenrkell]

In Documentation/init-order.txt we describe the motivation for marking liballocs DF_1_INITFIRST. However, being initfirst has been the cause of at least one glibc crash (#81) and I later found that glibc folklore records a prohibition on any library other than libpthread being marked with this flag. It's not clear whether that is current or where it is documented (except in a document advising against doing many obviously useful and supported things).

Is there a way to simplify our initialization so that we don't have to be initfirst? To summarise from init-order.txt, the issue is that "our 'new-style' approach to hooking malloc was to never need our own malloc [to service an incoming malloc request]. But if we want to initialize liballocs directly from the hook, we can't do that. The problem is that we are getting a really early malloc coming in, and liballocs isn't initialized yet".

One way to approach this: what initialization of liballocs might need to be done for "early" "incoming" events? If the events are just malloc and friends, and if we just need to mmap space for a bitmap, or even use the "linear malloc" index, then fix up later all the rest of the liballocs stuff around it, we could do that.

In other words we would have an "early indexing" implementation rather than an "early malloc". When we switch out of early mode, we create all the "non-early" indexing we deferred earlier. In our new code in allocsld (#98), we already have an implementation of early indexing, albeit currently specialised to the ld.so's malloc.

Can we generalise this into simply collecting an event log of "early" stuff that we process later? Not quite, because remember that our early malloc has to also interfere with malloc arguments. We can still do that, though, i.e. take a "rewrite and log" approach rather than initializing ourselves really early.

[stephenrkell]

Remember also we have an "early libs" hack in place.

Digging into the glibc code, I notice that dl_initfirst is a single global link map pointer in the dynamic linker. So it can only 'remember' a single initfirst object. It's not clear what happens if we clobber that pointer with another, from a second initfirst object.