allocscompilerwrapper.py would be better as one or more linker plugins

[stephenrkell]

We're very close to being able to ditch the wrapper script and move to 'toolsub'.

The linker plugin(s) should do the following

• 'link-used-types' fixup of .o files

• globalize any local-sym wrapped functions

• post-link metadata build including '-Wl,-q' hackery

• 'allocator mods' including both caller-side and callee-side wrappers/stubs

• setting the dynamic linker?

It's still nice to have these steps as separate scripts, invokable from the command line. So perhaps keep it structured that way. That means replacing the mess that is objdumpallocs is a separate project.

The linker plugins will get added by a toolsub-style command line rewriter, so anything that is purely a command-line thing (like the dynamic linker) may be best not done in a plugin. We should co-develop this with the necessary toolsub scripts. Similarly, the necessary compile-time interventions (dumpallocs.ml, force-setting '-g' and '-ffunction-sections') will need to be done there.

[stephenrkell]

How do multiple linker plugins interact? How do we make use of the 'xwrap' plugin?

Some of these plugins generate content for the link: allocator mods, used uniqtypes. Globalizing belongs here too. Let's make all these a single plugin. (Ideally we would use the dwarfidl name to identify the local allocator function, using '::', then globalize it under a new, generated name if we find a match.)

That content needs some xwraps. Let's imagine putting xwrap afterwards in the link. We could even check in the content-generating plugins that that xwraps are there, later in the command line.

We want one that does stuff after the link (metadata build, then strip the relocs / debuginfo as necessary). Probably this should be a standalone tool that can be run from a cleanup handler, assuming cleanup is done after output.

[stephenrkell]

See #14 for some relevant things here. But since that issue, most of the work has been done!

I think -z dynamic-undefined-weak is really useful. It might even let us eliminate link-used-types. The idea is that if a binary references its own metadata, it should depend on its own metadata using DT_NEEDED. Using -z dynamic-undefined-weak we can leave the reference unresolved but include a DT_NEEDED pointing at the metadata. (Ideally we'd have DT_USEFUL not just DT_NEEDED.

One problem is that the place is not a standard library path... even under /usr/lib/meta the path depends on the origin path of the binary. So can we use $ORIGIN? This is robust to file moves, but not to file renames, since the metadata binary name would be based on the name of the base DSO as originally created. We could switch to naming meta-DSOs by an opaque identifier. Then we hvae a potential for phase-order problems... does doing a .dynamic fix-up (populating a spare dynamic tag) change the build ID? Perhaps if it were to be recomputed, but of course it wouldn't change the one embedded in the file, so I think that is OK. Maybe build IDs plus a symlink is the right way to organise /usr/lib/meta.

[stephenrkell]

The remainder of this issue has been done in eb2cdd2 and predecessors (19addd3, 8cef4ca, e104d54). However, we are not using the linker plugin by default (only in a few test cases, e.g. plugin-hello and also alloca-clang). This is owing to a few small issues, e.g. link failures if our environment variables name allocation functions not defined in the link. At some point I will fix these and switch the entire test suite over to the linker plugin, but I will keep this issue open until then.