Add embedder defined allocator support (mimalloc example)

[jschwe]

This is a PR to explore adding support for using an allocator provided by the servo / the embedder.
An example is provided (examples/embedder_allocator), which uses mimalloc to show how an embedder is expected to provide a custom allocator for Spidermonkey to use. To summarize:

Example: Using mozjs with an embedder-provided allocator

1. In your project setup an include directory containing servo_embedder_allocator.h defining
   SERVO_EMBEDDER_MALLOC_PREFIX and all required allocation functions with the corresponding prefix.
   See mimalloc/include for an example based on mimalloc.
2. Enable the feature custom-alloc and set the environment variable SERVO_CUSTOM_ALLOC_INCLUDE_DIR to
   point to the include directory created in step 1.

This design ensures that the allocator functions can be inlined, and minimize boilerplate for allocators which provide prefixed versions of standard allocator APIs.

This PR does not eliminate all usages of standard malloc / free, but covers the most important ones. This is not a major issue, as long as the remaining usages are isolated in the respective third-party library.
Most significantly, icu is not covered, however that is exposed via JS_SetICUMemoryFunctions, so we could initialize that from the embedder / servo side.

Notes:

• -DJS_USE_CUSTOM_ALLOCATOR doesn't seem to be actually useful / maintained, since any custom implementation still need to copy lots of code from Utility.h
• Some code does make assumption that moz_arena_ allocated pointers can be freed / re-alloced by unprefixed free(), this lead to a patch to StringBuffer.h

[sagudev]

DJS_USE_CUSTOM_ALLOCATOR doesn't seem to be actually useful / maintained, since any custom implementation still need to copy lots of code from Utility.h

Indeed. I think JS_USE_CUSTOM_ALLOCATOR should guard the code in this block: https://searchfox.org/firefox-main/rev/af3c77e22f88ca06e1b4ac822c1d056f263a64fe/js/public/Utility.h#355 not the thread code too.

Some code does make assumption that moz_arena_ allocated pointers can be freed / re-alloced by unprefixed free(), this lead to a patch to StringBuffer.h

Sounds like we could upstream some fixes.

This design ensures that the allocator functions can be inlined

I understand the need for this (we cannot inline without LTO - maybe we should investigate thinlto support in mozjs artifacts), but this design is complex (hate environment variables), I am wondering how we could make use of this in servo (for linux where we use jemalloc). One thing that I loved about webeef solution is that it's easy to pass overrides, and we could possibly do this without changing artifacts (always use prefixed alloc, but provide "own" in mozjs as default). Does inlining really makes difference in practice?

[jschwe]

Does inlining really makes difference in practice?

Yes - the issue is that the happy path of allocators is highly optimized to be just a few instructions. Adding a function call is measurable overhead (since it saves and restores registers on the stack). How measurable depends of course on the frequency of allocations, but it really adds up. (This is based on discussions with a colleague who is responsible for our in-house custom allocator - I didn't measure myself).

I am wondering how we could make use of this in servo (for linux where we use jemalloc)

I have a half-baked PR for servo, which uses this feature. The main issue is that it adds another thing to mach (due to the environment variable). Otherwise, nothing much is really required on the servo side.

One thing that I loved about webeef solution is that it's easy to pass overrides, and we could possibly do this without changing artifacts

This is mainly a tradeoff between baking in the function calls statically (via static inline) or having real function declarations that are only resolved by the linker (which then requires LTO to inline the function calls again).
However, this approach does allow combining the two, by (the embedder / servo) providing a header file to SM which provides static inline function definitions for the allocator APIs, which call a servo_embedder_<malloc> function, which is forward declared and can be resolved at link-time.

Some code does make assumption that moz_arena_ allocated pointers can be freed / re-alloced by unprefixed free(), this lead to a patch to StringBuffer.h

Sounds like we could upstream some fixes.

I'm not sure if they would appreciate it. The intention seems to be that if the arena allocator is OOM, then they fall back to malloc, and thus need to always free with free. That's also why in practice the SM code seems to rely on the fact that jemalloc will be used unprefixed in many places.

[jschwe]

If we published servo-allocator, then mozjs-sys could also depend on it and access information via DEP_ syntax (it's a bit hacky since it requires adding a links key, but I believe we already abuse this in a different servo crate).
An embedder could choose a different allocator via patch. Not quite as simple as via environment variables, but perhaps an option if we want more robustness, and offer a couple built-in solutions like jemalloc and mimalloc?

I haven't thought this compleltely through, so perhaps something prevents that solution. It would make the publish pipeline for alloc related changes quite ugly though.