[emval] Move reference counting to C++

[RReverser]

C++ is super eager with copying values by default and asking questions later instead of moving, and each of those copies had to cross JS boundary on creation/deletion just to increase/decrease reference count correspondingly.

Moving reference count from JS to C++ allows to make all this copying substantially cheaper as it becomes a single integer increment/decrement on the Wasm side which, in most cases, can even be completely eliminated by LLVM.

This does mean that we need to heap-allocate a small object with value's metadata on the C++ side, and I know it can be a controversial change, but:

1. We had to do that on the JS side as well with the extra object wrapper around the value itself, and I'm putting my bet on C++ allocation being somewhat cheaper.
2. With some relatively minor tricks, we can convince LLVM to optimise most of those allocations away for common case of local values, just like refcounts. I'm even going as far as adding test to ensure we don't regress on those optimisations in the future.
3. This prevents implicit copies from triggering the cross-thread access assertion.
4. It prepares better ground for reference types and doing all handle management on Wasm side.

Note that reference counting and allocation optimisation for local values only applies to speed modes (-O1 and higher) and not to -Os where LLVM avoids inlining functions, but microbenchmarks show that even in situations where copies can't be eliminated, this approach is still faster:

benchmarks.cpp

#include <emscripten.h>
#include <emscripten/val.h>
#include <stdio.h>

using namespace emscripten;

EM_JS(void, sideEffect, (EM_VAL v), {
  globalThis.sideEffect = Emval.toValue(v);
});

template<typename Func> void measure(const char* name, Func&& func) {
  // preheat
  for (int i = 0; i < 1'000'000; i++) {
    sideEffect(func().as_handle());
  }
  double time = emscripten_get_now();
  // measure
  int count = 100'000'000;
  for (int i = 0; i < count; i++) {
    sideEffect(func().as_handle());
  }
  time = emscripten_get_now() - time;
  printf("%s: %d ns\n", name, (int)(1e6 * time / count));
}

int main() {
  measure("Allocation cost", []() { return val(42); });

  measure("Copying cost", [v = val(42)]() { return val(v); });
}

Before (timings and JS+Wasm byte sizes for different modes):

-Os:

Allocation cost: 61 ns
Copying cost: 20 ns
43164

-O1:

Allocation cost: 60 ns
Copying cost: 21 ns
82809

-O2:

Allocation cost: 62 ns
Copying cost: 23 ns
44955

After:

-Os:

Allocation cost: 68 ns
Copying cost: 11 ns
43022

-O1:

Allocation cost: 49 ns
Copying cost: 8 ns
82431

-O2:

Allocation cost: 46 ns
Copying cost: 8 ns
44746

The only small slowdown is for initial allocation & freeing in -Os mode, but it's typically less common than copying, and it's in exchange for a small code size win, which is, well, point of that mode anyway.

[RReverser]

Marking as draft for now as there's one unresolved issue.

[RReverser]

Hm the remaining issues might be easier to solve after #20383 and another planned subsequent PR. I'll leave this as a draft for now.

[mrolig5267319]

Have you considered using the same array approach on the C++ side for refcount as the JS-side? That is, having a single vector<val_metadata> with refcounts indexed by handle, such that most new handles won't require an allocation?

Ah, because in multi-threaded mode you could have handle collisions between threads? Could have an array per thread.

[RReverser]

Have you considered using the same array approach on the C++ side for refcount as the JS-side?

Yes, I played with it, but that prevents compiler optimisations that allow to completely eliminate allocation/free pairs, which are extremely valuable as most of Embind values are temporary. Compilers are happy to optimise malloc/free pairs because those are simple well-known APIs but they can't see through all the mechanics of something as complex as a vector (as it has its own list that can move around memory on reallocs).

[mrolig5267319]

Sorry if I'm missing something obvious, but how do you get away with removing the incref/decref from the C++ toWireType and JS fromWireType? Won't the C++ destructor call free before the JS fromWireType reads Emval.toValue?