Implement -s SINGLE_FILE_BINARY_ENCODE=1 option by juj · Pull Request #25599 · emscripten-core/emscripten

juj · 2025-10-20T18:29:32Z

Implement -s SINGLE_FILE_BINARY_ENCODE=1 option to embed Wasm binary as binary-encoded form instead of base64 form in SINGLE_FILE mode. Continuation of #21478.

For comparison of code size, see #21426 (comment).

…as binary-encoded form instead of base64 form in SINGLE_FILE mode. Continuation of emscripten-core#21478.

sbc100

Very nice!

Do you have some rough number you could put in the PR description?

How about adding a codesize check that asserts the overall result is smaller?

Assuming no unforseen issues do you expect we could just make this setting always-enabled one day? (and remove the old base64 fallback?)

src/preamble.js

test/test_other.py

juj · 2025-10-20T20:25:01Z

Do you have some rough number you could put in the PR description?

Added.

How about adding a codesize check that asserts the overall result is smaller?

Added.

Assuming no unforseen issues do you expect we could just make this setting always-enabled one day?

Maybe one day. For now I think it's good to have the opt-out to give people the fallback if issues come up.

sbc100 · 2025-10-20T20:30:07Z

src/preamble.js

+#elif SINGLE_FILE
  return base64Decode('<<< WASM_BINARY_DATA >>>');
+#elif AUDIO_WORKLET || !EXPORT_ES6
+  // For an Audio Worklet, we cannot use `new URL()`.


Its hard to see why the locateFile path is only used in EXPORT_ES6 mode... do you know why EXPORT_ES6 is here?

I don't know. It is indeed a little bit odd code flow, which is why I cleaned the ifdefs a bit here as a drive-by. Something to ponder further in a separate PR.

sbc100

Maybe worth hinting in the docs that the plan is to remove this new setting if nobody uses it.

src/binaryDecode.js

sbc100 · 2025-10-20T20:33:08Z

src/preamble.js

 #endif

+#if SINGLE_FILE && SINGLE_FILE_BINARY_ENCODE && !WASM2JS
+#include "binaryDecode.js"


Do we not want to support this in MINIMAL_RUNTIME too? If so maybe this could go in runtime_common.js instead?

Thanks, we do. I lost this when resurrecting the code. Added back, and updated test to cover minimal runtime as well.

sbc100 · 2025-10-20T20:34:49Z

Are there any cases where SINGLE_FILE_BINARY_ENCODE could make the total payload larger?

tools/link.py

sbc100 · 2025-10-20T21:39:30Z

src/binaryDecode.js

+    o[i] = bin.charCodeAt(i) - 1;
+  }
+  return o;
+}


Since this is only 5 lines i wonder if its worth creating a completely new flie? Maybe just inline into runtime_common.js?

juj · 2025-10-20T21:48:51Z

Are there any cases where SINGLE_FILE_BINARY_ENCODE could make the total payload larger?

base64 encoding encodes each byte uniformly with +33% overhead.

Binary encoding encodes compatible bytes in the original byte (+0% overhead), but incompatible bytes (09h, 0Ch, 21h, 5Bh, 80h-FFh) will be encoded as two bytes, so with +100% overhead.

If the distribution of input bytes were uniform, this comes out to a +49.61% overhead. So a larger size in uncompressed form.

But the trick here is that

a) since all those +100% code points are byte-delimited, they compress efficiently with gzip and brotli compression, and
b) in practice those incompatible bytes are more rare (MSB is more common to be unset than set), so the end result when encoding a .wasm code file ends up being a major win in uncompressed form as well.

sbc100 · 2025-10-20T22:20:18Z

test/codesize/test_codesize_minimal_O0.expected.js


 function findWasmBinary() {
-    return locateFile('a.out.wasm');
+  // For an Audio Worklet, we cannot use `new URL()`.


Maybe the new location of this comment is misplaced?

Hmm the code size before was relying on the comment being swallowed by virtue of being at the end of an #ifdef.. moved it back up there.

This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (1) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_unoptimized_code_size.json: 180812 => 180812 [+0 bytes / +0.00%] Average change: +0.00% (+0.00% - +0.00%) ```

kripken · 2025-10-30T17:32:08Z

@juj @sbc100 This caused a regression I bisected to here, reported on the mailing list:

https://groups.google.com/g/emscripten-discuss/c/E_HmYqXGjN8

sbc100 · 2025-11-01T22:58:55Z

@juj can you take a look at this early next week? If you don't have time, or there is no easy fix I suggest we just flip the default for SINGLE_FILE_BINARY_ENCODE back for false until we have a fix.

juj · 2025-11-03T14:10:26Z

Thanks for the ping - posted PR #25704 to address this.

Implement -s SINGLE_FILE_BINARY_ENCODE=1 option to embed Wasm binary as binary-encoded form instead of base64 form in SINGLE_FILE mode. Continuation of emscripten-core#21478. For comparison of code size, see emscripten-core#21426 (comment).

juj added 2 commits October 20, 2025 21:28

Implement -s SINGLE_FILE_BINARY_ENCODE=1 option to embed Wasm binary …

511a59e

…as binary-encoded form instead of base64 form in SINGLE_FILE mode. Continuation of emscripten-core#21478.

docs

e577792

sbc100 reviewed Oct 20, 2025

View reviewed changes

src/preamble.js Outdated Show resolved Hide resolved

test/test_other.py Outdated Show resolved Hide resolved

juj added 3 commits October 20, 2025 23:20

Add test

88f3570

Add missing file

543554e

Use booleans

0f8fce3

sbc100 reviewed Oct 20, 2025

View reviewed changes

sbc100 approved these changes Oct 20, 2025

View reviewed changes

sbc100 reviewed Oct 20, 2025

View reviewed changes

juj added 5 commits October 21, 2025 00:24

Add code size file.

091f207

Add comment about option becoming permanent in the future

a8e4cbf

whitespace

c3515f3

Add binary encoding to minimal runtime.

046c5ba

eslint

e9c0d0e

sbc100 reviewed Oct 20, 2025

View reviewed changes

tools/link.py Outdated Show resolved Hide resolved

sbc100 reviewed Oct 20, 2025

View reviewed changes

sbc100 approved these changes Oct 20, 2025

View reviewed changes

Optimize encoding.

dbb0ed7

juj added 5 commits October 21, 2025 00:49

Review

0783a62

Inline binaryDecode.js

5f8df5f

Rebaseline code size tests

f651178

Merge remote-tracking branch 'origin/main' into utf8_binary_encode

25477b9

code size

1c26859

sbc100 reviewed Oct 20, 2025

View reviewed changes

juj added 3 commits October 21, 2025 01:22

Move comment back to end of line

88cd3e8

code size

8a27910

code size

6ddaa56

juj enabled auto-merge (squash) October 21, 2025 00:15

Fix proxy_to_worker

a4144de

juj merged commit 9110edd into emscripten-core:main Oct 21, 2025
34 checks passed

sbc100 mentioned this pull request Dec 10, 2025

lz4 compressed wasm with -s SINGLE_FILE=1 #16056

Open

Conversation

juj commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sbc100 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

juj commented Oct 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sbc100 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sbc100 commented Oct 20, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juj commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kripken commented Oct 30, 2025

Uh oh!

sbc100 commented Nov 1, 2025

Uh oh!

juj commented Nov 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

juj commented Oct 20, 2025 •

edited

Loading

juj commented Oct 20, 2025 •

edited

Loading