std: Implement TLS for wasm32-unknown-unknown #54951

alexcrichton · 2018-10-10T06:17:03Z

This adds an implementation of thread local storage for the
wasm32-unknown-unknown target when the atomics feature is
implemented. This, however, comes with a notable caveat of that it
requires a new feature of the standard library, wasm-bindgen-threads,
to be enabled.

Thread local storage for wasm (when atomics are enabled and there's
actually more than one thread) is powered by the assumption that an
external entity can fill in some information for us. It's not currently
clear who will fill in this information nor whose responsibility it
should be long-term. In the meantime there's a strategy being gamed out
in the wasm-bindgen project specifically, and the hope is that we can
continue to test and iterate on the standard library without committing
to a particular strategy yet.

As to the details of wasm-bindgen's strategy, LLVM doesn't currently
have the ability to emit custom global values (thread locals in a
WebAssembly.Module) so we leverage the wasm-bindgen CLI tool to do
it for us. To that end we have a few intrinsics, assuming two global values:

__wbindgen_current_id - gets the current thread id as a 32-bit
integer. It's wasm-bindgen's responsibility to initialize this
per-thread and then inform libstd of the id. Currently wasm-bindgen
performs this initialization as part of the start function.
__wbindgen_tcb_{get,set} - in addition to a thread id it's assumed
that there's a global available for simply storing a pointer's worth
of information (a thread control block, which currently only contains
thread local storage). This would ideally be a native global
injected by LLVM, but we don't have a great way to support that right
now.

To reiterate, this is all intended to be unstable and purely intended
for testing out Rust on the web with threads. The story is very likely
to change in the future and we want to make sure that we're able to do
that!

rust-highfive · 2018-10-10T06:17:06Z

r? @bluss

(rust_highfive has picked a reviewer for you, use r? to override)

alexcrichton · 2018-10-10T06:17:22Z

r? @SimonSapin

rust-highfive · 2018-10-10T06:31:27Z

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.


[00:04:13] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:14] tidy error: /checkout/src/libstd/sys/wasm/thread_local_atomics.rs:42: TODO is deprecated; use FIXME
[00:04:14] tidy error: /checkout/src/libstd/sys/wasm/thread.rs:98: TODO is deprecated; use FIXME
[00:04:15] some tidy checks failed
[00:04:15] 
[00:04:15] 
[00:04:15] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:15] 
[00:04:15] 
[00:04:15] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:15] Build completed unsuccessfully in 0:00:42
[00:04:15] Build completed unsuccessfully in 0:00:42
[00:04:15] make: *** [tidy] Error 1
[00:04:15] Makefile:79: recipe for target 'tidy' failed

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:2203cbf0
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
---
travis_time:end:0072fa74:start=1539152573695837699,finish=1539152573699600477,duration=3762778
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:00c48a92
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:360961b8
travis_time:start:360961b8
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:32b53a88
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rust-highfive · 2018-10-10T06:50:11Z

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.


[00:04:15] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:16] tidy error: /checkout/src/libstd/sys/wasm/thread_local_atomics.rs:42: TODO is deprecated; use FIXME
[00:04:17] some tidy checks failed
[00:04:17] 
[00:04:17] 
[00:04:17] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:17] 
[00:04:17] 
[00:04:17] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:17] Build completed unsuccessfully in 0:00:48
[00:04:17] Build completed unsuccessfully in 0:00:48
[00:04:17] Makefile:79: recipe for target 'tidy' failed
[00:04:17] make: *** [tidy] Error 1

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:09d5c42f
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
---
travis_time:end:04e1f27c:start=1539153692745016423,finish=1539153692750064385,duration=5047962
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:1072f38c
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:091b0c84
travis_time:start:091b0c84
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:034a6106
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

SimonSapin · 2018-10-10T09:29:45Z

I know some of those words, but I feel very much not qualified to review this and don’t know who would be.

alexcrichton · 2018-10-10T17:34:25Z

@sfackler sorry I don't mean to dump all reviews on you, but you might be most familiar with all this as well, mind taking a look? If you don't have time though just let me know!

sfackler · 2018-10-10T18:56:13Z

src/libstd/sys/wasm/thread_local_atomics.rs

Why don't we want to hand out 0?

Ah this is a bit cryptic (and perhaps backwards), but it's due to this block where out platform-agnostic thread local code can't deal with a TLS descriptor of 0 so it just asks for another one if that happens

sfackler · 2018-10-11T14:32:54Z

src/libstd/sys/wasm/thread_local_atomics.rs

This can technically overflow after 4 billion repeated failures but that may not be worth worrying about?

Heh I'm not too worried about overflow but it's easy enough to add a store beneath this in the failure case back to MAX_KEYS, so done now!

This adds an implementation of thread local storage for the `wasm32-unknown-unknown` target when the `atomics` feature is implemented. This, however, comes with a notable caveat of that it requires a new feature of the standard library, `wasm-bindgen-threads`, to be enabled. Thread local storage for wasm (when `atomics` are enabled and there's actually more than one thread) is powered by the assumption that an external entity can fill in some information for us. It's not currently clear who will fill in this information nor whose responsibility it should be long-term. In the meantime there's a strategy being gamed out in the `wasm-bindgen` project specifically, and the hope is that we can continue to test and iterate on the standard library without committing to a particular strategy yet. As to the details of `wasm-bindgen`'s strategy, LLVM doesn't currently have the ability to emit custom `global` values (thread locals in a `WebAssembly.Module`) so we leverage the `wasm-bindgen` CLI tool to do it for us. To that end we have a few intrinsics, assuming two global values: * `__wbindgen_current_id` - gets the current thread id as a 32-bit integer. It's `wasm-bindgen`'s responsibility to initialize this per-thread and then inform libstd of the id. Currently `wasm-bindgen` performs this initialization as part of the `start` function. * `__wbindgen_tcb_{get,set}` - in addition to a thread id it's assumed that there's a global available for simply storing a pointer's worth of information (a thread control block, which currently only contains thread local storage). This would ideally be a native `global` injected by LLVM, but we don't have a great way to support that right now. To reiterate, this is all intended to be unstable and purely intended for testing out Rust on the web with threads. The story is very likely to change in the future and we want to make sure that we're able to do that!

sfackler · 2018-10-11T17:07:29Z

@bors r+

bors · 2018-10-11T17:07:30Z

📌 Commit cbe9f33 has been approved by sfackler

bors · 2018-10-13T04:22:35Z

⌛ Testing commit cbe9f33 with merge 3431fa629099c29a8ade8fc1ad5eb9019e076527...

bors · 2018-10-13T06:48:07Z

💔 Test failed - status-travis

rust-highfive · 2018-10-13T06:48:10Z

The job x86_64-gnu-distcheck of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[02:20:14] warning: spurious network error (2 tries remaining): curl error: Could not resolve host: github.com
[02:20:14] ; class=Net (12)
[02:20:35] warning: spurious network error (1 tries remaining): curl error: Could not resolve host: github.com
[02:20:35] ; class=Net (12)
[02:20:55] error: failed to load source for a dependency on `rand`
[02:20:55] Caused by:
[02:20:55]   Unable to update registry `https://github.com/rust-lang/crates.io-index`
[02:20:55] 
[02:20:55] Caused by:
[02:20:55] Caused by:
[02:20:55]   failed to fetch `https://github.com/rust-lang/crates.io-index`
[02:20:55] 
[02:20:55] Caused by:
[02:20:55]   curl error: Could not resolve host: github.com
[02:20:55] ; class=Net (12)
[02:20:55] 
[02:20:55] 
[02:20:55] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "generate-lockfile" "--manifest-path" "/checkout/obj/build/tmp/distcheck-src/rust-src/lib/rustlib/src/rust/src/libstd/Cargo.toml"
[02:20:55] 
[02:20:55] 
[02:20:55] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test distcheck
[02:20:55] Build completed unsuccessfully in 2:17:46
---
travis_time:end:06aca06c:start=1539413165398035422,finish=1539413165404781177,duration=6745755
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:003a863f
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:04a7c2d0
travis_time:start:04a7c2d0
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:23062fa2
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

alexcrichton · 2018-10-13T12:43:54Z

@bors: retry

bors · 2018-10-13T14:28:54Z

⌛ Testing commit cbe9f33 with merge f517527...

std: Implement TLS for wasm32-unknown-unknown This adds an implementation of thread local storage for the `wasm32-unknown-unknown` target when the `atomics` feature is implemented. This, however, comes with a notable caveat of that it requires a new feature of the standard library, `wasm-bindgen-threads`, to be enabled. Thread local storage for wasm (when `atomics` are enabled and there's actually more than one thread) is powered by the assumption that an external entity can fill in some information for us. It's not currently clear who will fill in this information nor whose responsibility it should be long-term. In the meantime there's a strategy being gamed out in the `wasm-bindgen` project specifically, and the hope is that we can continue to test and iterate on the standard library without committing to a particular strategy yet. As to the details of `wasm-bindgen`'s strategy, LLVM doesn't currently have the ability to emit custom `global` values (thread locals in a `WebAssembly.Module`) so we leverage the `wasm-bindgen` CLI tool to do it for us. To that end we have a few intrinsics, assuming two global values: * `__wbindgen_current_id` - gets the current thread id as a 32-bit integer. It's `wasm-bindgen`'s responsibility to initialize this per-thread and then inform libstd of the id. Currently `wasm-bindgen` performs this initialization as part of the `start` function. * `__wbindgen_tcb_{get,set}` - in addition to a thread id it's assumed that there's a global available for simply storing a pointer's worth of information (a thread control block, which currently only contains thread local storage). This would ideally be a native `global` injected by LLVM, but we don't have a great way to support that right now. To reiterate, this is all intended to be unstable and purely intended for testing out Rust on the web with threads. The story is very likely to change in the future and we want to make sure that we're able to do that!

bors · 2018-10-13T17:03:42Z

☀️ Test successful - status-appveyor, status-travis
Approved by: sfackler
Pushing f517527 to master...

rust-highfive assigned bluss Oct 10, 2018

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 10, 2018

rust-highfive assigned SimonSapin and unassigned bluss Oct 10, 2018

alexcrichton force-pushed the more-wasm-threads branch from 4873556 to 7524ad7 Compare October 10, 2018 06:35

alexcrichton force-pushed the more-wasm-threads branch from 7524ad7 to 651ea5d Compare October 10, 2018 17:33

sfackler reviewed Oct 10, 2018

View reviewed changes

alexcrichton force-pushed the more-wasm-threads branch from 651ea5d to 08237b2 Compare October 11, 2018 02:29

sfackler reviewed Oct 11, 2018

View reviewed changes

alexcrichton force-pushed the more-wasm-threads branch from 08237b2 to d205e5b Compare October 11, 2018 16:56

alexcrichton force-pushed the more-wasm-threads branch from d205e5b to cbe9f33 Compare October 11, 2018 16:58

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 11, 2018

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Oct 13, 2018

bors removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 13, 2018

bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Oct 13, 2018

bors merged commit cbe9f33 into rust-lang:master Oct 13, 2018

bors mentioned this pull request Oct 13, 2018

Add libstd Cargo feature "panic_immediate_abort" #55011

Merged

ArtemGr mentioned this pull request Oct 18, 2018

[v2] Provisional WASM build GLEECBTC/komodo-defi-framework#126

Closed

alexcrichton deleted the more-wasm-threads branch October 21, 2018 01:55

Uh oh!

std: Implement TLS for wasm32-unknown-unknown #54951

std: Implement TLS for wasm32-unknown-unknown #54951

Uh oh!

Conversation

alexcrichton commented Oct 10, 2018

Uh oh!

rust-highfive commented Oct 10, 2018

Uh oh!

alexcrichton commented Oct 10, 2018

Uh oh!

rust-highfive commented Oct 10, 2018

Uh oh!

rust-highfive commented Oct 10, 2018

Uh oh!

SimonSapin commented Oct 10, 2018

Uh oh!

alexcrichton commented Oct 10, 2018

Uh oh!

sfackler Oct 10, 2018

Choose a reason for hiding this comment

Uh oh!

alexcrichton Oct 10, 2018

Choose a reason for hiding this comment

Uh oh!

sfackler Oct 11, 2018

Choose a reason for hiding this comment

Uh oh!

alexcrichton Oct 11, 2018

Choose a reason for hiding this comment

Uh oh!

sfackler commented Oct 11, 2018

Uh oh!

bors commented Oct 11, 2018

Uh oh!

bors commented Oct 13, 2018

Uh oh!

bors commented Oct 13, 2018

Uh oh!

rust-highfive commented Oct 13, 2018

Uh oh!

alexcrichton commented Oct 13, 2018

Uh oh!

bors commented Oct 13, 2018

Uh oh!

bors commented Oct 13, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants