Skip to content

Recent releases use clang++ #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
lewing merged 2 commits into from
Jun 30, 2021
Merged

Recent releases use clang++ #34

lewing merged 2 commits into from
Jun 30, 2021

Conversation

@lewing
Copy link
Member

@lewing lewing commented Jun 30, 2021 •
edited by radical
Loading

we need to test the workloads in use before upgrading too.

On windows, emcc ends up invoking "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.win-x64\6.0.0-preview.7.21323.1\tools\bin\clang++.exe", but that file is missing on the sdk package.

And that manifests as:

       WasmApp.Native.targets(268,5): error : Failed to compile C:\dev\bl1\obj\Debug\net6.0\wasm\System.Private.Uri.dll.bc -> C:\dev\bl1\obj\Debug\net6.0\wasm\System.Private.Uri.dll.o: C:\dev\bl1>setlocal [C:\dev\bl1\bl1.csproj]
       WasmApp.Native.targets(268,5): error : C:\dev\bl1>set errorlevel=dummy  [C:\dev\bl1\bl1.csproj]
       WasmApp.Native.targets(268,5): error : C:\dev\bl1>set errorlevel=  [C:\dev\bl1\bl1.csproj]
       WasmApp.Native.targets(268,5): error : C:\dev\bl1>emcc "@c:\dev\.nuget\microsoft.netcore.app.runtime.mono.browser-wasm\6.0.0-preview.7.21329.1\runtimes\browser-wasm\native\src\emcc-default.rsp" -O0 -s DISABLE_EXCEPTION_CATCHING=0 -v -s TOTAL_MEMORY=536870912 -c -o C:\dev\bl1\obj\Debug\net6.0\wasm\System.Private.Uri.dll.o C:\dev\bl1\obj\Debug\net6.0\wasm\System.Private.Uri.dll.bc  [C:\dev\bl1\bl1.csproj]
       WasmApp.Native.targets(268,5): error :  "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.win-x64\6.0.0-preview.7.21323.1\tools\bin\clang++.exe" -target wasm32-unknown-emscripten -DEMSCRIPTEN -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-cxx-exceptions -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=23 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL "--sysroot=C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.win-x64\6.0.0-preview.7.21323.1\tools\emscripten\cache\sysroot" -Xclang -iwithsysroot/include\compat -emit-llvm -O0 -v -c C:\dev\bl1\obj\Debug\net6.0\wasm\System.Private.Uri.dll.bc -o C:\dev\bl1\obj\Debug\net6.0\wasm\System.Private.Uri.dll.o [C:\dev\bl1\bl1.csproj]

       WasmApp.Native.targets(268,5): error : emcc: error: '"C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.win-x64\6.0.0-preview.7.21323.1\tools\bin\clang++.exe" -target wasm32-unknown-emscripten -DEMSCRIPTEN -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-cxx-exceptions -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=23 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL "--sysroot=C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.23.Sdk.win-x64\6.0.0-preview.7.21323.1\tools\emscripten\cache\sysroot" -Xclang -iwithsysroot/include\compat -emit-llvm -O0 -v -c C:\dev\bl1\obj\Debug\net6.0\wasm\System.Private.Uri.dll.bc -o C:\dev\bl1\obj\Debug\net6.0\wasm\System.Private.Uri.dll.o' failed: [WinError 2] The system cannot find the file specified

radical
radical reviewed Jun 30, 2021
View reviewed changes
@lewing @radical
Update eng/emsdk.proj
8364f1d 
Co-authored-by: Ankit Jain <radical@gmail.com>
@lewing
Copy link
Member Author

lewing commented Jun 30, 2021

we need to test the workloads in use before upgrading too.

symlinks would be very nice.

@lewing lewing merged commit f534976 into dotnet:main Jun 30, 2021
@lewing lewing deleted the clang-clang branch June 30, 2021 17:57
@lewing
Copy link
Member Author

lewing commented Jul 2, 2021 •
edited
Loading

Adding some background. After upgrading emscripten windows AOT builds using these packages started failing because they were calling clang++ (not clang) and on windows clang++ was not included in the package. clang and clang++ are actually the same binary but since there is no way to express a symbolic link (or a literal duplicate) in a nupgk(zip) we have to make a copy. Unfortunately making a copy appears to push us over the nuget.org size limit on Windows because of enormous binaries.

this appears to have pushed us over the nuget.org package size limit.

@radekdoulik
Copy link
Member

radekdoulik commented Jul 12, 2021

I tried to strip the clang*exe files with llvm-strip. That doesn't change the size, so I think these are already stripped.

@radekdoulik
Copy link
Member

radekdoulik commented Jul 12, 2021

The cache directory in the nuget is quite big, cca 95M uncompressed. I compared it to my local cache and it is smaller, cca 45M.

I wonder if we can reduce it.

Another possibility would be to split the nuget and have cache in a separate one. That would bring the size back under 250M.

@radekdoulik
Copy link
Member

radekdoulik commented Jul 12, 2021

The original cache:

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---           7/12/2021  8:00 PM           1110 crt1_reactor.o
-a---           7/12/2021  8:00 PM           1264 crt1.o
-a---           7/12/2021  8:00 PM           2430 crtbegin.o
-a---           7/12/2021  8:00 PM          16680 libal.a
-a---           7/12/2021  8:00 PM           9774 libasan_js.a
-a---           7/12/2021  8:00 PM        1871736 libasan_rt-mt.a
-a---           7/12/2021  8:00 PM        1869984 libasan_rt.a
-a---           7/12/2021  8:00 PM         105732 libasmfs-mt.a
-a---           7/12/2021  8:00 PM         105848 libasmfs.a
-a---           7/12/2021  8:00 PM         111148 libc_rt_wasm-asan-optz.a
-a---           7/12/2021  8:00 PM         114662 libc_rt_wasm-asan.a
-a---           7/12/2021  8:00 PM          95664 libc_rt_wasm-optz.a
-a---           7/12/2021  8:00 PM          99190 libc_rt_wasm.a
-a---           7/12/2021  8:00 PM        3629750 libc-asan.a
-a---           7/12/2021  8:00 PM        4055446 libc-mt-asan.a
-a---           7/12/2021  8:00 PM        3066964 libc-mt.a
-a---           7/12/2021  8:00 PM        2730656 libc.a
-a---           7/12/2021  8:00 PM        6091526 libc++-except.a
-a---           7/12/2021  8:00 PM        6091518 libc++-mt-except.a
-a---           7/12/2021  8:00 PM        5869882 libc++-mt-noexcept.a
-a---           7/12/2021  8:00 PM        6270678 libc++-mt.a
-a---           7/12/2021  8:00 PM        5870216 libc++-noexcept.a
-a---           7/12/2021  8:00 PM        6296166 libc++.a
-a---           7/12/2021  8:00 PM         948066 libc++abi-except.a
-a---           7/12/2021  8:00 PM         968018 libc++abi-mt-except.a
-a---           7/12/2021  8:00 PM         910896 libc++abi-mt-noexcept.a
-a---           7/12/2021  8:00 PM         919338 libc++abi-mt.a
-a---           7/12/2021  8:00 PM         891002 libc++abi-noexcept.a
-a---           7/12/2021  8:00 PM         898318 libc++abi.a
-a---           7/12/2021  8:00 PM         527666 libcompiler_rt-mt.a
-a---           7/12/2021  8:00 PM         519362 libcompiler_rt.a
-a---           7/12/2021  8:00 PM          84672 libdlmalloc-debug-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          83348 libdlmalloc-debug-noerrno.a
-a---           7/12/2021  8:00 PM          85286 libdlmalloc-debug-tracing.a
-a---           7/12/2021  8:00 PM          83848 libdlmalloc-debug.a
-a---           7/12/2021  8:00 PM          84352 libdlmalloc-mt-debug-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          83292 libdlmalloc-mt-debug-noerrno.a
-a---           7/12/2021  8:00 PM          84836 libdlmalloc-mt-debug-tracing.a
-a---           7/12/2021  8:00 PM          83734 libdlmalloc-mt-debug.a
-a---           7/12/2021  8:00 PM          81168 libdlmalloc-mt-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          80110 libdlmalloc-mt-noerrno.a
-a---           7/12/2021  8:00 PM          81610 libdlmalloc-mt-tracing.a
-a---           7/12/2021  8:00 PM          80510 libdlmalloc-mt.a
-a---           7/12/2021  8:00 PM          81428 libdlmalloc-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          80104 libdlmalloc-noerrno.a
-a---           7/12/2021  8:00 PM          82002 libdlmalloc-tracing.a
-a---           7/12/2021  8:00 PM          80562 libdlmalloc.a
-a---           7/12/2021  8:00 PM         287162 libembind-rtti.a
-a---           7/12/2021  8:00 PM         277136 libembind.a
-a---           7/12/2021  8:00 PM          72054 libemmalloc-debug-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          70538 libemmalloc-debug-noerrno.a
-a---           7/12/2021  8:00 PM          72216 libemmalloc-debug-tracing.a
-a---           7/12/2021  8:00 PM          70620 libemmalloc-debug.a
-a---           7/12/2021  8:00 PM          73574 libemmalloc-memvalidate-mt-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          72310 libemmalloc-memvalidate-mt-noerrno.a
-a---           7/12/2021  8:00 PM          73638 libemmalloc-memvalidate-mt-tracing.a
-a---           7/12/2021  8:00 PM          72366 libemmalloc-memvalidate-mt.a
-a---           7/12/2021  8:00 PM          72410 libemmalloc-memvalidate-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          70876 libemmalloc-memvalidate-noerrno.a
-a---           7/12/2021  8:00 PM          72572 libemmalloc-memvalidate-tracing.a
-a---           7/12/2021  8:00 PM          83598 libemmalloc-memvalidate-verbose-mt-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          82478 libemmalloc-memvalidate-verbose-mt-noerrno.a
-a---           7/12/2021  8:00 PM          83662 libemmalloc-memvalidate-verbose-mt-tracing.a
-a---           7/12/2021  8:00 PM          82534 libemmalloc-memvalidate-verbose-mt.a
-a---           7/12/2021  8:00 PM          82440 libemmalloc-memvalidate-verbose-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          81064 libemmalloc-memvalidate-verbose-noerrno.a
-a---           7/12/2021  8:00 PM          82602 libemmalloc-memvalidate-verbose-tracing.a
-a---           7/12/2021  8:00 PM          81146 libemmalloc-memvalidate-verbose.a
-a---           7/12/2021  8:00 PM          70958 libemmalloc-memvalidate.a
-a---           7/12/2021  8:00 PM          72874 libemmalloc-mt-debug-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          71608 libemmalloc-mt-debug-noerrno.a
-a---           7/12/2021  8:00 PM          72938 libemmalloc-mt-debug-tracing.a
-a---           7/12/2021  8:00 PM          71664 libemmalloc-mt-debug.a
-a---           7/12/2021  8:00 PM          57022 libemmalloc-mt-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          56602 libemmalloc-mt-noerrno.a
-a---           7/12/2021  8:00 PM          57086 libemmalloc-mt-tracing.a
-a---           7/12/2021  8:00 PM          56658 libemmalloc-mt.a
-a---           7/12/2021  8:00 PM          56896 libemmalloc-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          56226 libemmalloc-noerrno.a
-a---           7/12/2021  8:00 PM          57058 libemmalloc-tracing.a
-a---           7/12/2021  8:00 PM          82910 libemmalloc-verbose-mt-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          81786 libemmalloc-verbose-mt-noerrno.a
-a---           7/12/2021  8:00 PM          82974 libemmalloc-verbose-mt-tracing.a
-a---           7/12/2021  8:00 PM          81842 libemmalloc-verbose-mt.a
-a---           7/12/2021  8:00 PM          82110 libemmalloc-verbose-noerrno-tracing.a
-a---           7/12/2021  8:00 PM          80730 libemmalloc-verbose-noerrno.a
-a---           7/12/2021  8:00 PM          82272 libemmalloc-verbose-tracing.a
-a---           7/12/2021  8:00 PM          80812 libemmalloc-verbose.a
-a---           7/12/2021  8:00 PM          56308 libemmalloc.a
-a---           7/12/2021  8:00 PM          19748 libfetch-mt.a
-a---           7/12/2021  8:00 PM          17786 libfetch.a
-a---           7/12/2021  8:00 PM          81128 libgl-emu-full_es3.a
-a---           7/12/2021  8:00 PM          81128 libgl-emu-ofb-full_es3.a
-a---           7/12/2021  8:00 PM          52376 libgl-emu-ofb.a
-a---           7/12/2021  8:00 PM          81506 libgl-emu-webgl2-full_es3.a
-a---           7/12/2021  8:00 PM          81506 libgl-emu-webgl2-ofb-full_es3.a
-a---           7/12/2021  8:00 PM          80826 libgl-emu-webgl2-ofb.a
-a---           7/12/2021  8:00 PM          80826 libgl-emu-webgl2.a
-a---           7/12/2021  8:00 PM          52376 libgl-emu.a
-a---           7/12/2021  8:00 PM          63978 libgl-full_es3.a
-a---           7/12/2021  8:00 PM          81416 libgl-mt-emu-full_es3.a
-a---           7/12/2021  8:00 PM         257160 libgl-mt-emu-ofb-full_es3.a
-a---           7/12/2021  8:00 PM         160106 libgl-mt-emu-ofb.a
-a---           7/12/2021  8:00 PM          81794 libgl-mt-emu-webgl2-full_es3.a
-a---           7/12/2021  8:00 PM         257538 libgl-mt-emu-webgl2-ofb-full_es3.a
-a---           7/12/2021  8:00 PM         254414 libgl-mt-emu-webgl2-ofb.a
-a---           7/12/2021  8:00 PM          81114 libgl-mt-emu-webgl2.a
-a---           7/12/2021  8:00 PM          52592 libgl-mt-emu.a
-a---           7/12/2021  8:00 PM          64266 libgl-mt-full_es3.a
-a---           7/12/2021  8:00 PM         240010 libgl-mt-ofb-full_es3.a
-a---           7/12/2021  8:00 PM         142956 libgl-mt-ofb.a
-a---           7/12/2021  8:00 PM          64636 libgl-mt-webgl2-full_es3.a
-a---           7/12/2021  8:00 PM         240380 libgl-mt-webgl2-ofb-full_es3.a
-a---           7/12/2021  8:00 PM         237256 libgl-mt-webgl2-ofb.a
-a---           7/12/2021  8:00 PM          63956 libgl-mt-webgl2.a
-a---           7/12/2021  8:00 PM          35442 libgl-mt.a
-a---           7/12/2021  8:00 PM          63978 libgl-ofb-full_es3.a
-a---           7/12/2021  8:00 PM          35226 libgl-ofb.a
-a---           7/12/2021  8:00 PM          64348 libgl-webgl2-full_es3.a
-a---           7/12/2021  8:00 PM          64348 libgl-webgl2-ofb-full_es3.a
-a---           7/12/2021  8:00 PM          63668 libgl-webgl2-ofb.a
-a---           7/12/2021  8:00 PM          63668 libgl-webgl2.a
-a---           7/12/2021  8:00 PM          35226 libgl.a
-a---           7/12/2021  8:00 PM          32018 libhtml5.a
-a---           7/12/2021  8:00 PM          13292 libjsmath.a
-a---           7/12/2021  8:00 PM         224852 liblsan_common_rt-mt.a
-a---           7/12/2021  8:00 PM         225078 liblsan_common_rt.a
-a---           7/12/2021  8:00 PM         477276 liblsan_rt-mt.a
-a---           7/12/2021  8:00 PM         476838 liblsan_rt.a
-a---           7/12/2021  8:00 PM          47546 libprintf_long_double-asan.a
-a---           7/12/2021  8:00 PM          47618 libprintf_long_double-mt-asan.a
-a---           7/12/2021  8:00 PM          31790 libprintf_long_double-mt.a
-a---           7/12/2021  8:00 PM          31726 libprintf_long_double.a
-a---           7/12/2021  8:00 PM        5591852 libsanitizer_common_rt-mt.a
-a---           7/12/2021  8:00 PM        5587782 libsanitizer_common_rt.a
-a---           7/12/2021  8:00 PM          34010 libsockets_proxy-mt.a
-a---           7/12/2021  8:00 PM          34128 libsockets_proxy.a
-a---           7/12/2021  8:00 PM          38122 libsockets-mt.a
-a---           7/12/2021  8:00 PM          36682 libsockets.a
-a---           7/12/2021  8:00 PM          82406 libstandalonewasm-memgrow.a
-a---           7/12/2021  8:00 PM          81850 libstandalonewasm.a
-a---           7/12/2021  8:00 PM         294562 libstb_image.a
-a---           7/12/2021  8:00 PM          29986 libubsan_minimal_rt_wasm-mt.a
-a---           7/12/2021  8:00 PM          29920 libubsan_minimal_rt_wasm.a
-a---           7/12/2021  8:00 PM         670908 libubsan_rt-mt.a
-a---           7/12/2021  8:00 PM         669972 libubsan_rt.a
-a---           7/12/2021  8:00 PM           5120 libunwind-except.a
-a---           7/12/2021  8:00 PM           5170 libunwind-mt-except.a
-a---           7/12/2021  8:00 PM            444 libunwind-mt-noexcept.a
-a---           7/12/2021  8:00 PM            444 libunwind-mt.a
-a---           7/12/2021  8:00 PM            372 libunwind-noexcept.a
-a---           7/12/2021  8:00 PM            372 libunwind.a
-a---           7/12/2021  8:00 PM         160370 libwebgpu_cpp-mt.a
-a---           7/12/2021  8:00 PM         160298 libwebgpu_cpp.a

the MINIMAL cache (emcc --clear-cache; embuilder build MINIMAL)

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---           7/12/2021  8:13 PM           1236 crt1.o
-a---           7/12/2021  8:13 PM          16604 libal.a
-a---           7/12/2021  8:13 PM          94170 libc_rt_wasm-optz.a
-a---           7/12/2021  8:13 PM          97700 libc_rt_wasm.a
-a---           7/12/2021  8:13 PM        2688406 libc.a
-a---           7/12/2021  8:13 PM        6036326 libc++-except.a
-a---           7/12/2021  8:13 PM        5815096 libc++-noexcept.a
-a---           7/12/2021  8:13 PM        6241028 libc++.a
-a---           7/12/2021  8:13 PM         937710 libc++abi-except.a
-a---           7/12/2021  8:13 PM         881876 libc++abi-noexcept.a
-a---           7/12/2021  8:13 PM         889430 libc++abi.a
-a---           7/12/2021  8:12 PM         502286 libcompiler_rt.a
-a---           7/12/2021  8:13 PM          83658 libdlmalloc-debug.a
-a---           7/12/2021  8:13 PM          80458 libdlmalloc.a
-a---           7/12/2021  8:13 PM          70378 libemmalloc-debug.a
-a---           7/12/2021  8:13 PM          80904 libemmalloc-memvalidate-verbose.a
-a---           7/12/2021  8:13 PM          70716 libemmalloc-memvalidate.a
-a---           7/12/2021  8:13 PM          80570 libemmalloc-verbose.a
-a---           7/12/2021  8:13 PM          56054 libemmalloc.a
-a---           7/12/2021  8:13 PM          35142 libgl.a
-a---           7/12/2021  8:13 PM          31962 libhtml5.a
-a---           7/12/2021  8:13 PM          35950 libsockets.a
-a---           7/12/2021  8:13 PM          81006 libstandalonewasm.a
-a---           7/12/2021  8:13 PM           5090 libunwind-except.a

I think we might try to remove few libs to shrink the nuget size.

libsanitizer_common_rt*.a might be good start.

@radekdoulik
Copy link
Member

radekdoulik commented Jul 12, 2021 •
edited
Loading

Plus all the *san*.a I think.

ls -1 *san*
libasan_js.a
libasan_rt-mt.a
libasan_rt.a
libc-asan.a
libc-mt-asan.a
libc_rt_wasm-asan-optz.a
libc_rt_wasm-asan.a
liblsan_common_rt-mt.a
liblsan_common_rt.a
liblsan_rt-mt.a
liblsan_rt.a
libprintf_long_double-asan.a
libprintf_long_double-mt-asan.a
libsanitizer_common_rt-mt.a
libsanitizer_common_rt.a
libubsan_minimal_rt_wasm-mt.a
libubsan_minimal_rt_wasm.a
libubsan_rt-mt.a
libubsan_rt.a

radekdoulik added a commit to radekdoulik/emsdk that referenced this pull request Jul 12, 2021
@radekdoulik
Delete sanitizer libs from cache to shrink size
6202925 
Delete these libs on windows, where we are over the nuget size limit.
Context: dotnet#34 (comment)

That should affect build times when using
[debug sanitizers](https://emscripten.org/docs/debugging/Sanitizers.html),
which is not a common scenario I think.
@radekdoulik radekdoulik mentioned this pull request Jul 12, 2021
Merged
@radekdoulik
Copy link
Member

radekdoulik commented Jul 12, 2021

Another option might be to try set CLANG_CXX to clang and don't pack clang++?

https://github.com/emscripten-core/emscripten/blob/main/tools/shared.py#L823

@radekdoulik
Copy link
Member

radekdoulik commented Jul 12, 2021

Related emscripten change emscripten-core/emscripten@2470230

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@radical radical radical approved these changes

@akoeplinger akoeplinger Awaiting requested review from akoeplinger

@radekdoulik radekdoulik Awaiting requested review from radekdoulik

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@lewing @radekdoulik @radical