WebAssembly build

[birkskyum]

Hi, I'm interested in a WebAssembly build of PROJ.

The intention is to mitigate differences across PROJ and proj4js, by simply running PROJ in the browser.

I'm curious if there are known blockers to this.

I've previously managed to make large C++ codebases run online as WebAssembly.

It's sometimes a bit slow still, but there're are all the time small performance gains coming from improved browser support for wasm, and improvements in emscripten to leverage it.

[jjimenezshaw]

Hi @birkskyum
I want to do it as well... but I do not know anything about WebAssembly. It would be great to have it (and include it in https://spatialreference.org ;)

There is no "hard" blocker. There is an implementation of GDAL in WebAssembly: https://github.com/bugra9/gdal3.js
So it is doable.

This gdal implementation is disabling curl, so it cannot use grid files.
About the curl part, somebody got this emscripten-core/emscripten#3270 (comment) , but we can leave that for a later step.

In the release of PROJ 9.6.0 (planned for March) there is an option to include the proj.db file embedded in the library. That would make it even easier. #4265

If you want to do it, please contact me.

[dg0yt]

I don't know if it is usable at runtime, but I can say that the default build of proj:wasm32-emscripten succeeds in vcpkg. This is the list of dependencies (host: x64-linux):

$ ./vcpkg depend-info proj:wasm32-emscripten
vcpkg-cmake: 
vcpkg-cmake-config: 
vcpkg-cmake-get-vars: vcpkg-cmake
openssl:wasm32-emscripten: vcpkg-cmake, vcpkg-cmake-config, vcpkg-cmake-get-vars
zlib:wasm32-emscripten: vcpkg-cmake
libjpeg-turbo:wasm32-emscripten: vcpkg-cmake, vcpkg-cmake-config
liblzma:wasm32-emscripten: vcpkg-cmake, vcpkg-cmake-config
curl[openssl, ssl, non-http]:wasm32-emscripten: openssl:wasm32-emscripten, vcpkg-cmake, vcpkg-cmake-config, zlib:wasm32-emscripten
nlohmann-json:wasm32-emscripten: vcpkg-cmake, vcpkg-cmake-config
sqlite3[json1]:wasm32-emscripten: vcpkg-cmake, vcpkg-cmake-config
sqlite3[json1, tool]: vcpkg-cmake, vcpkg-cmake-config
tiff[lzma, jpeg, zip]:wasm32-emscripten: libjpeg-turbo:wasm32-emscripten, liblzma:wasm32-emscripten, vcpkg-cmake, vcpkg-cmake-config, zlib:wasm32-emscripten
proj[tiff, net]:wasm32-emscripten: curl:wasm32-emscripten, nlohmann-json:wasm32-emscripten, sqlite3:wasm32-emscripten, sqlite3, tiff:wasm32-emscripten, vcpkg-cmake, vcpkg-cmake-config

[birkskyum]

thanks @dg0yt , that's great news! I think we can get this working, but as you point out it's hard to tell what runtime performance it'll have. It's probably easiest to just try it out and see what it brings. I'm still familiarizing myself a bit with the normal build pipeline ( I do have a build ), running tests ( few tests breaks still ) , running demos so the goal is clear - and then make an attempt at compiling it.

[willcohen]

I've had a version of this knocking around my computer for a year or two, and have gotten it quite close to release, but it's not 100% ready yet. I cleaned it up for initial proof-of-concept. It currently builds using emscripten, and works to allow for using a native version of PROJ on the JVM while falling back to WebAssembly using graalwasm when the native version isn't available for a platform. The framework for getting this to work on node is close, but there's still a few webpack issues I need to work through.

I'll probably publish an initial build of this as a .jar to Maven Central for initial usage on the JVM, and aspire to get a first version posted to node soon. Any assistance getting the JS issues resolved would be much appreciated!

https://github.com/willcohen/clj-proj

[willcohen]

More to the point -- if any want to help me get what I've got as step 3 working, it would be immensely helpful.

1. I'm not great at JS or webpack -- proj-emscripten is the basic package that puts an entry point into what emscripten spits out, to deal with loading the database and wasm files for graal, node, web. It seems like it mostly works but might need some love on node and web. It might be worth looking into rollup or some other tool to handle this more gracefully, but I am not an expert there either.
2. proj.cljc is the clojure/clojurescript api that does native/graal/js interfacing to PROJ. A major benefit here is that by staying lined up in one file with native FFI, it'll be much easier to ensure that this WASM library is 'drop-in compatible' with an actual native build and works on all end users of WASM, which I suspect will pay dividends when it comes to debugging. If it works on native but not WASM, we know it's a webassembly or emscripten problem. If it doesn't work on native, we know it's an interop problem related to interfacing with the PROJ blob and can diagnose there first.

[willcohen]

Your notes above solved the problem — the workaround for now is that I can build the git version, embed the database, and export it as a single file where the WASM is embedded in the js, thus no longer needing webpack.

I can successfully load PROJ in js and call its C api, returning accurate results. I’ll get this cleaned up and pushed to my repo, with an npm package that at least provides the initial functions to create transforms from CRS and transform points accordingly.

It would still be nice to eventually get this resolved so the WASM can be its own file without base64 expansion, but that is no longer a blocker. Thank you all!

[willcohen]

Additionally, an added benefit of piggybacking off of the native and graal bindings is that I can reuse all the logic for handling the PROJ contexts and pointer array expansions and object lifecycles, making it pretty seamless to abstract away all the C parts of PROJ behind the mechanisms of the JVM and JS VM so it still feels idiomatic here.

[willcohen]

NPM package is now available for testing: https://www.npmjs.com/package/proj-wasm

[willcohen]

I'm pleased to report that proj-wasm, at a long overdue 0.1.0-alpha2, now works in browser. A WebAssembly build of PROJ 9.6.2 is used on the client side in the project's demo page here: https://willcohen.github.io/clj-proj/

[kbevers]

This is proper cool, Will! Great effort, this will be useful to a lot of people I am sure. Do you want to advertise you work on https://proj.org/en/stable/development/bindings.html ?

[birkskyum]

@jjimenezshaw , would it be possible to use https://spatialreference.org/ as a testbed for maturing the WASM build that Will has made?

[hobu]

My ideal world would have WASM builds of PROJ released in sync with normal PROJ releases and distributed via https://cdn.proj.org

[palmerj]

Really cool @willcohen. What grids are included?