Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardize build for packages while building for webassembly against emscripten #392

Open
anutosh491 opened this issue Dec 13, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@anutosh491
Copy link
Collaborator

anutosh491 commented Dec 13, 2024

Description of new feature

This is mainly for supporting xeus-cpp-lite.

Xeus-cpp-lite needs the following

  1. cppinterop (which is dependent on llvm)
  2. xeus, xeus-lite and other packages mentioned here https://github.com/compiler-research/CppInterOp/blob/main/environment-wasm.yml

Now what is being done in our CI is

  1. We build llvm from source with almost the exact configs as present on emscripten-forge
    emcmake cmake -DCMAKE_BUILD_TYPE=Release \
    -DLLVM_HOST_TRIPLE=wasm32-unknown-emscripten \
    -DLLVM_ENABLE_ASSERTIONS=ON \
    -DLLVM_TARGETS_TO_BUILD="${{ matrix.llvm_targets_to_build }}" \
    -DLLVM_ENABLE_LIBEDIT=OFF \
    -DLLVM_ENABLE_PROJECTS="${{ matrix.llvm_enable_projects }}" \
    -DLLVM_ENABLE_ZSTD=OFF \
    -DLLVM_ENABLE_LIBXML2=OFF \
    -DCLANG_ENABLE_STATIC_ANALYZER=OFF \
    -DCLANG_ENABLE_ARCMT=OFF \
    -DCLANG_ENABLE_BOOTSTRAP=OFF \
    -DCMAKE_CXX_FLAGS="-Dwait4=__syscall_wait4" \
    ../llvm
    emmake make clang -j ${{ env.ncpus }}
    emmake make clang-repl -j ${{ env.ncpus }}
    emmake make lld -j ${{ env.ncpus }}

and we use this llvm to build cppinterop against emscripten.

  1. For the other packages we just simply fetch it from emscripten-forge.

This is actually really error prone. The llvm build we are doing is against emsdk 3.1.45 and luckily emscripten-forge master is also using emsdk 3.1.45 ( a migration to latest 3.1.73 is planned soon).

This system makes our build very vulnerable. Building for webassmebly using emscripten is very sensitive even if a single package is built using a different emsdk version and can lead to lot of errors like

  1. symbol undefined
  2. symbol signature mismatch
  3. symbol not exported

Also demands more effort in cases like

  1. What is emscripten-forge master switches its emsdk version ?
  2. Or what if we want to use another emsdk version ?

Under the cases we won't even realize but our deploy pages build for xeus-cpp-lite would be broken.

Hence to really have a concrete build we have two ways to go

  1. Build every dependency from source against an emsdk of choice in the CI for consistency
  2. Fetch any dependency possible from emscripten-forge's main
    i.e. except cppinterop that we should build from source in our CI to check compatability with latest changes , every other thing like llvm, xeus-lite etc should be fetched from emscripten-forge. The only thing to consider is that this cppinterop that we build should be against the emsdk being used by emscripten-forge's main and we would be good.
@anutosh491 anutosh491 added the enhancement New feature or request label Dec 13, 2024
@anutosh491
Copy link
Collaborator Author

I am highly in favour of the 2nd option because

  1. Xeus-cpp's repo anyways fetches cppinterop from emscripten-forge ... which means we anyways need to stick to the emsdk version on emscripten-forge and it doesn't make sense to use different builds for xeus-cpp-lite (1 in the xeus-cpp repo and 1 in the cppinterop repo)
  2. A migration to latest emsdk is planned and xeus-cpp-lite looks really promising with 3.1.73 (3.1.73 also allows us to get a static build for cppinterop)
  3. emscripten-forge is very flexible. Whatever build config we want to try out with respect to any package in our ci can be done there. We are the main users of these packages when it comes to webassembly and we should be able to use any config we are interested in there itself.

@mcbarton
Copy link
Collaborator

@anutosh491 I think I can change the ci so it is using the emsdk that emscripten forge uses rather than use picking a version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants