[AUDIO_WORKLET] tests in the CI don't appear to be run (in test_browser)

[cwoffenden]

EDIT: short version, since I think the error is misleading: they're compiling but they're instantly exiting without running.

Testing with the latest Emscripten locally (3.1.74) and seeing the same in the result from CI. Open any of the CI runs and look at the browser tests, e.g.:

https://app.circleci.com/pipelines/github/emscripten-core/emscripten/38495/workflows/353c8dd2-db7a-4691-a83b-c1b8088f8959/jobs/859340

Search for test_audio_worklet and the parameterised tests all have the same 404 error with missing files:

  test_audio_worklet (test_browser.browser) ... cache:INFO: generating system asset: symbol_lists/c9305f6ae33005a7b005afb88a5689b698a1f55c.json... (this will be cached in "/root/cache/symbol_lists/c9305f6ae33005a7b005afb88a5689b698a1f55c.json" for subsequent builds)
cache:INFO:  - ok
127.0.0.1 - - [11/Dec/2024 12:08:43] code 404, message File not found: /root/project/test.aw.js

I noticed this locally, and thought it was just the way my system was set up, then given that I'd seen there was no way the CI was passing tests that should be failing, I dug deeper. The test runner doesn't find test.aw.js and if that is copied (or a link created) then it next fails on test.js.

The tests are all passing though, I'm guessing because the main page opens and exits (interestingly it's only the Audio Worklets missing files, so 14 404s in test_browser's log).

[cwoffenden]

It seems to stem from the change from btest() to btest_exit(). At a guess, I'd say the server was being shut down before the file was served (if I stop the test harness from existing, via various means, there's no error). The error is therefore not the problem, it's just a side effect of the test instantly exiting.

[cwoffenden]

@sbc100 As per the discussion here:

#23508 (comment)

I'd like to get these tests running (at least not silently failing). I've put some fixes in already (e.g. STRICT), with the remaining fixes in #23508.

My question is, if I fix these they'll break the CI until other fixes are merged. What's the best way to do this? Would you like a separate PR that breaks things, or do they want rolling into #23508?

The more I delve into it the more issues I'm finding, the latest being audio params (WebAudioParamDescriptor) don't appear to be completely wired up* (and no tests exist). I can fix the code to correctly read structs based on the correct offsets, but without any tests it's a shot in the dark.

*edit here: it is in the AudioWorkletProcessor constructor but even with the API specs in front of me it's a bit of an odd design.

[sbc100]

If you fix the tests so they correctly start failing you can temporarily mark them as @disabled(<bug_url>) and then re-enable them when the bug is fixed

[cwoffenden]

And it is what I wrote a few comments back, it's the test being shut down with btest_exit() since the work is done on the worklet not the main thread. I'll look at how the regular worklet tests are done (and hopefully find they're not broken 😁).

[sbc100]

I would love to continue to use btest_exit where possible. I though I even had PR up to convert the remaining worklet testa to btest_exit but I can't find it now.

My hope is that all browser tests would use btest_exit one day.

[sbc100]

Wait, I thought it was impossible to run any audio tests during CI because they require user-interaction to start the audio? Don't they all need to interactive?

[cwoffenden]

It'd save me some time if you'd already had a sample of the main thread waiting on a worklet finishing.

As for any audio tests running, good point. Chrome at least will start the AW code, which is enough to catch most of the failures. I'll look at it.

[sbc100]

Getting btest_exit to behave nicely with AUDIO_WORKLETS (and WASM_WORKERS in general) can be tricky because IIUC you cannot called exit from a wasm worker (like many JS function it is proxied back to the main thread, which is not possible in a wasm worker, so it just one of many APIs you cannot call). So you need to use wasm worker style run-on-thread to run the exit itself back on the main thread.

Oh I found the PR I was working on: #22776

[cwoffenden]

Wait, I thought it was impossible to run any audio tests during CI

I did a quick test, I built this with WEBAUDIO_DEBUG and a bunch of other options:

https://wip.numfum.com/cw/2025-01-30/index.html

It won't play audio but if you look at the console it will create the worklet and perform most of the steps. It's enough to know in CI that initialisation works (esp. struct offsets and the message passing). I'll take a detour to look at what could be done.