Wanted: Web Assembly distribution

[MRYingLEE]

So that tokenizers can be used in web environment, such as Pyodide/Jupyterlite.

Thanks,

[Narsil]

You can build it already. We're not going to distribute it though, because it requires a change of regexp engine, and although both claim to be ISO, we know itś unlikely to be 100% really.

https://github.com/huggingface/tokenizers/tree/main/tokenizers/examples/unstable_wasm