Browser realtime rendering using WebGPU

[aalmada]

It would be interesting if you could support realtime rendering in browser using WebGPU.
https://developer.chrome.com/blog/webgpu-io2023/

[JamesPerlman]

Hey there @aalmada ! WebGPU is indeed an exciting development in modern technology. Currently the NeRF algorithm in TurboNeRF is rather slow and a poor fit for realtime rendering, but as new NeRF research is published I'll be upgrading it to allow for faster presentation.

Currently, the idea behind TurboNeRF is to get as realistic-looking renders as possible. However I've definitely entertained the idea of baking these into web-viewable formats. There are many approaches. MERF looks like a fun one to implement, and I'll likely be tackling that in the next refactor of the core training/rendering system.

The workflow will probably go as follows: 1. train your NeRF & bake it using TurboNeRF, 2. There will have to be a separate repository for a viewer component to load and visualize these in realtime in the web (using WebGPU).

I'm open to all ideas surrounding the best way to implement this. Thanks for sharing!