Question: change host folder location at runtime?

[imsnif]

Hey wasmtime 👋 ,

Been a happy wasmtime user in Zellij for a short while now and I have a question as I'm considering adding some new features:
Is it possible to change the location of a preopened_dir (opened with https://docs.rs/wasmtime-wasi/latest/wasmtime_wasi/struct.WasiCtxBuilder.html#method.preopened_dir) at runtime without reloading the wasi environment (a plugin in our case)?

To clarify, this would mean that /host on the guest side starts up pointing to eg. /tmp/folder1 on the host machine and is changed to point to /tmp/folder2 at runtime. For the removal of doubt, this would 100% happen when no filesystem entities of any sort are open.

I have looked through the API docs and a little bit through issues here and could not find the answer. Leading me to think this is either not possible, not advisable or that I'm asking the wrong questions. Your input would be appreciated, thanks!

[alexcrichton]

The API of WasiCtx is pretty bare at this point so updating it to have more features I think is definitely reasonable. One option you have though is that you could replace the entire WasiCtx outright in-memory and I think that would work today. That would only work if the program has no open handles, but it sounds like you might fall into that use case?

Otherwise though adding functions to remap/reopen things I think is good to do too

[pchickey]

I didn’t expose a mechanism to manipulate the preopens once the WasiCtx is constructed because, at least for wasi-libc-based guests (which is a lot of them) the libc doesn’t expect the preopens to change once initialized. If it did have to start expecting preopens can change, it would could make every open call more expensive by needing to refresh the preopens. So rather than worry about all of those circumstances I decided to treat preopens as a constant across a stores lifetime - at one point the idea was we would have component model value imports to express that concept, but those didn’t materialize in time for 0.2.

So, if your scheme requires manipulating the preopens, I’d advise against it, since it presents a pretty major compatibility risk with guests today. It’s also not the way we expect the standard to move- the consensus here WebAssembly/wasi-filesystem#128 is that the whole preopens concept isn’t serving us well and we may replace it with a rooted filesystem in a future revision.

The wasmtime-wasi crate provides a direct mapping between the host filesystem entities underneath a preopen and the wasi filesystem entities. We chose this implementation because it was the simplest and puts the most responsibility on the host’s filesystem. So, rather than manipulate the preopens, can you implement your scheme by manipulating the host? For example, assuming your host is Linux, can you create and manipulate the filesystem you want to expose to wasi by mount https://manpages.ubuntu.com/manpages/bionic/man8/mount.8.html ? Another alternative is that you make your own wasi-filesystem implementation where you control the mapping to the host in your library code, rather than the host filesystem.

[imsnif]

One option you have though is that you could replace the entire WasiCtx outright in-memory and I think that would work today.

I just tried this out and from a cursory test, this seems to work quite well. I'm essentially doing an std::mem::replace on the whole WasiCtx instance.

@pchickey - thanks for the explanation and suggestions. Unfortunately controlling the mounts on the host machine is not possible for me (these are user machines and the host directory change represents the users giving the plugin access to a different folder on their machine). Creating my own fs implementation is a possibility, but obviously one I'd like to avoid if possible (even if it's just a proxy)... do you think the above solution is not in my best interests? I only tested it with read_dir so far and the listing changes as it should. Should I expect more trouble if I go down this route?

[pchickey]

Swapping out the WasiCtx will change the value corresponding to the preopens, so any descriptors owned by the guest won't change (those are in the ResourceTable, not WasiCtx), and will create the compatibility risk I described. You'd need to define your own wasi-filesystem implementation (as you say, as a proxy on the existing one) that allows you to swap out the values of the open Descriptor in the ResourceTable as part of your manipulation.

[bjorn3]

I didn’t expose a mechanism to manipulate the preopens once the WasiCtx is constructed because, at least for wasi-libc-based guests (which is a lot of them) the libc doesn’t expect the preopens to change once initialized. If it did have to start expecting preopens can change, it would could make every open call more expensive by needing to refresh the preopens.

In Zellij's case I think replacing the preopen fd in-place is enough. Only adding or removing preopens would need wasi-libc to be aware that the set of preopens changed. If the preopen /host has fd 5, then the replace would only change the target of fd 5 such that calling fd_open will open files in the new target of /host and fd_readdir will list files in the new target of /host. As far as wasi-libc is concerned, it would be indistinguishable from all files in /host getting deleted and a set of new files being created in /host.

[pchickey]

If the preopen /host has fd 5, then the replace would only change the target of fd 5 such that calling fd_open will open files in the new target of /host and fd_readdir will list files in the new target of /host.

The unix fd 5 is stored in the Descriptor object in the ResourceTable in the Store, not in WasiCtx.

[bjorn3]

Zellij is using wasip1, so the fd is contained in WasiP1Ctx, which currently doesn't expose any way to swap out what an fd points to.

[pchickey]

Oh, ok, I totally missed this was a WasiP1Ctx. In that case, sure, swap it out.

[imsnif]

Oh, ok, I totally missed this was a WasiP1Ctx. In that case, sure, swap it out.

My bad, I did not mention it because I wasn't sure it was relevant.

Thanks everyone for weighing in. As a tl;dr for this thread, I'm going to be std::mem::replaceing the WasiCtx on demand as a solution for this.