mmap safetensor reads?

[richardm1]

Regarding this issue/conversation: Is the "Model load using streams" checkbox used to enable/disable memory mapped reads? I'm on 586ef9a3 and this checkbox appears to do nothing at all.

As a sanity check I tried ReForge and confirmed toggling memory-mapped reads restores the usual performance (20 second vs 45 second SDXL checkpoint reads over the LAN). SD.Next is reading these in 45 seconds no matter what.

In sd_models_utils.py I've uncommented the four instances of shared.log.debug() but I'm not seeing any of their verbiage in debug log. I tried really hard to self-diagnose (as a non-programmer) and not bother anyone here but I've run out of ideas..

[vladmandic]

original feature was implemented pre-diffusers, only for original backend and it was never ported to diffusers backend which is now default.
i just ported it to diffusers backend, its now available in dev branch.

[richardm1]

Thanks for this. Does "load directly to GPU" have relevance to this feature? I'm on dev@6358276 and trying to confirm non-mmap is working.

[vladmandic]

load overrides the same function, so they should not be used at the same time.

[richardm1]

Thanks. Here to confirm this is working. Full speed reads from my storage array across the LAN:

New times:
12:56:55 INFO Load model: time=total=26.54 load=19.07

Old times:
11:36:36 INFO Load model: time=total=55.27 load=46.52