What can I do to optimize/tweak tortoise to enable use with only 4GB of VRAM?

[rancortooth]

My Setup:
Windows 11, running in WSL
NVIDIA GeForce RTX 4070, 4GB VRAM
NVIDIA-SMI 535.103, Driver Version: 537.13, CUDA Version: 12.2

Trying to run do_tts.py with any voice other than "random" results in RuntimeError: cuFFT error: CUFFT_INTERNAL_ERROR

python tortoise/do_tts.py --text "I'm going to speak this" --voice william --preset ultra_fast

The same code runs better in google colab (still crashes often, but better) with stronger GPU hardware.

[rancortooth]

I dramatically increased the ram and cpus avaialble to WSL but it still fails. I re-installed everything directly on windows and it works! Not sure what's wrong with WSL. The docker version provided in this repo also failed in exactly the same way as WSL, so perhaps there's a hypervisor related issue or something with accessing GPU hardware? Not sure.