Add Support for Flex.1 Alpha #1891
Comments
|
This was also implemented into diffusion-pipe through this commit: tdrussell/diffusion-pipe@90ab3f9 |
|
I've created a minimal working example here: https://github.com/stepfunction83/sd-scripts/tree/sd3 I was able to quickly train a 1000 step finetune of flex and was able to test it in Comfy to validate that the training does take and the guidance module is not destroyed in the process. Additionally, the sampling was corrected as well and now works as expected. Just add the |
|
i use the GUI and it says bypass_flux_guidance unknown |
|
flux_train.py: error: unrecognized arguments: -bypass_flux_guidance |
|
Looks like you're missing a "-". It should be |
|
If it still doesn't work, in your kohya_ss folder, delete the sd-scripts folder and then navigate to the kohya_ss in a terminal and run this:
It will replace the sd-scripts folder with the version with the update. |
[--cpu_offload_checkpointing] |
|
Can you provide the full error please? |
19:21:11-815746 INFO Executing command: |
|
arning: LD_LIBRARY_PATH environment variable is not set. If you use WSL2 you may want to: export LD_LIBRARY_PATH=/usr/lib/wsl/lib/ 2025-01-22 19:19:53.640904: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 19:19:59-238859 INFO Submodule initialized and updated.
To create a public link, set |
|
Let me try it with a fresh install... |
|
Ah! Figured it out. Kohya_ss gets the latest version of the official sd-scripts repo when it first starts up. Try deleting the sd-scripts folder and cloning my fork while the server is running. (You can probably tell I don't have much experience with this...) |
|
nor do i ..good job so far thank you |
|
|
|
lets see how this puppy is |
|
This fix only works for LoRA training - flux_network_train is used for LoRA training. I'll try to find the proper way to intercept in flux_train. (edit: not implying that I have tested LoRA training, I mean theoretically) Also, if anyone is loading using the diffusers model in /transformers this will fail - right now, the code in flux_utils.py looks for diffusers model in 00001-00003, but Flex uses 00001-00002. You can just change every instance of 00002 to 00003 for a quick fix. |
Thanks for pointing that out. I've expanded it to flux_train.py as well in the PR. Interestingly, it still seemed to train well even though it wasn't bypassing properly. Perhaps the guidance module weights were just not drifting enough to cause major issues. Maybe that was the cause of some of the quality degradation. That would also explain why my logging tests were not giving any output... That one completely stumped me. Once this training run I'm currently doing reaches its next checkpoint, I'll cut it off and start a new run to compare the difference. |
|
Apparently this is non-trivial. I'm not entirely sure how the code translates to the Flux() model class. It'll take longer than I thought to figure this out. |
|
do you have the one from last night? t least that worked |
|
xamples / サンプル数: 2380 |
|
Comment out Realistically, you can just use the raw sd-scripts to train. It was effectively training with no changes previously. |
Creating an Issue from the related discussion: #1882
Ostris has created a new Flux model that is both distilled to a lower parameter count and has a custom trained guidance module which allows the model to be much more easily and flexibly finetuned.
https://huggingface.co/ostris/Flex.1-alpha
From my experience so far creating a few LoRAs, the hype is 100% real and this is trainable in a way that base Flux Dev is simply not.
To train this, the newly created guidance module needs to be bypassed during training and then restored afterwards. This is currently implemented in ai-toolkit and would need to be implemented in sd-scripts to be able to support training of the model.
This seems to be the key snippet of code to bypass the guidance block in ai-toolkit:
bypass_flux_guidanceis applied before noise prediction, whilerestore_flux_guidanceis applied after noise prediction and also before sampling.The text was updated successfully, but these errors were encountered: