A ComfyUI workflow to train LoRA's for FLUX.1 Dev model.
An easy-to-use workflow that allows you to train your own LoRA's on ComfyUI. Based on Kohya (Kohya-ss backend) and Kijai (ComfyUI custom nodes) incredible works.
----------------
To train a LoRA (Low Rank Adaptation) for FLUX these are the steps you should follow before clicking on Queue:
1) Prepare learning data - that is an images set (min 10, 20-30 is fine, but for some specific LoRA's the more is better). The image quality is VERY important, use only high-res images with good quality (no blurred imgs, out of focus, etc.)
2) You don't need to create the caption .txt files, FLUX model's LoRA's can be trained on images only, but on some specific kind of training caption is needed (like for example style LoRA's)
3) Check you have set the input (training images) and the output (saved LoRA's) folders correctly. For ComfyUI windows-portable version the "training" directory must be created in the same directory with "ComfyUI_windows_portable" one.
4) Set your LoraTrigger word (optional)
5) Add at least on prompt (up to 4 prompts) for Training Validation in the "Prompts for Validation" green node, at the bottom. Prompts must be separated by "|".<br>
6) Adjust training settings (or leave default ones)
-Max training steps: 1000-4000
-Network_dim: can be 4,8,16,32,64,128 (high values mean better quality but larger lora files)
-Network_alpha: default is 1
-Learning rate: suggested 0.0004
-Blocks_to_swap: default 18
For more details on settings here are a couple of links:<br>
Now click "Queue" and wait a few hours... (Unless you have a nVidia H100 as GPU)
At the end of the trainig you will have a few different LoRA's, chose the best one and enjoy it in your next workflow!
Description
FAQ
Comments (25)
Hi, masked training possible?
No, not with this workflow.
you can put a 'Image Rembg' node between load image and tagger/save image nodes in Set up dataset group
Hi, how to train specific blocks? 7 and 20
I am not sure if it is possible... maybe you should modify the workflow for that. Check on Kijai (the author of the nodes) if it is possible: https://github.com/kijai/ComfyUI-FluxTrainer
5) Add at least on prompt (up to 4 prompts) for Training Validation in the "Prompts for Validation" green node, at the bottom. Prompts must be separated by "|".<br>
What exactly is the prompt here? What are the requirements？thanks！
you should write at least one simple prompt to test how the training is going. every 1/4 of the process it will stop training, and generate an image with the given prompt (or prompts) to show how the training is going, then it resumes the training. So if you are training a lora for a character (let's say an actress) you would write a prompt like "a professional photo portrait of a woman sitting in a crowded bar". this way all the training process will stop at each 1/4 of the total steps, and show you how the trained lora (till that moment) would output the image of the woman in the bar, to compare them. Sometimes you will find out that the lora trained at half the total process gives you best results.
@Tenofas thank you for your advice. When I train lora with your workflow, it will automatically stop. I forgot to take the screenshot of the prompt, but the error prompt is on the first Flux Train Validate. I don't know if it is related to the situation you mentioned. ?
Great workflow!
Is there any chance you can develop one for SDXL?
For some reason kohya_ss never works for me.
Unfortunately this workflow is based on Custom Nodes that use kohya_ss as backend. So, I guess it's not possible to convert this to train SDXL LoRAs... But I will check if there are other nodes to make a new different one for SDXL.
How to set Save LoRA Every n Steps? Thank you
I see it : Simple Math = a/4
I'm getting this error, any help?: "InitFluxLoRATraining
'FluxNetworkTrainer' object has no attribute 'num_train_epochs'"
Have to check... Will let you know
That usually means your input path is wrong. It should be a full path to your input folder, eg C:\ComfyUI\Training\Input\YourInputFolder\
On comfyui, in my case, this error is caused by 2 reasons, the folder paths are not recognized when they start with "../" you have to type the full path "C:\..." and I was also forced to use the original model "flux1-dev.safetensors" 22gb.
Can this be modified to work with SD3.5
InitFluxLoRATraining
expected str, bytes or os.PathLike object, not NoneType
Help~
T __ T
it depends on many variables, how many images you use for training, the settings you use... it could take a little more than one hour with 10 images and very few steps, to several hours if you use a lot of images and increase the training steps.
Tenofas If I wanna make a medium quality real character lora, how many images at least I need
zczcg I would say at least 20-25. 10 is the bare minimum, but with only 10 images, you will sacrifice variety.
Tenofas Thanks a lot, I think I need train lora online..besides the civitai, any suggestion?
Does anyone know the best settings for flux? optimizer steps? scheduler?
enable full bf16 training.
enable fp8 training for U-Net.
unet_weight_dtype: torch.float8_e4m3fn
INFO set U-Net weight dtype to torch.float8_e4m3fn train_network.py:638
running training
I'm seeing the fp8 downscale comment when I'm training LoRA base on full fb16 Flux.1.dev somehow.