Compared CivitAI default FLUX LoRA vs my best FLUX and more developments

Compared CivitAI default FLUX LoRA vs my best FLUX, tested different regularization images trainings and T5 Attention Mask - next is different LoRA ranks impact on quality.

I am about to complete my Kohya SS GUI FLUX LoRA training. Next is hopefully full Fine-Tuning / DreamBooth

Only left thing is testing impact of different LoRA Ranks on quality

I find that T5 Attention Mask is improving quality slightly

Further reducing the LR didn't help

16 GB fast config has slightly worse quality than 10 GB slower config

CivitAI default config is lower quality as expected - bigger batch size and 512px

Regularization / classification images usage not helping. Tested different Prior loss weight as well

Windows training is still significantly slower than Linux. e.g. RTX 3090 is 5.5 second / it on Linux and 7.7 second / it on Windows. I am using exactly same libraries. Compared pip freeze. Trying to find the reason

Entire R&D with full history and grids and configs shared here : https://www.patreon.com/posts/110293257

Newest comparison full grids are here : Reg_T5_Attention_Mask_CivitAI_50_Epoch.jpg, Reg_T5_Attention_Mask_CivitAI_100_Epoch.jpg, Reg_T5_Attention_Mask_CivitAI_150_Epoch.jpg, T5_Attention_Mask_v1.jpg, Civit_Reg_15GB_150_Epoch_Compare.jpg

Full grids and more info : https://www.reddit.com/r/StableDiffusion/comments/1ezivfn/compared_civitai_default_flux_lora_vs_my_best/