PixelWave FLUX.1-schnell 04 - Apache 2.0!
Safetensor Files: 💾BF16 💾FP8 💾bnb FP4
GGUF Files: 💾Q8_0 🤗Q6_K 💾Q4_K_M
Model also available at: RunDiffusion and Runware.ai
PixelWave FLUX.1 schnell version 04 is an aesthetic fine tune of FLUX.1-schnell. The training images were hand picked to ensure the model has a bias to eye catching images, with beautiful colors, textures and lighting.
Trained on the original schnell model, so Apache 2.0 license!
No special requirements to run. Supports FLUX LoRAs
Euler Normal, 8 steps.
You can use more steps to improve finer details, but the output doesn't change much after 8 steps.
Shout out to RunDiffusion
Huge thank you to RunDiffusion (co-creators of Juggernaut) for sponsoring the compute that made training this model possible! Figuring out how to train schnell without de-distilling the model required a lot of experimenting, and being able to utilize RunDiffusion's cloud compute made it a lot easier.
For those needing API access for this model, we're partnering with Runware.ai
I have made the FLUX.1-dev 04 version exclusive to RunDiffusion and Runware for the time being. When I release version 05 in future, I plan to release the dev 04 open weights.
Grateful for their support in getting this model out there, please check them out!
Training
Training was done with kohya_ss/sd-scripts. You can find my fork of Kohya here , which also contains changes to the sd-scripts submodule, make sure you clone both.
Use the fine tuning tab. I found the best results with the pagedlion8bit optimizer which also could run on my 4090 GPU 24GB. I found other optimizers struggle to learn anything.
I have frozen the time_in, vector_in and mod/modulation parameters. This stops the 'de-distillation'.
I avoid training single blocks over 15. You can set which blocks to train in the FLUX section.
LR 5e-6 trains fast, but you have to stop after a few thousand steps as it starts to corrupt blocks and slow down learning.
You can then block merge with an earlier checkpoint, replacing the corrupt blocks, and then continue training further.
Signs of corrupt blocks: paper texture over most images, loss of background details.
Contact
For business or commercial inquiries please reach out to us at [email protected]. Licensing flux fine tunes. Customer training projects. Commercial AI development. The team can do it all!
PixelWave Flux.1-dev 03 fine tuned!
Safetensor Files: 💾BF16 💾FP8 💾NF4
GGUF Files: 💾Q8_0 🤗Q6_K 💾Q4_K_M
The 'diffusers' files are actually the Q8_0 and Q4_K_M GGUF versions. GGUF files also available on huggingface.
I fine tuned version 03 from base FLUX.1-dev for over 5 weeks on my 4090. It is able to do different art styles, photography, and anime. Trick I discovered to help with LoRAs.
I used dpmpp 2m sgm uniform 30 steps for the showcase images. If you want a neater/cleaner output, try increasing the guidance. Also mentioning a style can help, so the model doesn't have to guess.
I also recommend try adding the upscale latent by node, and scale the latent by 1.5, e.g. generating an image that is 1536x1536 instead of 1024x1024.
PixelWave Flux.1-schnell 03
GGUF Files: go to huggingface
I used dpmpp 2m sgm uniform 8 steps for the showcase images.
You can start with 4 steps, but there are less errors with anatomy if you run with more steps.
PixelWave Flux.1-dev 02
GGUF Files: 💾Q8_0 🤗Q6_K 💾Q4_K_M
Version 02 has greatly improved black and dark images, and more reliable outputs with fewer issues with hands.
I recommend using dpmpp_2s_ancestral, beta, 14 steps. Or euler, simple, 20 steps.
PixelWave 11 SDXL. A general purpose fine tuned model. Great for art and photo styles.
I use 20 steps, DPM++ SDE, CFG 4 to 6 or 40 steps, 2M SDE Karras
Accelerated Version - 5+ Steps, DPM++ SDE Karras, 2.5 CFG
PAG Recommended⚡Recommend 1.5 Scale, with CFG 3. Link to workflow
⭐Link to prompting guide.⭐ You don't need to use 'quality' terms such as 4K, 8K, masterpiece, high def, high quality, etc. Unless you want it, I recommend not using words such as 'vibrant, intense, bright, high contrast, neon, dramatic' for photographic styles if you a wanting a more natural look. This can cause images to look 'overcooked', but it's just the CLIP following your prompt. 🙂 If you do want vibrant, neon photos PixelWave will provide!
The focus for version 10 was to train the CLIP models, which improves the reliability, ensures you can produce a wide variety of styles, and better at following prompts.
Thanks to my friends who helped test: masslevel, blink, socalguitarist, klinter, wizard whitebeard.
Guide: Upscaling Prompts with LM Studio and Mikey Nodes
Guide: Add more details to your image using the skip step method
No need for the refiner model.
This model is not a mix of other models.
I also created Mikey Nodes which contains a lot of useful nodes. You can install it through comfy manager.
Description
I used ostris' LoRA training script to train 33 LoRA's and added them to the base model with different weights that produced a nice result. Then tweaked the block weights to fix issues with hands.
FAQ
Comments (18)
Congrats on the flux release, might test it later when I get home
Nice work! Thank you for making this.
Great!
Can you make Q8 ( as Q8 has closer quality to fp16 ) and Q4k_m ( the best version of 4 bit , much better than nf4 and better than "old" Q4 ) version .
Thank you
The author has already posted. I am now testing Q8 on Forge and adding photos to the gallery.
Nice.
Q4k_m also would be nice.
Has quality of "old" Q5.1
@mirek190 Q8 works great on the RTX 3060 12 GB
@mirek190 Q8_0 (fp8 diffusers) and Q4_K_M (nf4 diffusers) are the versions I already posted. The site doesn't support GGUF files, so I had to zip them and upload them. You can find the link to my huggingface repo in the description which has a lot more different quant types available.
@humblemikey thanks
I get this error while trying to run on Forge. How can I run it? Thanks in advance!
File "C:\FORGE\webui\backend\loader.py", line 59, in load_huggingface_component
assert isinstance(state_dict, dict) and len(state_dict) > 16, 'You do not have CLIP state dict!'
AssertionError: You do not have CLIP state dict!
You do not have CLIP state dict!
You need the clip_l and text encoders from here: https://huggingface.co/comfyanonymous/flux_text_encoders/tree/main
@Legend_of_Ray Thank you , after I download, where do I place them?
@AnaRose You can place them in the models/text_encoder You might also need the vae file as well its here: https://huggingface.co/black-forest-labs/FLUX.1-dev/tree/main/vae
@AnaRose I have a video here were I use a similar model with forge if that helps as well. https://www.youtube.com/watch?v=JlfS5EGbcmQ
@Legend_of_Ray Thank you so much! I'll try to follow the tutorial.
@Legend_of_Ray I followed the tutorial, but now after it seems to be working the output is a blue screen but I see the generation preview. How do I fix that?
@AnaRose one thing you might try is to download an example image, either from this checkpoint or another, then drag and drop that image into comfy, if it has the workflow in it then it should give you a complete setup. you might need to go through the clip models and the vae and the checkpoint and assign those to the ones you have installed. try running it and see if it works, also the workflow might give you some ideas. obviously you might need to install missing nodes, but probably not many. then see if you still get errors or blue outputs. another very important thing is to make sure your comfy is fully updated. mine won't auto update, so i have to go to the github page and clone it, and manually change out the files and folders that have been recently updated at the github.
@fablegenius I truly appreciate your help. However, I am not using Comfy UI, I'm using Forge.
@AnaRose put the ae.safetensors in the text encoder folder.