27/05 : Updated with SVD INT4 Quant for Flux https://github.com/mit-han-lab/ComfyUI-nunchaku
Special thanks @theunlikely for the work done on quantification (took 6 hours on a H100 GPU), and thanks to JIB (J1B Creator Profile | Civitai) for his WF and clarifications on using this type of model:
For using this model you need to install the nunchaku project following the instructions there. and the nunchaku ComfyUI custom nodes to get it to work. Download and unzip the archive from Civitai to: \comfyui.git\app\models\diffusion_models\svdq-int4-flux-dev-de-distill - A Nunchaku workflow you can use : https://civarchive.com/models/617562
This is a repost from hugging faces, I did not play any role at all in the creation of this epic, groundbreaking model from nyanko7. I just had to post it asap because it is F A N T A S T I C
Flux.dev as we have known until now was a distilled model, meaning it was trained by flux.pro as its teacher. These new models change everything ! This is the first experimental, effectively de-distilled version of Flux.dev, meaning it is much closer to what flux.pro is capable of. And it's just the start !
(NB : This is not stating it was trained with flux.pro - I don't know the exact method)
/!\ READ ALL IMPORTANT STUFF BELOW (AFTER THE EXAMPLES) IN THIS DESCRIPTION OR IT WILL BE A FAIL FOR YOU /!\
EXAMPLES MADE MYSELF
Distilled CFG 8 VS Real CFG 8 - Fixed seed, no prompt change except for the text
Each picture is the first shot on each model : no cherrypicking / cheating
Ear gauge LoRa : "Cute blonde raver young girl smiling, facing viewer, with emo makeup and puffy emo hairstyle, green shiny colored hairstyle. 3argauge, both ears have Ear gauge plug with very large circular hole in her lobe. She is wearing a black hoodie with text "DEDISTILLED MAKES MY LORAS WORK" in golden letters
Cum on face Lora (with dedistilled it actually works anywhere) : "COF, Young woman with white sticky cum on face,white sticky semen on face,white sticky sperm on face,face covered with white sticky cum, face covered with white sticky semen, face covered with white sticky sperm. She is wearing a black hoodie with text letters "DISTILLED" in silver font on it"
No LoRa : "Dutch shot view of a black car speeding away from a massive explosion in the streets of a futuristic city with large buildings, towards the viewer, escaping the blast of the explosion. The motion lines around the car give the impression of speed. The front plate of the car has text "DISTILLED" on it."
No Lora : "A small kitten playing with a ball of yarn is seen through an old wooden window of a rustic house. The scene is cozy, with weathered wooden furniture inside the house and soft afternoon light streaming through the window, casting gentle shadows. Outside, in the distance, a photographer is approaching, camera in hand, ready to capture the playful moment of the kitten. The photographer is wearing a brown jacket and is framed by the soft glow of the golden hour light, adding a sense of warmth and tranquility to the scene. The overall atmosphere is peaceful, with a touch of nostalgia from the vintage setting."
SUMMARY OF THE IMPORTANT STUFF (aka current knowledge about it)
Disclaimer : These models are very new. So, just gathering here what is known about them for now. Please share your experiences in the comment section so we can update this together.
PARAMETERS
You can now forget about Distilled CFG and use real CFG (I've tried up to like 14)
NEVER, ever use it with CFG = 1 - It will automatically be a complete disaster, and most of the time the reason why you don't get results
You SHOULD use at least 40 - 60 steps, depending on the CFG you use.
It will be much longer but SO worth it
Unfortunately the current hyperdev 8-steps Lora doesn't seem to work with it to reduce steps
Dedistilled allows NEGATIVE PROMPTS
BENEFITS
Prompt adherence will be EXCEPTIONAL, even with Loras.
Faces Loras will work better, details will be better, text will be WAY better...
Everything from the prompt will be better basically
/!\ IF YOU DON'T SEE ANY IMPROVEMENT FROM DISTILLED MODELS : VERIFY YOU ARE NOT ACTUALLY USING REAL CFG = 1 WITHOUT KNOWING. NOT FLUX GUIDANCE. IT IS TRICKY /!\
AS ALL WORKFLOWS WERE TAILORED FOR DISTILLED, CONSIDER TRYING WITH FORGE IF IT'S NOT WORKING FOR YOU IN COMFY ?
GUIDELINES FOR USE IN FORGE UI
Works in Forge without any change. Will be loaded as if it was Schnell model, disabling Distilled CFG (cool).
EDIT : uploaded all the new Quants, you should find at least one working for you
If you are new to Forge, make sure you use similar settings :Flux workflow
DeDistilled as the checkpoint
In VAE / Text encoder files, provide the vae (ae.sft / ae.safetensors) + clip_l (or a modified clip) + t5xxl (whatever the quant you are using, fp16, fp8, etc).
Otherwise it won't work as they are NOT bundled in the model files herePlease also set Diffusion in Low Bits = Automatic (FP16 Lora), otherwise you might be in trouble with LoRas. This applies to any checkpoint in Forge.
I then recommend these settings for Dedistilled :
GUIDELINES FOR USE IN COMFY UI
Works in ComfyUI using a pretty standard workflow, the one cited below uses the GGUF Loader, Dual CLIP Loader for t5xxl and clip_l prompts, and KSampler Efficient
Recommended settings for Comfy :
Dual CLIP Loader guidance: 3.5 KSampler cfg: 2 to 10 Steps: 50 to 60 Negative Prompt: Can be left blank or can be provided if needed, will affect the image if provided
Sampler: DDIM or euler Scheduler: beta or exponential
WORFKLOW HERE : https://gist.github.com/dasilva333/87bdd5b5b8ebba5515a9919ede0e3c05
Found this one also on reddit (drag & drop it into Comfy) : https://files.catbox.moe/y99yl7.png
TRAINING AND LORAS
I have just trained myself my first LoRa using De-distilled and guidance = 6, after failing hard with Distilled and guidance = 1. Results are awesome, it basically saved my LoRa. Works great with both De-distilled and distilled (but better with De-distilled).
I will be using it to train from now on.
The first checkpoint fined tune with dedistilled has been posted on civitai here : https://civarchive.com/models/690991/sapianf-nude-men-and-women-for-flux-now-de-distilled
Awaiting answers from the author to update here
Sources
Dedistilled model FP16 : https://huggingface.co/nyanko7/flux-dev-de-distill
Dedistilled model FP8 : https://huggingface.co/MinusZoneAI/flux-dev-de-distill-fp8/tree/main
Dedistilled FP8 GGUG & Q4_ K_M GGUG : https://huggingface.co/TheYuriLover/flux-dev-de-distill-GGUF/tree/main