27/05 : Updated with SVD INT4 Quant for Flux https://github.com/mit-han-lab/ComfyUI-nunchaku
Special thanks @theunlikely for the work done on quantification (took 6 hours on a H100 GPU), and thanks to JIB (J1B Creator Profile | Civitai) for his WF and clarifications on using this type of model:
For using this model you need to install the nunchaku project following the instructions there. and the nunchaku ComfyUI custom nodes to get it to work. Download and unzip the archive from Civitai to: \comfyui.git\app\models\diffusion_models\svdq-int4-flux-dev-de-distill - A Nunchaku workflow you can use : https://civarchive.com/models/617562
This is a repost from hugging faces, I did not play any role at all in the creation of this epic, groundbreaking model from nyanko7. I just had to post it asap because it is F A N T A S T I C
Flux.dev as we have known until now was a distilled model, meaning it was trained by flux.pro as its teacher. These new models change everything ! This is the first experimental, effectively de-distilled version of Flux.dev, meaning it is much closer to what flux.pro is capable of. And it's just the start !
(NB : This is not stating it was trained with flux.pro - I don't know the exact method)
/!\ READ ALL IMPORTANT STUFF BELOW (AFTER THE EXAMPLES) IN THIS DESCRIPTION OR IT WILL BE A FAIL FOR YOU /!\
EXAMPLES MADE MYSELF
Distilled CFG 8 VS Real CFG 8 - Fixed seed, no prompt change except for the text
Each picture is the first shot on each model : no cherrypicking / cheating
Ear gauge LoRa : "Cute blonde raver young girl smiling, facing viewer, with emo makeup and puffy emo hairstyle, green shiny colored hairstyle. 3argauge, both ears have Ear gauge plug with very large circular hole in her lobe. She is wearing a black hoodie with text "DEDISTILLED MAKES MY LORAS WORK" in golden letters
Cum on face Lora (with dedistilled it actually works anywhere) : "COF, Young woman with white sticky cum on face,white sticky semen on face,white sticky sperm on face,face covered with white sticky cum, face covered with white sticky semen, face covered with white sticky sperm. She is wearing a black hoodie with text letters "DISTILLED" in silver font on it"
No LoRa : "Dutch shot view of a black car speeding away from a massive explosion in the streets of a futuristic city with large buildings, towards the viewer, escaping the blast of the explosion. The motion lines around the car give the impression of speed. The front plate of the car has text "DISTILLED" on it."
No Lora : "A small kitten playing with a ball of yarn is seen through an old wooden window of a rustic house. The scene is cozy, with weathered wooden furniture inside the house and soft afternoon light streaming through the window, casting gentle shadows. Outside, in the distance, a photographer is approaching, camera in hand, ready to capture the playful moment of the kitten. The photographer is wearing a brown jacket and is framed by the soft glow of the golden hour light, adding a sense of warmth and tranquility to the scene. The overall atmosphere is peaceful, with a touch of nostalgia from the vintage setting."
SUMMARY OF THE IMPORTANT STUFF (aka current knowledge about it)
Disclaimer : These models are very new. So, just gathering here what is known about them for now. Please share your experiences in the comment section so we can update this together.
PARAMETERS
You can now forget about Distilled CFG and use real CFG (I've tried up to like 14)
NEVER, ever use it with CFG = 1 - It will automatically be a complete disaster, and most of the time the reason why you don't get results
You SHOULD use at least 40 - 60 steps, depending on the CFG you use.
It will be much longer but SO worth it
Unfortunately the current hyperdev 8-steps Lora doesn't seem to work with it to reduce steps
Dedistilled allows NEGATIVE PROMPTS
BENEFITS
Prompt adherence will be EXCEPTIONAL, even with Loras.
Faces Loras will work better, details will be better, text will be WAY better...
Everything from the prompt will be better basically
/!\ IF YOU DON'T SEE ANY IMPROVEMENT FROM DISTILLED MODELS : VERIFY YOU ARE NOT ACTUALLY USING REAL CFG = 1 WITHOUT KNOWING. NOT FLUX GUIDANCE. IT IS TRICKY /!\
AS ALL WORKFLOWS WERE TAILORED FOR DISTILLED, CONSIDER TRYING WITH FORGE IF IT'S NOT WORKING FOR YOU IN COMFY ?
GUIDELINES FOR USE IN FORGE UI
Works in Forge without any change. Will be loaded as if it was Schnell model, disabling Distilled CFG (cool).
EDIT : uploaded all the new Quants, you should find at least one working for you
If you are new to Forge, make sure you use similar settings :Flux workflow
DeDistilled as the checkpoint
In VAE / Text encoder files, provide the vae (ae.sft / ae.safetensors) + clip_l (or a modified clip) + t5xxl (whatever the quant you are using, fp16, fp8, etc).
Otherwise it won't work as they are NOT bundled in the model files herePlease also set Diffusion in Low Bits = Automatic (FP16 Lora), otherwise you might be in trouble with LoRas. This applies to any checkpoint in Forge.
I then recommend these settings for Dedistilled :
GUIDELINES FOR USE IN COMFY UI
Works in ComfyUI using a pretty standard workflow, the one cited below uses the GGUF Loader, Dual CLIP Loader for t5xxl and clip_l prompts, and KSampler Efficient
Recommended settings for Comfy :
Dual CLIP Loader guidance: 3.5 KSampler cfg: 2 to 10 Steps: 50 to 60 Negative Prompt: Can be left blank or can be provided if needed, will affect the image if provided
Sampler: DDIM or euler Scheduler: beta or exponential
WORFKLOW HERE : https://gist.github.com/dasilva333/87bdd5b5b8ebba5515a9919ede0e3c05
Found this one also on reddit (drag & drop it into Comfy) : https://files.catbox.moe/y99yl7.png
TRAINING AND LORAS
I have just trained myself my first LoRa using De-distilled and guidance = 6, after failing hard with Distilled and guidance = 1. Results are awesome, it basically saved my LoRa. Works great with both De-distilled and distilled (but better with De-distilled).
I will be using it to train from now on.
The first checkpoint fined tune with dedistilled has been posted on civitai here : https://civarchive.com/models/690991/sapianf-nude-men-and-women-for-flux-now-de-distilled
Awaiting answers from the author to update here
Sources
Dedistilled model FP16 : https://huggingface.co/nyanko7/flux-dev-de-distill
Dedistilled model FP8 : https://huggingface.co/MinusZoneAI/flux-dev-de-distill-fp8/tree/main
Dedistilled FP8 GGUG & Q4_ K_M GGUG : https://huggingface.co/TheYuriLover/flux-dev-de-distill-GGUF/tree/main
Description
FAQ
Comments (14)
the Github workflow doesn't work, gives an error: Invalid workflow against zod schema: Validation error: Required at "last_link_id"; Required at "nodes"; Required at "links"; Required at "version"; Required at "last_node_id"
Everything is up to date. Any ideas?
Also, I have a red "missing" node OverrideMODELDevice, but in the Manager, there's no more missing custom nodes, everything is installed. I assume OverrideMODELDevice is part of the ExtraModels along with ClipDevice and VAEDevice, but I have no ModelDevice node.
@Norby123 AFAIK the OverrideModelDevice node is just a selector for CPU or GPU. You can just connect the Unet loader straight into the lora loader and delete it.
Same issue. what's the deal
@beignets I dunno, never figured out. I decided to just "rebuild" my own version of the workflow.
Simulacrum seems to have just doubled in potency from this thing. What a find.
DPM++ 2M actually works, which is a bit surprising. A lot of the samplers that didn't work for any other flux model, work for this flux model.
Gotta run it at cfg 6-9 and distilled cfg 0, steps 20-50.
My more complicated descriptions don't seem to work.
Old loras that I thought didn't work, actually work on this thing. It's quite the powerful tool.
Are you on Forge or Comfy UI? Forge seems to have broken the day after Q8 released. Now it's a blurry mess with the highly detailed istructions followed to a tee with Q3 Q4 Q8 all posting the same slightly out of focus blurry result regardless of steps.
It's like the Distilled CFG is being forced to 1 when it's set at 0.0
@323f802 I'm having no issue with Forge. I'm running forge just standard fp8 version of DeDistilled. I discovered it today.
@AbstractPhila Interesting. As far as I can tell through my forge version, distilled = 0 should not matter, because in schnell mode it isn't passed along with the processing object.
Are you getting the message "Distilled CFG Scale will be ignored for Schnell" (with dcfg = 0 or dcfg > 0) before every run?
When did you start using forge, or what is its commit?
@firemanbrakeneck From what I've seen, this thing works fantastically. I'm currently testing merges for a full Simulacrum mix that I never expected to exist, using a series of seemingly dead models that shouldn't work, that work for some reason. It's quite the fantastic thing.
Whatever is happening here, is actually something beyond simple. All of my trainings work on it.
The result is blurry blob on white background (exponential) or blury image (normal). Only beta scheduler works. I am using CFG 8 50 steps. Advanced ksampler node. ConfyUi
You might need to use a flux guidance node. Try adding that and setting it to 0 manually.
I can verify that with a ClipTextEncodeFlux (e.g. flux guidance) node with the value set to 0.0 for each the positive and negative in Comfy will do the trick. I highly recommend building a custom sampler, it's not that tricky. For instance you can use a KSamplerSelect+BasicScheduler+ClipTextEncodeFlux+ClipTextEncodeFlux+CfgGuider+SamplerCustomAdvanced to make what you are wanting here, and the connections are pretty intuitive. The values of ClipTextEncodeFlux should be set to 0.0(i think 1.0 is also equivalent?) for both positive and negative, then the value of CFG can be used to control the strength of guidance. Start with Euler Beta and experiment!
With Comfy and a more advanced custom sampler setup that I use which involves some logic to enable/disable CFG/negative prompts and an adaptive negative guidance node, it's possible to use negative guidance and even CFG with normal Flux models, fwiw. I use perpendicular negative adaptive guidance to run with CFG 1.1-1.2 and negative prompt for ~25% steps with regular, not dedistilled Flux and it works great! It sits in the middle speed wise between base Flux and this de-distilled Flux.
@blyss Can probably go grab some of the basic node code and make your own pretty quick I'd wager.