27/05 : Updated with SVD INT4 Quant for Flux https://github.com/mit-han-lab/ComfyUI-nunchaku
Special thanks @theunlikely for the work done on quantification (took 6 hours on a H100 GPU), and thanks to JIB (J1B Creator Profile | Civitai) for his WF and clarifications on using this type of model:
For using this model you need to install the nunchaku project following the instructions there. and the nunchaku ComfyUI custom nodes to get it to work. Download and unzip the archive from Civitai to: \comfyui.git\app\models\diffusion_models\svdq-int4-flux-dev-de-distill - A Nunchaku workflow you can use : https://civarchive.com/models/617562
This is a repost from hugging faces, I did not play any role at all in the creation of this epic, groundbreaking model from nyanko7. I just had to post it asap because it is F A N T A S T I C
Flux.dev as we have known until now was a distilled model, meaning it was trained by flux.pro as its teacher. These new models change everything ! This is the first experimental, effectively de-distilled version of Flux.dev, meaning it is much closer to what flux.pro is capable of. And it's just the start !
(NB : This is not stating it was trained with flux.pro - I don't know the exact method)
/!\ READ ALL IMPORTANT STUFF BELOW (AFTER THE EXAMPLES) IN THIS DESCRIPTION OR IT WILL BE A FAIL FOR YOU /!\
EXAMPLES MADE MYSELF
Distilled CFG 8 VS Real CFG 8 - Fixed seed, no prompt change except for the text
Each picture is the first shot on each model : no cherrypicking / cheating
Ear gauge LoRa : "Cute blonde raver young girl smiling, facing viewer, with emo makeup and puffy emo hairstyle, green shiny colored hairstyle. 3argauge, both ears have Ear gauge plug with very large circular hole in her lobe. She is wearing a black hoodie with text "DEDISTILLED MAKES MY LORAS WORK" in golden letters
Cum on face Lora (with dedistilled it actually works anywhere) : "COF, Young woman with white sticky cum on face,white sticky semen on face,white sticky sperm on face,face covered with white sticky cum, face covered with white sticky semen, face covered with white sticky sperm. She is wearing a black hoodie with text letters "DISTILLED" in silver font on it"
No LoRa : "Dutch shot view of a black car speeding away from a massive explosion in the streets of a futuristic city with large buildings, towards the viewer, escaping the blast of the explosion. The motion lines around the car give the impression of speed. The front plate of the car has text "DISTILLED" on it."
No Lora : "A small kitten playing with a ball of yarn is seen through an old wooden window of a rustic house. The scene is cozy, with weathered wooden furniture inside the house and soft afternoon light streaming through the window, casting gentle shadows. Outside, in the distance, a photographer is approaching, camera in hand, ready to capture the playful moment of the kitten. The photographer is wearing a brown jacket and is framed by the soft glow of the golden hour light, adding a sense of warmth and tranquility to the scene. The overall atmosphere is peaceful, with a touch of nostalgia from the vintage setting."
SUMMARY OF THE IMPORTANT STUFF (aka current knowledge about it)
Disclaimer : These models are very new. So, just gathering here what is known about them for now. Please share your experiences in the comment section so we can update this together.
PARAMETERS
You can now forget about Distilled CFG and use real CFG (I've tried up to like 14)
NEVER, ever use it with CFG = 1 - It will automatically be a complete disaster, and most of the time the reason why you don't get results
You SHOULD use at least 40 - 60 steps, depending on the CFG you use.
It will be much longer but SO worth it
Unfortunately the current hyperdev 8-steps Lora doesn't seem to work with it to reduce steps
Dedistilled allows NEGATIVE PROMPTS
BENEFITS
Prompt adherence will be EXCEPTIONAL, even with Loras.
Faces Loras will work better, details will be better, text will be WAY better...
Everything from the prompt will be better basically
/!\ IF YOU DON'T SEE ANY IMPROVEMENT FROM DISTILLED MODELS : VERIFY YOU ARE NOT ACTUALLY USING REAL CFG = 1 WITHOUT KNOWING. NOT FLUX GUIDANCE. IT IS TRICKY /!\
AS ALL WORKFLOWS WERE TAILORED FOR DISTILLED, CONSIDER TRYING WITH FORGE IF IT'S NOT WORKING FOR YOU IN COMFY ?
GUIDELINES FOR USE IN FORGE UI
Works in Forge without any change. Will be loaded as if it was Schnell model, disabling Distilled CFG (cool).
EDIT : uploaded all the new Quants, you should find at least one working for you
If you are new to Forge, make sure you use similar settings :Flux workflow
DeDistilled as the checkpoint
In VAE / Text encoder files, provide the vae (ae.sft / ae.safetensors) + clip_l (or a modified clip) + t5xxl (whatever the quant you are using, fp16, fp8, etc).
Otherwise it won't work as they are NOT bundled in the model files herePlease also set Diffusion in Low Bits = Automatic (FP16 Lora), otherwise you might be in trouble with LoRas. This applies to any checkpoint in Forge.
I then recommend these settings for Dedistilled :
GUIDELINES FOR USE IN COMFY UI
Works in ComfyUI using a pretty standard workflow, the one cited below uses the GGUF Loader, Dual CLIP Loader for t5xxl and clip_l prompts, and KSampler Efficient
Recommended settings for Comfy :
Dual CLIP Loader guidance: 3.5 KSampler cfg: 2 to 10 Steps: 50 to 60 Negative Prompt: Can be left blank or can be provided if needed, will affect the image if provided
Sampler: DDIM or euler Scheduler: beta or exponential
WORFKLOW HERE : https://gist.github.com/dasilva333/87bdd5b5b8ebba5515a9919ede0e3c05
Found this one also on reddit (drag & drop it into Comfy) : https://files.catbox.moe/y99yl7.png
TRAINING AND LORAS
I have just trained myself my first LoRa using De-distilled and guidance = 6, after failing hard with Distilled and guidance = 1. Results are awesome, it basically saved my LoRa. Works great with both De-distilled and distilled (but better with De-distilled).
I will be using it to train from now on.
The first checkpoint fined tune with dedistilled has been posted on civitai here : https://civarchive.com/models/690991/sapianf-nude-men-and-women-for-flux-now-de-distilled
Awaiting answers from the author to update here
Sources
Dedistilled model FP16 : https://huggingface.co/nyanko7/flux-dev-de-distill
Dedistilled model FP8 : https://huggingface.co/MinusZoneAI/flux-dev-de-distill-fp8/tree/main
Dedistilled FP8 GGUG & Q4_ K_M GGUG : https://huggingface.co/TheYuriLover/flux-dev-de-distill-GGUF/tree/main
Description
FAQ
Comments (57)
Test your prompts with the regular distilled version. Finalize with the de-distilled version.
hey, i have made a comparison between sampler and schedule vs steps and CFG
very interesting, thanks. It appears we can save many steps if we don't care too much about details
@valentinkognito365 its for a simple prompt, i dont know how it doeas on more complex or with a lora ;)
I made a merged offshoot of this model. It's quite potent. It contains multiple new trainings.
There are many interesting experiments on it at the moment. Seems for instance that finetuning with your concept is very powerful, then you can extract back the LoRa.
Also, we got to keep an eye on what's happening with LibreFlux and OpenFlux which are both ongoing full de-distillation of Flux Schnell (which is even more distilled than Flux.dev but has the Apache 2.0 licence)
That's where the Pony author should have put his efforts, but once he's done with AuroraFlow, he might perhaps consider finetuning one of these, as they should be ready for that by then
@valentinkognito365 AuroraFlow looks promising too. One of the researchers who came up with the Lora came up with the concept for it, so I'm hoping that pans out to something substantial soon.
It seems Forge works better with this model, and faster. In ComfyUI I always have the problem that there is either not enough contrast or that the pictures get blurry. I am running tests now for about 1 day - sometimes I hate my 3060 for being so slow.
I'm getting blurry results in Forge. Haven't found the right settings yet, though it seems like I didn't get any blur until I started utilizing the negative prompt.
@thebollo How could you add a negative prompt in Forge? For me it is grayed out!
@hannahthorn I'm not sure of the exact trick because I've seen it disabled as well. It either unlocks when you have the de-distilled model actually running a job, or when you apply a style that includes both positive and negative prompts.
@thebollo If you set de CFG Scale over 1, the negative prompt stops being grayed out.
The Comfy workflow provided does not work.
'Invalid workflow against zod schema: Validation error: Required at "last_link_id"; Required at "nodes"; Required at "links"; Required at "version"; Required at "last_node_id"'
The workflow in the PNG file loads but has a missing node which I cannot find through install missing nodes or Googling.
'overrideMODELdevice' with an ID of #173 or #175 I think. If anyone knows where to get it let me know. I removed it which does not seem to affect generations but I would like to know what it does.
same here
Would you recommend training Loras on this model or on the dev2pro model ->https://huggingface.co/ashen0209/Flux-Dev2Pro?
I can't say myself, I have never trained on Fluxdev2pro, and I've just really started training on Dedistilled. My understanding is dedistilled is more de-distilled than fluxdev2pro but fluxdev2pro has had some finetuning that helps training. Some people have even trained on both separately then merged the results. So, this is still the experimental phase
Did the Selene Lora on the de-distilled fp8 model on Kohya and got decent results. Going to keep using it for now and see if there's a difference.
I've confirmed with multiple X/Y/Z plot tests that this model produces blurry results when values are present in the negative prompt field. Any ideas why?
never really got issue with blurry stuff, could be not enough steps or sampler/scheduler
Geez, two days were spent solving this blurry mess and here we go. After reading your post, removed negative prompts and its working. Thank you for sharing.
@Azaella7 It's a shame because this model lets me do wayyyyy more than base Flux, and for my use case I'm desperate for negative prompts, but the blurriness is just too bad when they're applied.
The negative prompt also gets fed into the T5, but it doesn't have the required negative inference to produce those negative effects. I've had better results simply negating prompts from the positive;
(cheese:-1), taco, chicken
@AbstractPhila This was a thought-provoking idea, but in my tests in both Forge and ComfyUI adding negated prompts to the positive prompt has no effect.
@thebollo https://civitai.com/articles/8569
@AbstractPhila Interesting. I'll keep trying to reproduce because I desperately want negative prompting with Flux!
Using beta shcheduller; 50 steps. Might help.
As my model continues to diverge, it becomes distinctly different from this model. The capability of using DeDistilled is shrinking the more I train.
It's gone beyond context being controlled by DeDistilled, to me having to control exactly what context I want DeDistilled to present, which defeats the purpose.
The quality has dropped exponentially as well the more training i run.
I keep getting blurred or not ideal pictures, even though I copied your settings exactly, on forge. Using a LORA too. Tried steps till 60, adjusting the cfg. Anyone have similar issues?
Same here. Pictures look "dirty" (not in the good way) or not well defined.
Loos really bad. You can see there is a better adherence to prompt but the pictures are all burred
@nemrrode Yes, it sticks much more to the prompt, but if you use the negative prompt, the image looks "blocky", while if you don't use it, the image looks like... when you resize a picture and then apply a filter to blur the pixelated image. Horrible.
To be honest with you few, it looks like the CFG is set way too high on his images. Don't follow his examples, and it should be a fine model.
@BenjiLORA Don't follow his examples. Well, even tried plyaing up and down with CFG, doesn't really fix, still blurry as a nightclub toilets at 6am.
For ComfyUI I used this workflow: https://civitai.com/articles/8419
It worked good with just cfg 3 and 30 steps.
.
@sgil No, sorry, no idea.
I created a simple workflow, I hope someone finds it useful
https://civitai.com/models/936177?modelVersionId=1047987
I get Cannot execute because a node is missing the class_type property.: Node ID '#45'
where is the fp16 gguf?
fp16 gguf is Q8 gguf
@superuser111 yes ok, I hadn't understood that. But I wonder why I have differences when I test Q8 against fp16...
@nanunana Q8 GGUF is close to FP16 quality, but not exactly the same, so there will be some slight differences. Q8 GGUF image quality is closer to FP16 than FP8 in my tests.
Can this be block merged with regular Flux Dev models without losing the dedistillation? If so do we need to merge only specific blocks?
.
No, it works great with LoRas trained on distilled models. In fact, it enhances them
.
@sgil I don't use Comfy but I'm going to switch back to it soon because there's a lot of things I want to test under it. So doing this I will also test more workflows for this model and update the description with them. Meanwhile look in comments there has been a few different links for workflows
This model is for training Lora's from what I understand. You train on this model and then use the Lora's on the Flux-dev model. People have a hard time training lora's on flux-dev but this version seems to help solve some of the issues when training.
@VelorianX it is better for training because you can train at higher CFG. But its also much better for generation, prompt adherence is way higher especially with LoRas. A lot of LoRas work better. It is a lot slower than distilled though.a hyper version has just been released which is a good tradeoff between both. I almost don't use distilled models anymore
Your version ID numbers are not populating correctly on downloaders.
When I try to download this via the comfy model downloader nodes, the only version ID number that works is for the FP16 version of the model. The fp8gugg model's version ID number points to a safetensors file named "princessBubbleGum" which is in no way shape or form a flux GGUF.
Since this has been out for a minute, I tried a few premade workflows with mixed results. After quite a bit of tweaking of a few workflows, I came up with one of my own that works absurdly well, only 30 steps at a CFG of 4.2. It is set up to work with FP16 but any dedistilled model version can be used if you use the proper node (gguf, for example). I will post some images with some workflows. I'm using the naked men and women version of this in most workflows but this setup works with both versions of dedistilled equally well. Will post images/workflow soon once some more tweaks are done. Also, this workflow omits the flux CFG entirely and adds the model sampling flux space with width and height nodes added. This workflow is based on two of the workflows posted in this comment section, will give shouts to the original contributors when I upload.
Here is the workflow I was talking about where I combined elements of a few other workflows, streamlined them a bit, used a couple of efficiency nodes, and BAM! This workflow is fast, simple, easy to edit, laid out in a way that everything's easy to reach and change, and it features the model sampling flux node, which makes all the difference when using flux. With that node, flux, in my humble opinion, outperforms most other image generators. Also has the "better" text encoder, which, well... that's subjective, but in my opinion it renders much more legible text and the images generally look better. Enjoy!
https://civitai.com/articles/13258/flux-dedistilled-workflow-with-flux-modeling-sampler
Any idea how to get Tea Cache / First Block Cache working with the Q8? Either accelerates other gguf models with minimal quality loss, but severely degrade the image on this, even at 90+ steps.
One of the best Flux models I've used! Very flexible with LoRAs and better quality than the distilled Flux Dev. Props to Nyanko7 for blessing us with such a great model!
Once discovering it i could never go back to distilled. Try the hyper 8 steps one, its faster, its my main model for flux now
silly question as I cannot find any information about "GUGG".
What is the difference between GUGG and GUFF? isnt it just the same?
Details
Files
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.