UM_T5xxl (FP32 Pruned)
This is the Text Encoder NOT the diffusion model
From FP32 Source (Google) unpruned model is 50GB
Works with I2V and T2V small (3B) and large (14B) models
You will still need to download the Diffusion Model, VAE and Vision Model if using I2V see the wiki
I could not find a pruned FP32 TE for WAN 2.1 posted anywhere. Makes you wonder if the "Scaled FP8" was quantized correctly.
Description
FAQ
Comments (33)
as always - stupid question - but, maybe, just maybe :: gguf Q_8 ???
It is not a standard T5 architecture, I am not sure it has been added
@Felldude there are gguf quants on huggingface - I'm always using one of them
@OliviaRossi You can define a custom architecture, I am not a gguf fan (Do to the speed, the quality is good) so I don't really keep up with it - I haven't updated in months
@Felldude ok, TY!
@OliviaRossi So even the latest GGUF does not appear to have WAN_T5 added yet
@Felldude interesting, I'm generating videos with it for weeks now (with gguf version of it)
@OliviaRossi ChatPig likely defined the architecture themselves - they list using the FP32 version as the base and likely would have if went through the trouble of writing custom script
gguf model would be amazing :D
currently the convert_hf_to_gguf.py will not convert the model to gguf - FP32 is 20% faster then GGUF for me so I don't delve into GGUF
@Felldude The upside of gguf is that with multigpu, you can allocate a small amount of virtual vram (I usually go with 1-2 for the clip, I think it's only used in the initial stage?) and still get a decent portion of the speedup, per pollock's writeup on the subject. With fp it's all or nothing.
Is the stated comparison when both are fully loaded?
@SAVITAR_SPEED_GOD Also, city96 published umt5-xxl ggufs so his patcher might work for it. Conversion colab here: https://colab.research.google.com/drive/1JFRdqy1EdsZNU6STYnf3ft2a4DUHvKXm?usp=sharing
@firemanbrakeneck The user chatpig did a gguf of umt5 and list using FP32 as the base - I have not tested any multigpu setups, with gguf I was able to use FLUX before CPU offloading was a thing, and for any models that require full loading at all times it might speed up certain things. But with CLIP once the initial load is done I have not had any issues. Now for the diffusion model I absolutely think Q8 is better quality then FP8 but for me it is slower then BF16 with offloading
could you post a few comparisons between this clip and the quantized one at same seed? would be interesting to see the difference in prompt understanding
Usually the difference are minor - Over 100 or so generations the FP32 version tends to have better hand positioning and other minor details. That is to say out of 100 most generations it will not matter, when it does matter the FP32 looks better most of the time
I put an example in the article https://civitai.com/articles/16388
@Felldude appreciate it, thanks!
What video card do you need for this to run at reasonable speeds?
With CPU offloading most video cards can handle the 3B model or even the 14B model - I guess it depends on how many minutes your willing to wait
@Felldude I have a 5080. But 14b even at 480p settings seem to take more than 10 minutes for a few second video. And im using teacache, torche, skip layer, with 20 steps. edit: unless I use gguf models.
@delta45424155 One minute per IT is more in the range of a 3050 or 3060 even on the BF16 14B model - You might have another issue
@delta45424155 are you using bf16, fp16 or fp8_scaled version?
@BBBAAA2 fp16.
using i2v_480P_14B_bf16, UM_T5-xxl(fp32), clip_vision h, wan2.1 vae, i'm getting 25.52s/it that goes down to 9.98s/it over 15 step 33frame video (480x832 resolution)
@delta45424155 You might not have enough RAM to support your video card and the offloading, so for a 24GB card you need 24GB of RAM then you need to fit the other models so that is over 60GB - You can manually set your virtual memory to something like 100GB assuming you have a NvME hardrive
@delta45424155 sorry for the delayed response, it didn't give me a notification for some reason.... i have personally had issues with other versions but good success using i2v_720P_14B_fp8_scaled or i2v_720P_14B_fp8_scaled_e4m3fn models.
So, do you mean that this text encoder model can better understand the ability of the prompt word? I have always suspected that there is something wrong with F8's text encoder. It often fails to understand the prompt word correctly, and the generated videos are often incomprehensible about how it works. Even the basic walking video is very abstract.
Yes it is the text encoder, you will need to source the other files from the guide linked
lmao why did you add that suggested resource
because this isn't the same text encoder you're thinking of. This is a better one than what most people are using and it's better.
If your referring to the lora it is one of 5 onsite and was used in the test in the article comparing the BF16 and FP32
Thank you for the FP32
Details
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.