Version 1.1
This Version does not change a whole lot. I have added Aligned Sampler for less Steps and better results just from the base HunyuanDiT model. I have also added optional translation to Chinese, since the model was trained on Chinese language.
Version 1.0
Repurposed my PixArt Sigma Workflow for HunyuanDiT. I did not put much time into finetuning the sampler, etc. for this, but it works. Please check out the Extra Models ComfyUI Extension to see how to install the model properly and use the ComfyUI Manager to get all the missing custom nodes. Have fun!
Description
1.1.1: Hotfixed AutoCFG not properly working with Hunyuan, so I removed it for now
Switched to Aligned Scheduler for much better base results with less steps
Added an optional Chinese Translation node with a switch. The model was trained on Chinese and seems to throw better results with translated text. Feel free to play around with it!
FAQ
Comments (17)
what mt5 did you use ?
Hi, thank you for making this. Do you know how to get this to work with new v1.1? https://huggingface.co/Tencent-Hunyuan/HunyuanDiT-v1.1-Diffusers-Distilled/tree/main ?
I will have a look at it tomorrow!
From the looks of it you can download the same stuff mentioned in the extra model node for setup from here: https://huggingface.co/Tencent-Hunyuan/HunyuanDiT-v1.1/tree/main/t2i
Have not tried it myself yet tho!
@xthisisbotoxx Thank you, I will give this a try hopefully late today. It's odd they didn't just update the main repo - confusing to me!
I tried but can't figure out a mapping. I am hopeful that hunyuan will update the base repository to make this more straightforward, or that extra models will update to handle easier.
@fauxrealistic Heyo, finally had some time to try it for myself, it seems to work if you just switch the model file from the original release to the 1.1 file from here: https://huggingface.co/Tencent-Hunyuan/HunyuanDiT-v1.1/tree/main/t2i/model
@xthisisbotoxx Thank you! Will give it another go
Hi, for awareness if you are interested in updating, 1.2 is now out: https://huggingface.co/Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers
Heyo, ill take a look if the workflow needs a change for it!
It looks like the folder structure is the same as the previous models, so it might work, have not tried it myself yet tho! https://huggingface.co/Tencent-Hunyuan/HunyuanDiT-v1.2/tree/main/t2i
You might be good with just downloading the model and switching that, worked for 1.1 as well!
@xthisisbotoxx Thanks. I def don't understand this stuff to your level. I just tried dropping in the updated model and also the updated clip text encoder and referred to them in your existing workflow. I also updated transformers and restarted. I receive:
Error occurred when executing HYDiTCheckpointLoader: Error(s) in loading state_dict for HunYuanDiT: Missing key(s) in state_dict: "style_embedder.weight". size mismatch for extra_embedder.0.weight: copying a param with shape torch.Size([5632, 1024]) from checkpoint, the shape in current model is torch.Size([5632, 3968]). File "D:\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ComfyUI\custom_nodes\ComfyUI_ExtraModels\HunYuanDiT\nodes.py", line 26, in load_checkpoint model = load_hydit( ^^^^^^^^^^^ File "D:\ComfyUI\custom_nodes\ComfyUI_ExtraModels\HunYuanDiT\loader.py", line 70, in load_hydit model.diffusion_model.load_state_dict(state_dict) File "D:\ComfyUI\venv\Lib\site-packages\torch\nn\modules\module.py", line 2228, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
@fauxrealistic Ill check it out when I have some free time and come back to you!
Okay, so I had some time to check it out and it seems like 1.2 is not supported by the extra model node so far. So all we can do right now is sit tight and wait unfortunately!
@xthisisbotoxx Thank you for the follow up. There is a feature request if you want to add your support to it! https://github.com/city96/ComfyUI_ExtraModels/issues/71
@fauxrealistic Just saw that it got implemented, the workflow should now work again if you updated the extra models custom node!