TO：https://github.com/StartHua/Comfyui_CXH_joy_caption
Note: If the Comfyui_image2prompt node is installed, you can check whether the direct transform version is forcibly rolled back!
1. Install dependencies requirements.txt (note: the version of transformers cannot be too low)
Run automatic download model (manual download recommended)
2.
(1).https://huggingface.co/google/siglip-so400m-patch14-384 and put it in clip/siglip-so400m-patch14-384
(2). It is recommended to download https://huggingface.co/unsloth/Meta-Llama-3.1-8B-bnb-4bit (if you have an A100, you can consider downloading meta-llama/Meta-Llama-3.1-8B) and put it in LLM /Meta-Llama-3.1-8B-bnb-4bit
(3). Must be downloaded manually: https://huggingface.co/spaces/fancyfeast/joy-caption-pre-alpha/tree/main/wpkklhc6 and put it under Joy_caption
FAQ
Comments (4)
Can you add instructions in english?
OK了
Hi! Not so sure If you could help me, got the following error when tried to use this workflow:
ValueError: Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit the quantized model. If you want to dispatch the model on the CPU or the disk while keeping these modules in 32-bit, you need to set load_in_8bit_fp32_cpu_offload=True and pass a custom device_map to from_pretrained. Check https://huggingface.co/docs/transformers/main/en/main_classes/quantization#offload-between-cpu-and-gpu for more details.
which I am not so sure where to fix.
this is great, it works on computers with dedicated (8GB or higher) GPUs only.. I wished it worked on eGPUs.. I wish that a future version will work within Stability Matrix which also supports Zluda, ROCm (for AMD GPUs) and eGPUs