Joy Caption Standalone
NF4 Model works extremely fast even on low VRAM Cards
Easy to use standalone GUI and model loading (No HuggingFace)
Word replacement can replace vagina with pussy or labia minora with inner pussy lips for PONY models (Optional)
Banned words can be omitted (Optional)
Automatically creates sized png (No latent errors from bad jpg) just select the training size eg: 512, 1024, 1536 or any other size. (Aspect Ratio is kept)
Updated bat file to create a VENV (You can also use Comfy UI or Forge VENV)
Tip in the user defined caption "Write a long detailed description for this image with some WD14 tags." or "Write a long detailed description for this image with some Booru-like tags."
METADATA UPDATE
This is an update of the base script to parse existing metadata from images. Joy caption can intelligently use this information.
Logic is in the script to append the user prompt with the correct phrase to use metadata. DO NOT write anything in the user prompt, the script handles it.
It is set to ignore 10 characters or less and crop to 200 if the length is longer then that to avoid truncation. This should not be an issue if the use 77 words or less is added to the user prompt.
Names of animals and people
Locations
Film Exposure
Etc
Getty Images is an excellent source of accurate metadata.
Description
FAQ
Comments (7)
Doesn't want to load, says something wrong with my bits and bytes or something. Not sure if i start changing things if it will break my other programs that use those versions, this install don't like. Any support would be appreciated.
[notice] A new release of pip is available: 23.0.1 -> 25.1.1
[notice] To update, run: python.exe -m pip install --upgrade pip
Running joycaption.py...
Using a slow image processor as use_fast is unset and a slow processor was saved with this model. use_fast=True will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with use_fast=False.
C:\AI\joyCaptionStandalone_v10\venv\lib\site-packages\transformers\quantizers\auto.py:222: UserWarning: You passed quantization_config or equivalent parameters to from_pretrained but the model you're loading already has a quantization_config attribute. The quantization_config from the model will be used.
warnings.warn(warning_msg)
The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.
Traceback (most recent call last):
File "C:\AI\joyCaptionStandalone_v10\joycaption.py", line 21, in <module>
llava_model = LlavaForConditionalGeneration.from_pretrained(
File "C:\AI\joyCaptionStandalone_v10\venv\lib\site-packages\transformers\modeling_utils.py", line 311, in _wrapper
return func(*args, **kwargs)
File "C:\AI\joyCaptionStandalone_v10\venv\lib\site-packages\transformers\modeling_utils.py", line 4839, in from_pretrained
) = cls._load_pretrained_model(
File "C:\AI\joyCaptionStandalone_v10\venv\lib\site-packages\transformers\modeling_utils.py", line 5260, in loadpretrained_model
caching_allocator_warmup(model_to_load, expanded_device_map, hf_quantizer)
File "C:\AI\joyCaptionStandalone_v10\venv\lib\site-packages\transformers\modeling_utils.py", line 5834, in caching_allocator_warmup
accelerator_device_map = {
File "C:\AI\joyCaptionStandalone_v10\venv\lib\site-packages\transformers\modeling_utils.py", line 5835, in <dictcomp>
param: torch.device(device) for param, device in expanded_device_map.items() if is_accelerator_device(device)
File "C:\AI\joyCaptionStandalone_v10\venv\lib\site-packages\transformers\modeling_utils.py", line 5809, in is_accelerator_device
return torch.device(device).type not in ["meta", "cpu"]
RuntimeError: Cannot access accelerator device when none is available.
Press any key to continue . . .
I think I made an error with the req file and it installed cpu torch, it should look like this instead
--extra-index-url https://download.pytorch.org/whl/cu128
torch
torchvision
torchaudio
transformers
accelerate
bitsandbytes
Pillow
@Felldude i pasted it in and reinstalled requirements but still same error
@Genie_AI_ delete the VENV in the joy caption folder before launching with the .bat again
@Felldude that worked. Awesome. you filled a void. I been avoiding most api so haven't been able to auto tag in natural language outside of tensor art conveniently. great tool!
@Genie_AI_ I needed a faster solution then comfy, but I was testing using the comfy VENV so I did not notice the cpu install. Glad it worked for you
@Felldude This has already been a fun tool. I believe you may of helped speed up my rate of training too. Props my man, props!