Hui Buh from German trash movies Hui Buh: The Goofy Ghost and Hui Buh and the Witchs Castle. The character design is rougly based on the appearence of Michael “Bully” Herbig.

Created as bf16 DoRa, so it might require a fairly modern inference engine and/or GPU.

Tags

👤 Primary tag

HuiBuh

🩼 Situational general tags

1boy, solo

🎯 Essential face tags

bob cut, dark brown hair

🩼 Situational face tags

blunt bangs, thick eyebrows

🎯 Essential clothing tags

metal collar, armor

🩼 Situational clothing tags

chain, breastplate

Inference

Recommended checkpoints:

• Pony Meme Diffusion

• Pony FaeTality

• Pony 2DN

Recommended settings:

• Sampler: Euler a

• Steps: 40

• CFG scale: 7

• Clip skip: 1 (yes, really)

Changelog

v1.0.11 → v1.2.25

• Decreased steps from 1040 to 832

• Increased network alpha from 12.5% dim to 25% dim

• Increased batch size from 1 to 4

v1.2.25 → v1.2.35

• Trained for 10 more epochs

v1.2.35 → v1.2.35-lite

• Reduced network dim from 32 to 8

• Increased network alpha from 25% dim to 50% dim

v1.2.35 → v1.3.35

• Decreased batch size from 4 to 1

v1.3.35 → v1.4

• Changed learning rate scheduler from “cosine” to “constant”

• Reduced learning rate from 0.001 to 0.0005

• Increased steps from 832 to 1248

• Disabled tag shuffling

Development

This is my first ever lora developed, so please expect some training mistakes. I created the dataset from movie screencaps. It’s publicly available here. The training was done on a Colab L4 VM (22.5 gb VRAM) using custom software based on kohya-ss/sd-scripts.

Kohya invocation:

accelerate launch /content/repos/kohya-ss/sdxl_train_network.py --pretrained_model_name_or_path /content/inputModel/checkpoint.safetensors --vae /content/inputModel/vae.safetensors --train_data_dir dataset/train/dreambooth --output_dir /content/model --logging_dir /content/log --resolution 1024,1024 --save_model_as safetensors --text_encoder_lr 0.0001 --unet_lr 0.001 --network_dim 32 --network_alpha 8 --output_name hui_buh --max_train_steps 832 --caption_extension .txt --cache_latents --train_batch_size 4 --seed 1910506442 --shuffle_caption --caption_separator ', ' --cache_info --no_half_vae --network_module lycoris.kohya --network_args preset=full algo=lora dora_wd=True --optimizer_type adafactor --optimizer_args scale_parameter=False relative_step=False warmup_init=False --save_every_n_epochs 1 --learning_rate 0.001 --lr_scheduler cosine --highvram --xformers --gradient_checkpointing --fused_backward_pass --mixed_precision no --save_precision bf16 --sample_sampler euler_a --sample_prompts prompt.txt --sample_at_first --sample_every_n_epochs 1 --log_with wandb --wandb_api_key 👌 --wandb_run_name hui_buh-1723892649 --log_config --huggingface_repo_id Jaidchen/sdxl-pony-lora-hui-buh --huggingface_path_in_repo model --huggingface_repo_type model --huggingface_token 👌 --save_state_to_huggingface --async_upload --metadata_title 'Hui Buh' --metadata_author Jaidchen

Collaboration

Created in collaboration with:

• Aki Hayakawa by JustTNP

• Canas and Volke by MadlyLaughing

• Kieran by ElizaPottinger

• Strahd von Zarovich by ClamJam

• Hibino Kafka by dwnsty

• Zack by Lamancha50002310

• Aadit and Hisao Nakai by NorthSpirit

• Dhar by Bshi

• Tatsuo Shinada by guy90

• Kensuke Aida by DegenerateDiffusion

• Ferdinand von Aegir and Hanneman by bsinky

• Tatsuya Suou by FP_plus

• Sephiroth by Vinbeez

• Stark and Jack Atlas by ChameleonAI

• Coomer by FallenIncursio

• Arbiter by richyrich515

• Space Dandy and Bobobo by Ravenfoot

• Ramza Beoulve by novowels

• Adamant by Hoseki

• Sir Ansbach by wrench1815

• Garron Paduk by doomguy1111

• Keyaru by Yami

• Alex Louis Armstrong and Senator Armstrong by Fandingo

• Kim Dokja, Yoo Joonghyuk and Min Ryu by Shufflebox

• Hui Buh by Jaidchen

• Ushiromiya Battler by Shalie

Thanks to mfcg for collecting the links!