This model makes bumps on pregnant bellies indicating fetal movement. It's a simple dataset I extracted from hyperfusion where images were labeled "fetal movement", and met a certain quality threshold.
It will trend towards bigger sized bellies based on the training data, but "medium belly" might work for smaller sized bellies.
Description
Same dataset as v1 but removed about 25% of the worst images.
Training notes:
Kohya's trainer
optimizer ADOPT
optimizer_args
"betas=(0.9, 0.9999)" "eps=1e-7"
weight_decay was actively harming DoRA so I trained without it
DoRa LoCon
frozen text encoder (increased training time, but I prefer to not touch the TE if possible)
lr 5e-4
dim 16
alpha 8
conv_dim 8
conv_alpha 4
batch 8
GA 16
3k images
flip
bucket
resolution 1024
tag dropout 0.1
dropout 0.2
caption_dropout 0.1
scale_weight_norms 6
ip_noise_gamma 0.02
min_snr_gamma 2
zsnr
v_pred
Extras:
soft_min_snr instead of the default formula
learned timestep loss weights, a small network to learn the loss scale for each timestep. similar goal to min_snr
sort important tags to the front and sort separately from others
tag implication dropout for all common implied tags ~40% drop