Flat Color - Style
Trained on images without visible lineart, flat colors, and little to no indication of depth.
ℹ️ LoRA work best when applied to the base models on which they are trained. Please read the About This Version on the appropriate base models and workflow/training information.
This is a small style LoRA I thought would be interesting to try with a v-pred model (noobai v-pred), for the reduced color bleeding and strong blacks in particular.
The effect is quite nice and easy to evaluate in training, so I've extended the dataset with videos in following versions for text-to-video models like Wan and Hunyuan, and it is what I am generally using to test LoRA training on new models now.
Recommended tags:
flat color, no lineart, blending, negative space, {{color}} backgroundDescription
[HIDREAM] LoRA
Trained with diffusion-pipe on HiDream-I1-Full
1480 steps on a 4090 over ~7hrs
Previews generated with ComfyUI_examples/hidream/#hidream-dev-workflow
Loading the LoRA with LoraLoaderModelOnly node and using the HiDream DEV fp8 model: hidream_i1_dev_fp8.safetensors
config.toml
# Dataset config file.
output_dir = '/mnt/d/hidream/training_output'
dataset = 'dataset-hidream.toml'
# Training settings
epochs = 50
micro_batch_size_per_gpu = 1
pipeline_stages = 1
gradient_accumulation_steps = 4
gradient_clipping = 1.0
warmup_steps = 100
blocks_to_swap = 20
# eval settings
eval_every_n_epochs = 5
eval_before_first_step = true
eval_micro_batch_size_per_gpu = 1
eval_gradient_accumulation_steps = 1
# misc settings
save_every_n_epochs = 10
checkpoint_every_n_minutes = 30
activation_checkpointing = true
partition_method = 'parameters'
save_dtype = 'bfloat16'
caching_batch_size = 1
steps_per_print = 1
video_clip_mode = 'single_beginning'
[model]
type = 'hidream'
diffusers_path = '../hidream-full'
llama3_path = '../llama-3.1'
llama3_4bit = true
dtype = 'bfloat16'
transformer_dtype = 'nf4'
max_llama3_sequence_length = 256
flux_shift = true
[adapter]
type = 'lora'
rank = 32
dtype = 'bfloat16'
[optimizer]
type = 'adamw_optimi'
lr = 5e-5
betas = [0.9, 0.99]
weight_decay = 0.02
eps = 1e-8dataset.toml
# Resolution settings.
resolutions = [1024]
# Aspect ratio bucketing settings
enable_ar_bucket = true
min_ar = 0.5
max_ar = 2.0
num_ar_buckets = 7
[[directory]] # IMAGES
# Path to the directory containing images and their corresponding caption files.
path = '/mnt/d/huanvideo/training_data/images'
num_repeats = 5
resolutions = [1024]FAQ
Comments (17)
It works really well, I love it.
Out of curiosity, considering the results and the dataset, what's your take on how well the model actually learned?
Hello, thank you for trying it out!
Judging by the representation of "flat color" and "no lineart" in the generations, the latest HiDream version does seem to have learned the concepts and styles of the dataset quite well.
Thank you for the Diffusion pipe link in the about section! Going to take a look and see if Kohyas had an update for it yet as well.
I like my GUIs lol.
No problem, yes GUIs for kohya and diffusion-pipe exist but they will be a bit behind the training code for new models
AMAZING!!!!! TY SO MUCH!!!!
what project did you used to make this lora? diffusion-pipe?
Hello, for some versions diffusion-pipe was used, yes.
Details can be found in the "about this version" section
Will there be a FLUX version?? I need it >.<
Hi, I've not looked too much into training Flux dev/schnell, so I'm not sure how well that would go. Hoping that a de-destilled model could be easier to train (Chroma, Flex etc).
At the moment my compute/effort is going towards fine-tuning SDXL for the next couple weeks at least. :)
@motimalu Okay, Understandable! :D
still, I'd like to see what they can do nicely...
Only v2 for illustrious works correctly, but it have one significant problem - it makes images purple for some reason.
这真是一个非常好看的画风！但有一个比较严重的问题，它不是很听提示词，当我描述了一些角色特征后，我就只能生成正面视角，而无法生成侧面，我想要解决这个问题只能先用其它的模型生成前半部分，再用这个lora来修改画风。不过也有个优点，就是画面不容易崩掉，当我使用通配符添加了一堆乱七八糟的提示词后画面依然很和谐。你的lora真的很棒！感谢你的付出！
#这是我无法使其侧身的角色提示词“1girl,blonde hair,long hair,high ponytail,ponytail,hair_bobbles,blue eyes,slender,navel,groin,white_panties,bow_panties,white_shirt,cropped_shirt,midriff,denim_shorts,thigh_strap,”。
This is really a very nice style of drawing! But there is a rather serious problem. It doesn't follow the prompt words very well. After I described some character features, I could only generate a frontal view and couldn't generate a side view. To solve this problem, I can only use other models to generate the first half and then use this lora to modify the style. However, there is also an advantage. The picture is not easy to break down. Even when I added a bunch of random prompt words using wildcards, the picture remained harmonious. Your lora is really great! Thank you for your efforts!
#This is the character prompt I can't make her pose sideways: "1girl,blonde hair,long hair,high ponytail,ponytail,hair_bobbles,blue eyes,slender,navel,groin,white_panties,bow_panties,white_shirt,cropped_shirt,midriff,denim_shorts,thigh_strap,".
Thank you - maybe I misunderstand your goal, but for a sideways pose, usually the tags to use would be "from side, profile" to reliably achieve it: https://civitai.com/images/90172286
（翻译）谢谢 - 也许我误解了你的目标，但对于侧身姿势，通常使用的标签是“从侧面，轮廓”才能可靠地实现它
motimalu 好吧，我犯了一个蠢错误，我使用的是“from_side”，它带有“_”，而使用你的lora后，模型无法识别含有“_”的提示词了，所以它没有生效，我以为只有Checkpoint上才会有这种差异，没想到lora也会有这种情况。
OK, I made a silly mistake. I used "from_side", which contains "_". After using your LoRA, the model can no longer recognize prompt words with "_", so it didn't take effect. I thought this kind of difference only existed on Checkpoint, but I didn't expect LoRA to have this issue as well.
denghuiw1160 No problem, yes the "_" is usually trimmed for training, at least for my LoRA and I believe for the illustrious base model as well.
（翻译）没问题，是的，“_”通常会被修剪以进行训练，至少对于我的 LoRA 来说是这样，我相信对于“illustrious ”基础模型也是如此。
motimalu 我之前使用的基于“illustrious”微调的模型和lora都可以正常识别“_”，以至于没有怎么注意过这点，可能是在微调的过程中加入了含有“_”的标签。你的lora还是很棒的，画风很好看。不过……它在“nsfw”方面貌似表现得不是很好……你的lora更适合生成美丽的图像。
The model I previously used that was fine-tuned based on "illustrious" and lora could both recognize "_" normally, so I didn't pay much attention to this point. It might be that labels containing "_" were added during the fine-tuning process. Your lora is still great. The art style is very nice. However... It seems not to perform very well in terms of "nsfw"... Your lora is more suitable for generating beautiful images.