Arknights: Endfield | Operators Collection
Intended to generate non-commercial fan works of Operators from the Arknights: Endfield video game.
ℹ️ LoRA work best when applied to the base models on which they are trained. Please read the About This Version on the appropriate base models and workflow/training information.
Version 3.0 (Latest)
This was re-trained with several attempts to reach the same level of quality as the full-finetune which included the same dataset:
The LoRA is not quite there, even when adjusting hyperparameters and extending the training time.
Little disappointed about this, but it may be an expected limitation of LoRA training.
Operators (game version 1.0):
female endministrator \(arknights\)
male endministrator \(arknights\)
perlica \(arknights\)
chen qianyu \(arknights\)
akekuri \(arknights\)
alesh \(arknights\)
antal \(arknights\)
arclight \(arknights\)
ardelia \(arknights\)
avywenna \(arknights\)
catcher \(arknights\)
da pan \(arknights\)
estella \(arknights\)
fluorite \(arknights\)
gilberta \(arknights\)
laevatain \(arknights\)
last rite \(arknights\)
lifeng \(arknights\)
snowshine \(arknights\)
wulfgard \(arknights\)
xaihi \(arknights\)
yvonne \(arknights\)
Operators (game version 1.1):
mi fu \(arknights\)
rossi \(arknights\)
tangtang \(arknights\)
zhuang fangyi \(arknights\)
Unreleased Operators (unconfirmed designs):
arcane \(arknights\)
camille \(arknights\)
feranmut proxy \(arknights\)Antagonists:
ardashir \(arknights\)
nefarith \(arknights\)
Works best in combination with NL if you name a character, then describe their basic appearance.
A vibrant and dynamic illustration of Yvonne from Arknights: Endfield, featuring her with long pink hair styled in twintails, pointy ears, and small horns, along with a playful tail...
To be fixed:
Pogranichnik had a mistake for his labelling
Ember did not seem to learn her outfit/features
Description
Trained on Anima Base 1 with diffusion-pipe
Dataset updated to cutoff of 2026/06/14
Added partially supported characters:
arcane \(arknights\)
camille \(arknights\)
feranmut proxy \(arknights\)
*these characters have not released, so their designs are unknown/inconsistent.
Training config:
output_dir = '/mnt/d/anima/training_output/anima-base-1-arknights-v31'
dataset = 'dataset-anima-arknights-s1.toml'
# training settings
epochs = 5
# Per-resolution batch sizes
micro_batch_size_per_gpu = [[512, 64], [1024, 32], [1536, 16]]
pipeline_stages = 1
gradient_accumulation_steps = 1
gradient_clipping = 1
warmup_steps = 250
lr_scheduler = 'cosine'
# misc settings
save_every_n_epochs = 1
activation_checkpointing = true
partition_method = 'parameters'
save_dtype = 'bfloat16'
caching_batch_size = 1
map_num_proc = 8
steps_per_print = 1
compile = true
[model]
type = 'anima'
transformer_path = '/mnt/c/workspace/models/diffusion_models/anima-base-v1.0.safetensors'
vae_path = '/mnt/c/workspace/models/vae/qwen_image_vae.safetensors'
llm_path = '/mnt/c/workspace/models/text_encoders/qwen_3_06b_base.safetensors'
dtype = 'bfloat16'
llm_adapter_lr = 1e-6
sigmoid_scale = 1.3
[adapter]
type = 'lora'
rank = 64
dtype = 'bfloat16'
[optimizer]
type = 'adamw_optimi'
lr = 4e-5
betas = [0.9, 0.99]
weight_decay = 0.01
eps = 1e-8FAQ
Comments (7)
so good
Hi! Funny enough, I'm writing under the same post as a couple of months ago, and just like back then, if it's not too much trouble, I'd like to ask a couple of questions :d
I'm still training characters on small datasets in diffusion-pipe and came up with this config: https://pastebin.com/xPejTm6z. Based on my observations with a dataset of 20 images, the character learns quite well by around step 500 with a batch size of 4 and [512, 1024] resolutions. It would be great if you could take a quick look at the config and tell me how reasonable my approach is :d. Here are some more specific questions:
I really want to know if you still use a mixed captioning mode where combinations of tags and nl alternate? In my specific case with small datasets, training on tags only or a mixed mode increased overfitting and generally reduced model flexibility compared to using only nl.
Continuing the topic of captions, I noticed that in the "tag to nl" conversion script, you changed the initial structure from "Artwork of Fern, with long hair..." to just "Fern with long hair...". Did you make this change to give more weight to the activation token because you work with quite big datasets? I understand this is a correct and logical approach, but I don't know what to do since in my case, all models look better specifically with NL captions in the "Artwork of [act_token], with..." format :(
I'm curious if you've tried disabling training for specific layers, and if so, how useful it is? I tried disabling self_attn, but I didn't really like the result. Despite some improvement in generalization, it feels like the model became worse at reproducing certain unique details, which was expected.
I'd be very grateful for an answer, and thank you for your models. They are highly impressive on their own and give a lot of inspiration for working on own ideas
Hi degurshaft
The config you shared does look good for a small dataset.
I do still use a mixed captioning method, as like you say I am less concerned with overfitting on a larger dataset.
I removed the "Artwork of" from the example as I want to give the VLM a bit more freedom in describing what it is looking at based on the tag grounding, e.g. is it a traditional media painting, digital illustration, mixed media, photo background etc.
It's likely the "Artwork of" tokens are extremely common in the base models captioning which might be why you would have trouble when removing it for a smaller LoRA, any divergence from what the model already knows would require more training to overcome/fit to a new paradigm.
I haven't tried disabling specific layers, I'm not really sure what the motivation behind doing so would be - memory/perf saving optimizations, or minimizing updates to the model?
@motimalu Specifically in my case, I really wanted to minimize model updates to potentially reduce the risk of overfitting on small datasets, and as a side benefit, the weight of the final model indeed decreases. As a result, I noticed that generalization improved slightly and there were minor improvements to the backgrounds. Unfortunately, I don't know how much this experience can be extrapolated to training with large datasets, so I don't risk giving any recommendations :D
So about that Ember...
The v2 version has a special filter that makes the picture look more textured. Do you have any ideas?
how u trainiend this lora ?