Didn't like other controlnets since they gave off distortions and couldnt copy pose, so i did my own.
LLite model trained with various images scrapped from twitter and previous models datasets.
Previews generated with pixel perfect copy resolution with "My prompt is more important option".
To generate cond images script was used:
from PIL import Image
import os
imgs = [img for img in os.listdir(os.getcwd()) if (os.path.isfile(os.path.join(os.getcwd(),img)) & (os.path.splitext(img)[1] in ['.jpg','.png','.jpeg']))]
from controlnet_aux.processor import Processor
os.makedirs(str("roast"), exist_ok=True)
for img in imgs:
temp = Image.open(img).convert('RGB')
width, height = temp.size
processor_id = 'lineart_realistic'
processor = Processor(processor_id)
temp = processor(temp, to_pil=True)
temp.save('./roast/' + os.path.splitext(img)[0] + '.jpg', 'jpeg', quality=95)To make controlnet_aux package generate pixel perfect cond images file controlnet_aux\lineart\__init.py was modified at 138 line like following:
...
device = next(iter(self.model.parameters())).device
if not isinstance(input_image, np.ndarray):
input_image = np.array(input_image, dtype=np.uint8)
h1, w1, _ = input_image.shape / starting here
detect_resolution = min(h1, w1)
image_resolution = min(h1, w1) / ending here
input_image = HWC3(input_image)
input_image = resize_image(input_image, detect_resolution)
...Description
trained on anima-base-v1, LR = 0.000073, 7 epochs
FAQ
Comments (21)
Could you please share the workflow?
i am using sd-neo-forge
Sad! jeje
Thanks for making an Anima Controlnet
heya, how does it exactly work? i have tested it on Forge neo but all i get are ''apend'' errors
put it in controlnet folder in models directory and use it as controlnet model.
i am currently using commit 6009ffff99b5d5b4312dc8a8f6476ec0a69b37b1 and don't update much cuz it tends to break things sometimes.
Thank you! I'm using SD Forge NEO! which preprocessor do i set it to?
any line art except for invert
Could you please create a openpose controlnet for it?
my experiments with dataset show that modern openpose preprocessors create bad conditioning images
it works so excellent~THX!!! Any line-based preprocessor will work. I recommend Lineart Realistic and softEdge Hed. Set both the weight and timestep to 1.
If possible, I hope you can make an improved version in the future. The current version still struggles with complex scenes. Thank you.
now cooking with 1800 more images, although with this big size of sample i had to revert to canny detector for batch generating masks. will release if it will turn out good
@loopback_trainer look forward to it!!
How to use in comfyUI?
i dunno man i use forge, look up for workflow and replace model
you will probably need to install LLLite node and load it via that - the file goes into the controlnet folder
@gabrielscarduane1371 you can either install kohya's anima llite custom node or the native llite node supported by comfyui. If you're using the native node, download the controlnet weights and place them in ComfyUI/models/model patches. If using Kohya's anima llite node, place the weights in ComfyUI/models/controlnet.
hello, gotta say this works really nicely, Im combining it with the v2 anything control net that was released earlier at lower strength and it basically behaves like controlnet from SDXL and IL now







