Doujinshi Uncensor Lora: https://civarchive.com/models/2395643
Manhwa Uncensor Lora: https://civarchive.com/models/2685875/manhwa-uncensor-lora-flux-2-klein-9b-blank-censor
Manga colorizer FLux 2 Klein (inferior): https://civarchive.com/models/2395415/manga-colorizer-with-image-reference-flux-2-klein-9b-trained-on-nsfw-doujinshi
V1 Overview:
This lora was trained on a 7k duos dataset (5k NSFW and 2k SFW) of manga pages. Colorizes pages without prompt, but you can use brief prompting or tags to customize colorization (careful it can degrade results with long or bad prompts).
Recommend to use it with the official Turbo Lora, takes less steps and is more colorful, but has more inconsistencies.
If you dont see the NSFW examples, check your civitai settings.
Workflow: https://pastebin.com/pEP64H1M
You will need two nodes (one is just for image resizer, you can use any resizer):
Recommendations:
Prompt: Empty or use descriptive sentences or just booru tags, the less intrusive the better.
Resolution: 1536 (with resizer node, if you go lower the text gets messed up)
Steps: 20 (8 with the turbo lora)
Strength: 1
CFG: 5 (1 with the turbo lora)
Description
Colorizes manga pages. Did not apply the colored blob method or second image reference for limitations of the model.
FAQ
Comments (27)
It's quite impressive. It seems to work just as well with the turbo lora applied with 4 steps cfg 1. Sometimes additional details are added (ie: sparkles in pure black backgrounds), lower step counts seem to minimize additional details.
Great to know it works with the turbo lora, didnt try that till right now. It makes it more colorful which I like, will include that detail in the description thanks for letting me know.
Mind sharing how you trained this lora?
Sure, since i cannot code I asked CODEX to setup for me this fork of kohya that has conditioning implemented: https://github.com/Mirumo0u0/sd-scripts
My parameters were in summary
For the first 12800 steps: lr: 5e-5, constant, AdamW8bit, 512p, Rank/DIM: 32, weighting_scheme: uniform, timestep_sampling: sigmoid, discrete_flow_shift: 1.0
Next till the 15800 step at 1024p with lr of 2e-5
Next till the 19800 step at 1536p with lr of 2e-5
I have an rtx 5070ti, for 2000 steps at 512p takes 20 minutes and 7.5 GB VRAM (Windows takes 2 GB of that). At 1024 takes 50 minutes and 8.2 GB VRAM. At 1536 takes 1:20 hours and 9 GB VRAM with gradient checkpointing and flash attention (for the 1024 and 1536 times i cannot remember them precisely but it was around that length)
this is good! i wish it can do more like transforming the manga into 3d renders
Hi. Will there be an "Uncensor Lora" version for Anima?
I have them already, not working as good as the flux Klein ones yet. If I can make them at least as good I will publish them.
I'm actually going to give it a try right now. Should be ready by tomorrow but will only share if its any good. Using OP's recommended sd-scripts fork.
EDIT: that fork (or maybe sd-scripts itself) has a problem when using '--cache_latents --cache_latents_to_disk --vae_disable_cache' that results in tensor shape mismatch errors due to it loading the conditionals as images instead of using the already vae-encoded and cached latents. took me a while to figure out the problem and solve it but I'm posting this here to raise awareness that those running on that repo may face the same issue with those args
@huj0ps1t6 Great! We can’t wait!
@artaibyanuartoraigyrov It was a fail.
It took the model ~1000 steps at LR=3e-5 to align the weights to conditional format plus understanding its supposed to output the same input (for the most part) and to 'understand where its supposed to apply changes'.
At step 3000 the output was near perfect (still had some VAE issues due to VAE Encoding+Decoding but thats normal) in the areas that do not contain mosaic censoring / bar censors / gaussian blur - it was trying to fix them but it just couldn't get it right.
Fast forward to step 20000 - still identical to step 3000. The model knows what its supposed to do and its able to figure out the right areas to change but does not know what to change them with. Its not able to 'understand the input as a whole and fill in the missing data based on it'.
It may be possible to achieve this but it would probably require a finetune-level training with higher batch size and trained for much, much longer (200k+ steps at the very least). I ain't gonna do that. Its also possible that the conditioning training code is flawed in some way - which would mean even such a large training would end up failing.
What I am trying now is ControlNet-LLLite - currently at step 2000 and the outputs are still very noisy but I gotta say that it does seem like its already more capable at 'filling in the missing data' than the conditional LoRA was at step 20000.
@huj0ps1t6 Thank you for your efforts!
Maybe it will all work out in the future!
black images, and the apply cosmos node has the wrong input
I noticed that node bugs out adding a latent input that shouldnt exist when trying to share the workflow, deleting the node and adding it solves it.
@Norian still black
@ntrlover ComfyUI-Cosmos-Reference was broken by the author around two weeks ago. Anyone who installed this node after that update will not see the expected result from this workflow, to put it mildly.
will the updated version include more data ? i'm not sure if it is the turbo lora but shaded art and dark skinned/tanned characters get colorized in blue/purple
Yes, I also had a hard time getting it to do a tan, it is a dataset issue, I will need to fix that for next version. In the meantime you could try without the turbo lora and adding in the prompt that the character is tanned to see if it listens.
@Norian i was testing and it seems reducing the strength of the turbo lora to 0.05 produce result similar to without it and no speed loss
Perhaps it's not the fault of turbo lora, because I also encountered this situation without using turbo lora,I've uploaded the picture. Is that the case?
Does using the artist's tags actually make a noticeable difference
Another issue is that the text box becomes blurry. Even when using high-definition nodes, this problem persists. Is there any way to solve it?
Yes, artist tags have some effect, I tried with @Khyle and worked in my test. As for the blurry text there is not much to do, i can try training at 2048p but I think it is because the qwen vae cannot restore the text pixel perfect at 1536p.
@Norian Yes, I tried to zoom in to 2560P, but in the end, a black image was generated, and the effect was very poor. Using SDVR2 to zoom in also makes the text blurry. It's really frustrating.By the way, I'm still looking forward to your flux2 V2 colorized lora
@Norian It's so hard to find the artist TAG that matches the initial comic cover painting. Could you create a reference image reference style node or function like klein and add it to V2?
I played this lora one day and found that if the original image has a lot of ambient lines or shadow dots (noise), then the final image will be particularly dark in color. But if the original image's line drawing is clean lines without dots, noise or shadows, this situation won't occur. I've already uploaded it to my gallery. You can take a look. Is this a problem with the training set? I hope version V2 can be improved. I'm really looking forward to this lora update
I played this lora one day and found that if the original image has a lot of ambient lines or shadow dots (noise), then the final image will be particularly dark in color. But if the original image's line drawing is clean lines without dots, noise or shadows, this situation won't occur. I've already uploaded it to my gallery. You can take a look. Is this a problem with the training set? I hope version V2 can be improved. I'm really looking forward to this lora update
Thank you very much for the great lora. its really good. but the problem I had with it is inconsistency, how can I define the color of the hair skin and clothes to match the author ones ? I tried with the prompt it gives you the color but messes up the whole panel, tried changing the seed but still, is there a way to give it a reference image for the male character and a reference image for the female character and it takes the outfit and hair and skin color from it ? thx