LoRAs just do that. You can try inpainting your old images, only changing the areas you want the LoRA to affect, but for generating an entire image, the mere presence of a LoRA changes everything.

@A1322 Is there any tutorial specially of doing that?

@ProvenFlawless I've kind of figured it out on my own without a tutorial, but the way I'd summarize it is that when you're inpainting, you don't have to keep your original prompt, because that image is already safely generated. This means you can use inpainting to give specific micro-prompts to smaller areas.

Doing this enables a few things. One is that it lets you have things affect one character instead of all characters in the image. Normally if you mention sunglasses, everyone will be wearing them. You can instead leave them out of the original prompt and inpaint them in later. But you can also use it to trade out LoRAs, only using the ones you need for what you're doing right now and setting aside the ones you don't. You can even use this to transform people in your image with character LoRAs. (You'll probably want to use ControlNet's OpenPose if you do that. It's actually shockingly easy to use and extremely powerful, so it's worth learning anyway!)

Getting a realistic face to really match when you inpaint it can be tricky. ADetailer is great for making anime faces fit into an image, but not realistic ones. The mediapipe models may give better results than the face_yolo models for realistic images, but TBH I've never really tried it. (Restore faces is always trash for everything.)

Another thing that will help with faces is using "only masked" mode instead of "whole picture", but the learning curve for "only masked" is steep. It gives you much, much more detail for small areas, which is important for a face, but consistently getting good results with it is its own tutorial.

Also, be 100% absolutely sure you're using an inpainting model. If a checkpoint's inpainting model isn't available on Civitai, they're actually very easy to make: How to turn any model into an inpainting model

That's a lot more than you asked for but I hope it helps!

Or you can use ADetailer, it kinda inpaints automatically. Or use/create LoRAs with databases excluding faces.

@olaz @A1322 you can even have Adetailer use a different checkpoint for a more realistic face. It works really well.

@Lja I've actually been doing this quite a bit lately. Pony models can be incredibly finicky about inpainting (which is what ADetailer does) and it's often easier to load up a SD1.5 model with a similar aesthetic and take things from there.

This isn't really useful for regular inpainting, but it's fantastic for ADetailer.