@TheEliteChief01 Thanks! I don't think that matters much, to be honest. Unless of course you want to avoid the AI learning the subject's teeth for whatever reason. ;P

@Benkei244 As @asdfreasy was saying, the training images will mostly look pretty bad. They're just an indication, at best, of how the training progresses, and you'll need to test different steps in order to ascertain how good the TI is and what step is the best among them.

What I usually do is training until step 1000, then check that step to get an idea. Usually it needs more training, but at times it'll look like it's a bit overtrained (I judge this based on different signs, particularly the eyes, but here experience is what eventually will help you most). If it needs more training, I'll give it 500 more steps and check again. If it looks overtrained, I'll test previous steps. Every now and then, step 1000 looks amazing and I will settle for it. 😉

@Benkei244 Yes, the token count's effects on the TI's quality has been the subject of quite a lot of discussion. Apparently, the official documentation and some of the earliest tutorials on how to train embeddings recommended not going over 4 or 5, and preferably sticking to a maximum of 2 tokens (which is the number I'm now using, and I'm quite happy with the results). But then again there are countless examples of excellent TIs that use a higher count.

In principle, a very high count of tokens should make the TI less flexible. Partly, at least, because it leaves less space for other instructions in the prompt. But, apart from that, I'm not sure just how much of an effect it has on the quality of the embedding.

@summer4love263 Unfortunately that application is not a magic recipe either. It might be a good indicator in case you're working with standard settings (batch size and gradient = 1), but as soon as your batch size is greater than 1, vector strength isn't as good an indicator anymore. For instance, many of my TIs look better (more accurate and more consistent) around vector strength 1.1-1.3, and training them further usually leads to worse results. But I've had TIs that peaked around vector strength 0.07 and even lower.

@Benkei244 Sorry, what comparison? :D I've never trained neither LoRAs nor with Dreambooth.

Hey @0226! I've uploaded Madison Ivy's dataset to MediaFire. Hope you find it useful! Without seeing your results maybe it's too much inferring on my side, but I often get "cartoonish" results when I've overtrained a model. So maybe checking previous steps would help too. You probably do that already, but just in case. :)

@jr3 Yes, always train on the base SD model, whether it's 1.4 or 1.5 I personally prefer 1.5, but some other creators (like @Bozack3000) like 1.4 better.

@solo_lee Hey, thanks for your comment! Regarding your questions:

1) Batch size definitely affects the results, but that doesn't mean training with batch size 1 or 2 will lead to bad results. For instance, @Bozack3000 trains with batch size 1, I think, and he's one of the best TI creators I know. It's about finding the settings that work best for you and, of course, that your GPU can handle. What I'm now doing is I keep batch size at 3 but increase the gradient up to the number of pics included in the dataset (generally between 12 and 18). I'm pretty sure you could do something similar with batch size 1. For instance, batch size 1 and gradient 15. Your GPU will probably be able to handle it, even if training seems a bit slow. You'll probably start getting decent results between 500 and 1000 steps with those settings, but you'd need to experiment by yourself, as I have 0 experience with those exact parameters. :)

2) Training should always be done with either v1.5 or v1.4. Custom models are usually not a good idea for training, be it Asian, European or any other ethnicity/nationality/race. That said, maybe Chillout does work for training Asian models. I've never tested it, so I can't say. But, in principle, always use one of the base SD models for training.

Cheers,

Jernau

@grink89 With regards to the subject's expression, I tend to avoid any that distort the subject's face in a way that's basically unrecognisable or that would pose an issue for the AI to learn (I try to be as consistent as possible in the pics I choose). If they're smiling, sad or overly serious, for instance, I may state that in the captions, though. With regards to the gaze direction, as you mentioned, I try to get most pictures with the subject directly looking at the camera. Otherwise, I usually don't mention it anyway. I think it's not necessary (at least if most of the pics in your dataset present the subject looking straight at the camera).

And the contents of my custom subject filewords are as follows:

a photo of [name], [filewords]

a photo of [name], [filewords]

a photo of [name], [filewords]

a photo of [name], [filewords]

Yep, the same line 4 times. Apparently there is or there used to be an issue while training where the line could be dropped/disappear all of a sudden, and if you didn't have anything else in the file you kept training with a blank phrase. Repeating the line is supposed to solve the issue.

@grink89 Good luck, mate!

@Kinkau Yep. I never use xformers (training or otherwise). It is indeed said they affect results for the worse.

@solo_lee TIs perform differently under different models, yes. Sometimes, the changes are minimal, but often they're quite noticeable and you'll probably prefer one model's results over the rest.

I've tried numerous models these past few months. My current favorites for realistic images are ICBINP, ConsistentFactor, Avalon TruVision, Zovya's model and Cyberrealistic. I also like RealisticVision and Reliberate and use them every now and then. But if I had to choose just one, I'd go for ICBINP.

@solo_lee Steps are not very important. Epochs are a bit more so, as they relate to the number of times the whole dataset is processed every step. In my case, most of my TIs seem to reach their 'peak' at between 300-500 epochs (with exceptions, but as a general rule). With my current settings, that usually entails around 150 steps or so. Why? Because, as you said, I'm keeping my batch size at 3, but my gradient now equals the number of images in my dataset (usually around 15). Which means that, for every step, I run 3 full epochs. Previously, my datasets often consisted of 18 images. And I trained with batch size 3 and gradient 2, which meant that I ran 1 epoch every 3 steps. Basically the opposite of my current numbers.

The reason for the change is, simply put, that results seem better. I also feel like I have more control on when to expect good results (always between steps 130 and 200, often between 140 and 160). Since I'm saving every 10 steps, I can quickly determine which is the best version. So far I haven't trained a single TI with this method that I felt wasn't at least decent, so I'm really happy with it.

@tanengast Hey there! You can support me through Paypal (http://paypal.me/jernaugurgeh) or through my website (https://www.buymeacoffee.com/jernaugurgeh). If you got a specific request for a TI, please let me know! :)

@formertwitteremployee I wouldn't say there's a strict definition of "overtrained". At least in my case, I say a model is overtrained depending on a variety of factors. For instance, when the images of a subject have been processed so many times by the AI that the embedding progressively starts to ressemble said subject less and less, I think it's safe to say the model is "overtrained". But also when some artifacts appear they may be a sign of overtraining. Sometimes it's really difficult to judge when an embedding has peaked, but if you can pinpoint the exact step, obviously everything that comes after is overtraining.

In case you train an embedding with default settings (batch size = 1, gradient = 1), it is usually said that vector strength 0.2 signals the doorstep of overtraining, so to speak. You can install this little script to check the vector strength of your embedding.

@hourglassfcupslimwaist For the first embedding I ever tried to train I used a custom model (I think it was Protogen, back in around January this year). It looked horrible and I learned the lesson. Training can only be done with a standard SD model (1.4, 1.5, etc.). Using a custom model will usually lead to disastrous results (even though I think my friend @Scottymac has had some degree of success using certain custom models, but I can't tell for sure which or how replicable the method is).

@taterdots Both batch size and gradient affect how the TI is trained. For instance, my usual numbers (batch size 3 and gradient 15) mean that the AI processes a total of 45 images every step. Since my datasets consist of 15 images, that means 3 epochs per step (an epoch is a cycle of the AI examining/processing all the images in the dataset). This is a considerably high amount of training, compared to normal settings.

I discovered through practice that my TIs always peak at around 400 epochs when I train with batch size 3 (even when my gradient was as low as 2--back then, I used to need around 1200 steps to get the best of my embeddings, wich equaled 400 epochs too because my datasets consisted of 18 images). Since then I've basically disregarded vector strength, which I think is only useful as a measure of how well trained the embedding is if you're working with the standard parameters (including batch size 1 and gradient 1).

@hourglassfcupslimwaist I really can't say why they don't work. My 'theory' is that custom models basically just modify what the base model does, by means of applying a certain style. So perhaps they're so much leaning towards that certain style that whatever you try to train will never look realistic. I'm pretty sure there's a better explanation out there somewhere, but I never cared enough to look for it, to be honest. Since, with a little luck and skill, you can achieve very good results with the standard models, I was always happy to stick to them.

I'm not sure whether you're asking about training an embedding with a given model or about testing an embedding with it, though. If it's the latter, as long as the TI has been well trained (which of course includes training it with a standard model), it should work pretty well with most models. There are some, however, that exert a greater influence on the composition, and thus may alter your TI in a way that makes it look less accurate. I've had that experiennce with several well-regarded models, some of which I do enjoy a lot, but which definitely do not treat embeddings (at least subject embeddings) as good as I'd like.

@formertwitteremployee 

4. No, the trigger text is mandatory, you cannot train an embedding without one. In Automatic1111 you have an "Initialization text" field. You can see a screenshot here. And here is the explanation of what said initialization text is for. It's a text that will be used for training the embedding (preceding the text of the image captions if there's any). The explanatory text warns you not to use an excessively long initialization text--the number of words should never be greater than the number of tokens you've selected. I leave it empty for several reasons: first, because I prefer to start with totally empty vectors, so that my TI does not 'inherit' any of the vectors atttached to the words in the initialization text. (The vectors are the instructions the AI follows when trying to generate an image of a specific word you typed in a prompt.) And second because I always use captions, and these include all the text I need to train the TI.

6. Not the filenames, no. My dataset folders include a text file for every image, named exactly the same as the corresponding image. These textfiles are a more or less precise description of what the corresponding image shows. Here's an example of one of my latest TIs.

By the way, I've edited the opening post with my current settings. It was a bit outdated.

@taterdots Fixed that. Thanks for the heads-up! :)

@TheUnpossibleDream Thanks for the comment, mate! :)

So, as far as I know, the initialization text represents the vectors that will be used when you start training your embedding. For example, if you use the words "tom cruise", your embedding will start its training using the vectors of Tom Cruise already present in SD 1.5 (or whichever model you use to train it). That means your embedding will resemble Tom Cruise at the beginning of the training process. Then it will start processing the images in your dataset. If your images are of Tom Cruise himself, then maybe it was not a bad idea (although, if you want to create a completely new version of Tom Cruise, you shouldn't do it either). But if your dataset contains pictures of, let's say, Brad Pitt, it's obviously not a good idea unless you're aiming to get a fictional character based on the looks of both Cruise and Pitt. Hope that makes sense. :D

I do remove the star (*). Why? Because even the star has some vectors associated to it. If you want a clean slate, if you want your embedding to resemble the pics in your dataset and those pics only, then it's better to leave the initialization text completely empty. It does little difference to leave the star anyway, but details matter. :)

Hey @masfam! I always have the VAE (vae-ft-mse-840000-ema-pruned.ckpt) and the cross optimizations turned on while training indeed.

Hi @saintmaresgerard760! A1111 works perfectly for me (my Python version is 3.10.6, in case that helps). About other options, some people use Comfy UI, but I've never tried it. Perhaps someone who's used it could offer you some advice on this regard. Good luck, mate!

@saintmaresgerard760 The important part is that you find a method that works for you. I don't use only closeups, though. My datasets are usually composed of upper body/from the chest up shots (like 60/70%), the occasional full body shot and some closeups. This also varies depending on the subject. For instance, for a recent TI (Raica Oliveira) I used like 50% full body shots. I gotta say my method works for me 99% of the times. And when it fails I know for a fact that the dataset is at fault. With better pics, it would work. The only problem, when you're beginning, is determining which is the best step, but with some experience and patience you'll learn to locate it. For me it's always between steps 120 to 145, and normally between 135 and 140.

But, as I was saying, your mileage may vary depending on so many factors, and the only thing that matters is that you find a consistent method that you can rely on.

@saintmaresgerard760 Because that was like 7 months ago, mate, and my method has changed a bit since then, when I used to train with batch 3, gradient 2 and a dataset of 18 images. Which meant that I was running 1 epoch every 3 steps... Therefore, step 1000 was like 333 epochs, which is quite close to where my current "sweet zone" begins. Usually I would find the best step around step 1200, which is exactly 400 epochs. Today, my best step is often 135 or 140 (405 and 420 epochs, respectively). So ultimately not much has actually changed since then.

@masfam Why not. I mean, the most important part is that they're quality pictures. The full body shots are generally less useful for my embeddings, who are all celebrities (usually actresses or singers). Getting their faces right is the most important part. I don't follow strict numbers, but I do give priority to closeups and medium shots compared to full body shots. Most of the times the AI gets the general body type quite well with just the medium shots.

@mangaba12000275 Hey mate! No, I don't use neither xformers nor the TensorRT extension. For 140 iterations, my usual training times are around 8-10 minutes. 160 iterations would just add 1 or 2 minutes tops. I guess the hardware does make a lot of difference in this case. If it proves too much for yours, I'd reduce the gradient number to 5 (provided you're reproducing exactly all of the other steps in my original post). By doing so, with a dataset of 15 images, you'd be running 1 full epoch every step. Thus, I would probably train until about step 450 and pay special attention to steps 400 until 420, cause that's probably where you'll often find the best version of your TI.

@dilectiogames I've never used kohya, so unfortunately I can't offer any advice on the subject.