i did add a LoRa

Send me an email to ClassicRPGArt -At- proton.me

it is, for now, 1.5 as a base but once all the training data is in I can also do a 2.X version, 1.5 is better, for now, to work with during development and has a wider range of data in it.

These are the main points that you have two keep in mind when working with training subjects outside of the photorealistic realm.

1. make sure that your dataset is consistent in regard to brightness and contrast. If you're working with line art drawings, black should be black, and white should be white; otherwise, you are training the model to generate the noise of your paper and the imperfections of the ink as part of the learning data, which is an artefact that later on will inherently reduce the usefulness of the model.

2. Clean up all images that you work with. Images shouldn't contain anything that is not part of what you like a model to learn. A typical mistake to avoid is having lines of image borders, speech bubbles, or several images from the comic together on one training data page. Work either with a white or black background and simply photoshop everything away that you don’t want to have as part of the training data. it’s better to have empty space on a training dataset image than any kind of artefact because blank background spaces (hex value FF.FF.FF or 00.00.00 ) will not be learned if your tag description mentions like a white background. While artefacts will compromise your network.

3. remove noisy and low resolution images from your dataset. Either by not including them or using AI to upress your image (SD batch scaler in A1111 or Topaz Gigapixel AI). I would recommend that the images in your dataset should be two times the size that you are training the model at. This means if your model is working at 512x512 you should have a dataset of 1024x1024 images.

4. Make sure that every one of your images has a correct set of tag’s. you can automatically generate tags with the smart process add-on for A1111. I would recommend setting the threshold to something like 0.25; the default is 0.85 and by far too high for working with intricate art . don't be afraid to use your text editor or the tools in A1111 Two added to the text files remove tags that you don't like can definitely add tags of the subject domain. Meaning names of places, persons or events that are visible in the images so that you can later on recall those concepts from your promps. as an example, if you have an image of Arnold Schwarzenegger and don't tag him as terminator you won't be able to use the word terminator and the picture of Mr Schwarzenegger will just have been trained as muscular body build up on a movie set. Which is completely useless as training information if you want to have useful prompts.

5. Always work with classification images in order to not overtrain and break the checkpoint with your training. Have at least 10 classification images for every image in the dataset if your dataset is large and has a great amount of tags make sure that you have between 10 and 50 times the images in your classification directory geenerated for each of your images. Make sure that your classification images have a correct instance prompt ( Instance Prompt ) always use ,[filewords] in order to extract the description and make it part of the training do that for Instance Prompt, Class Prompt and Sample Image Prompt

6. make sure that your instance prompt is a continuous string that is not part of the actual clip of your checkpoint otherwise, you destroying parts of the checkpoint with your training data, in most cases a combination of the first two letters of a sentence that describes your training data gets a rather random string that is not overwriting anything. This is really important if you make a mistake at this point; your model will only show you back your training data and not allow you to mix it with anything else it knows in the correct ratio.

7. use negative classification image prompt is two avoid art styles or artists or concepts that are counter-indicated to what you’re trying to train for example, if you are training a western art style in the marvel look make sure that you have, for example, manga as a negative prompt in order to not get the training results confused.

8. Make sure that you don’t overtrain your checkpoint. What you should do is let’s say for 300 training images set the “Training Steps Per Image (Epochs)” two 500 iterations per image and put the model saving frequency to something like 50 and the preview frequency also. Make sure that you say in the setting that it should savour checkpoint every time while training. And activate the checkbox for safe training data for later use by setting “Save separate diffusers snapshots when training completes.” what will happen now is it will go from 0 to 500 iterations and create 10 checkpoints on the way. You should also set the preview sample number to something like eight images so that you see something. Now you just let it run and you will see in the sample images at what point your network gets overtrain by your results degrading again. Then take the checkpoint that looked good before the one that you saw the degeneration and try to merge both if the resut odd looks better than each one of them then you have the best result that you can possibly get if the result looks worse, just go with the last good looking checkpoint and that is it. You’ve trained your checkpoint on your data in the correct way and have the “Best” result for the time being.

9. Very important last message info make sure that you have gigabytes of free hard drive space each checkpoint is between 4 and 8 GB and in the current setup of the example this will result in about 40 GB of checkpoints if you run out of disk space while training everything crashes and fails!!!!

10. don’t be afraid to hit the cancel button if anything looks odd or the model training is not progressing in the sample images. Just cancel everything and start over fresh there is no point letting your GPU waste energy for eight hours on an already failing set up! I mean it if something looks rather ood, it probably is wrong start over. However, don’t do this before you had at least two checkpoints look at but if you don’t see the improvements in the smaple images, just have a look at the setup and do it again with better values.

if you have any further questions feel free to write me an email “ClassicRPGArt -at- proton . me”

風の谷のナウシカ (宮崎 駿)