Preface

I used the first version of these models in version v1.0 for learning and testing purposes. In parallel I wrote an article how a panama hat can be modified by a LoRA model [1].

Approach

So far I am using AUTOMATIC1111 and the Extension TrainTrain for the training of the LoRA models. Other approaches for a LoRA training did not work or did not work as expected on my local installation.

What the Model Does

The LoRA model creates a panama hat, when the weight is set to 1.0.

If the weight is less than 1.0 then the LoRA model can change a panama hat in a given image.

The model is not working at the moment as I would expect it.

Trigger Word

The required trigger word is

panama hat

In version 1.3 I changed the trigger word to

panamahat

Versions

The version which is not working so well is version v1.0 The other following versions can be tried in parallel. It feels like they produce better results.

Version v1.3

Maximal training steps were 5000. I stored versions in steps of 500. Trying version by version the version with 1000 training steps resulted in the best results.

Test Case

Using following Prompts are good testing cases for the LoRA model:

 panama hat, <lora:PanamaHatConcept_v13:[1.0...1.4]>
 panamahat, <lora:PanamaHatConcept_v13:[1.0...1.4]>

A weight of 1.0 up to 1.4 is working. With a value of 1.5 overfitting takes place.

One should get a panama hat as in the following image:

Training Images

I started the training with 20 images and I used finally 36 images. The used training images shows a panama from different viewing angles and positions. The background of all images is bright and single-coloured.

Limitation

You have to bear in mind what a LoRA model does when it comes to limits. If the LoRA model is not used and the base model is not knowing what a panama hat ist, then it is not possible to change a not existing panama hat.

Final Words

Have a nice day. Happy model testing. Have fun. Be inspired!

Reference

[1] https://civarchive.com/articles/7481/using-lora-to-change-details-in-a-photo

Donation

About each Buzz I am happy! Yellow ones are very welcome!

I am also happy to accept donations in one of 250+ crypto currencies:

https://nowpayments.io/donation/zentrocdot

Leave optional a comment like improve documentation or improve model etc. when doing so, that I know in what you are interested in.