This embed is designed to nudge SD towards a better fighter jet solution. Is it perfect? Far from it. I've yet to get something that is spot on and at best it produces something "fighter jet-esque" but it does seem to work better than just putting in "fighter jet" alone.
As with most things, your model and seed will really impact this. The images here were created using Deliberate 1.1 in Automatic1111, save for the last image, which was done in A1111 and Invoke with a lot of inpainting.
As always, your mileage may vary. Posting here in case it helps someone.
Description
This model was trained using a 120 frame animation of an F-15 fighter jet. The model was animated in Blender and rendered at 512x512 resolution. The animation was created after several failed attempts at using stock photos. I figured the lack of coherence between photos was making it hard for SD to understand the true form of the aircraft, so I animated the ship to give the best training chance possible.
I ran the training up to 10,000 steps, but found that after about 3,000 steps the model was overtrained, I wasn't getting a better F-15 image and it overpowered any prompt I put in. After several tests, it turned out that the 250 step model was actually the best. Go figure.
FAQ
Comments (12)
Ok, right off the bat, I gotta say awesome! Definite download for me. If you're on a kick, may I suggest WWII mono-wings? I have such a soft-spot, and doing some awesome WWII dogfight stuff sounds cool.
Happy to give it a go. Can't make any guarantees as to the end quality. TIs don't seem capable of nailing down the complexities of hard surface objects. Shoot me the names of a few planes though. If I can find some 3D models, I'll put out the animations and do the TI training.
@SteveWarner That would be awesome! Thank ya in any case. Here are some suggestions:
https://sketchfab.com/3d-models/p-51-mustang-dbb4a717a4c141f9bf0869bf1ce74529
https://sketchfab.com/3d-models/supermarine-spitfire-8349f26e1e88455da75dd7352b02b794 (Couldn't find it animated within quality threshold)
May work better as a lora. I trained a lora on Super Tucano attack plane, but didn't post lora here yet, need improve some things.
WW2 planes - Yak 3/6/9 fighter, La 5/7, Mitsubishi Zero, P47 thunderbolt, FW 190, Me 109,,, etc
@peele Do you have any tutorials, youtube videos, etc. on Lora training? I've read several guides and it doesn't seem like my 8GB vRAM is going to be enough. I'm trying to avoid having to use an outside service for training at the moment.
I think the issue right now is that I'm thinking of "objects" of "subjects" as 3D objects, meaning full turnarounds from every angle. When you do a matchmove or a photogrammetry solve, you're trying to give the software as many camera angles as possible for it to "understand" the 3D shape of an object. I don't think that's what TI or even Lora training is doing. I think it can handle some small amount of variation in angle or orientation, but not full 360 rotations, and that's how I've been approaching things.
I'm going to try to train an F-15 on just a few images (rather than 100+) and I'm going to limit the view to top isometric and see if I can get it to understand that. If that works, then I'll do separate TIs for each main view. Not ideal, but I have this nagging feeling that SD simply doesn't "think" in 3D like I'm used to with other software.
@SteveWarner If you have LORAs set up that you couldn't train cause of vram, I may be able to help. I haven't even looked into those yet, but I do have a 4090 with 24gb VRAM.
@WAS That's awesome! Thanks for the offer. I'll certainly keep you posted!
@SteveWarner , I used free google colab, compute time was about 30 min or so... used this guide https://i.imgur.com/J8xXLLy.png, using this notebook GitHub - Linaqruf/kohya-trainer: Adapted from https://note.com/kohya_ss/n/nbf7ce8d80f29 for easier cloning
Saw a video somewhere about using 7gb vram on youtube, don't have a good Nvidia GPU myself so didn't try it.
@peele Awesome. Thank you so much! I'm out for the day but will dive into this and see. 30 minute training beats the 2+ hours I have to dedicate to a TI.
Great stuff... we need more aircraft and other weapons , base SD is quite poor at these
Couldn't agree more. I think this TI was a very loose success, in that it produces something airplane-esque. I'm still working on learning the TI process. Hoping to be able to avoid building a whole model for these things, as TIs allow you to work on top of any model.
Details
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.