How to Train LoRA Using Flux1-dev2pro with ai-toolkit

Hey there! So, you’re pumped about training LoRA models but not sure how to make it happen with ai-toolkit? No worries, I’ve got you covered! Whether you’re just starting out or already familiar with LoRA training, this guide will show you how to integrate the powerful Flux1-dev2pro into your setup. Let’s dive in and make it fun!

What You Need:

Before we get started, make sure you have the following handy:

1. Flux1-dev2pro (More on this: Why Flux LoRA is Hard to Train and How to Overcome It)

2. ai-toolkit (Get it from GitHub)

3. Flux-dev (From Hugging Face: FLUX.1-dev)

4. Flux1-dev2pro (Also from Hugging Face: Flux-Dev2Pro)

5. https://civitai.com/models/808669/flux-dev2pro-fp8-special-use-for-training-flux-lora

Step-by-Step Guide:

1. Install ai-toolkit

First things first, you need to have ai-toolkit installed. Just clone it from GitHub if you haven’t already:

git clone https://github.com/ostris/ai-toolkit
cd ai-toolkit

Install any required dependencies and you’re good to go!

2. Update the .yaml File

Now that you’ve got ai-toolkit up and running, let’s configure it for Flux1-dev2pro. Open the .yaml configuration file in your project. You’ll want to modify the name_or_path field to point to your local Flux model.

Example:

model:
  name_or_path: /path/to/local/fluxmodel

This tells ai-toolkit to use the local version of Flux1-dev2pro.

3. Swap the Transformer Model

Here’s where the magic happens. To optimize your LoRA training setup, you need to swap out the default FLUX.1-dev transformer with the upgraded Flux1-dev2pro transformer. Locate the transformer directory in ai-toolkit and update it.

Replace this:

FLUX.1-dev/transformer

With this:

Flux1-dev2pro/transformer

Boom! You’re now using the enhanced Flux1-dev2pro model for training LoRA. 🚀

4. (Optional) Use fluxgym

If you want to take things to the next level, you can also try using fluxgym. It’s a gym environment tailored for advanced users who want more control and insight into their training process. You can find it here: fluxgym.

Wrapping Up

That’s it! You’re all set to train LoRA using Flux1-dev2pro with ai-toolkit. This method gives you the flexibility to customize and supercharge your models, making it easier to achieve great results. Now go have fun experimenting!

感謝 ai-toolkit / @jaretburkett ||

----

💡 Spread the Love!

If this guide helped you out, don’t forget to hit the ❤️ button and drop me a little tip on Civitai.com. Your support gives me the motivation to keep sharing cool training tips and tricks! Thanks a ton—LOVE YOU ALL! 🙏