Anime name is all you need: A fully automatic process for anime text-to-image dataset construction

I have improved my dataset construction pipeline (github: https://github.com/cyber-meow/anime_screenshot_pipeline) to make it fully automatic!

You can now get everything done by just entering the anime name.

python automatic_pipeline.py \
--anime_name name_of_my_favorite_anime \
--base_config_file configs/pipelines/base.toml \
--config_file configs/pipelines/screenshot.toml configs/pipelines/booru.toml

Here is a demonstration.

The dataset construction is split into 8 stages

• Anime and fanart downloading

• Frame extraction and similar image removal

• Character cropping

• Character classification

• Dataset image selection and resizing

• Tagging, captioning, and generating wildcards and embedding initialization information

• Dataset arrangement

• Repeat computation for concept balancing

The script contains more than 100 arguments that allow you to configure the entire process on your own. It is compatible with all mainstream trainers including Everydream2, Kohya trainer, and HCP-diffusion. It is designed with pivotal tuning in mind.

Moreover, you can decide yourself which stage to start from and which stage to end at to perform manual inspection between different stages to further improve the quality of the dataset!