CivArchive
    ← All articles
    Published December 20, 2024by Tranchillo

    Image Filter

    58 views3 reactions0 comments on CivitAI2 collected
    image dimensionsautomationtraining datadata preppython utilitydataset preparationimage processingdata preparationimage filtering

    # Image Filter Script: Simplifying Dataset Preparation for LoRA Models

    When working with machine learning models like LoRA, preparing high-quality datasets is crucial. One common challenge during this process is the time-consuming task of filtering images to meet specific dimension requirements. To address this, the Image Filter Script was developed.

    This Python script automates the process of selecting images based on their dimensions, ensuring that only those meeting the minimum criteria (512x512 pixels) are retained in a separate folder. With parallel processing capabilities, it efficiently handles large image collections, making it an essential tool for data preparation.

    ---

    ## Why This Script Was Created

    During the preparation of datasets for LoRA models, it is common to download entire galleries of images, often containing files of varying sizes. Manually reviewing and selecting images that meet the minimum size requirements can be tedious and error-prone. This script was designed to:

    - Eliminate manual filtering efforts

    - Ensure consistency in dataset quality by adhering to specific size requirements

    - Provide a fast and automated solution for dataset preparation

    - Process large collections of images efficiently using parallel execution

    ---

    ## Key Features

    The Image Filter Script offers the following features:

    - Simple Usage: Just place the script in your image folder and run it

    - Automated Filtering: Scans the current working directory and checks each image's dimensions

    - Dimension Validation: Ensures that both width and height are >= 512 pixels

    - Safe Copying: Copies valid images into a new subfolder named "ok," preserving the original files

    - Parallel Processing: Utilizes multiple threads for faster execution

    - Real-Time Feedback: Displays progress and error messages during execution

    - No Configuration Needed: Automatically uses the current directory for processing

    ---

    The script can be downloaded directly here in the article as a ZIP file or from the following GitHub link[Image Filter GitHub Repository]: (https://github.com/Tranchillo/image-filter/tree/main).

    Feel free to customize the script to fit your specific needs and share your experiences or improvements with the community!

    Attachments