Why ?

Deep Learning generative AI training requires a large image dataset; the quality of the dataset significantly influences the quality of the results. Many datasets are available for download, but most of them are synthetic (AI-generated). The goal here is to be able to generate a large image dataset from the real world.

The main problem we face is the difficulty of legally sharing large image datasets, as most images on the internet are copyrighted.

The tool we propose here automates the following processes:

• Download a large dataset of images based on automation of a set of searches on web image search engines.

• Exclude images not suitable for training:

  • Remove images too similar based on their perceptual hash distance.

  • Remove watermarked images.

  • Filter using lavis img2text question/expected answer, allowing to keep only the images representing what you want to train for.

• Crop & Resize:

  • Downsample very large images.

  • Face crop.

  • Remove borders.

MagicDataset, available at https://github.com/Cloud-Tomat/magicDataset/,

Key Features

• Automated Downloads: MagicDataset automates image downloads from Yandex Image Search

• Duplicate Filtering: N identifies and filters out duplicates downloaded images

• Watermark Detection: Watermarked images are filters out

• Smart Crop & Resize: This featurecrops and resizes images ensuring that the faces of people remain in the crop.

• Q&A Filtering: Using img2txt BLIP MagicDataset filters images based on question / answers to only include relevant image based on what you plan to do with the dataset.

• Smart Captioning: the tool also offers captioning based on custom templates.

How ?

Search

First, you need images. Suppose you're interested in gathering a wide range of images depicting men . In your YAML configuration file :

search:
  enable: true
  engine: yandex
  targetDir: &outDir "output"
  minResolution: 2
  searches:
    - [photo of a young man, 150]
    - [photo of an elderly man, 150]
    - [photo of a man smiling, 100]

This setup tells MagicDataset to automatically download 400 images from Yandex searches, each with a least 2M pixel resolution.

Duplicate Filtering

After download process you will likely have many duplicate image downloaded from different sources. This filter allow to discard duplicate image. detection is based on hash distance:

duplicateFilter:
  enable: true
  sourceDir: *outDir
  threshold: 5

Watermark Detection

This filter just detects if there is a text in the central part of the image and isolate them

watermarkFilter:
  enable: true
  sourceDir: *outDir
  searchCropPercent: 70

Smart Crop & Resize

This part does the following:

• Crop to the maximum possible size to have the specified homothetic ratio.

• Resize the image to the specified dimension.

cropAndResize:
  enable: true
  sourceDir: *outDir
  resize:
    enable: true
    dimension: [512, 512]
    keepFace: True

This configuration ensures that images are resized to 512x512 pixels while prioritizing keeping the face in the image.

Q&A Filtering

Q&A Filtering uses img2txt technology, BLIP, to filter images based on semantic content. Here’s how you can effectively use this feature:

• Defining Clear Questions: example : “What emotion is the person displaying?”

• Expected Answers: Alongside each question, you provide a list of acceptable answers. This list guides the filtering process, ensuring that only images meeting your criteria are retained. For example, if you’re only interested in images where the subject is happy, your configuration might look like this:

lavisFilterAndCaption:
  enable: true
  questions:
    - question: "What emotion is the person displaying?"
      expectedAnswers: ["happy", "smiling"]

Smart Captioning

Smart Captioning automatically generates descriptive captions for images based on a template you define.

• Creating Detailed Templates: Your caption template should include placeholders { } for the BLIP model to fill in, based on the image content.

• Example of Smart Captioning Configuration: Suppose you want to include details about the subject's appearance and the setting in your captions. Your YAML configuration might look like this:

lavisFilterAndCaption:
  enable: true
  caption: "This is a photo of a man, showing {What emotion is the person displaying?,1,2} emotion. He is wearing {describe the clothes.,2,5}. The photo is taken in {describe the setting,2,5}."

In this template, {What emotion is the person displaying?,1,2} instructs the tool to identify the emotion displayed by the person in the photo answer will be minimum one word length and maximum two words describing this emotion in the caption. Similarly, placeholders for clothing description and setting provide additional context.

Open Source and Contributions Welcome

MagicDataset is a free, open-source tool, and we welcome contributions from the community. Whether you're a developer looking to add new features, fix bugs, or improve the readme, or you're a user with feedback and ideas on how to make MagicDataset even better, your input is invaluable.

Getting involved is simple:

• Check out the project on GitHub: https://github.com/Cloud-Tomat/magicDataset/

• Look through the issues to see if there's something you're interested in working on.

• Feel free to fork the repository and submit pull requests.