CivArchive
    ← All articles
    Published September 21, 2024by zentrocdot

    Common Used File Types

    89 views4 reactions0 comments on CivitAI0 collected
    resource guidefile structure

    Foreword

    While working on some small tool for analysing models I had to learn, that it makes sense to make a small list with the file types and how they are organised.

    if you are not busy with the different file types around the clock, you can get sometimes a little bit confused confused. This is my first attempt to summarize what I have seen and learned.

    File Types

    Possible file types while working with models are:

    • .pt

    • .pth

    • .pwf

    • .bin

    • .ckpt

    • .safetensors

    File Types Explained

    The file types .pt and .pth are nowadays zip archives. In the past they were binary files. Today such files have the file extension .bin and are binary files.

    The new file type extension .safetensor is related to a new secure binary file format. This is quite different from the previous named .pt or .pth.

    The files with the file extension .ckpt I have seen are zip archives. .ckpt is simply the same as .pt or .pth. It is a different way of file name convention.

    The file extension .pwf can be found and is nothing else as .pt and .pth. It seems to be named by PyTorch Weight Format to state that there are weights in it.

    Magic Number

    So far I have seen no file type related to AI model creation that has a so-called magic number about the file can be identified.

    Side Note

    The file extensions says nothing about a file and how the file is organized. Everybody can use file types he likes.

    To-Do

    I will update this article from time to time

    Finally

    Have a nice day. Have fun. Be inspired!

    Reference

    [1] https://pytorch.org/tutorials/beginner/saving_loading_models.html

    [2] https://github.com/pytorch/pytorch/blob/main/torch/csrc/jit/docs/serialization.md

    [3] https://github.com/huggingface/safetensors