Convert Hypernetworks to fp16

This short guide will tell you how to get fp16 hypernetworks like mine with a small python script. Knowledge of command line and running python scripts is required.

There's no reason for hypernetworks to always be fp32.

Why fp16?

Most likely the hypernetworks you find out there will be in 32-bit float tensors, also called fp32 or "full" precision. That's what Automatic1111 WebUI is using for training, which is fine. I use it as well for training, but it's overkill for inference (inference means using your neural network as opposed to training it, i.e. generating images), and actually makes it slower to load because most people already use fp16 for inference (and thus it has to do extra conversion), plus larger disk size.

During training you'll sometimes need the exponent provided by fp32 to prevent some neurons from exploding, and the extra precision is nice as well, but fp16 is enough for inference in the vast majority of cases.

Some people use bf16 for training, which is another 16-bit format ("brain" floating point). It is like fp16 except it uses 8 bits for exponent (same as fp32), so it won't blow up during training. The problem is that it has only 8 bits left for precision, so it loses precision also for inference (you can't convert bf16 to fp16 and retrieve the missing precision, it will be a pointless conversion, can't create data out of nothing). It is, of course, twice as fast as fp32 training and uses half the VRAM but it still loses precision.

Anyway, your hypernetwork is most likely fp32 at this point. And you want to shave off half its file size and convert it to fp16 because that's what everyone is using for inference. What do?

A small python script to convert hypernets to fp16

Here's a small python script I made to convert your hypernetwork to fp16:

# Simple script that converts a generic hypernetwork to fp16 (use it in same python env as where you use the hypernetwork to guarantee success). Example usage:
#
#   python hypernet16.py input.pt output.pt
#
# Please name the output file appropriately, don't just rename it later with your file manager, it gets stamped into the file!

import sys, torch
from torch import Tensor

def conv_fp16(t):
    if type(t) is dict or type(t).__name__ == "OrderedDict":
        for k, v in t.items():
            t[k] = conv_fp16(v)
        return t
    if type(t) is tuple:
        x = list(t)
        for k, v in enumerate(x):
            x[k] = conv_fp16(v)
        return tuple(x)
    if not isinstance(t, Tensor):
        return t
    return t.half()

def main():
    m = torch.load(sys.argv[1], map_location="cpu")
    out = {}
    for k, v in m.items():
        out[k] = conv_fp16(v)

    torch.save(out, sys.argv[2])
    print("Done.")

if __name__ == "__main__":
    main()

Copy-paste that into text editor and save it as hypernet16.py or something, or just download it from the attachment.

So assuming your fp32 hypernetwork is called "input.pt" in the current directory and your saved script is also there, just run it like described in the comment from the command line using the WebUI python environment, i.e. python hypernet16.py input.pt HypernetworkName.pt. You can use full paths if you want.

That's all.