Thanks—a Q4_K_M .gguf would be great! :) For the poor among us. ^^

@ratwatat you can use ggufy to create gguf versions: https://github.com/qskousen/ggufy . I have not used it myself but I heard it wasn't too hard.

We are both on 3090s so we haven’t tried that. But from what I can tell based upon recent research I have read, the later layers appear to affect the output most. So if you were to be selective with the process, I would imagine it would have very little impact up to layer 12 or so. But I’m just speculating here. We are also not going to be able to do it ourselves for at least 2 months due to being on vacation. So feel free to modify the model yourself to fit your hardware budget.

Your next question is a bit easier to answer.

While we haven’t tested every text encoder, we can definitively tell you what the difference is between our two releases and most other versions.

Specifically we modified the code for AI Toolkit to allow for simultaneous training of both the text encoder and the transformer inside of the image generation pipeline on a single 3090.

Comparatively, other text encoder versions would have been trained with text only, inside of a standard LLM training pipeline.

This essentially means that our model is specifically fine tuned with image generation in mind, whereas any improvement in image generation from other models would almost certainly be coincidental and/or luck dependent.

@Triple_Headed_Monkey Great, thanks for the reply. So would those benefits apply to any checkpoint used with your Text Encoder, or only if we use your Checkpoint? I'll run some tests but it sounds very interesting.