WARNING!
I just discovered that my rendering program (Stable Swarm) seems to do different things, based on what the embedding file is NAMED.
I can take the exact same file, name it "cat.safetensors", and get different results from if it is named "synthcat.safetensors".
My sample image was generated with "cat.safetensors"
This is a strange experiment that I created by hand via python code directly. It was SUPOSED to just generate cats. But sometimes it generates female portraits, sometimes catgirls.. and yes, sometimes actually cats. But rarely.
So far, works best with aniverse 1.5 and NUKEcolormaxAnime.
If you use keyword 1girl or 1woman, it can make for more interesting results.
Note that, in stable swarm at least, there is a HUGE difference in results between
"1girl<embeding:synthcat>"
"1girl <embeding:synthcat>"
"1girl,<embeding:synthcat>"
Warning: using easynegative tends to make it slutty.
Description
FAQ
Comments (6)
I must admit, I'm wildly curious about the direct-from-code story. Any additional details you'd be willing to share about the project?
Code is at huggingface.co/datasets/ppbrown/tokenspace/blob/main/generate-embedding.py
Lemme know if you do anything fun with it :)
@phil866 So!
Playing around with your linked code sent me on quite a tangent. I searched around for any literature or similar code and installed the Automatic1111 extension Embedding Merge (https://github.com/klimaleksus/stable-diffusion-webui-embedding-merge) that I think(?) accomplishes much the same thing (generates a custom set of vectors based on input prompts) by querying CLIP.
It spits out embeddings in .pt format, so I use a modded version of this workbook to convert to .safetensors format (https://colab.research.google.com/github/DiffusionDalmation/pt_to_safetensors_converter_notebook/blob/main/pt_to_safetensors_converter.ipynb#scrollTo=UwH1lXmGw9XP), altered slightly so it runs locally, plus some minor enhancements so I can pass files in directly from the command line.
A lot of my curiosity stems from wanting a way to reverse-engineer tokens from a LoRa (presumably merged with a base model) that are more-widely applicable, but unfortunately, LoRas don't update the text layer that I'm aware of.
I do wish more folks would train textual inversions (tell a model how to find a thing it's definitely seen before but might not have words to describe; e.g., a specific human being, of which it's seen millions) rather than defaulting to LoRa creation (used to add something truly new to a model; e.g., a specific building it's never seen).
Thank you for sharing this!
@sendmefreestuffa8246 "It spits out embeddings in .pt format, so I use a modded version of this workbook to convert to .safetensors format" ??
Booo, inefficient! My coder heart cries at this. Just mod the github code to write it out in safetensors format to start with! :-)
"unfortunately, LoRas don't update the text layer that I'm aware of."
Yah. Not that I really understand loras that much :) but I think its more like an img2img process.
Its just that you have multiple image fragments, rather than a single source image.
@phil866 Updating the extension code occurred to me, but gets overwritten with a git pull, plus I have other .pt that needed converting; having a standalone conversion script behooves me at the moment :)
I may still end up modding the extension and simply saving my edits in a non-clobbered directory. Guess I could send 'em to the author, too, hah.
@sendmefreestuffa8246 the nice, proper github way, is to fork the code, add the feature in a FLEXIBLE way (eg: only trigger if filename ends in .safetensors), and then submit a pull request to the mainline.
If it is accepted: great! the functionality gets added.
If it doesnt.. just git clone from your repo from then on. you can automatically merge in changes from "upstream" and it will keep your change intact. You win either way.
Details
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.