Stable Diffusion Tokenizer in the browser

See the tokens used in your prompt. Useful for seeing which tokens are used for trigger words in training. Or just general knowledge about what the tokenizer is doing to break down your words into tokens.

https://sd-tokenizer.rocker.boo/

And open source: https://github.com/rockerBOO/sd-tokenizer
Uses Hugging Face Tokenizer in the browser using Rust and WASM. So it is done all locally in your browser and no servers involved. All private and no tracking.

Open to feedback or if you have any questions. Thank you!