<p>This article is a brief summary of how to get access to and use the Groq LLM API <strong>for free</strong>, and how to use it inside ComfyUI.</p><p>The article version can be found <a target="_blank" rel="ugc" href="https://civarchive.com/articles/4997">here</a>.</p><h3 id="heading-42"><span style="color:#82c91e">There's nothing to download. This is not a model. It's just the text below in text-format and a link back the article.</span></h3><h1 id="heading-7420">Create an account</h1><p>Create your groq account <a target="_blank" rel="ugc" href="https://console.groq.com/login">here</a>. This is all free, and you can use the API for free with some <a target="_blank" rel="ugc" href="https://console.groq.com/settings/limits">rate limits</a> to how many times per minute, per day and the number of tokens you can use. The limits are currently very generous.</p><p>As of the time of writing (2024-04-19), the rate limits are:<br /><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/92535e6a-3d62-4821-96f9-9e3da8a6238a/width=525/92535e6a-3d62-4821-96f9-9e3da8a6238a.jpeg" /></p><h1 id="heading-7421">Create an API Key</h1><p>Go to the <a target="_blank" rel="ugc" href="https://console.groq.com/playground">GroqCloud Playground</a> and find the <a target="_blank" rel="ugc" href="https://console.groq.com/keys">API Keys</a> section in the menu on the left side.</p><p><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/22e8d2e9-baa0-4244-b0ce-b5a705a84434/width=525/22e8d2e9-baa0-4244-b0ce-b5a705a84434.jpeg" />Copy the API key from the next dialogue that pops up. This is the only time you will see this code. Make sure you keep a backup in case you want to use exactly this one later.</p><p>It doesn't really matter as you can just create a new key and switch to that one.</p><p></p><h1 id="heading-7422">Testing the API online</h1><p>Go to the <a target="_blank" rel="ugc" href="https://console.groq.com/playground">GroqCloud Playground</a> and enter a system message in the field on the left side. You are now ready to test your customized chat model.</p><p>Enter your chat message in the <code>User</code> field and press CTRL + ENTER to send the message.</p><p><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/012d1fc2-87c3-4347-a73e-8527ac9453fe/width=525/012d1fc2-87c3-4347-a73e-8527ac9453fe.jpeg" /></p><h1 id="heading-7423">Get the python code</h1><p>Towards the bottom of the Playground page, there should be a "View code" button. This is showing you the python script running your query and instructions in the background.</p><p></p><p>This should pop up a window with the current code:</p><pre><code>from groq import Groq

client = Groq()
completion = client.chat.completions.create(
    model="mixtral-8x7b-32768",
    messages=[
        {
            "role": "system",
            "content": "You are an expert python programmer and your mission is to help the user with their programming troubles."
        },
        {
            "role": "user",
            "content": "I need help to write a script. Please write a hello world script for me."
        },
        {
            "role": "assistant",
            "content": "Sure, I'd be happy to help you write a \"Hello, World\" script in Python! Here's an example:\n```python\nprint(\"Hello, World!\")\n```\nWhen you run this script, it will display the message \"Hello, World!\" in the console. Let me know if you have any questions or if there's anything else I can help you with!"
        }
    ],
    temperature=1,
    max_tokens=1024,
    top_p=1,
    stream=True,
    stop=None,
)

for chunk in completion:
    print(chunk.choices[0].delta.content or "", end="")</code></pre><p>If you remove the reply, you actually have a valid code snippet that you can run locally using python to send a query to the API. Removing it, now the code looks like this:</p><pre><code>from groq import Groq

client = Groq()
completion = client.chat.completions.create(
    model="mixtral-8x7b-32768",
    messages=[
        {
            "role": "system",
            "content": "You are an expert python programmer and your mission is to help the user with their programming troubles."
        },
        {
            "role": "user",
            "content": "I need help to write a script. Please write a hello world script for me."
        }
    ],
    temperature=1,
    max_tokens=1024,
    top_p=1,
    stream=True,
    stop=None,
)

for chunk in completion:
    print(chunk.choices[0].delta.content or "", end="")</code></pre><h3 id="heading-7424">Adding your API key to your script</h3><p>The only thing you need to do now is to add your API key to your script. Add this to the <code>client = Groq()</code> line.</p><pre><code>client = Groq(api_key="gsk_xxxxxxxxxxxxxxxxxxxxxxxxxx")</code></pre><p><strong><span style="color:rgb(130, 201, 30)">Running this script in python should now work.</span></strong></p><p></p><p>Note that you will need to install the <code>groq</code> library first. You can do this by running <code>pip install groq</code>. Ideally you should have created a separate environment before running all this, but that's a separate topic. You can find a tool I wrote to help you <a target="_blank" rel="ugc" href="https://github.com/MNeMoNiCuZ/venv_create">create a virtual environment here</a>. Alternatively follow any guide online on how to set up a virtual environment for python.</p><p></p><p>Additional note: It's not recommended to hard-code your API key into code like this. It's better to store it as an <a target="_blank" rel="ugc" href="https://console.groq.com/docs/quickstart">environmental variable</a> in your system, or create a .ini-file which contains your code using the <code>configparser</code> library for example. ChatGPT can help you with this.</p><p></p><h1 id="heading-9291">Using Groq in ComfyUI</h1><p>Using the above method, I created a node to use in ComfyUI. It's available on my node pack on Github: <a target="_blank" rel="ugc" href="https://github.com/MNeMoNiCuZ/ComfyUI-mnemic-nodes?tab=readme-ov-file#-groq-llm-api-node">https://github.com/MNeMoNiCuZ/ComfyUI-mnemic-nodes?tab=readme-ov-file#-groq-llm-api-node</a>.</p><p>Install the node pack using ComfyUI Manager, or by cloning the repository manually into your <code>/ComfyUI/custom_nodes/</code>-directory.</p><p></p><p>Once installed, you will need to enter your API key into the config-file (<code>ComfyUI\custom_nodes\ComfyUI-mnemic-nodes\nodes\groq</code>), and then you should be good to go! Restart ComfyUI and find the <code>Groq LLM API</code> node and add it to your workflow.</p><p>Additional information can be found on the <a target="_blank" rel="ugc" href="https://github.com/MNeMoNiCuZ/ComfyUI-mnemic-nodes?tab=readme-ov-file#-groq-llm-api-node">Github page</a>.</p><p></p><h2 id="heading-9292">ComfyUI Groq LLM API Screenshots</h2><p>Below are a few screenshots to show what the node is capable of.</p><p><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/73afb24a-71a6-41fd-a045-a70062faa3b9/width=525/73afb24a-71a6-41fd-a045-a70062faa3b9.jpeg" /><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/4ad1778e-56b2-485b-b1d0-21c18879199d/width=525/4ad1778e-56b2-485b-b1d0-21c18879199d.jpeg" /><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/ecc58ae6-c468-4027-bb59-293324c4e727/width=525/ecc58ae6-c468-4027-bb59-293324c4e727.jpeg" /><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/176e6b0c-9b22-4d21-8bff-226e50e2b6e7/width=525/176e6b0c-9b22-4d21-8bff-226e50e2b6e7.jpeg" /><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/2171a976-0744-4d52-959e-8e92c32d78c1/width=525/2171a976-0744-4d52-959e-8e92c32d78c1.jpeg" /><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/9b1ecf22-289c-455f-b822-9f40a4cca091/width=525/9b1ecf22-289c-455f-b822-9f40a4cca091.jpeg" /></p><p>Don't hesitate to ask questions below.</p>

Files with SHA256: c57827833cf125863765c0915678e2177dedbaad41a557f98e3de373047246c7