@huchukato Thank you! ❤️

@huchukato No worries!

@huchukato Ah yes, I've not used the webterminal before, usually used the Juypeter one opened in each file to quickly run a wget command for huggingface whilst using the civic node for the odd hosted model to play with if needed. I'll take a look at the webterminal now, I got by using a HF-Downloader haha!

@huchukato Yes.
In the model_name (in the qwen3-vl gguf node), I tried:

qwen3-vl-8b-instruct-c_abliterated-v3.q4_k_m.gguf

qwen3-vl-8b-instruct-c_abliterated-v3.q6_k.gguf

qwen3-vl-4b-instruct-c_abliterated-v2.q4_k_m.gguf

However, I encountered the same error with all of them.

And following Gemini’s advice, I installed CUDA, the NVIDIA CUDA Toolkit, and Microsoft Visual Studio. I also showed the ComfyUI CMD (console) window to Gemini and ChatGPT, but there was no problem and the result was the same.

Also the manager inside ComfyUI did not detect any missing files. It seems that all the required files are installed.

@huchukato Also, I have a question. In the "qwen3-vl autoprompt" group, there are four prompts (prompt1–4). I noticed that I can’t directly edit the text written inside them. Is that how it’s supposed to be?

(I’m referring to the text that says “two woman with vibrant green eyes~”.)

(I already have the “comfyui-easy-use” node.)

The QwenVL GGUF nodes need llama-cpp-python with vision support. A normal pip install llama-cpp-python often doesn’t include handlers.. Install the vision-capable llama-cpp-python
python -m pip install --upgrade --no-cache-dir --force-reinstall ` "llama-cpp-python @ git+https://github.com/JamePeng/llama-cpp-python.git"

@drernestbrown474 After following your instructions and reinstalling llama-cpp-python with vision support, qwen3-vl seems to be working now. Thank you for that!

@huchukato I didn’t know that—what an incredible feature!

I’m using it well, but I have a question.

When writing a prompt, how can I make the 'qwen3-vl gguf' node structure the prompt by time segments?

For example, how would I give a command like, “From 0 to 5 seconds, walk and from 5 to 10 seconds, run”?

@huchukato Thank you!

t. A simillar work flow called Smooth uses florence2run. Init's case to try to make video from a picture and the ai tries to figure out what'd be going on. In it's case you can disable those nodes manually and describe action yourself. The also did a all in one wokflow kind of thing where you can pick what you want to do. One I grabbed didn't pick up on what ever the author has set up for dependencies, and my reaction is: I have better things to do then fuck around with this.

I encountered the same error when trying the OneClick-I2V-Story workflow. The latest version of the ComfyUI desktop app is currently 0.14.1 (it doesn't automatically update, even though 0.14.2 is out). I encountered the same issue with that version, and the error disappeared when I unpacked all subgraphs. Try unpacking all subgraphs.

According to ChatGPT, if you're not using the latest ComfyUI version, something is causing the issue with the subgraphs.

@fopof4264449 what do you mean by "unpacked" all subgraphs?

@huchukato Me too. Didn't fiddle too much, got the same error message.

@huchukato Yeah I experience same, longer videos loses its point when one have to speed it up , hence again creating a relatively short video😅 I hope you figure out something clever🙌🏻

I have the same issue, it lights up red almost instantly when generation starts, the prompt isn't even generated and still it's red, and in console I can see at least "Failed to validate prompt for output..." error.

I spent a moment more looking into why this doesn't work in my case;

* WanMoeKSamplerAdvanced ...

- Return type mismatch between linked nodes: scheduler, received_type(['simple', 'sgm_uniform', 'karras', 'exponential', 'ddim_uniform', 'beta', 'normal', 'linear_quadratic', 'kl_optimal', 'bong_tangent']) mismatch input_type(['simple', 'sgm_uniform', 'karras', 'exponential', 'ddim_uniform', 'beta', 'normal', 'linear_quadratic', 'kl_optimal', 'bong_tangent', 'beta57'])

Output will be ignored

Looks like at least in my case there's something changed somewhere, and the enum doesn't anymore match - there's beta57 in another of the enums, which comes from RES4LYF I think. But if change the sampler to be manually configured within the subgraph, at least this error/issue is gone.

@huchukato

also, where the negative prompt box?

@huchukato Yes, it's not a problem for me, I can get around it easy enough but just thought to point it out that there's possible incompatibilities.

Same here, but I'm on 0.15 which probably is troublesome.

@i_m_l_O how do you get around it easily? do you replace the ksampler? do I need 2 ksamplers now cause the regular clownsharksampler only got 1 model connection.

ok cock fucker. Prompt outputs failed validation: AILab_QwenVL_GGUF_Advanced: - Value not in list: preset_prompt: '🍿 Wan 2.2 NSFW I2V' not in ['🖼️ Tags', '🖼️ Simple Description', '🖼️ Detailed Description', '🖼️ Ultra Detailed Description', '🎬 Cinematic Description', '🖼️ Detailed Analysis', '📹 Video Summary', '📖 Short Story', '🪄 Prompt Refine & Expand'] WanMoeKSamplerAdvanced: - Return type mismatch between linked nodes: scheduler, received_type(['simple', 'sgm_uniform', 'karras', 'exponential', 'ddim_uniform', 'beta', 'normal', 'linear_quadratic', 'kl_optimal', 'bong_tangent']) mismatch input_type(['simple', 'sgm_uniform', 'karras', 'exponential', 'ddim_uniform', 'beta', 'normal', 'linear_quadratic', 'kl_optimal', 'bong_tangent', 'beta57']) . what piece of shit.

@huchukato huchukato

OP

9 days ago

" I wrote about RES4LYF in the readme, didn't tested 0.15 yet" wtf are you even trying to say?

I'm also stuck at "Loading checkpoint shards: 100%" when using the Qwen3-VL-8B-Instruct-Ablitered model. Waited over 20 minutes. I'm able to generate a 5s video with Qwen3-VL-8B-Instruct and Qwen3-VL-8B-Instruct-FP8 models however.

The shitstain didn't even bother to test some stuff it's just a broken cunt ass add.

Probably. the shit stain has no idea wtf he's doing

@huchukato Specifically WAN2.2-I2V-AutoPrompt.json, it's not story or full, I just want the basic Image to Video, but it keeps throwing that weird memory error.

@huchukato Same error unfortunately :
[QwenVL] Loading Qwen3-VL-4B-Instruct-Abliterated (8-bit (Balanced), attn=sdpa)

Loading weights: 0%| | 0/713 [00:00<?, ?it/s]

!!! Exception during processing !!! Allocation on device 0 would exceed allowed memory. (out of memory)

Currently allocated : 9.12 MiB

Requested : 741.88 MiB

Device limit : 31.37 GiB

Free (according to CUDA): 22.41 GiB

PyTorch limit (set by user-supplied memory fraction)

: 17179869184.00 GiB

Traceback (most recent call last):

File "/ComfyUI/execution.py", line 524, in execute

output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/ComfyUI/execution.py", line 333, in get_output_data

return_values = await asyncmap_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/ComfyUI/execution.py", line 307, in asyncmap_node_over_list

await process_inputs(input_dict, i)

File "/ComfyUI/execution.py", line 295, in process_inputs

result = f(**inputs)

^^^^^^^^^^^

File "/ComfyUI/custom_nodes/ComfyUI-QwenVL-Mod/AILab_QwenVL.py", line 706, in process

return self.run(model_name, quantization, preset_prompt, custom_prompt, image, video, frame_count, max_tokens, temperature, top_p, num_beams, repetition_penalty, seed, keep_model_loaded, attention_mode, use_torch_compile, device, keep_last_prompt)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/ComfyUI/custom_nodes/ComfyUI-QwenVL-Mod/AILab_QwenVL.py", line 585, in run

self.load_model(

File "/ComfyUI/custom_nodes/ComfyUI-QwenVL-Mod/AILab_QwenVL.py", line 468, in load_model

self.model = AutoModelForVision2Seq.from_pretrained(model_path, **load_kwargs).eval()

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/opt/comfyui-env/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 374, in from_pretrained

return model_class.from_pretrained(

^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/opt/comfyui-env/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4137, in from_pretrained

loading_info, disk_offload_index = cls._load_pretrained_model(model, state_dict, checkpoint_files, load_config)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/opt/comfyui-env/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4256, in loadpretrained_model

loading_info, disk_offload_index = convert_and_load_state_dict_in_model(

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/opt/comfyui-env/lib/python3.12/site-packages/transformers/core_model_loading.py", line 1212, in convert_and_load_state_dict_in_model

realized_value = mapping.convert(

^^^^^^^^^^^^^^^^

File "/opt/comfyui-env/lib/python3.12/site-packages/transformers/core_model_loading.py", line 678, in convert

collected_tensors = self.materialize_tensors()

^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/opt/comfyui-env/lib/python3.12/site-packages/transformers/core_model_loading.py", line 654, in materialize_tensors

tensors = [future.result() for future in tensors if future.result() is not None]

^^^^^^^^^^^^^^^

File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result

return self.__get_result()

^^^^^^^^^^^^^^^^^^^

File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result

raise self._exception

File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run

result = self.fn(*self.args, **self.kwargs)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/opt/comfyui-env/lib/python3.12/site-packages/transformers/core_model_loading.py", line 800, in _job

return materializecopy(tensor, device, dtype)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/opt/comfyui-env/lib/python3.12/site-packages/transformers/core_model_loading.py", line 789, in materializecopy

tensor = tensor.to(device=device, dtype=dtype)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

torch.OutOfMemoryError: Allocation on device 0 would exceed allowed memory. (out of memory)

Currently allocated : 9.12 MiB

Requested : 741.88 MiB

Device limit : 31.37 GiB

Free (according to CUDA): 22.41 GiB

PyTorch limit (set by user-supplied memory fraction)

: 17179869184.00 GiB

Memory summary: |===========================================================================|

| PyTorch CUDA memory summary, device ID 0 |

|---------------------------------------------------------------------------|

| CUDA OOMs: 0 | cudaMalloc retries: 0 |

|===========================================================================|

| Metric | Cur Usage | Peak Usage | Tot Alloc | Tot Freed |

|---------------------------------------------------------------------------|

| Allocated memory | 83589 KiB | 8418 MiB | 0 B | 0 B |

| from large pool | 0 KiB | 0 MiB | 0 B | 0 B |

| from small pool | 0 KiB | 0 MiB | 0 B | 0 B |

|---------------------------------------------------------------------------|

| Active memory | 83589 KiB | 8418 MiB | 0 B | 0 B |

| from large pool | 0 KiB | 0 MiB | 0 B | 0 B |

| from small pool | 0 KiB | 0 MiB | 0 B | 0 B |

|---------------------------------------------------------------------------|

| Requested memory | 0 B | 0 B | 0 B | 0 B |

| from large pool | 0 B | 0 B | 0 B | 0 B |

| from small pool | 0 B | 0 B | 0 B | 0 B |

|---------------------------------------------------------------------------|

| GPU reserved memory | 131072 KiB | 8480 MiB | 0 B | 0 B |

| from large pool | 0 KiB | 0 MiB | 0 B | 0 B |

| from small pool | 0 KiB | 0 MiB | 0 B | 0 B |

|---------------------------------------------------------------------------|

| Non-releasable memory | 0 B | 0 B | 0 B | 0 B |

| from large pool | 0 B | 0 B | 0 B | 0 B |

| from small pool | 0 B | 0 B | 0 B | 0 B |

|---------------------------------------------------------------------------|

| Allocations | 0 | 0 | 0 | 0 |

| from large pool | 0 | 0 | 0 | 0 |

| from small pool | 0 | 0 | 0 | 0 |

|---------------------------------------------------------------------------|

| Active allocs | 0 | 0 | 0 | 0 |

| from large pool | 0 | 0 | 0 | 0 |

| from small pool | 0 | 0 | 0 | 0 |

|---------------------------------------------------------------------------|

| GPU reserved segments | 0 | 0 | 0 | 0 |

| from large pool | 0 | 0 | 0 | 0 |

| from small pool | 0 | 0 | 0 | 0 |

|---------------------------------------------------------------------------|

| Non-releasable allocs | 0 | 0 | 0 | 0 |

| from large pool | 0 | 0 | 0 | 0 |

| from small pool | 0 | 0 | 0 | 0 |

|---------------------------------------------------------------------------|

| Oversize allocations | 0 | 0 | 0 | 0 |

|---------------------------------------------------------------------------|

| Oversize GPU segments | 0 | 0 | 0 | 0 |

|===========================================================================|

i'm not crashing with an OOM message, but my 5090 runs full, and then it just stops processing, the 4B or 8B model doesn't matter.

@huchukato i downloaded them literally earlier today, after some more testing, i guess only the 8B-MAX VL Version doesn't like to run consistently. fresh launch and it runs, next run it just just stops or takes ages, idk, i stopped it after like 10 minutes last time. can i somehow convert it to gguf or nvfe4 to save on some vram? i mean, 8B is huge anyway.

@huchukato mine makes me restart the backend, the backend never restarts and when i manually restart it need installing again. Maybe Comfyui just isn't for me lol

lol, same for me

@huchukato make sure to hardcode more repos that are half baked in while your at it. oh and more cunt-um nodes. I don't think people are trolled enough make sure it never garabage collects to while you vibe code.

Having the same issue

your step size is too low. You need to adjust your stepsize to higher like 20, or use a lightning LORA

because the stupid guy on the node 5s / 10s / 15s / 20s . Is sending wan moe decode latent *purplee dot to > vram cleanup input.

When he should be sending latent to > wan vae decode > samples.

from there send wan vae decode output to image

delete vram cleanup.

Because i dont know where in the hell i should connect that

also disconnect the node in wan moe decode > sample name > just leave it without the green cable. Otherwise will thow error of mismatch.

and now it works. Repeat this in all others nodes, meaning 10s 15s 20s.

@messinajonathan2459 Thank you for the solution. I delete the vramcleanup node and it works fine. Still a wonderful workflow.

Same. Just keep spitting out empty video files and two png's.

i discover that i have some issues with sage, i bypass the node and woks fine now.

@dantufis664 Good to know. Thanks!

what is the comfyui virtual environment?

https://github.com/yuvraj108c/ComfyUI-Upscaler-Tensorrt/issues/64

From user swdvb:

"How to install TensorRT

I use ComfyUI on Windows11.

ComfyUI version: 0.3.68
Python version: 3.12.10
pytorch version: 2.8.0+cu128

I read all your comments and the methods each of you used, but none of you specified exactly what needs to be done to make everything work.
I used the hints (ai-capybara), but that’s not the whole process.

What needs to be done:

Download tensorrt-10.13.2.6 from the NVidia website: (https://developer.nvidia.com/tensorrt/download/10x)

Choose your Python version (mine is Python version: 3.12.10). To install, open the folder with (python_embeded), right-click and choose (open in terminal) from the menu, then enter these commands one by one:
( python.exe -s -m pip install wheel-stub )
( python.exe -m pip install --upgrade pip setuptools wheel )
( python.exe -m pip install nvidia-cudnn-cu12==9.2.0.82 onnx polygraphy --no-cache-dir )
Once these libraries are installed, proceed to install tensorrt.
If you downloaded the file, for example (tensorrt-10.13.2.6-cp312-none-win_amd64.whl),
and it’s located in another folder or on another drive, you don’t need to copy it to the python folder—just enter this command ( python.exe -m pip install ) and then drag and drop the file into the terminal window and press Enter.

If the file is located in the python folder, then run this command:
( python.exe -m pip install tensorrt-10.13.2.6-cp312-none-win_amd64.whl )
Just copy the name of the file you want to install and replace (tensorrt-10.13.2.6-cp312-none-win_amd64.whl) with your version.

After all this, you need to copy all dll files from the folder (TensorRT-10.13.2.6\lib) to the python_embeded directory.
That’s the whole process. After that, everything should start up. If these nodes don’t work (ComfyUI-Upscaler-Tensorrt) and (ComfyUI_TensorRT), you need to install (numpy-1.26.4.dist-info). How to install it:
Go to (https://pypi.org/project/numpy/1.26.4/#files) and download the (whl) file for your Python version and install it the same way as tensorrt—just enter this command ( python.exe -m pip install ) and then drag and drop the file into the terminal window and press Enter."