Wanted to share my Wan 2.1 first-last Frame workflow, loosely based around previous LTX workflow. Generation times are MUCH longer than LTX workflow but there is a lot more consistent video production as well.
A few tips:
Start with a single portion of the sequence (say, Image 1-2 portion), and then use the preview to guide both your prompt and the seed generation to save compute time. You can usually tell based on the Stage 1 render if the final is going to suck or not :)
Once you are happy with your base prompt and base generation, move forward with the rest of the generations. If you don't touch the base "Controller" portion you won't have to regenerate your previously generated sections!
If you are unhappy with any portion of the sequence, just regenerate that portion using a "Prompt Addend" or different seed. You may also hook in additional LoRA for any portion of the sequence. Just attach a LoRA to that Purple line "model" section.
A few other tips for successful generations:
It seems that the AI picks up on any differences (even differences that are impossible to discern such as differences in grain or noise) and uses this to decide if an area gets motion or not.
If you don't want any motion, literally use your favorite photo editor and copy/paste that area between frames to keep it completely consistent.
If you want subtle motion, just run a low 1-2% noise pass on the area between frames, again using your favorite photo editor.
If you are using Flux or SDXL for base image generation, another tip is to use different prompts or LoRA on the same seed to generate similar but not identical generations.
Have fun!
Description
Added a "Simple" version with a single start/end frame. Also added a looping workflow to the "Multi" Workflow.
FAQ
Comments (17)
Im using Q4 gguf, with and without tea c it creates a large flash at the end and/or large random blobs of color noise, please help.
i've discovered for teacache using the "14b" rather than i2v 720p works better.
@katsumend what do you mean , im using this https://huggingface.co/city96/Wan2.1-FLF2V-14B-720P-gguf/tree/main
@dscvff https://imgur.com/a/yBNa4kC
The other problem could be VAE. I am working with a fair amount of VRAM but maybe move some of the techniques from one of the other low VRAM workflows to see how they are tiling the VAE. That GGUF is great, I haven't used the Q4 but I do use the Q6 without issue.
@katsumend and what sampler?
@dscvff I've been using Euler_A
Hey dscff, did you ever solve this? I'm seeing the same thing. I've been experimenting with tiling and different versions of the GGUF. Thanks!
@SinglePage do you have the flash or color noise or both?
in my experience this was happening because i set width/height to a different (smaller) resolution than the source image. setting it to match the input image fixed the glitchy artifacts
So what does this workflow specific ?
Sorry I'm new, what are the height and width nodes doing here? Should I leave them at 1200x800? I'm trying to solve a OOM issue. Thanks!
It will resize your image to this size. If OOM decrease the image size, length, and use lower numbers for GGUF quants (i.e. 4K M or 6 quants.
Where do you adjust the CFG to this WF? I'm trying to add the x2V self enforcing Lora to speed it up and don't se where to select the CFG...
It works perfect, I achieved a 5 sec decent video, I just set as disabled the pack sage attention kj Node, I think it is because my python version, but works excellent
Per your note about Unit 4 always running first, I looked into it as I was also confused and would prefer sequential processing. The base FLF2V 1.5 workflow daisy chains all of the loops together so all data is available to each loop at runtime, causing the compiler to decide which a loop to run first. However, I did manage to find a workaround using the "highway" nodes in the ComfyUI-0246 package.
What I did to set up:
1. Load ComfyUI-0246 https://github.com/ARZUMATA/ComfyUI-0246 into custom nodes folder (it's also on ComfyRegistry)
2. Per: https://github.com/Trung0246/ComfyUI-0246/issues/62 - appears these nodes broke with updates and so the package will fail to import. ARZUMATA in the thread provided edited scripts. Download and replace the versions in the custom nodes folder with the edited versions here:
https://github.com/ARZUMATA/ComfyUI-0246/commit/46d487ac3713c58309944c906ee5576640d95f43
3. Go to ComfyUI settings > 0246 > AlternateDOMWidget and disable it. This fixed bars covering the screen for some reason.
4. Retooled the workflow and connections for each Loop using the highway
5. I think i broke the auto-combiner, but I use premiere
I used Comfy 0.3.67 in stability matrix
The highway nodes are like a daisy chain, but with fewer lines. They store all the data they receive across all the daisy chained nodes in a 2-way relationship. However, each nodes's 'output' will ONLY release data once ALL data on the list is available in the pipeline.
This lets us force processing of clips in-order by using the previous gen's pass 2 output as a trigger to give the next loop data. The pipeline will have all the generation data (clip, VAE, LxW, etc) after loading, but before it passes to a generation loop, it needs to wait for the prior loop to finish and give it's second pass forcing a sequential generation.
I figure I( must be doing something wrong. I load the image and create a prompt and I get a completely different video. Is there some setting I am missing? I did disable sage as I cannot make it work. But it should only delay the generation.