This is an evolution of my AnimateLCM workflow. Cut down and put together for ease of use.
This workflow should at MOST require 12-14GB of VRAM making it a lot more approachable for folks with smaller GPUs.
This workflow utilizes and requires an Alpha Mask that matches your video input to work properly. It will take the mask, resize is (shoutout to @Kijai for helping me get passed the initial CUDA errors), and invert it.
I recommend using MACHINEDELUSION'S PhotonLCM model with this workflow
How you go about getting that AlphaMask is entirely up to you. I personally like rotoscoping manually in Adobe After Effects for precision.
The first two IPAdapter images will be for your subject (the white part of your alpha mask).
The second two IPAdapter images will be for your background (the mask is being automatically inverted to reverse this).
Running the HighResFix at Bilinear will keep you from getting a CUDA on longer frame videos. At 1.5 you will get a clean 768x1344 output after the HighResFix is finished.
Appreciate everyone and all the support I receive.
Special shoutout to @cerspense @Purz @jeru @Fill @MidJourney.Man @PatchesFlows for allowing me to babble about my AnimateDiff wants and desires and helping me navigate how to get there when I hit a wall and hit it often.
Find me on IG @jboogx.creative
Description
This version has been rearranged with the same functionality, to work with the newly refactored and updated IPAdapter nodes.
FAQ
Comments (21)
GOAT
So I also removed the lora, and the process went into writing this link to the lora.
Error occurred when executing Efficient Loader: 'NoneType' object has no attribute 'lower' File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) Efficient Loader:I get an error at the Efficient Loader stage, tell me what could be wrong?
I suspect one of your IPAdapter models is wrong, I'm sure I had that issue. I Tried to backtrack to find the exact answer but since the IP adapters has changed I sadly deleted them. I had 2 issues one was IPadapter model (so just redownloaded from manager) (and check the motion model) the other fault I had was with The QRCode checkpoint being spelt different. Hope it can be of some help. Is any Node showing up red?
(Am re-doing workflows for practice so if that fault pops up ill be sure to pop back)
Will this workflow work on MAC? Has anyone launched it?
Nope, going to need a GPU with dedicated VRAM.
This workflow can be run on a Mac system, but it requires large memory. Otherwise, the size or resolution of the video will be very small, even 512*512 cannot be reached. A configuration with memory greater than 32g or higher, 96g, is more suitable. , the running speed will also be faster.
@35828375355 thanks!
yup im running it on my m2 ultra 192gb
on 24gb vram getting cuda out of memory with 700frames at 90% being finished. Havent had this issue before
I have have an ERROR please help!!
Error occurred when executing KSampler (Efficient): 'float' object has no attribute 'repeat' File "D:\AI\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\AI\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
you are a ComfyUI master!!! well done!!
I have an error with the textbox, what module you use to use the textbox node? :) Fixed but other error appear:
Error occurred when executing Efficient Loader: 'NoneType' object has no attribute 'lower' File "/workspace/ComfyUI/execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "/workspace/ComfyUI/execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "/workspace/ComfyUI/execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "/workspace/ComfyUI/custom_nodes/efficiency-nodes-comfyui/efficiency_nodes.py", line 151, in efficientloader model, clip = load_lora(lora_params, ckpt_name, my_unique_id, cache=lora_cache, ckpt_cache=ckpt_cache, cache_overwrite=True) File "/workspace/ComfyUI/custom_nodes/efficiency-nodes-comfyui/tsc_utils.py", line 372, in load_lora lora_model, lora_clip = recursive_load_lora(lora_params, ckpt, clip, id, ckpt_cache, cache_overwrite, folder_paths) File "/workspace/ComfyUI/custom_nodes/efficiency-nodes-comfyui/tsc_utils.py", line 366, in recursive_load_lora return recursive_load_lora(lora_params[1:], lora_model, lora_clip, id, ckpt_cache, cache_overwrite, folder_paths) File "/workspace/ComfyUI/custom_nodes/efficiency-nodes-comfyui/tsc_utils.py", line 363, in recursive_load_lora lora_model, lora_clip = comfy.sd.load_lora_for_models(ckpt, clip, comfy.utils.load_torch_file(lora_path), strength_model, strength_clip) File "/workspace/ComfyUI/comfy/utils.py", line 13, in load_torch_file if ckpt.lower().endswith(".safetensors"):
Same here.
I think the Derfuu comfyUI nodes broke?
re add the text from Derfuu menu , or use any text box !
For people having issues with Text box. They are just regular text box, you can delete and add new ones. Jboogx mentioned that he doesn't even use them anymore. He just writes in the prompt travel at 0 without the text box.
Error occurred when executing Efficient Loader: 'NoneType' object has no attribute 'lower'
I really don't know what to do
Done everything like in the flow.
Just starting on this, 4 months old, any optimized/updated workflows from this workflow now?
nope. this is the same workflow I currently use. I make little changes and add / subtract nodes based on what i'm currently doing in a particular video. but this is it.
@jboogx_creative Thanks, workflow is a beast. My character is looking more human than noodle like, any tips for dialing settings to help with that?
I have updated this workflow to have automatic alpha mask creation. Tell me how to get it to you if you want it.. I also added a little logic for longer videos(multiply frame load cap by a number that increases every queue to get the skip frames number)