VACE VIDEO CONTROLNET IMAGE TO VIDEO WORKFLOW!
Added 2025-05-22 17:24:47 +0000 UTCHey everyone! I've just released a new video on "THE NEW AI VIDEO CONTROLNET KING! ", called "VACE". Simply THE MOST POWERFUL UPGRADE TO AI VIDEO GENERATION EVER! A ControlNet-style adapter for WAN 2.1 that gives you perfect control over your video generations using OpenPose, Depth, Segmentation AND WHEN Paired with the special LoRA CausVid, you can now generate videos up to 4x faster than before… It's just amazing!
You can check out the video right here: https://youtu.be/zhNKpdL03PA
I also created a 1-click installer for the Webui for my Patreon supporters for Local AND Runpod, that you can find right here: 1-Click INSTALL VACE VIDEO CONTROLNET IMAGE TO VIDEO ComfyUI WebUI! | Patreon
Here the link to the manual install guide: https://pastebin.com/i1jzaRna
Down below you will also find the VACE_IMAGE-TO-VIDEO_CONTROLNET.json for free for you to use!
As always, supporting me on Patreon allows me to keep creating helpful resources like this for the community. Thank you for your support - now go have some fun😉!
Comments
I fixed the pose and depth issues via manual download and placement. if you have this issue pop the error logs into chat gpt it will take a while but you will get it fixed
Tyraecial John
2025-07-19 05:17:44 +0000 UTChey all I'm getting this error anyone know how to repair it? the depth-anything folder is empty...# ComfyUI Error Report ## Error Details - **Node ID:** 170 - **Node Type:** DepthAnythingV2Preprocessor - **Exception Type:** FileNotFoundError - **Exception Message:** [Errno 2] No such file or directory: 'C:\\Users\\john\\Desktop\\comfy\\ComfyUI_windows_portable\\ComfyUI\\custom_nodes\\comfyui_controlnet_aux\\ckpts\\depth-anything\\Depth-Anything-V2-Small\\.cache\\huggingface\\download\\Wgzz53Hpl7-m749TyT1Y6_XeYfg=.715fade13be8f229f8a70cc02066f656f2423a59effd0579197bbf57860e1378.incomplete'
Tyraecial John
2025-07-19 04:13:51 +0000 UTCHi, I've tried to make a test video, and I keep getting the same kind of error : When I tried with the medium model, it gave me this error : MPS backend out of memory (MPS allocated: 33.17 GiB, other allocations: 164.62 MiB, max allowed: 36.27 GiB). Tried to allocate 3.15 GiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure). And now I've tried with the lowest model, and it took like a full 24 hours (it's a 7 second video that I've reduced to 720x480 and 2 seconds), and it gives me this error : MPS backend out of memory (MPS allocated: 16.67 GiB, other allocations: 19.58 GiB, max allowed: 36.27 GiB). Tried to allocate 759.38 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure). I have a M1 Max Mac Studio with 32GB. Is it just not enought ? Or is there a wrong parameter somewhere ? Thanks for your help :)
Giant Jack
2025-06-13 22:20:21 +0000 UTChi love your creations, i'm getting this error when trying to do video to image :LoaderGGUF cannot reshape array of size 13834627 into shape (5120,2880) comfyui
F C
2025-06-05 23:59:18 +0000 UTCdm me your parameters and gpu, you are probably running out of memory
Aitrepreneur
2025-05-29 21:45:38 +0000 UTCyou are running out of memory unfortunately, make sure you are running the latest nvidia drivers as well with sysmem fallback activated
Aitrepreneur
2025-05-29 21:44:52 +0000 UTCI just upgraded my GPU to 5070 Ti 16 GB VRAM, but I get this error after the Controlnet finishes the process: KSampler Allocation on device (in ComfyUI) And the following in the console: =================== File "C:\AI\ComfyUI_windows_portable\ComfyUI\comfy\ldm\flux\math.py", line 43, in apply_rope xk_out = freqs_cis[..., 0] * xk_[..., 0] + freqs_cis[..., 1] * xk_[..., 1] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ torch.OutOfMemoryError: Allocation on device Got an OOM, unloading all loaded models. Prompt executed in 1.62 seconds =================== I even downloaded the "Wan2.1-VACE-14B-Q4_K_S.gguf" thinking maybe the 5 is heavy, but sadly the same error. Any idea what is going on? Is there something wrong with my new GFX Card ?
TomiTom1234
2025-05-29 20:23:42 +0000 UTCIt doesn't work for me. It generates the depth preview but there is no output. It also doesn't seem to even begin generating the video but just stops after pose preview. No, wait. It says something about composing but it's gone too fast to be real. And yeah, then... no video output.
Mailman
2025-05-29 13:51:58 +0000 UTCFor me, it does the pose preview animation but it doesn't output a final video. i mean, nothing. no error, just no video where it is supposed to show up.
Mailman
2025-05-29 13:40:13 +0000 UTCWhat do we do when we are using runpod and there are missing nodes due to clean installation? Is there an easy way to download all missing nodes? I don't see that in the instructions anywhere.
Nicolas Diaz
2025-05-28 03:07:16 +0000 UTCDear Aitrepreneur, thanx for your content! I haven't practiced AI for a long time, and now I'm trying to catch up with the changes over the past 2 years). It feels like several epochs have changed before my eyes))). But I have a question. You're saying that the model is uncensored. And nswf works can be found on the Internet. But all my attempts to generate uncensored content fail, even if all input files are nsfw. Can you tell me what I'm doing wrong?
Berk
2025-05-25 21:27:42 +0000 UTC
