Best open source Image to Video CogVideoX1.5-5B-I2V is pretty decent and optimized for low VRAM machines with high resolution - native resolution is 1360px and up to 10 seconds 161 frames - audios generated with new open source audio model
Added 2024-12-25 00:31:54 +0000 UTCFull YouTube tutorial for CogVideoX1.5-5B-I2V : https://youtu.be/5UCkMzP2VLE
1-Click Windows, RunPod and Massed Compute installers : https://www.patreon.com/posts/112848192
https://www.patreon.com/posts/112848192 - installs into Python 3.11 VENV
Official Hugging Face repo of CogVideoX1.5-5B-I2V : https://huggingface.co/THUDM/CogVideoX1.5-5B-I2V
Official github repo : https://github.com/THUDM/CogVideo
Used prompts to generate videos txt file : https://gist.github.com/FurkanGozukara/471db7b987ab8d9877790358c126ac05
Demo images shared in : https://www.patreon.com/posts/112848192
I used 1360x768px images at 16 FPS and 81 frames = 5 seconds
+1 frame coming from initial image
Also I have enabled all the optimizations shared on Hugging Face
pipe.enable_sequential_cpu_offload()
pipe.vae.enable_slicing()
pipe.vae.enable_tiling()
quantization = int8_weight_only - you need TorchAO and DeepSpeed works great on Windows with Python 3.11 VENV
Used audio model : https://github.com/hkchengrex/MMAudio
1-Click Windows, RunPod and Massed Compute Installers for MMAudio : https://www.patreon.com/posts/117990364
https://www.patreon.com/posts/117990364 - Installs into Python 3.10 VENV
Used very simple prompts - it fails when there is human in input video so use text to audio in such cases
I also tested some VRAM usages for CogVideoX1.5-5B-I2V
Resolutions and here their VRAM requirements - may work on lower VRAM GPUs too but slower
512x288 - 41 frames : 7700 MB , 576x320 - 41 frames : 7900 MB
576x320 - 81 frames : 8850 MB , 704x384 - 81 frames : 8950 MB
768x432 - 81 frames : 10600 MB , 896x496 - 81 frames : 12050 MB
896x496 - 81 frames : 12050 MB , 960x528 - 81 frames : 12850 MB
1024x576 - 81 frames : 13900 MB , 1280x720- 81 frames : 17950 MB
1360x768 - 81 frames : 19000 MB
Our Gradio APP is extremely advanced and working perfect
Comments
yes please follow this video without skipping a second. or upgrade gold tier and i can connect your pc and install requirements : https://youtu.be/DrhUHnYfwC0
Furkan Gözükara
2025-02-06 11:36:40 +0000 UTCI get these errors. Any thoughts on why this happens? WARNING: Failed to find MSVC. WARNING: Failed to find Windows SDK. WARNING: Failed to find CUDA. A matching Triton is not available, some optimizations will not be enabled Traceback (most recent call last): File "D:\CogVid\CogVideo\venv\Lib\site-packages\xformers\__init__.py", line 57, in _is_triton_available import triton # noqa ^^^^^^^^^^^^^ File "D:\CogVid\CogVideo\venv\Lib\site-packages\triton\__init__.py", line 20, in from .runtime import ( File "D:\CogVid\CogVideo\venv\Lib\site-packages\triton\runtime\__init__.py", line 1, in from .autotuner import (Autotuner, Config, Heuristics, autotune, heuristics) File "D:\CogVid\CogVideo\venv\Lib\site-packages\triton\runtime\autotuner.py", line 9, in from ..testing import do_bench, do_bench_cudagraph File "D:\CogVid\CogVideo\venv\Lib\site-packages\triton\testing.py", line 7, in from . import language as tl File "D:\CogVid\CogVideo\venv\Lib\site-packages\triton\language\__init__.py", line 4, in from . import math File "D:\CogVid\CogVideo\venv\Lib\site-packages\triton\language\math.py", line 1, in from . import core File "D:\CogVid\CogVideo\venv\Lib\site-packages\triton\language\core.py", line 10, in from ..runtime.jit import jit File "D:\CogVid\CogVideo\venv\Lib\site-packages\triton\runtime\jit.py", line 12, in from ..runtime.driver import driver File "D:\CogVid\CogVideo\venv\Lib\site-packages\triton\runtime\driver.py", line 1, in from ..backends import backends File "D:\CogVid\CogVideo\venv\Lib\site-packages\triton\backends\__init__.py", line 50, in backends = _discover_backends() ^^^^^^^^^^^^^^^^^^^^ File "D:\CogVid\CogVideo\venv\Lib\site-packages\triton\backends\__init__.py", line 43, in _discover_backends compiler = _load_module(name, os.path.join(root, name, 'compiler.py')) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\CogVid\CogVideo\venv\Lib\site-packages\triton\backends\__init__.py", line 12, in _load_module spec.loader.exec_module(module) File "D:\CogVid\CogVideo\venv\Lib\site-packages\triton\backends\amd\compiler.py", line 2, in from triton._C.libtriton import ir, passes, llvm, amd ImportError: DLL load failed while importing libtriton: Произошел сбой в программе инициализации библиотеки динамической компоновки (DLL). import error: DLL load failed while importing libtriton: Произошел сбой в программе инициализации библиотеки динамической компоновки (DLL). ИНФОРМАЦИЯ: не удается найти файлы по заданным шаблонам. * Running on local URL: http://127.0.0.1:7860 To create a public link, set `share=True` in `launch()`. Downloading shards: 100%|██████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 4001.24it/s] Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 4/4 [00:08<00:00, 2.04s/it] Loading pipeline components...: 0%| | 0/5 [00:00
VDR
2025-02-06 11:17:15 +0000 UTCawesome thanks for comment
Furkan Gözükara
2025-01-06 15:01:08 +0000 UTCI think that I know what he speaking about. I've had this happen to me a lot. Typically I open your installer as edit and then enter the lines myself to be sure everything gets into the venv correctly. Personally I don't mind, it helps me learn off your work. Thank you for all your hard work.
Lenny
2025-01-06 14:58:33 +0000 UTCwell if venv exists it will not delete and remake it. in my tests above code was not working but surely i can add it. the bat system is unfortunately limited. if you have improved any of the existing install can you email me so i can apply that logic to all : monstermmorpg@gmail.com
Furkan Gözükara
2024-12-31 00:23:16 +0000 UTCI like your one-click installs, but I've had a few instances where the venv isn't created correctly and the script keeps going and installs packages over my base installation anyway. Could you add something like the following after the "call .\venv\Scripts\activate.bat" line in the future? if %ERRORLEVEL% neq 0 ( echo Failed to create the virtual environment. pause exit /b ) I think a more robust one-click install would really enhance the quality. Thanks! (to be clear, this is in Windows_Install.bat)
Wyatt George
2024-12-31 00:14:21 +0000 UTCyes a lot of people waiting
Furkan Gözükara
2024-12-29 09:45:18 +0000 UTCI'm interesting too
Begoña Serrano
2024-12-29 09:42:24 +0000 UTCyou are welcome
Furkan Gözükara
2024-12-26 01:53:54 +0000 UTCThank you! Looking forward to your tutorial.
peng beryl
2024-12-26 01:11:01 +0000 UTCKohya said he is working on Hunyuan-Video lora training. i am waiting him atm
Furkan Gözükara
2024-12-25 19:17:42 +0000 UTCI've been following your tutorials—thanks so much for all the guidance! Now, it's possible to train LoRA using Hunyuan-Video， and it works for both images and videos. Could you create a tutorial specifically on training Hunyuan-Video？
peng beryl
2024-12-25 16:22:35 +0000 UTC
