NokiMo
Furkan Gözükara
Furkan Gözükara

patreon


Tencent AI Lab - V-Express Image to Animation Gradio Web APP and 1-Click Installers for Windows, Massed Compute, RunPod and Kaggle

Patreon exclusive posts index to find our scripts easily, Patreon scripts updates history to see which updates arrived to which scripts and amazing Patreon special generative scripts list that you can use in any of your task.

Join discord to get help, chat, discuss and also tell me your discord username to get your special rank : SECourses Discord

Please also Star, Watch and Fork our Stable Diffusion & Generative AI  GitHub repository and join our Reddit subreddit and follow me on LinkedIn (my real profile)

=======

Latest Zip file : V_Express_Installers_V9.zip

Tutorial : https://youtu.be/xLqDTVWUSec

14 July 2025 Update

21 June 2024 Update

13 June 2024 Update

12 June 2024 Update

7 June 2024 Update

6 June 2024 Update

You can see 34 comparison tests results and their configs in this file : https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/Comparison_Tests.zip

I think best config is as below but you should look at each test

Windows Requirements

RunPod Instructions

Massed Compute Instructions

Tencent AI Lab - V-Express Image to Animation Gradio Web APP and 1-Click Installers for Windows, Massed Compute, RunPod and Kaggle Tencent AI Lab - V-Express Image to Animation Gradio Web APP and 1-Click Installers for Windows, Massed Compute, RunPod and Kaggle Tencent AI Lab - V-Express Image to Animation Gradio Web APP and 1-Click Installers for Windows, Massed Compute, RunPod and Kaggle

Comments

hello. can you try demo video and images and see if they work or not and let me know?

Furkan Gözükara

i need help 2024-08-12 17:47:57.291772: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-08-12 17:48:05.436774: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. WARNING:tensorflow:From A:\V-Express\venv\lib\site-packages\tf_keras\src\losses.py:2976: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead. Running on local URL: http://127.0.0.1:7860 To create a public link, set `share=True` in `launch()`. Exception in callback _ProactorBasePipeTransport._call_connection_lost(None) handle: Traceback (most recent call last): File "C:\Python31011\lib\asyncio\events.py", line 80, in _run self._context.run(self._callback, *self._args) File "C:\Python31011\lib\asyncio\proactor_events.py", line 165, in _call_connection_lost self._sock.shutdown(socket.SHUT_RDWR) ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host generating generate_kps_sequence_and_audio... INFO:albumentations.check_version:A new version of Albumentations is available: 1.4.13 (you have 1.4.12). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1. Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: A:\V-Express\model_ckpts\insightface_models\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: A:\V-Express\model_ckpts\insightface_models\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: A:\V-Express\model_ckpts\insightface_models\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: A:\V-Express\model_ckpts\insightface_models\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5 set det-size: (512, 512) ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 9.2.1 (GCC) 20200122 configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 libpostproc 55. 5.100 / 55. 5.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:\Users\Metr\AppData\Local\Temp\gradio\0cd61b9c0e97d47d65b66d273a1dfd3dbbe6759b\snow_itspuffpuff_1653493703_7101701373607021867_E_Improved_nosmooth1_pads-U-5D-5L-5R-5_mask-S0.1F5_mt.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf60.3.100 Duration: 00:00:07.04, start: 0.000000, bitrate: 4354 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 720x1280 [SAR 1:1 DAR 9:16], 4298 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default) Metadata: handler_name : VideoHandler encoder : Lavc60.3.100 libx264 Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 129 kb/s (default) Metadata: handler_name : SoundHandler Stream mapping: Stream #0:1 -> #0:0 (aac (native) -> mp3 (libmp3lame)) Press [q] to stop, [?] for help Output #0, mp3, to 'A:\V-Express\temp_process\ComfyUI_00303__20240812175043\target_audio.mp3': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 TSSE : Lavf58.29.100 Stream #0:0(und): Audio: mp3 (libmp3lame), 44100 Hz, stereo, fltp (default) Metadata: handler_name : SoundHandler encoder : Lavc58.54.100 libmp3lame size= 111kB time=00:00:07.08 bitrate= 128.9kbits/s speed=50.4x video:0kB audio:111kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.313149% Traceback (most recent call last): File "A:\V-Express\scripts\extract_kps_sequence_and_audio.py", line 56, in assert len(faces) == 1, f'There are {len(faces)} faces in the {frame_idx}-th frame. Only one face is supported.' AssertionError: There are 0 faces in the 0-th frame. Only one face is supported. auto cropping... Using GPU for RetinaFace detection. 2024-08-12 17:51:06.201158: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. No faces detected. starting inference... INFO:albumentations.check_version:A new version of Albumentations is available: 1.4.13 (you have 1.4.12). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1. WARNING:tensorflow:From A:\V-Express\venv\lib\site-packages\tf_keras\src\losses.py:2976: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead. Some weights of Wav2Vec2Model were not initialized from the model checkpoint at ./model_ckpts/wav2vec2-base-960h/ and are newly initialized: ['wav2vec2.masked_spec_embed'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. Loaded weights of Reference Net from ./model_ckpts/v-express/reference_net.bin. Loaded weights of Denoising U-Net from ./model_ckpts/v-express/denoising_unet.bin. Loaded weights of Denoising U-Net Motion Module from ./model_ckpts/v-express/motion_module.bin. Loaded weights of V-Kps Guider from ./model_ckpts/v-express/v_kps_guider.bin. Loaded weights of Audio Projection from ./model_ckpts/v-express/audio_projection.bin. Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: ./model_ckpts/insightface_models/models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: ./model_ckpts/insightface_models/models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: ./model_ckpts/insightface_models/models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: ./model_ckpts/insightface_models/models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5 set det-size: (1280, 720) Traceback (most recent call last): File "A:\V-Express\inference.py", line 291, in main() File "A:\V-Express\inference.py", line 209, in main reference_kps = app.get(reference_image_for_kps)[0].kps[:3] File "A:\V-Express\venv\lib\site-packages\insightface\app\face_analysis.py", line 59, in get bboxes, kpss = self.det_model.detect(img, File "A:\V-Express\venv\lib\site-packages\insightface\model_zoo\retinaface.py", line 224, in detect scores_list, bboxes_list, kpss_list = self.forward(det_img, self.det_thresh) File "A:\V-Express\venv\lib\site-packages\insightface\model_zoo\retinaface.py", line 194, in forward bboxes = distance2bbox(anchor_centers, bbox_preds) File "A:\V-Express\venv\lib\site-packages\insightface\model_zoo\retinaface.py", line 38, in distance2bbox x1 = points[:, 0] - distance[:, 0] ValueError: operands could not be broadcast together with shapes (1760,) (1840,) generating generate_kps_sequence_and_audio... INFO:albumentations.check_version:A new version of Albumentations is available: 1.4.13 (you have 1.4.12). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1. Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: A:\V-Express\model_ckpts\insightface_models\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: A:\V-Express\model_ckpts\insightface_models\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: A:\V-Express\model_ckpts\insightface_models\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: A:\V-Express\model_ckpts\insightface_models\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5 set det-size: (512, 512) ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 9.2.1 (GCC) 20200122 configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 libpostproc 55. 5.100 / 55. 5.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:\Users\Metr\AppData\Local\Temp\gradio\0cd61b9c0e97d47d65b66d273a1dfd3dbbe6759b\snow_itspuffpuff_1653493703_7101701373607021867_E_Improved_nosmooth1_pads-U-5D-5L-5R-5_mask-S0.1F5_mt.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf60.3.100 Duration: 00:00:07.04, start: 0.000000, bitrate: 4354 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 720x1280 [SAR 1:1 DAR 9:16], 4298 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default) Metadata: handler_name : VideoHandler encoder : Lavc60.3.100 libx264 Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 129 kb/s (default) Metadata: handler_name : SoundHandler Stream mapping: Stream #0:1 -> #0:0 (aac (native) -> mp3 (libmp3lame)) Press [q] to stop, [?] for help Output #0, mp3, to 'A:\V-Express\temp_process\ComfyUI_00303__20240812175754\target_audio.mp3': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 TSSE : Lavf58.29.100 Stream #0:0(und): Audio: mp3 (libmp3lame), 44100 Hz, stereo, fltp (default) Metadata: handler_name : SoundHandler encoder : Lavc58.54.100 libmp3lame size= 111kB time=00:00:07.08 bitrate= 128.9kbits/s speed= 123x video:0kB audio:111kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.313149% Traceback (most recent call last): File "A:\V-Express\scripts\extract_kps_sequence_and_audio.py", line 56, in assert len(faces) == 1, f'There are {len(faces)} faces in the {frame_idx}-th frame. Only one face is supported.' AssertionError: There are 0 faces in the 0-th frame. Only one face is supported. auto cropping... starting inference... INFO:albumentations.check_version:A new version of Albumentations is available: 1.4.13 (you have 1.4.12). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1. WARNING:tensorflow:From A:\V-Express\venv\lib\site-packages\tf_keras\src\losses.py:2976: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead. Some weights of Wav2Vec2Model were not initialized from the model checkpoint at ./model_ckpts/wav2vec2-base-960h/ and are newly initialized: ['wav2vec2.masked_spec_embed'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. Loaded weights of Reference Net from ./model_ckpts/v-express/reference_net.bin. Loaded weights of Denoising U-Net from ./model_ckpts/v-express/denoising_unet.bin. Loaded weights of Denoising U-Net Motion Module from ./model_ckpts/v-express/motion_module.bin. Loaded weights of V-Kps Guider from ./model_ckpts/v-express/v_kps_guider.bin. Loaded weights of Audio Projection from ./model_ckpts/v-express/audio_projection.bin. Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: ./model_ckpts/insightface_models/models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: ./model_ckpts/insightface_models/models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: ./model_ckpts/insightface_models/models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: ./model_ckpts/insightface_models/models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5 set det-size: (790, 416) Traceback (most recent call last): File "A:\V-Express\inference.py", line 291, in main() File "A:\V-Express\inference.py", line 209, in main reference_kps = app.get(reference_image_for_kps)[0].kps[:3] File "A:\V-Express\venv\lib\site-packages\insightface\app\face_analysis.py", line 59, in get bboxes, kpss = self.det_model.detect(img, File "A:\V-Express\venv\lib\site-packages\insightface\model_zoo\retinaface.py", line 224, in detect scores_list, bboxes_list, kpss_list = self.forward(det_img, self.det_thresh) File "A:\V-Express\venv\lib\site-packages\insightface\model_zoo\retinaface.py", line 194, in forward bboxes = distance2bbox(anchor_centers, bbox_preds) File "A:\V-Express\venv\lib\site-packages\insightface\model_zoo\retinaface.py", line 38, in distance2bbox x1 = points[:, 0] - distance[:, 0] ValueError: operands could not be broadcast together with shapes (10192,) (10296,)

Hash Bookies

great. could you let me know the error so if someone gets similar i can tell them

Furkan Gözükara

I discovered the issue. It was user-error. Please disregard. Thank you!

Brett Baker

yes you can undervolt your CPU and GPU - and lower clock speeds. normally no app can cause it so you have instability issues unrelated to this app

Furkan Gözükara

sadly this doesnt tell the error. can you try demo material to see if working?

Furkan Gözükara

Hi, I just started using V-express again after a couple of weeks and I'm suddenly getting the error below when trying to generate the video. I've tried images and audio that worked in the past and even those are causing this error now. Any help is appreciated. Thank you! "2024-07-25 16:34:31.735470: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2024-07-25 16:34:32.140433: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. WARNING:tensorflow:From C:\AI\V_Express\V-Express\venv\lib\site-packages\tf_keras\src\losses.py:2976: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead. Running on local URL: http://127.0.0.1:7860 To create a public link, set `share=True` in `launch()`. Using GPU for RetinaFace detection. 2024-07-25 16:35:57.594380: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. ffmpeg version 2023-06-21-git-1bcb8a7338-essentials_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers built with gcc 12.2.0 (Rev10, Built by MSYS2 project) configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libvpl --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband libavutil 58. 13.101 / 58. 13.101 libavcodec 60. 21.100 / 60. 21.100 libavformat 60. 9.100 / 60. 9.100 libavdevice 60. 2.100 / 60. 2.100 libavfilter 9. 8.102 / 9. 8.102 libswscale 7. 3.100 / 7. 3.100 libswresample 4. 11.100 / 4. 11.100 libpostproc 57. 2.100 / 57. 2.100 [aist#0:0/pcm_f32le @ 000002106d02fa00] Guessed Channel Layout: mono Input #0, wav, from 'C:\Users\bbaker\AppData\Local\Temp\gradio\7163b867f3339b759c9351387783f92a35e0cf38\Ben_Franklin_Voice_Audio.wav': Duration: 00:00:08.87, bitrate: 1416 kb/s Stream #0:0: Audio: pcm_f32le ([3][0][0][0] / 0x0003), 44100 Hz, 1 channels, flt, 1411 kb/s Stream mapping: Stream #0:0 -> #0:0 (pcm_f32le (native) -> mp3 (libmp3lame)) Press [q] to stop, [?] for help Output #0, mp3, to 'temp_process\Ben_Franklin512_20240725163613\target_audio_converted.mp3': Metadata: TSSE : Lavf60.9.100 Stream #0:0: Audio: mp3, 44100 Hz, mono, fltp Metadata: encoder : Lavc60.21.100 libmp3lame [out#0/mp3 @ 000002106d02fe00] video:0kB audio:102kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.215534% size= 103kB time=00:00:08.85 bitrate= 94.9kbits/s speed= 174x auto cropping... starting inference... INFO:albumentations.check_version:A new version of Albumentations is available: 1.4.11 (you have 1.4.8). Upgrade using: pip install --upgrade albumentations WARNING:tensorflow:From C:\AI\V_Express\V-Express\venv\lib\site-packages\tf_keras\src\losses.py:2976: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead. Some weights of the model checkpoint at ./model_ckpts/wav2vec2-base-960h/ were not used when initializing Wav2Vec2Model: ['lm_head.weight', 'lm_head.bias'] - This IS expected if you are initializing Wav2Vec2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model). - This IS NOT expected if you are initializing Wav2Vec2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). Some weights of Wav2Vec2Model were not initialized from the model checkpoint at ./model_ckpts/wav2vec2-base-960h/ and are newly initialized: ['wav2vec2.masked_spec_embed'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. Loaded weights of Reference Net from ./model_ckpts/v-express/reference_net.pth. Loaded weights of Denoising U-Net from ./model_ckpts/v-express/denoising_unet.pth. Loaded weights of Denoising U-Net Motion Module from ./model_ckpts/v-express/motion_module.pth. Loaded weights of V-Kps Guider from ./model_ckpts/v-express/v_kps_guider.pth. Loaded weights of Audio Projection from ./model_ckpts/v-express/audio_projection.pth. Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: ./model_ckpts/insightface_models/models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: ./model_ckpts/insightface_models/models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: ./model_ckpts/insightface_models/models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: ./model_ckpts/insightface_models/models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0 Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CUDAExecutionProvider': {'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0', 'cudnn_conv1d_pad_to_nc1d': '0', 'has_user_compute_stream': '0', 'gpu_external_alloc': '0', 'enable_cuda_graph': '0', 'gpu_mem_limit': '18446744073709551615', 'gpu_external_free': '0', 'gpu_external_empty_cache': '0', 'arena_extend_strategy': 'kNextPowerOfTwo', 'do_copy_in_default_stream': '1', 'cudnn_conv_use_max_workspace': '1', 'tunable_op_enable': '0', 'tunable_op_tuning_enable': '0', 'tunable_op_max_tuning_duration_ms': '0', 'enable_skip_layer_norm_strict_mode': '0'}, 'CPUExecutionProvider': {}} find model: ./model_ckpts/insightface_models/models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5 set det-size: (512, 512) Length of audio is 391068 with the sampling rate of 44100. The corresponding video length is 266. Traceback (most recent call last): File "C:\AI\V_Express\V-Express\inference.py", line 277, in main() File "C:\AI\V_Express\V-Express\inference.py", line 230, in main kps_sequence = retarget_kps(reference_kps, kps_sequence, only_offset=True) UnboundLocalError: local variable 'kps_sequence' referenced before assignment"

Brett Baker

This AI tool cant face swap on more than 5 minute long 4k video at Rtx 4090, due to overheating, the PC is getting restarted again and again. is there any solution of this

Anshul Gupta

hello it turns out that original authors had changed the folder structure in hugging face repo. i have updated all installers please reinstall now works ty

Furkan Gözükara

hello it turns out that original authors had changed the folder structure in hugging face repo. i have updated all installers please reinstall now works ty

Furkan Gözükara

sorry for late reply. it is accurate template. i am gonna test again now. the demo files are in attachments of this post. e.g. : Demo_Images_And_Videos_Part2.zip

Furkan Gözükara

I picked RunPod Pytorch 2.1 https://postimg.cc/crYCjN9d Where can i find demo content?

Schlecky Silberstein

Sadly no idea about it :/

Furkan Gözükara

I have 2 gpu 3090 dual link, do the following to run it on 2GPU, do you have any way?

Đạt Nguyễn

That means install error. You have to have python 3.10, Cuda 11.8 and c++ tools and ffmpeg installed on Linux as well. You can see all installed on massed compute virtual machine

Furkan Gözükara

The corresponding video length is 1600. Traceback (most recent call last): File "/media/dev_ai/LinuxData2/V_Express_Installers_V4/V-Express/inference.py", line 277, in main() File "/media/dev_ai/LinuxData2/V_Express_Installers_V4/V-Express/inference.py", line 217, in main assert os.path.exists(args.kps_path), f'{args.kps_path} does not exist' AssertionError: ./temp_process/tttttt(11)_20240612150754/kps.pth does not exist how to fix it ?

Đạt Nguyễn

I am sorry for late reply. A6000 GPU and coupon code is SECourses. it will be 31 cents per hour

Furkan Gözükara

Hello, should I select your deploy creator wise GPU to run this on mass compute? Please help as I am new to all this but want to use this tech GPU Type: L40 GPU Quantity: 1 Category: Creator Image: SECourses SSH Key: Coupon Code Verify Deploy 4 Available Specs: RAM: 128 GB, Storage: 625 GB, vCPU: 26 Price: $0.99/hour

Sahil Sangani

you are welcome

Furkan Gözükara

I will install as recommended and run the windows update bat file. Thank you so much for looking into this!

Brett Baker

it was due to you having space character in your folder path. the error was from the original repo authors. just made update. but still it is best if you install into like c:/my_Ai. now run windows update bat file and it should be fixed

Furkan Gözükara

Very probably since you have space character in your folder path. I am gonna test.

Furkan Gözükara

Thank you for your quick reply! Yes, I have ffmpeg installed accurately. My python version is Python 3.10.10. Here is a video capture of the process that is happening. https://www.youtube.com/watch?v=naQxO0GEcKE

Brett Baker

can you show me screenshot of your error getting? i tested all combinations never had any error what are the file names? did you install ffmpeg accurate? type ffmpeg and let me know what you see. also your python version?

Furkan Gözükara

Thank you! The install worked perfectly. I was able to generate a video using an image and an .mp3 audio file using the fix_face Retarget Strategy, When I use an image and an .mp4 video using either offset or naive retarget strategy, I get the error " raise RuntimeError(f"File not found: {filename}") RuntimeError: File not found: temp_process\ref_20240602195005\target_audio.mp3" Do you know what might be causing this error?

Brett Baker


Related Creators