Innovate Futures @ Benji

(Workflow) Flux Ace Plus with SkyReels GoWithTheFlow LoRA Version 20250224

Added 2025-02-24 13:00:23 +0000 UTC

Summary of Workflow: Flux Ace Plus with SkyReels GoWithTheFlow LoRA

This workflow demonstrates how to combine the capabilities of Flux Ace Plus and SkyReels GoWithTheFlow LoRA for advanced video editing and motion capture animations.

Step-by-Step Workflow:

Initial Setup:
- Use SkyReels, a fine-tuned model for image-to-video generation.
- Download the necessary GoWithTheFlow LoRA models from Hugging Face.
- Ensure you have the correct JSON files and connect the model loader in your ComfyUI setup.
Image Preparation:
- Load your target image that needs animation using the "load image from path" node.
- Prepare a reference video containing the desired motions (e.g., jumping rope, head turning).
Flux Ace Plus Image Editing:
- Bypass other groups and focus on the image editing stage.
- Enable the video loader to bring in the reference footage.
- Create masks to modify specific parts of the image (e.g., changing outfits or backgrounds).
- Apply text prompts for context-aware edits, such as transferring styles or swapping objects.
- Run the initial image through Flux Ace Plus to generate the modified base image.

Motion Capture with GoWithTheFlow:
- Transfer the edited image to the SkyReels GoWithTheFlow group.
- Capture simple motions from the reference video using the GoWithTheFlow framework.
- Animate the single input image by applying these captured motions.
- Utilize two sampler passes: the first for the base animation and the second for refinement.

Final Output:
- Generate the final animated video by combining the modified image with the motion data.
- Preview the results and make any necessary adjustments.
- Export the high-quality video after upscaling if needed.

This workflow is ideal for creating short, looping animations with simple motions, making it suitable for tasks like character outfit swaps or basic action sequences.

Go-with-the-Flow Lora

Go-with-the-Flow

https://github.com/Eyeline-Research/Go-with-the-Flow

VideoNoiseWarp

https://github.com/kijai/ComfyUI-VideoNoiseWarp

Spacepxl 's GoWithTheFlow-lora

https://huggingface.co/spacepxl/HunyuanVideo-GoWithTheFlow-unofficial/tree/main

Attached workflow file below, have fun :)