This workflow uses SAM to invert the colors of the character to be replaced. By doing so, it makes character replacement significantly more consistent that Minimax H3 alone.

How to use it

You must

1. (v2,v3 or v4) Install https://github.com/kijai/ComfyUI-SolAttn_triton if you want the speedup.

2. (v3 or v4) Install https://github.com/1038lab/ComfyUI-QwenVL for QwenVL video to text description.

3. (v4 only) Download the custom node h3-split-video-inputs.zip. You can download this one in the same menu as the workflow.

4. (v1 only) Change the text prompt to describe both the scenario and the subject. Your goal is that Minimax H3 reproduces the original video as similar as possible.

5. Change the video and the picture, and make sure to change as well the resolution to match the original video.

6. (Optional, v2) change the SAM prompt until it masks only the subject to replace.

7. (Optional, v3) change the hint given to the Qwen node.

Why SAM and masking with H3

When using H3, it seems that the model will either recreate the original video as it is, without changing the subject at all (follows the <Video 1> too literally), or it will create a totally new video (follows the <Picture 1> too literally).

However, when inverting the color of the subject in the video, H3 is forced to produce a new video because the subject looks weird. So it can't follow the <Video 1> too literally anymore, otherwise it would produce a video with inverted colors.

Custom node

The custom node used on V4 feeds different perturbed subject videos to the Qwen and the VAE parts of the model. This has been tested in many different scenarios and it has good accuracy.