
# 

## Model Introduction

This model is trained using the [Civision/model training](https://modelscope.ai/civision/modelTraining) environment and computing resources provided by ModelScope.

* Model Type: LoRA
* Base Model: [MiniMax/MiniMax-H3](https://modelscope.ai/models/MiniMax/MiniMax-H3)
* Training Code: [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio)
* Data amount：22
* Total training steps：2000
* License: Apache-2.0

## Inference Code

Install [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio):

```bash
pip install diffsynth
```

Inference:

```python
import torch
from diffsynth.pipelines.minimax_h3_audio_video import MiniMaxH3Pipeline, ModelConfig
from diffsynth.utils.data.audio_video import write_video_audio

vram_config = {
    "offload_dtype": "disk",
    "offload_device": "disk",
    "onload_dtype": "disk",
    "onload_device": "disk",
    "preparing_dtype": torch.bfloat16,
    "preparing_device": "cuda",
    "computation_dtype": torch.bfloat16,
    "computation_device": "cuda",
}
pipe = MiniMaxH3Pipeline.from_pretrained(
    torch_dtype=torch.bfloat16,
    device="cuda",
    model_configs=[
        ModelConfig(model_id="MiniMax/MiniMax-H3", origin_file_pattern="FL2VA/text_encoder/model*.safetensors", **vram_config),
        ModelConfig(model_id="MiniMax/MiniMax-H3", origin_file_pattern="FL2VA/transformer/model*.safetensors", **vram_config),
        ModelConfig(model_id="MiniMax/MiniMax-H3", origin_file_pattern="FL2VA/video_vae/source/model.safetensors", **vram_config),
        ModelConfig(model_id="MiniMax/MiniMax-H3", origin_file_pattern="FL2VA/audio_vae/model.safetensors", **vram_config),
    ],
    processor_config=ModelConfig(model_id="MiniMax/MiniMax-H3", origin_file_pattern="FL2VA/processor/"),
    vram_limit=torch.cuda.mem_get_info("cuda")[1] / (1024 ** 3) - 2,
)
pipe.load_lora(pipe.dit, ModelConfig(model_id="aiartworks/mmh3-tanya2", origin_file_pattern="mmh3-tanya2_c2-st2000.safetensors"))

prompt = "A cat"
video, audio = pipe(
    prompt=prompt,
    height=480, width=832, num_frames=124, num_inference_steps=50, seed=0,
)
write_video_audio(
    video=video, audio=audio,
    output_path="t2va.mp4", fps=24, audio_sample_rate=32000,
)
```


mmh3-tanya2

mmh3-tanya2_c2-st100.safetensors

c2-st100

c2-st200

c2-st300

c2-st400

c2-st500

c2-st600

c2-st700

c2-st800

c2-st900

c2-st1000

c2-st1100

c2-st1200

c2-st1300

c2-st1400

c2-st1500

c2-st1600

c2-st1700

c2-st1800

c2-st1900

c2-st2000

Files with SHA256: 7fc4d547378edb2a301f5118c9df85c00d495e4c7257f4874a54dbe92e3975b4