[Code documentation] Using SkyReels Hunyuan T2V & I2V - pre-quantized int4 models

[nitinmukesh]

@tin2tin

SkyReels - Text 2 Video - BnB int4 pre-quantized model

import torch
import torch._dynamo.config
from diffusers import HunyuanVideoPipeline, HunyuanVideoTransformer3DModel
from diffusers.utils import export_to_video

model_id = "newgenai79/HunyuanVideo-int4"
transformer_model_id = "newgenai79/SkyReels-V1-Hunyuan-T2V-int4"
transformer = HunyuanVideoTransformer3DModel.from_pretrained(
    transformer_model_id,
    subfolder="transformer",
	torch_dtype=torch.bfloat16
)
pipe = HunyuanVideoPipeline.from_pretrained(model_id, transformer=transformer, torch_dtype=torch.float16)
pipe.enable_model_cpu_offload()
pipe.vae.enable_tiling()

output = pipe(
    prompt="FPS-24, A cat walks on the grass, realistic",
    # negative_prompt="Aerial view, aerial view, overexposed, low quality, deformation, a poor composition, bad hands, bad teeth, bad eyes, bad limbs, distortion",
    height=480,
    width=480,
    num_frames=25,
    guidance_scale=6.0,
    # true_cfg_scale=6.0,
    num_inference_steps=30,
).frames[0]
export_to_video(output, "output.mp4", fps=15)

100%|███████████████████████████████████████████| 30/30 [05:24<00:00, 10.80s/it]

With these settings I am able to do the inference. width, height and num_frames can be modified as per available VRAM/RAM.
Recommended settings for video generation: 960 x 544 x 97f