How to use VAE as single file in HunyuanVideo pipeline

[nitinmukesh]

So I am not a developer/coder.

I want to use this VAE as it is much smaller. Please help how to use this instead of default.

vae_path = (
"https://huggingface.co/Kijai/HunyuanVideo_comfy/resolve/main/hunyuan_video_vae_bf16.safetensors"
)

import torch
from diffusers import HunyuanVideoPipeline, HunyuanVideoTransformer3DModel
from diffusers import GGUFQuantizationConfig
from diffusers.utils import export_to_video

model_id = "hunyuanvideo-community/HunyuanVideo"
transformer = HunyuanVideoTransformer3DModel.from_pretrained(
    model_id, 
    subfolder="transformer", 
    torch_dtype=torch.bfloat16
)

pipe = HunyuanVideoPipeline.from_pretrained(model_id, transformer=transformer, torch_dtype=torch.float16)
pipe.vae.enable_tiling()
pipe.enable_model_cpu_offload()

output = pipe(
    prompt="A cat walks on the grass, realistic",
    height=320,
    width=512,
    num_frames=61,
    num_inference_steps=10,
).frames[0]
export_to_video(output, "output.mp4", fps=15)

[nitinmukesh]

Tried but output is very bad. Maybe I'm using it incorrectly or it is meant for Comfy only.

vae = AutoencoderKLHunyuanVideo(
in_channels=3,
out_channels=3,
block_out_channels=(128, 256, 512, 512),
layers_per_block=2,
scaling_factor=0.08333
)

vae_path = "Kijai/HunyuanVideo_comfy/hunyuan_video_vae_bf16.safetensors"
vae_state_dict = load_file(vae_path)
vae.load_state_dict(vae_state_dict)
vae = vae.to(torch.bfloat16)