@TencentHunyuan's Hy4-preview runs in vLLM from day 0, verified on NVIDIA GPUs. 🎉 - 770B total, 49B...
By @vllm_project
vLLM project announces day-zero verified support for Tencent Hunyuan Hy4-preview on NVIDIA GPUs, detailing architectural specifics including 770B total/49B active parameters, 256 routed experts plus one shared, 1M context with sparse attention to 2048 tokens, HPC-Ops kernels, and a 10B MTP speculative decoding layer.
- 770B total, 49B active, 256 routed experts plus one shared
- 1M context, but each query attends to just 2048 tokens
- Only 21 of the 78 layers compute their own sparse index, the other 57 reuse one
- A 10B MTP layer ships inside the checkpoint, 0.7B of it active, draft depth 3
Source details
- 770B total, 49B active, 256 routed experts plus one shared
- 1M context, but each query attends to just 2048 tokens
- Only 21 of the 78 layers compute their own sparse index, the other 57 reuse one
- A 10B MTP layer ships inside the checkpoint, 0.7B of it active, draft depth 3