Tencent Ships a 770B MoE Text Model Under Apache-2.0 — and the Benchmarks Are Its Own

What happened

Tencent published Hy4-preview to Hugging Face on August 27, 2026. It is a 770B-parameter Mixture-of-Experts text-generation model with 49B activated parameters per token, a 1M-token context window, and an Apache-2.0 licence.

Context

Tencent's Hy Team has been building the Hunyuan line, and the model card explicitly describes the Gated DSA attention design as inspired by DeepSeek and GLM. Two arXiv papers (2512.02556 for the attention mechanism, 2603.12201 for IndexCache) back the architectural choices. The model is co-designed with Tencent's CodeBuddy and WorkBuddy products, and post-training data was drawn from work performed by Tencent's own software engineers, game developers, finance analysts, and security experts. This is a corporate-ecosystem model that ships with open weights, distributed across Hugging Face, ModelScope, cnb.cool, and GitCode.

How it works

The backbone is 78 layers. Layer one is a standard dense FFN; layers two through 78 are MoE, each containing 256 routed experts plus one shared expert. A token activates the top-8 routed experts and the shared expert, which produces the 49B activated figure from the 770B total. Attention uses Gated DSA: 64 heads, hidden size 6144, query compression to 2048, key-value compression to 512. An indexer with 32 heads (head dimension 128, top-k 2048) selects which keys participate in attention. IndexCache reuses sparse indices across layers so they are computed once rather than per layer. Residual streams use iHC (identity Hyper-Connections) with 4 streams to widen inter-layer information flow. A separate 10B MTP layer (0.7B activated) sits alongside the backbone for speculative decoding. Weights are in safetensors; deployment targets are transformers, vLLM, and SGLang.

Our read

The benchmarks need a caveat. The model card reports a single internal evaluation: 163 Tencent experts scoring 203 engineering tasks. Hy4 preview scored 2.99 against GLM 5.3's 2.92 and Kimi K3's 2.94. The gap to GLM 5.3 is 0.07, which is close enough that task composition and prompt framing matter as much as architecture. No third-party benchmark appears in the sources.

The memory arithmetic should land harder than the benchmark. 770B parameters at FP16 is roughly 1.54 TB of weights, before the 10B MTP layer. At INT4, around 400 GB. The 49B activated figure cuts per-token compute, not the memory footprint: every expert's weights must be resident to route to. A single workstation does not serve this. The sources do not state a minimum hardware configuration or a hosted endpoint with pricing.

The model card calls Gated DSA "inspired by DeepSeek and GLM." That is the honest read: an engineering integration of published techniques at a scale few labs reach. Post-training data comes from Tencent's internal engineers, game developers, finance analysts, and security experts. The model will be strong where that data is dense and weaker where it is not.

171 likes and 0 downloads at collection time is a day-old snapshot. The pattern suggests the community is reading the architecture before anyone has replicated the evaluation.

What this changes

For a studio running ComfyUI and local models: the stack does not change. Hy4-preview is a text-generation model, not a visual model. It does not appear as a node in a ComfyUI workflow. The one feature with direct studio relevance is the 1M-token context window: ingesting a full season of episode scripts, a shot list, and a style bible in a single pass, then asking for continuity checks or variant dialogue. That assumes API access, and the sources do not name a hosted endpoint with published pricing or rate limits. If you need a local text model for prompt templates or QA, this is overkill. No smaller Hy4 variant is mentioned in the sources.

License

Apache-2.0. Commercial use, fine-tuning, and redistribution are permitted without attribution, revenue share, or usage caps. The model card includes Finetuning and Quantization sections consistent with this. The sources do not state additional data-usage restrictions beyond the licence.

Key takeaways

• Tencent shipped a 770B-parameter MoE text model (49B activated per token) under Apache-2.0 on August 27, 2026, distributed across Hugging Face, ModelScope, cnb.cool, and GitCode.
• The benchmark data is a single internal evaluation (163 experts, 203 tasks); no independent third-party results appear in the sources.
• Memory requirements (roughly 1.5 TB at FP16, around 400 GB at INT4) place this well beyond single-workstation deployment; the sources do not state a minimum hardware configuration.
• Post-training data is drawn from Tencent's internal engineering, game-dev, finance, and security teams, which shapes where the model will be strong and where it will trail open-domain models.
• No operational change to a ComfyUI-based video pipeline results from this release; the model is text-in/text-out and the sources do not name a hosted API endpoint.

Sources

1. tencent/Hy4-preview — text-generation on Hugging Face — tier 2