What is LongCat 2.5 Preview and how is its 1.6-trillion-parameter MoE structured?

LongCat 2.5 Preview is Meituan's flagship multimodal Mixture-of-Experts (MoE) foundation model, scaling to 1.6 trillion total parameters across fine-grained feed-forward experts. Despite its colossal parametric memory, LongCat 2.5 utilizes dynamic sparse routing to activate only ~48 billion parameters per token. This gives developer agents the expansive world knowledge and semantic precision of a multi-trillion-parameter frontier model while retaining the generation throughput, KV cache economics, and inference latency of a mid-tier dense network.

What is LongCat Sparse Attention (LSA) and what fundamental bottlenecks does it solve?

Standard full attention scales with quadratic time and memory complexity O(L^2), making 1-million-token contexts computationally prohibitive. Prior sparse schemes like DeepSeek Sparse Attention (DSA) and Lightning Indexers suffered from two critical bottlenecks: quadratic O(L^2) indexer scoring overhead and fragmented, non-coalesced High Bandwidth Memory (HBM) read patterns. LongCat Sparse Attention (LSA) is a hardware-algorithm co-designed attention framework that achieves near-linear complexity O(L) by combining Streaming-Aware Indexing (SI), Cross-Layer Indexing (CLI), and Hierarchical Indexing (HI).

How does Streaming-Aware Indexing (SI) enforce contiguous, coalesced HBM memory access?

Dynamic sparse attention usually picks arbitrary, scattered tokens across the context history, resulting in uncoalesced memory reads that severely degrade memory bus utilization. Streaming-Aware Indexing reallocates a guaranteed slice of the attention selection budget to two contiguous zones: an initial attention sink (the first 128-512 tokens) and a local sliding window (e.g., the most recent 2,048 tokens). Only the remaining budget is dynamically selected across middle history. This ensures that the majority of memory transactions are sequential burst reads from HBM.

How does Cross-Layer Indexing (CLI) reduce inference computation without degrading retrieval accuracy?

Empirical observation demonstrates that token saliency and routing relevance remain highly correlated across adjacent transformer layers. Instead of computing expensive indexer scores independently at every single layer, CLI computes indexing selections once every K layers (e.g., K=2 or 4) and reuses the index map across subsequent layers. During pre-training, this behavior is enforced and stabilized using cross-layer distillation loss, reducing indexing compute overhead by over 60%.

How does Hierarchical Indexing (HI) perform two-stage coarse-to-fine token selection?

Evaluating every token's relevance at 1M context is itself an O(L) bottleneck. Hierarchical Indexing splits the sequence into coarse blocks (e.g., pages of 64 tokens) and first runs a lightweight pooled scoring pass to shortlist the top candidate blocks. In the second stage, fine-grained token-level selection is performed strictly within those shortlisted blocks. This reduces total indexer operations by up to 8x without requiring custom architectural retraining.

What infrastructure powers LongCat 2.5's training and deployment?

Unlike Western frontier models trained almost exclusively on NVIDIA Hopper/Blackwell clusters, LongCat 2.5 was trained and deployed end-to-end on domestic AI ASIC superpods. Through custom distributed kernels, ring-AllReduce communication topologies, and 8-bit/4-bit quantization routines, LongCat achieves robust sub-25ms inter-token latencies across million-token agentic coding and multimodal vision workflows.

LongCat 2.5 Preview: Inside the 1.6T Sparse MoE Architecture, LongCat Sparse Attention (LSA), Hierarchical Cross-Layer Indexing, and Domestic ASIC Superpod Inference

Simplified Overview (ELI5)

Written by Abrar Akhunji

Imagine you are tasked with studying an entire warehouse containing 1,000,000 pages of technical documentation, code, and architectural blueprints—and answering a complex programming question in less than two seconds.

How would you do it?

                     THE 1,000,000 TOKEN INFERENCE DILEMMA
                     
    Classic Attention: "The Exhaustive Reader"
    Query Token ──► [ Reads Every Single Token from 1 to 1,000,000 ] ──► Melting GPUs
                    Quadratic Complexity: O(L^2) = 1,000,000,000,000 comparisons!
                    High Bandwidth Memory (HBM) completely saturated.
                    
    LongCat 2.5 LSA: "The Master Archivist"
                     ┌─ 1. Fixed Memory Sink (First 256 tokens) ───── Contiguous Read
    Query Token ──►  ├─ 2. Local Sliding Window (Recent 2k tokens) ── Contiguous Read
      (Hierarchy)    └─ 3. Hierarchical Block Indexer (Page Shortlist)► Fine-grained picks
                     
    Near-Linear Complexity: O(L) | 94% Memory Bandwidth Saved!
    Active Parameters: Only 48B out of 1.6 Trillion!

If you read every single page from page 1 to page 1,000,000 every time someone asked you a question (which is what standard Full Attention does), your brain would fry. That is $O(L^2)$ quadratic complexity: as the document gets 10 times longer, the compute cost spikes 100 times higher.

Earlier models tried to fix this with "sparse indexers," like a student scanning random highlighted lines across thousands of books. But in a physical GPU, jumping randomly between page 4, page 9,821, and page 854,120 causes scattered memory stalls. The GPU spends 90% of its time waiting for the memory bus to fetch tiny fragments of data!

Enter Meituan's LongCat 2.5 Preview.

The 4 Breakthroughs That Make LongCat 2.5 Extraordinary

1. The 1.6 Trillion Parameter Giant That Acts Like a Nimble 48B Model

LongCat 2.5 is a massive Mixture-of-Experts (MoE) network with 1.6 trillion total parameters. Think of it as a university faculty of 1,600 specialized professors (a Linux kernel hacker, a distributed systems architect, an ancient Latin scholar, a CSS layout wizard, a compiler engineer, etc.).

When your prompt arrives, an intelligent mathematical router summons only ~48 billion active parameters (the exact 4 to 8 professors needed). You get the superhuman world knowledge of a 1.6T brain, but the inference speed, low latency, and energy bill of a mid-sized 48B model.

2. Streaming-Aware Indexing (SI): The "Contiguous Reading" Trick

Instead of scattering memory requests randomly across a million tokens, LongCat guarantees that the GPU always reads large, contiguous blocks:

The Anchor Sink: Always keeps the first 256 tokens (the project goals, system prompt, and core instructions) warm in fast cache.
The Recent Focus Window: Always reads the last 2,048 tokens contiguously (what just happened in the shell or chat).
The Surgical Target: Uses its indexer only for the middle historical references.

This transforms fragmented memory reads into coalesced burst reads, allowing GPUs and ASICs to run at peak hardware efficiency.

3. Cross-Layer Indexing (CLI): Stop Repeating the Indexer

In a standard deep neural network with 64 or 80 layers, why should layer 42 recalculate which tokens are important if layer 41 already figured it out?

LongCat's Cross-Layer Indexing calculates the attention relevance map once, and then shares that exact roadmap across consecutive layers! A special distillation process during training ensures that the downstream layers understand the shared map without dropping a single percentage point of accuracy.

4. Hierarchical Indexing (HI): The Table of Contents First

If you're searching a million tokens for a bug, you don't scan all 1,000,000 words one by one. You first look at the chapters and pages (coarse blocks). Once you find the 5 pages that discuss the database transaction pool, you read those specific words.

Hierarchical Indexing does exactly this in hardware: a lightweight pass shortlists the top 64-token pages, and the detailed mathematical attention calculation only runs inside those shortlisted blocks!

Let's dive deep into the production systems architecture, mathematical formulations, Triton kernels, and benchmark evaluations behind this 1.6T milestone.

LongCat 2.5 LSA Pipeline & MoE Systems Architecture

1.6T Parameter Dynamic Expert Selection: A high-precision router computes affinity logits across specialized SwiGLU experts, dynamically routing input tokens to activate only ~48B parameters per forward pass.
Sink & Local Sliding Window Pinning: Reallocates token selection budget into an initial attention sink (128-512 tokens) and contiguous local sliding window (2k tokens), enforcing burst HBM memory reads.
Shared Attention Saliency Roadmaps: Computes token saliency once every K layers and shares the resulting indexing mask across adjacent transformer blocks via cross-layer distillation loss.
Coarse-to-Fine Two-Stage Attention: First evaluates coarse 64-token block poolings to prune 98% of the sequence, then executes fine-grained dot-product attention strictly within surviving candidates.

Inference Latency & KV Cache Memory: LSA vs. Full GQA Attention

Comparative analysis of per-token generation latency (ms) and KV cache memory footprint (GB) across context lengths up to 1,000,000 tokens.

contextLength	Full GQA Attention (ms)	LongCat LSA (ms)	Full GQA KV Memory (GB)	LongCat LSA KV Memory (GB)
64k Tokens	18.2	11.4	24.5	8.2
128k Tokens	46.8	13.1	49.1	11.6
256k Tokens	128.4	15.6	98.2	15.4
512k Tokens	412	19.8	196.4	21.8
1M Tokens	1480	24.6	392.8	32.4

4. Production Kernel Implementation: Triton Hierarchical Block Selector

Below is a production-grade Triton kernel implementing the two-stage Hierarchical Indexing Block Selector with streaming-aware memory coalescing:

import torch
import triton
import triton.language as tl

@triton.jit
def _hierarchical_block_selection_kernel(
    Q_idx_ptr,          # [Batch, NumHeads, 1, DimIdx]
    K_block_mean_ptr,   # [Batch, NumBlocks, DimIdx]
    Block_Scores_ptr,   # [Batch, NumBlocks]
    stride_qb, stride_qh, stride_qd,
    stride_kb, stride_kn, stride_kd,
    stride_sb, stride_sn,
    DIM_IDX: tl.constexpr,
    BLOCK_SIZE: tl.constexpr
):
    """
    Stage-1 Coarse Block Scoring Kernel.
    Computes dot-product affinities between the query index vector
    and coarse pooled block representations.
    """
    pid_b = tl.program_id(axis=0)  # Batch index
    pid_n = tl.program_id(axis=1)  # Block index
    
    # Compute memory offsets
    q_offset = pid_b * stride_qb + tl.arange(0, DIM_IDX) * stride_qd
    k_offset = pid_b * stride_kb + pid_n * stride_kn + tl.arange(0, DIM_IDX) * stride_kd
    
    # Load query and block key vectors into registers
    q_vec = tl.load(Q_idx_ptr + q_offset)
    k_vec = tl.load(K_block_mean_ptr + k_offset)
    
    # Compute block score via reduction
    score = tl.sum(q_vec * k_vec, axis=0)
    
    # Store block score
    out_offset = pid_b * stride_sb + pid_n * stride_sn
    tl.store(Block_Scores_ptr + out_offset, score)

class LongCatHierarchicalSelector(torch.nn.Module):
    """
    Production Hierarchical Token Selector for LongCat 2.5 Sparse Attention.
    Combines Streaming-Aware Sink, Local Window, and Top-M Coarse Block Pruning.
    """
    def __init__(
        self,
        dim_idx: int = 64,
        block_size: int = 64,
        sink_tokens: int = 256,
        local_window: int = 2048,
        top_m_blocks: int = 128
    ):
        super().__init__()
        self.dim_idx = dim_idx
        self.block_size = block_size
        self.sink_tokens = sink_tokens
        self.local_window = local_window
        self.top_m_blocks = top_m_blocks

    @torch.inference_mode()
    def forward(
        self,
        q_idx: torch.Tensor,         # [B, 1, D_idx]
        k_block_pools: torch.Tensor, # [B, NumBlocks, D_idx]
        seq_len: int
    ) -> torch.Tensor:
        batch_size, num_blocks, _ = k_block_pools.shape
        block_scores = torch.empty((batch_size, num_blocks), device=q_idx.device, dtype=torch.float32)

        # Launch Triton coarse block grid
        grid = (batch_size, num_blocks)
        _hierarchical_block_selection_kernel[grid](
            q_idx, k_block_pools, block_scores,
            q_idx.stride(0), 0, q_idx.stride(2),
            k_block_pools.stride(0), k_block_pools.stride(1), k_block_pools.stride(2),
            block_scores.stride(0), block_scores.stride(1),
            DIM_IDX=self.dim_idx,
            BLOCK_SIZE=self.block_size
        )

        # Retrieve Top-M candidate blocks
        _, top_blocks = torch.topk(block_scores, k=min(self.top_m_blocks, num_blocks), dim=-1)
        
        return top_blocks

5. Benchmark Evaluation & SWE-Bench Verified Teardown

Meituan evaluated LongCat 2.5 Preview across premier autonomous software engineering and long-context benchmarks. Evaluated against contemporary frontier systems (such as Claude 3.5 Sonnet, GPT-4o, and Meta Muse-Spark-Open-816B):

+---------------------------------------------------------------------------------------------------------------+
| BENCHMARK PERFORMANCE: LONGCAT 2.5 PREVIEW VS. FRONTIER BASELINES                                             |
+-----------------------------------+--------------------+--------------------+---------------------------------+
| Benchmark Evaluation              | LongCat 2.5 (1.6T) | Muse-Spark-Open 816| Claude 3.5 Sonnet (Frontier)    |
+-----------------------------------+--------------------+--------------------+---------------------------------+
| SWE-bench Verified (Resolved)     | 61.8%              | 60.0%              | 51.2%                           |
| Terminal-Bench 2.1 (Terminus 2)   | 78.2%              | 76.4%              | 68.9%                           |
| MCP-Atlas (Tool Orchestration)    | 91.4%              | 89.2%              | 84.1%                           |
| GPQA Diamond (Science / Reasoning)| 88.6%              | 89.0%              | 85.0%                           |
| LongBench v2 (1M Context Accuracy)| 84.6%              | 81.2%              | 72.8%                           |
| TTFT @ 1M Context (Seconds)       | 1.42s              | 3.86s              | 14.20s                          |
| Generation Throughput (tok/sec)   | 42.8 tok/s         | 24.2 tok/s         | 28.5 tok/s                      |
+-----------------------------------+--------------------+--------------------+---------------------------------+

Key Engineering Observations:

Time-To-First-Token (TTFT) Dominance: At 1,000,000 tokens of prompt context, LongCat 2.5 achieves an astounding 1.42 seconds TTFT, compared to 14.2 seconds for dense frontier systems. The combination of Hierarchical Indexing and coalesced burst memory loading completely bypasses the prefill compute stall.
Autonomous Tool Calling (MCP-Atlas): LongCat 2.5 reaches 91.4% on Model Context Protocol benchmarks. The model's fine-grained MoE routing isolates bash syntax, AST parsing, and JSON protocol serializations into distinct sub-experts, preventing cross-domain syntax leakage.
Linear KV Cache Scaling: Under 1M context, LongCat requires only 32.4 GB of KV cache memory per concurrent stream, compared to ~392 GB for dense GQA architectures. This enables high-density serving on standard multi-accelerator nodes without requiring offloading to host system RAM.

6. Architectural Takeaways for Senior Systems Engineers

The release of LongCat 2.5 Preview validates three critical evolutionary directions in AI infrastructure:

Hardware-Algorithm Co-Design Over Pure Mathematical Sparsity: Developing sparse attention mechanisms without considering HBM burst sizes and DRAM bus coalescing leads to poor real-world speedups. LongCat's Streaming-Aware Indexing (SI) proves that pinning contiguous memory zones (sinks and sliding windows) is essential for saturating hardware throughput.
Cross-Layer Representation Sharing Is Underexploited: Most transformer architectures treat every layer as an independent compute island. Cross-Layer Indexing (CLI) demonstrates that structural attention geometry can be shared across multiple layer depths via distillation, eliminating redundant indexing overhead.
The Rise of Domestic Superpod Inference: LongCat 2.5 proves that multi-trillion parameter MoE models can be trained and served at frontier performance levels outside the traditional NVIDIA Blackwell/CUDA monoculture, utilizing custom AI ASICs, high-speed RoCEv2 topologies, and specialized Triton-compatible kernels. :::