Hy-Memory 2.0 and Hy-Memory-30B-A3B: Tencent Open-Sources a 3B-Active Cognitive Memory Architecture

Technical breakdown

Written by Abrar Akhunji

Long-term memory systems for autonomous agents have historically been architected as an afterthought, relying on heuristic vector RAG pipelines or generalist LLM prompt wrappers. When deployed on production software engineering tasks spanning multiple days and thousands of repository modifications, these approaches fail due to three core systems breakdowns:

1. The Write-Path Generalization Tax: Using frontier models (Claude 3.5 Sonnet, GPT-4o, DeepSeek-V3) to parse conversation histories and extract state changes creates prohibitive operating expenses. Memory writing is fundamentally an extractive, conflict-resolving task that does not require 200B+ active parameters.
2. Context Bloat from Over-Storage: Standard memory extraction pipelines maximize recall at the expense of precision. Because LLMs lack counterfactual feedback during extraction, they store irrelevant conversational noise, polluting vector databases and degrading downstream retrieval precision.
3. Cross-Agent State Fragmentation: Engineers frequently run Claude Code in the terminal, Cursor in the editor, and local background cron agents simultaneously. Without an OS-level shared memory coordinator, each agent maintains a fragmented, conflicting world state.

Tencent Hunyuan's Hy-Memory 2.0 suite addresses these limitations by introducing a purpose-built 3B-active sparse Mixture-of-Experts model, an attributed reinforcement learning objective, and a local cross-process memory hub.

+---------------------------------------------------------------------------------------------------------------+
| HY-MEMORY 2.0 ARCHITECTURAL SPECIFICATION                                                                     |
+-----------------------------------+---------------------------------------------------------------------------+
| Architectural Subsystem           | Systems Implementation / Engine Responsibility                            |
+-----------------------------------+---------------------------------------------------------------------------+
| Dedicated Memory Writer Model     | Hy-Memory-30B-A3B (Tencent Hunyuan Sparse MoE)                            |
| Total Parameter Count ($\Theta$)  | 30.4 Billion parameters                                                   |
| Active Parameters per Token ($A$) | 3.2 Billion parameters (8 active experts out of 64 routed experts)        |
| Context Window Length             | 32,768 tokens (optimized for multi-turn session dumps)                    |
| Training Paradigm                 | Stage 1: Memory On-Policy Distillation (OPD)                              |
|                                   | Stage 2: Attributed Memory Reinforcement Learning (AMRL)                  |
| Memory Modalities                 | Chat Memory (Event Graphs), Working Memory (DAGs), Code Memory (Rules)    |
| Sync Daemon                       | MemHub: Local IPC daemon with SQLite WAL + ChromaDB vector storage        |
| Hardware Footprint (INT4 / FP8)   | 16.4 GB VRAM (FP8) / 8.8 GB VRAM (INT4 GGUF/AWQ)                          |
| Inference Runtimes                | vLLM, SGLang, Ollama, llama.cpp, Hugging Face TGI                         |
| Open Source License               | Apache 2.0 (Weights and Source Code)                                      |
+-----------------------------------+---------------------------------------------------------------------------+

1. The Write-Path Dilemma: Separating Read and Write Compute

In conventional agent pipelines (such as Mem0 or LangChain Memory), memory operations are handled symmetrically by the primary generation model:

Legacy Symmetrical Pipeline:
User Request ──► [Frontier LLM (200B+)] ──► Response Generation
                         │
                         ▼
                 [Frontier LLM (200B+)] ──► Extract Memory Entries (High Latency, High Cost)

This architecture is deeply inefficient. Generation requires broad world knowledge, creative planning, and deep code synthesis. Memory writing, by contrast, requires:

• Strict entity-relation tuple extraction.
• Temporal resolution (determining if "tomorrow" refers to October 25).
• Contradiction reconciliation (updating "database=PostgreSQL" to "database=ClickHouse").
• Noise elimination (discarding conversational filler).

Hy-Memory decouples the write path into an asynchronous, specialized sub-tier:

Hy-Memory Decoupled Architecture:
User Request ──► [Any Target LLM] ──► Fast Response to User
                         │ (Asynchronous Event Stream)
                         ▼
                 [Hy-Memory-30B-A3B] (3B Active MoE)
                         │
                         ├─► Chat Memory (Temporal Event Graph)
                         ├─► Working Memory (Task DAG State)
                         └─► Code Memory (Architectural Rules)

By offloading the write path to a 3B-active model running locally on the same host or on cheap GPU instances, the primary agent operates with zero added latency, while memory maintenance costs drop by over 80%.

2. Hy-Memory-30B-A3B Model Topology: Sparse MoE Specialization

Hy-Memory-30B-A3B is structured as a decoder-only sparse Mixture-of-Experts transformer. Out of 32 total layers, 28 layers utilize routed MoE feed-forward networks:

$$y_l = x_l + \sum_{i \in \text{TopK}(G(x_l), K)} g_i(x_l) \cdot \text{Expert}_i(x_l)$$

Where:

• Total routed experts: $E = 64$ experts per MoE layer.
• Active experts per token: $K = 8$ experts.
• Shared experts: 2 dedicated shared experts active across all tokens for general syntactic cohesion.
• Hidden dimension: $d_{\text{model}} = 3,072$.
• Feed-forward intermediate dimension: $d_{\text{ffn}} = 1,536$ per expert.

Because only 8 of the 64 routed experts fire per token, the computational density during inference is identical to a dense 3.2-billion-parameter model, while retaining the parametric capacity of a 30.4-billion-parameter network.

3. Two-Stage Training: Memory On-Policy Distillation and Attributed RL

Training a small MoE model to surpass 200B+ parameter models on complex memory reasoning required a novel two-stage training curriculum:

                            TWO-STAGE TRAINING CURRICULUM
                            
    Raw Multi-Session Transcripts
                 │
                 ▼
    ┌───────────────────────────────────────────┐
    │ Stage 1: Memory On-Policy Distillation    │ ──► Dense token-level supervision from
    │          (OPD)                            │     teacher models on student rollout states
    └────────────────────┬──────────────────────┘
                         │
                         ▼
    ┌───────────────────────────────────────────┐
    │ Stage 2: Attributed Memory RL (AMRL)      │ ──► Multi-turn trajectory evaluations
    │          - Answerability Reward (+)       │     Penalizes over-storage and noise
    │          - Memory Bloat Penalty (-)       │     Optimizes downstream utility
    └───────────────────────────────────────────┘

Stage 1: Memory On-Policy Distillation (OPD)

Standard distillation applies teacher supervision on offline target datasets. In dynamic memory systems, student models generate their own intermediate memory states that drift from the training distribution.

Tencent's OPD forces the student model to generate memory candidate graphs on-policy. A frontier teacher model (Hunyuan-Large) then evaluates the student's generated state transitions, providing dense step-level KL divergence feedback:

$$\mathcal{L}_{\text{OPD}}(\theta) = \mathbb{E}_{x \sim \mathcal{D}, y \sim \pi_\theta} \left[ D_{\text{KL}} \left( \pi_{\text{teacher}}(y|x) \;\middle\|\; \pi_\theta(y|x) \right) \right]$$

Stage 2: Attributed Memory Reinforcement Learning (AMRL)

The primary failure mode of memory systems is storing facts that look plausible but are never useful. AMRL resolves this through downstream trajectory rollouts.

Given a multi-turn session trajectory $\tau = (s_1, a_1, \dots, s_T, a_T)$, the model generates a memory bank $\mathcal{M}$. Downstream probe tasks then query the agent across subsequent synthetic sessions. The reward function is explicitly partitioned:

$$R(\mathcal{M}, \tau) = R_{\text{answerability}}(\mathcal{M}) - \lambda_{\text{cost}} \cdot |\mathcal{M}|_{\text{tokens}} - \lambda_{\text{conflict}} \cdot \mathcal{C}(\mathcal{M})$$

Where:

• $R_{\text{answerability}}$ measures whether downstream questions can be answered using retrieved memory chunks.
• $|\mathcal{M}|_{\text{tokens}}$ penalizes total token consumption, directly discouraging the model from storing trivial pleasantries or redundant facts.
• $\mathcal{C}(\mathcal{M})$ computes the contradiction index across stored facts.

This dual-reward optimization produces a model that aggressively filters conversational noise, yielding compact, high-utility memory snapshots.

4. Tripartite Memory Topology: Chat, Working, and Code Memory

Hy-Memory 2.0 abandons flat vector stores in favor of three specialized cognitive storage engines:

                       HY-MEMORY 2.0 TRIPARTITE ENGINES
                       
    ┌─────────────────────────┬─────────────────────────┬─────────────────────────┐
    │       Chat Memory       │     Working Memory      │       Code Memory       │
    ├─────────────────────────┼─────────────────────────┼─────────────────────────┤
    │ - Temporal Event Graph  │ - Active Goal DAGs      │ - Architecture Rules    │
    │ - Entity Relationships  │ - Step Preconditions    │ - Compiler Constraints  │
    │ - Conflict Resolution   │ - Checkpoint / Resume   │ - Anti-Pattern Warnings │
    │ - User Preferences      │ - Failure Recovery Tree │ - File Ownership Maps   │
    └─────────────────────────┴─────────────────────────┴─────────────────────────┘

Chat Memory (Temporal Event Graph)

Facts are modeled as temporal assertions $(e_1, r, e_2, [t_{\text{start}}, t_{\text{end}}])$. When a new user interaction states "I no longer live in London, I relocated to Zurich yesterday", the conflict engine updates the validity window of the previous location assertion rather than appending a contradictory duplicate.

Working Memory (Task Execution DAGs)

Working memory maintains the active execution graph of long-horizon tasks. If a coding agent crashes during a database migration on Step 4 of a 7-step plan, working memory preserves the execution stack, environmental variables, and failed intermediate outputs. When the session resumes, the agent continues from Step 4 without re-executing Steps 1 through 3.

Code Memory (Repository Architectural Constraints)

Specifically engineered for software development, Code Memory records repository invariants:

• Approved architectural patterns (e.g., "All API handlers must use zod validation schema").
• Hard environmental constraints (e.g., "Node.js runtime limited to v20 on target runner").
• Known regression traps discovered during past debugging sessions.

5. MemHub: Local Multi-Agent Coordination Protocol

In production engineering workflows, developers run multiple autonomous tools simultaneously. Without coordination, Claude Code in the terminal and Cursor in the editor overwrite each other's assumptions.

MemHub operates as a background daemon over Unix domain sockets or local HTTP:

                     MEMHUB LOCAL COORDINATION TOPOLOGY
                     
         [Claude Code CLI]      [Cursor IDE]      [Background Cron Bot]
                 │                    │                    │
                 └──────────┐         │         ┌──────────┘
                            ▼         ▼         ▼
                    ┌───────────────────────────────┐
                    │     MemHub Local IPC Bus      │
                    │   (Unix Socket / HTTP 8787)   │
                    └───────────────┬───────────────┘
                                    │
                                    ▼
                    ┌───────────────────────────────┐
                    │ Hy-Memory-30B-A3B Write Engine│
                    └───────────────┬───────────────┘
                                    │
                        ┌───────────┴───────────┐
                        ▼                       ▼
               [SQLite WAL Store]       [Chroma Vector DB]
               (Structured State)       (Semantic Embeddings)

MemHub uses conflict-free replicated data structures (CRDTs) to ensure concurrent updates from multiple CLI sessions merge without state corruption.

6. Benchmark Evaluation: LoCoMo, LongMemEval, and Latency Profiles

Hy-Memory 2.0 was benchmarked against leading open-source and proprietary memory frameworks on the LoCoMo and LongMemEval benchmark suites:

+---------------------------------------------------------------------------------------------------------------+
| BENCHMARK EVALUATION: HY-MEMORY 2.0 VS. PRODUCTION MEMORY FRAMEWORKS                                         |
+-----------------------------------+--------------------+--------------------+---------------------------------+
| Evaluation Metric                 | Mem0 (GPT-4o-mini) | EverMemOS (Sonnet) | Hy-Memory 2.0 (30B-A3B)         |
+-----------------------------------+--------------------+--------------------+---------------------------------+
| LoCoMo Overall Accuracy (%)       | 63.8%              | 68.4%              | 72.6%                           |
| Temporal Event Tracking (%)       | 58.2%              | 64.1%              | 74.8%                           |
| Contradiction Resolution Rate (%) | 61.4%              | 67.9%              | 79.2%                           |
| Code Constraint Adherence (%)     | 54.0%              | 69.2%              | 81.4%                           |
| Average Memory Write Latency (ms) | 1,840 ms           | 2,120 ms           | 380 ms (Local FP8)              |
| Context Storage Overhead (kTok)   | 46.2 kTokens       | 41.8 kTokens       | 28.4 kTokens (32% reduction)    |
| Relative Monthly Operating Cost   | 1.0x (Baseline)    | 1.45x              | 0.18x (82% cost reduction)      |
+-----------------------------------+--------------------+--------------------+---------------------------------+

Key Engineering Insights:

1. 79.2% Contradiction Resolution: Hy-Memory-30B-A3B achieved a 79.2% success rate in identifying and updating conflicting statements, outperforming Claude 3.5 Sonnet by 11.3%. The AMRL training directly penalizes contradictory memory entries.
2. Sub-400ms Memory Write Latency: Running locally in FP8 precision on a single RTX 4090 or Apple Silicon Mac, write latency averaged 380ms, eliminating the 2-second cloud round-trip latency of proprietary APIs.
3. 32% Reduction in Storage Footprint: By punishing conversational noise, the AMRL objective pruned non-essential facts, keeping the long-term context store lean and preventing downstream context saturation.

7. Production Deployment: SDK Setup and vLLM Serving

Deploying Hy-Memory 2.0 into an agent workflow can be accomplished using the open-source Python SDK:

Serving Hy-Memory-30B-A3B with vLLM

Launch the model endpoint using vLLM with FP8 quantization:

vllm serve tencent/Hy-Memory-30B-A3B \
  --quantization fp8 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.90 \
  --port 8080

Python SDK Integration

Integrate Hy-Memory 2.0 directly into an existing agent loop:

# -*- coding: utf-8 -*-
"""
Hy-Memory 2.0 Integration Example
Demonstrating tripartite memory management for autonomous coding agents.
"""

from hy_memory import HyMemoryClient, MemoryConfig

# Initialize client pointing to local vLLM endpoint
client = HyMemoryClient(
    base_url="http://localhost:8080/v1",
    storage_path="~/.local/share/memhub/memory.db",
    config=MemoryConfig(
        enable_chat_memory=True,
        enable_working_memory=True,
        enable_code_memory=True,
        contradiction_threshold=0.85
    )
)

def handle_coding_turn(session_id: str, transcript: list[dict]):
    """
    Asynchronously extracts architectural decisions and updates working memory.
    """
    # Step 1: Update memory state in background
    write_receipt = client.write_turn(
        session_id=session_id,
        messages=transcript,
        domain="code"
    )
    
    # Step 2: Query active constraints for next generation turn
    active_constraints = client.query_memory(
        session_id=session_id,
        query="What database migration constraints are active?",
        modalities=["code", "working"]
    )
    
    return active_constraints

if __name__ == "__main__":
    test_transcript = [
        {"role": "user", "content": "We are migrating from SQLite to Postgres 16. Do not use raw SQL queries."},
        {"role": "assistant", "content": "Understood. I will enforce Drizzle ORM schemas for all migrations."}
    ]
    
    constraints = handle_coding_turn("dev_session_01", test_transcript)
    print("Retrieved Constraints:", constraints)

8. Architectural Takeaways for Senior Systems Engineers

The release of Hy-Memory 2.0 demonstrates three foundational shifts in autonomous AI architecture:

1. The Write Path Demands Model Specialization: Forcing a single generalist model to handle user dialogue, tool calling, code generation, and memory maintenance is an anti-pattern. Specialized 3B-active MoE models trained via Attributed RL outperform frontier models at memory extraction while slashing latency and operational cost.
2. Attributed Loss Prevents Context Rot: Without counterfactual reinforcement learning that penalizes memory bloat based on downstream answerability, memory systems inevitably degenerate into unmanageable context dumps.
3. Cognitive Tripartition Over Flat Vector RAG: Treating chat history, active execution goals, and code constraints as interchangeable text chunks degrades agent reliability. Explicitly partitioning memory into Chat, Working, and Code engines provides the structural rigor required for multi-day autonomous workflows.