EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory

Hongru Cai1, Ran Wei2, Wenjie Wang3, Chengfa Wu2, Ning Song2, Yongqi Li1, Wenjie Li1
1The Hong Kong Polytechnic University 2Hangzhou Diagens Biotechnology Co., Ltd 3University of Science and Technology of China
{henry.hongrucai, liyongqi0}@gmail.com

Overview

Conditional memory expands an LLM’s capacity through learned n-gram embeddings. At each token position, the model uses token sequences of different lengths ending at that position to look up embeddings that participate in its computation. EngramEdit enables factual knowledge updates through this memory. It first computes the memory representations needed to predict revised facts, then jointly updates the corresponding embeddings across expressions and edits. Stronger penalties on frequently reused embeddings help preserve unrelated knowledge. Updated facts remain usable across different expressions and in multi-hop reasoning, while general capabilities are largely preserved.
Conditional memory structure, evidence from disabling Engram memory, and decoupled knowledge updates using EngramEdit.
Conditional memory and knowledge updating with EngramEdit. Click any figure to view it at full resolution.

Key Features

  • Effective knowledge updates. Revise facts through conditional memory alone, with high editing success.
  • Cross-expression recall and reasoning. Use revised facts across unseen expressions and in multi-hop reasoning.
  • Knowledge and capability preservation. Largely preserve unrelated knowledge and general capabilities through sequential updates.

Abstract

Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Beyond model scaling, this architecture has demonstrated the potential to decouple factual knowledge storage from general-purpose computation, offering a promising route to updating factual knowledge while keeping the Transformer backbone fixed. Realizing this potential is challenging because different expressions of a fact may activate different n-gram embeddings, while updating shared embeddings can unintentionally change the model’s predictions about other facts. We propose EngramEdit for decoupled knowledge updates through conditional memory. EngramEdit first computes target memory representations that make the model predict the updated fact across multiple expressions. It then jointly updates the shared n-gram embeddings to match these targets across expressions and edits, penalizing updates to frequently reused embeddings more strongly to preserve unrelated knowledge. Experiments show that EngramEdit enables independent factual knowledge updates through conditional memory, achieving near-perfect editing success. Revised knowledge is usable across unseen expressions and in multi-hop reasoning, with nearly 3× the strongest baseline's accuracy under chain-of-thought (CoT) prompting. Unrelated knowledge and general capabilities are largely preserved even as factual updates accumulate. These findings show that EngramEdit turns conditional memory into an editable knowledge interface, extending its role beyond model scaling to support decoupled knowledge updates.

Method

Different expressions of a fact can activate different n-grams, while different facts may share embeddings. EngramEdit first computes how memory representations should change, then jointly allocates those changes to the shared embeddings.

  1. Compute targets. Generate additional expressions of each fact. Optimize a temporary perturbation shared across their memory representations so that the model predicts the revised fact.
  2. Map expressions to memory. Map each target representation to the activated n-grams, identifying embeddings shared across expressions and edits.
  3. Update memory jointly. Solve for one update per embedding to match the targets. Stronger penalties on frequently reused embeddings help preserve unrelated knowledge.
EngramEdit computes target representations, maps them to shared n-gram embeddings, and jointly allocates regularized updates.
EngramEdit's target computation, memory mapping, and joint memory updating.

Results

Experiments on LongCat-Flash-Lite test whether facts can be updated through conditional memory, used across expressions and in reasoning, and revised while largely preserving unrelated knowledge and general capabilities.

Effective Knowledge Updates

EngramEdit achieves near-perfect editing success and strong generalization on CounterFact and ZsRE, with the highest Utility among the evaluated methods on both datasets.

Results after 2,000 sequential edits. Subscripts show 95% confidence-interval half-widths.
MethodCounterFactZsRE
Eff. ↑Gen. ↑Spec. ↑Util. ↑Flu. ↑Cons. ↑Eff. ↑Gen. ↑Spec. ↑Util. ↑
Pre-edited9.5±1.2811.2±1.1986.8±0.9435.8544.0±1.6715.0±0.3736.9±1.3036.5±1.2941.6±1.1938.3
FT94.9±0.9663.6±1.8343.8±1.6367.4505.1±2.0413.3±0.3676.7±1.1376.8±1.1543.1±1.2765.5
FT-L15.3±1.5812.5±1.2483.3±1.1137.0542.0±1.7014.7±0.3658.0±1.4754.4±1.4845.8±1.2452.7
AdaLoRA61.1±2.1327.5±1.5551.1±1.3546.6544.3±1.5712.9±0.3570.9±1.4064.7±1.4646.8±1.2760.8
UnKE84.5±1.5958.0±1.8847.5±1.6163.3546.5±1.6415.0±0.3586.9±1.1379.0±1.3349.9±1.3271.9
MoEEdit99.1±0.4167.1±1.6963.5±1.2876.6544.7±1.6514.7±0.3385.1±0.9178.7±1.2144.4±1.2669.4
MFT-S99.0±0.4559.1±1.9285.2±0.9081.1555.2±1.3815.3±0.3666.9±1.3556.5±1.5336.9±1.1753.4
MFT-A93.9±1.0440.7±1.7566.6±1.1267.1496.7±4.0411.5±0.4062.8±1.4951.3±1.5733.7±1.1349.3
EngramEdit99.5±0.3197.0±0.6985.2±0.9293.9565.2±1.1016.1±0.3697.3±0.4093.7±0.7938.3±1.1876.4

Using Updated Facts in Reasoning

On MQuAKE, updated facts support multi-hop answers. With chain-of-thought prompting, EngramEdit reaches nearly three times the strongest baseline's accuracy and leads across the evaluated hop counts.

Direct and chain-of-thought multi-hop accuracy, including results by hop count on MQuAKE.
MQuAKE results with direct and chain-of-thought prompting, including chain-of-thought accuracy by hop count.

Preserving General Capabilities

General capabilities are largely preserved through 5,000 sequential CounterFact edits. The evaluation covers six tasks with 100 examples per task and reports mean F1 and changes on individual tasks.

Mean general-capability F1 during sequential editing and task-level changes after 5,000 edits.
Mean F1 during sequential editing and task-level changes after 5,000 edits, relative to pre-edit scores.

Citation

Please cite our paper as:

@misc{cai2026engramedit,
  title  = {EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory},
  author = {Hongru Cai and Ran Wei and Wenjie Wang and Chengfa Wu and Ning Song and Yongqi Li and Wenjie Li},
  year   = {2026},
  eprint = {2610.10533},
  archivePrefix = {arXiv},
  primaryClass = {cs.CL},
  url    = {https://arxiv.org/abs/2610.10533}
}

Acknowledgements

We thank the contributors and open-source projects that support EngramEdit with models, editing utilities, datasets, and evaluation resources.

The code is available under the MIT License. Third-party models, datasets, and dependencies retain their own licenses.

Related Projects