Replacing full rewrites with minimal edit units to avoid context collapse
ACE's first key mechanism is incremental Delta updates: treating context as a collection of bullets, where the Curator generates only a small set of candidate bullets each time, then deterministically merges them into the existing Playbook.
Full rewriting requires the LLM to summarize all known information at every step. As context grows longer, it tends to "compress into a short summary," leading to context collapse.
Delta updates counter this: always edit only relevant bullets, never regenerate the whole.
Bullet-based design brings three benefits.
The bullet concept is similar to memory entries in Dynamic Cheatsheet and A-MEM, but ACE additionally requires:
In offline mode, ACE supports multiple epochs to repeatedly digest the training set:
Why Not Rewrite Everything
Full rewriting requires the LLM to summarize all known information at every step. As context grows longer, it tends to "compress into a short summary," leading to context collapse.
Delta updates counter this: always edit only relevant bullets, never regenerate the whole.
Three Key Properties
Bullet-based design brings three benefits.
| Property | Description | Effect |
|---|---|---|
| Localization | Only affected bullets are updated | Avoids rewriting entire context |
| Fine-grained retrieval | Generator can focus on most relevant bullets | Improves reasoning efficiency |
| Incremental adaptation | Supports efficient merging, pruning, deduplication | Ensures long-term evolution feasibility |
Delta Generation and Merging
1
Reflector Outputs Experiences
Distills specific experiences from the latest trace and execution feedback.
2
Curator Generates Delta
Organizes experiences into a small set of candidate bullets (additions or modifications).
3
Deterministic Merging
Non-LLM logic matches by id, appends, or increments counters.
4
Parallel Merge
Multiple deltas can be merged in parallel, supporting batch adaptation.
Relationship to LLM Memory Frameworks
The bullet concept is similar to memory entries in Dynamic Cheatsheet and A-MEM, but ACE additionally requires:
- Explicit helpful / harmful counters
- Each bullet carries only one small unit (one strategy, one concept, one failure pattern)
- Updates go through the delta channel, not full replacement
Supporting Multi-Round Adaptation
In offline mode, ACE supports multiple epochs to repeatedly digest the training set:
- Each round generates deltas, progressively strengthening the Playbook
- Ablation experiments show multiple epochs significantly improve TGC on AppWorld compared to single-round
Why Use Non-LLM Logic for Merging
- Merging is just "append + count," requiring no model reasoning
- Ensures determinism, parallelism, reproducibility
- Avoids semantic drift introduced by LLM merging