How delta updates significantly reduce adaptation latency and cost
Delta incremental updates and non-LLM merging give ACE significant advantages over GEPA / Dynamic Cheatsheet in both adaptation latency and token cost.
Compared to GEPA:
Compared to Dynamic Cheatsheet (DC):
Although ACE-generated Playbooks are typically longer than GEPA prompts, inference cost during serving does not increase linearly. Reasons:
Offline Scenario (AppWorld)
Compared to GEPA:
| Metric | GEPA | ACE | Change |
|---|---|---|---|
| Adaptation Latency (seconds) | 53,898 | 9,517 | -82.3% |
| Number of Rollouts | 1,434 | 357 | -75.1% |
Online Scenario (FiNER)
Compared to Dynamic Cheatsheet (DC):
| Metric | DC | ACE | Change |
|---|---|---|---|
| Adaptation Latency (seconds) | 65,104 | 5,503 | -91.5% |
| Token Cost ($) | 17.7 | 2.9 | -83.6% |
Sources of Latency Gains
1
Delta Instead of Full Rewrite
Generates only a small set of candidate bullets each time; model output size is small.
2
Non-LLM Logic Merging
Merging is just id matching + counter updates; no LLM calls needed.
3
Parallel Merging
Multiple deltas can be merged in parallel for batch adaptation.
4
Fewer Rollouts
A single rollout accumulates multiple experiences; no need for repeated trial-and-error.
Long Context Does Not Equal High Serving Cost
Although ACE-generated Playbooks are typically longer than GEPA prompts, inference cost during serving does not increase linearly. Reasons:
| Mechanism | Effect |
|---|---|
| KV Cache Reuse | Identical prefixes avoid repeated prefill |
| KV Cache Compression | Smaller memory footprint |
| KV Cache Offload | Can be loaded from disk/remote storage |
Modern serving stacks have made system-level optimizations for long-context workloads, making ACE's long Playbooks feasible in production environments.
When Savings Are Greatest
| Scenario | Significant Savings |
|---|---|
| Offline optimization | ~75% fewer rollouts vs GEPA |
| Online adaptation | ~90% less latency vs DC |
| Domain tasks requiring long-term evolution | High efficiency of delta accumulation |