Save this runbook if you use Claude + Codex locally.
On February 19, 2026, one measured pass cut zsh -i -c exit from 1.794s to 0.386s(-78.5%). [E1]
Everyone says "fix prompts"; in practice, this is a systems issue, and teams without baselines + retrieval gates ship false confidence.
This is the same repeatable 5-step sequence I now reuse: baselines, gates, memory tiers, retrieval benchmarks, and drift checks.
Evidence Snapshot (Audited February 20, 2026)
- launch agents loaded: 8/8 [E2]
- daily memory coverage: 108 files (2025-09-27 through 2026-02-20) [E3]
- extracted totals: 7596 prompts, 22858 tool calls [E3]
- benchmark A (2026-02-20T04:16:45Z): hit_rate=1.0000, mrr=0.7500,ndcg=0.8143, precision=0.1250, recall=1.0000, triage=0 [E4]
- benchmark B (2026-02-20T04:56:59Z): hit_rate=0.9500, mrr=0.7125,ndcg=0.7727, precision=0.1188, recall=0.9500, triage=2 [E5]
- latest benchmark (2026-02-20T06:58:19Z): dense model nomic-embed-text:latest, vector_dim=768, dense enabled, gates passing [E6]
5-Step Saveable Implementation
Step 1: Baseline First
Measure before any config edits.
- zsh -i -c exit: 1.794s -> 0.386s (-78.5%) [E1]
- codex --help: 0.210s -> 0.116s (-44.8%) [E1]
- claude --help: 0.464s -> 0.312s (-32.8%) [E1]
Step 2: Add Reliability Gates
Replace reminder-based quality with execution checks:
- preflight checks
- apply + verify loops
- browser validation for UI work
- explicit done criteria
Step 3: Split Memory into 3 Tiers
- Tier 1: raw session history
- Tier 2: curated working memory
- Tier 3: hybrid retrieval + graph layer
This prevents noisy logs from polluting high-signal memory.
Step 4: Benchmark Retrieval Like Infra
Track:
- Hit@K
- MRR@K
- NDCG@K
- Precision@K
- Recall@K
- triage issue count
Back-to-back snapshots caught quality drift (triage: 0 -> 2) while gates remained green. [E4][E5]
Step 5: Fix Drift in Ingestion Paths
Incident found:
- active drafts in ~/docs/content/drafts
- sync defaults still pointed to old root
Fix:
- set canonical drafts root
- keep legacy remap compatibility
- re-sync and rebuild index
Outcome: graph alignment became explicit and auditable. [E7]
Copy-Paste Run Sequence
~/control/automation/ai-dev-optimization/scripts/manage_knowledge_launchagents.sh status python3 ~/control/automation/ai-dev-optimization/scripts/run_snapshot.sh 24 python3 ~/control/automation/ai-dev-optimization/scripts/sync_article_memory.py python3 ~/control/automation/ai-dev-optimization/scripts/build_memory_rag_index.py --backend hybrid --allow-sparse-fallback --output ~/control/knowledge/knowledge-memory/data/rag/hybrid-index.json python3 ~/control/automation/ai-dev-optimization/scripts/benchmark_memory_retrieval.py
Weekly Cadence (Use This As a Checklist)
- Verify launch agents are loaded.
- Refresh extraction + sync.
- Rebuild retrieval index.
- Run benchmark and compare triage delta.
- Promote only high-signal memory updates.
Failure Patterns to Watch
- Health checks green, but capability degraded.
- “One memory file” turning retrieval into noise.
- Path drift between active drafts and sync roots.
Conclusion
The point of this setup is simple:
- less waiting
- less guesswork
- fewer silent regressions
Prompt quality still matters, but the durable gains came from instrumentation and operating discipline.
Use this as a working runbook before your next sprint, then iterate the thresholds and checks to match your stack.