Abstract (plain-language)
We ran a 20-iteration optimization campaign on a single 8-billion-parameter open model (qwen3:8b, 4-bit, one consumer 8 GB GPU) under a pre-registered, hash-locked benchmark protocol. The final architecture — dual-pool aggregation with a mechanical merge — reached 10/10 on our frozen ten-task reasoning/context/memory suite, replicated eight consecutive times, the same suite on which claude-opus-5 scores 10/10 under identical conditions, and 87% of the combined frozen-suite composite, at roughly ten seconds per task.
The load-bearing findings are negative: (1) the selection bottleneck — across six mixture architectures whose candidate pools provably contained the correct answer, a small-model judge failed to select it every time, scoring at or below its own best member; and (2) instruction-presence poisoning — an auxiliary instruction anywhere in the prompt measurably degrades knife-edge and sequential answers, making single-prompt designs zero-sum. The architecture that survived removes model judgment entirely: candidates are generated in two isolated pools, and the pool to aggregate is chosen by one bit — did deterministic code substitute a value? — that no model sets or sees.
Headline results
| configuration | tier-1 (10) | tier-2 (7) | composite | s/task |
|---|---|---|---|---|
| original 2-model MoA | 9.0 (10→9 on replication) | — | — | 36.0 |
| Self-MoA-Seq, 1 model | 9.0 stable | 4.0 | 13.0 | 7.8 |
| + executable primitives | 9.0 stable | 5.0 | 14.0 | 6.4 |
| dual-pool 5/2 (shipped) | 10.0 ×8 | 4.75 | 14.75 (87%) | ~10.3 |
| claude-opus-5 (same harness) | 10 | 7 | 17 | (API) |
What failed — the graveyard is the contribution
Eight orchestration mechanisms were built, measured, and discarded, six of them failing at the identical joint: a small model asked to choose (judge panels, cross-adjudication, fold-as-arbiter, answer verification, action re-commit, evidence adjudication). Every surviving gain came from bounding what the model sees per call and moving decisions into deterministic code — the executable-primitive guards were forced by probes that caught confidently-wrong computed values passing lenient checkers.
Integrity protocol — and the three retractions
Task files were SHA-256 hash-locked with an add-only freeze; every mechanism carried pre-registered predictions with committed failure branches; no single-pass number was ever treated as evidence; held-out suites were authored and locked before first model contact. The protocol caught our own errors, and we published all three retractions the same day each was found — an inflated agentic result that depended on a truncated evidence window, an unstable model score that didn't replicate, and a benchmark win we had misattributed to the wrong mechanism. If you evaluate orchestration vendors: ask them what they've retracted.
Honest scope
The parity claim is scoped to our frozen single-turn suite and to the exact conditions published. It does not extend to long-horizon agentic control — our own agentic benchmarks show the gap and we say so — and we do not claim our sidecar improves frontier-model scores (we tested that thesis and retracted it when the evidence failed).
Read the full report
The complete paper (methods, ablations, threat analysis, references) ships in the repository as docs/papers/MOAMAO_V1.md, with the full iteration log — retractions verbatim — in docs/AUTORESEARCH_LOG.md at tag v1. The shipped engine is the benchmark code with its guards intact: how it becomes the product.
Build on this: the orchestration guide · what is mixture-of-agents · choosing a platform