CASE STUDY · INFERENCE INFRASTRUCTURE · AGENTIC SYSTEMS
Multi-Agent Cluster — $0 Cloud/API Cost
multi-agent concurrent inference · 48GB pooled across 2 nodes · 0 API calls · 40 overnight cycles
2AM. 40-cycle autoresearch loop running. Cloud/API billing tab: $0 — self-hosted. Multi-agent swarm, M1 Max + M4 Mini, no API calls. The loop finishes before anyone else logs in.
The Problem
Running a multi-agent research loop on cloud inference costs real money at scale. The same managed-API workload this cluster replaces ran $868/day in inference spend: $26,400/month at steady state. Over a year of overnight research cycles, that compounds to $316,800 in API spend alone, before compute, storage, or orchestration overhead.
The second problem is latency. Cloud API rate limits throttle concurrent agent workloads: a multi-agent swarm hitting the same endpoint competes with itself for token budget and rate-limit windows. At 40 cycles per overnight run, that contention compounds into a 3–5x wall-clock multiplier versus on-device inference.
The third problem is data egress. Every agent prompt that leaves the device crosses a third-party API boundary. For research workloads that include proprietary codebase context, internal metrics, or unreleased architecture decisions. That boundary is a compliance and IP risk, not just a cost.
The Approach
The 2-node cluster (M1 Max 32GB + M4 Mini 16GB) pools 48GB of unified memory addressable by both CPU and GPU. At open-weights model sizes used in production research loops (7B–14B parameter range), 48GB supports multiple simultaneous model instances with full KV-cache allocation. With aggressive KV-cache prefix sharing across agents working on the same codebase context, the architecture runs multiple concurrent agents, each with its own inference context, system prompt, and tool chain.
# Cluster configuration
primary_node: M1 Max · 32GB unified
peer_node: M4 Mini · 16GB unified · Thunderbolt 4 fabric
total_unified: 48GB pooled
# Agent routing: multi-agent concurrent
agents: multiple independent context windows
model: 30B Q4 (served)
kv_cache: prefix-shared on common codebase context
api_calls: 0 (all inference local)
# Autoresearch loop
cycles_per_run: 40+ (8hr overnight window)
gate: binary yes/no criteria per cycle
commit_on_pass: git commit after each passing cycle
revert_on_fail: git revert on regression
The autoresearch harness runs a ratchet loop: each cycle runs a binary success-or-fail gate against a measurable target. Pass → commit the improvement. Fail → revert and try a different approach. The loop runs unattended overnight, building a git commit trail of every successful improvement. 40 cycles in an 8-hour window produces 40 independent experiments with full rollback capability. The same workload on managed cloud inference bills $868/day.
DELIBERATION PIPELINEblock on any single agent fail · 140/140 caught · 0 bypasses
The overnight autoresearch architecture changes the research cadence from experiment-per-day (cloud, rate-limited) to experiment-per-cycle (local, rate-unlimited). 40 cycles overnight is 40 binary experiments with full commit provenance, each gated by a binary pass/fail oracle — no rate limits, no inference spend.
The cloud/API cost floor is $0/month — self-hosted; the real cost floor is electrical draw plus hardware amortization, not zero. There is no invoice. Every dollar saved on inference API spend is a dollar that stays in the system — re-deployed into more cycles, more agents, or more aggressive success criteria. Cloud inference spending is the most compressible cost in a modern AI stack, and it is one of the least frequently optimized.
