puodziukas.dev · remote · available Q3 2026

MLOps / Infrastructure Engineer

389ms P50. Thunderbolt 4 fabric. Apple Silicon MLX. Zero-copy pipeline. Production.

389ms P50. Measured on the operator cluster. The benchmark number is real because the architecture is real.

reproducible · verified · available Q3 2026

The problem

A model benchmarks at 200ms and runs at 800ms under load. The Kubernetes config is correct and GPU utilization sits at 40%, which points to the data path rather than the model — an infrastructure problem wearing an ML costume.

What exists, reproducible

389ms P50 measured on the operator cluster. Reproducible. Thunderbolt 4 fabric, Apple Silicon MLX, zero-copy pipeline. The bottleneck analysis that gets you from 800ms to 389ms is documented and reproducible.

Relevant projects

• →Exo distributed inference cluster: upstream lineage, Thunderbolt 4 mesh, Apple Silicon MLX, 389ms P50
• →Zero-copy inference pipeline: Thunderbolt 4 fabric, documented bottleneck analysis
• →CI/CD for ML: adversarial regression suite on every model push, zero-downtime deploys
• →Prometheus / Grafana: drift detection, auto-retrain trigger, inference SLA alerting
• →Inference API cost: $26,400/mo → $0 (self-hosted). Documented methodology, reproducible.
• →Multi-node topology design: Thunderbolt 4 fabric, peer-loss survivable (asymmetric fallback)
• →Thermal gate: CPU/VRAM load monitoring with auto-pause, no inference under thermal pressure

What you receive in 30 days

1. CI/CD pipeline for model: build → adversarial eval → deploy, zero-downtime tested (Week 1)
2. Observability stack: Prometheus metrics, Grafana dashboard, drift alert configured
3. First model deployed with rollback tested and latency baseline established

Companies at this intersection

Modal · Replicate · CoreWeave · Lambda Labs · RunPod · Baseten · SambaNova · Cerebras

All results under NDA. Representative figures. SHA-256 hash-chained audit trail available on request. · puodziukas.dev