- Brought in across 6+ fast-growing platforms, from Series B to late-stage teams, to build the multi-tenant substrate for agent systems serving 250-450 concurrent instances across AWS Bedrock, AgentCore, and Azure environments.
- Led incident response for a production multi-agent cost explosion projected at $40K+ per day, implementing circuit breakers, per-session spend caps, and retrospective governance controls within 72 hours with zero recurrence.
- Defined Service Level Objective (SLO) frameworks across 3 client platforms using task completion rate, mean tool calls, hallucination escape rate, and end-to-end latency; improved SLO compliance from 71% to 96% over two quarters.
- Audited and built retrieval and governance systems supporting 1.8M+ tool invocations, with deterministic policy validation and zero known bypasses across deployments using Pinecone and OpenSearch.
- Built Go, Python, and TypeScript middleware, Azure DevOps templates, and deployment automation for 4 distributed engineering teams, reducing release-cycle overhead by roughly 40%.
- Refactored client integrations with Anthropic SDK, n8n, and Copilot Studio to enforce token optimization, caching, and low-infrastructure patterns, delivering an aggregate $200K annual reduction in cloud and LLM spend.
- Standardized data lakes and model registries across 2 client platforms, integrating Databricks and Domino Data Lab with classical ML workloads in scikit-learn, PyTorch, and TensorFlow.
Nick Cerutti
AI Architect & Consultant · Production LLMs & Agents
I design and harden AI systems that have to work in production. Agents, inference, and multi-tenant platforms that survive contact with real users and real budgets.
Open to new engagements
Book a 15-min callExperience
Professional experience
Systems, infrastructure, and advisory work across trading, production AI, and distributed engineering organizations.
Projects
Selected open-source work
Tooling for agent memory, governance, orchestration, and the operational realities around production deployment. Building in public as proof of work.
Drop-in persistent memory for AI agents. Cut token consumption by 97% with a single zero-dependency binary. No Docker, no external databases, no cloud storage. Pure pragmatism. +450 GitHub stars and 35 forks.
Golang / Agent Memory / Hybrid Retrieval / MCP Server / Persistent State
- Built a storage layer using bbolt for facts and metadata plus chromem-go for vector search, with durable reconciliation and background reindexing.
- Implemented hybrid recall using vector similarity, keyword scoring, recency weighting, and reciprocal rank fusion tuned for prompt injection into live agent runs.
- Built ingestion through direct remember flows, extraction-assisted memory writes, agent self-curation, and asynchronous consolidation with decay and pruning.
- Shipped a full MCP server with auto-onboarding, memory search, checkpointing, and real-time Bubble Tea observability dashboards.
02Epica
Embeddable Rust library for formal causal belief revision in LLM agents. Grounded in 5 arXiv papers, it delivers dual-process uncertainty quantification and cryptographically auditable trails.
Rust / PyO3 / Causal Graphs / Agent Governance / MCP
- Built an embeddable 12-crate Rust runtime implementing formal AGM contraction and semantic-equivalence postulates to enforce contradiction-aware belief updates across tool cycles.
- Implemented dual-process uncertainty via System 1 Noisy-OR causal propagation and System 2 asynchronous LLM reflection, derived directly from foundational variational free-energy research.
- Developed a tamper-evident audit ledger using BLAKE3 Merkle hash chains, enabling decentralized third-party verification of agent memory and governance history via an offline CLI tool.
- Engineered a native MCP server with 16 routes supporting OAuth 2.1 authentication and Prometheus telemetry, exposing Python SDK bindings compiled via PyO3.
03Meridian
Inference-time compute scheduler for reasoning models, optimizing token-generation phases via dual-queue dispatch and entropy-driven budget control.
Rust / Python / CUDA / Inference Infrastructure / vLLM
- Designed and engineered a phase-aware vLLM scheduling layer for reasoning models including DeepSeek-R1, Qwen3, and o3 to decouple think-decode and output-decode workloads at runtime without codebase forks.
- Implemented a dual-queue dispatch system alongside a three-tier phase-aware KV cache eviction mechanism, neutralizing inter-token latency bottlenecks caused by heavy reasoning batch dynamics.
- Integrated entropy-driven budget forcing leveraging EAT and RPDI signals via custom CUDA kernels running on secondary GPU streams to dynamically inject termination boundaries based on convergence.
- Developed a drop-in vLLM plugin architecture using reversible attribute delegation, adding disaggregated KV transfer hooks to interface with high-performance prefill-decode fabrics.
04Stratum
Local dual-tier LLM serving stack featuring sub-millisecond adaptive complexity routing and real-time VRAM orchestration.
Python / vLLM / llama.cpp / Infrastructure Optimization / ASGI
- Designed and deployed a dual-tier LLM serving stack capable of supporting 200 concurrent users via optimized container orchestration.
- Built an ASGI-compatible adaptive proxy middleware executing sub-millisecond multi-dimensional routing based on prompt complexity, SLA latency targets, and hardware availability.
- Developed an asynchronous hardware monitoring system utilizing raw NVML bindings to track GPU state and dynamically enforce real-time circuit-breaking policies under load.
- Implemented benchmarking harnesses for prefix caching evaluation and multi-user token-throughput stress testing across heterogeneous engine setups.
05Drako
Open-core security and governance platform for production AI agents, spanning static analysis, policy authoring, and deterministic runtime enforcement.
Python / Policy-as-Code / Runtime Governance / AI Agent Security
- Designed and implemented a 97-rule, zero-LLM policy engine across security, compliance, and FinOps, achieving 100% deterministic evaluation with sub-2ms overhead (<0.3% of an LLM call).
- Built static analysis features featuring reachability analysis to eliminate false positives, Agent BOM generation, baseline tracking, and native GitHub Actions/SARIF CI integration.
- Built a 13-stage production runtime pipeline covering Presidio-based Data Loss Prevention (DLP), cryptographic audit trails, circuit breakers, spend controls, and EU AI Act (Art. 14) human-in-the-loop review.
- Added autopilot config generation, behavioral drift detection, and 100% offline desktop MCP scanning for Cursor and Claude with framework guardrails for LangGraph, CrewAI, and AutoGen.
06Forge
Universal harness that wraps existing agent systems to add observability, memory, budget controls, and safe self-evolution without forcing framework migration.
Python / Multi-Agent Orchestration / Self-Optimizing Runtime / Observability
- Designed adapters for LangGraph, CrewAI, AutoGen, and generic async callables on top of a shared runtime and event bus.
- Implemented OpenTelemetry tracing, per-agent and per-model cost tracking, real-time budget enforcement, and REST plus SSE monitoring APIs.
- Developed cross-run memory that combines vector retrieval, knowledge graph persistence, provenance tracking, and conflict-aware versioning.
- Built a snapshot-safe self-evolution loop for prompts, parameters, models, and topology with rollback and A/B safeguards.
Work with me
Production-minded engagement.
Five ways to engage. One standard: production.
Ranges scale with system size and team count; final scope is set on the call.
Production AI Audit
Fixed price · 2-3 weeks
I audit agent and LLM systems for cost, reliability, and governance. Every finding priced in dollars, not just severity levels.
From $15K
Platform Design & Build
Fixed-scope project · ~6 weeks
I design and build the multi-tenant substrate agent systems run on: routing, memory, evaluation, and spend control, on Bedrock, AgentCore, or Azure. Delivered to production at 250-450 concurrent instances, then handed to your team.
$45-75K
EU AI Act readiness
Fixed-scope project · 4-8 weeks
Annex III systems must comply by 2 December 2027. I build the technical controls the Act actually asks for: Article 14 human-in-the-loop review, cryptographic audit trails, data-loss prevention, and deterministic policy enforcement — controls that run in production, not a compliance binder.
$30-60K
Fractional AI Architect
Retainer · 2-3 days/month
Ongoing architecture, evaluations, SLOs, and stack decisions, working directly with your CTO or VP Engineering. For teams already shipping.
$12-18K/mo
Production emergency
Incident response · priced after triage
If agents are burning money right now, I contain it: circuit breakers, spend caps, and governance within days.
Email me directly — skip the calendar.
Same-day
If you're building something where AI has to work in production, let's talk.
Book a 15-min callAlso open to full-time Head of AI and AI Architect roles. Download the resume or ask in the call.
Notes
Engineering notes
Short technical notes from real infrastructure work. The kind of details that usually stay inside private Slack channels or incident reviews. Last updated .
Build notes from KVSentry: why per-tenant cache isolation is not the zero-cost mitigation it gets quoted as, how to price it from lost cache hit rate instead of a published percentage, and the one metric that moves the answer by two orders of magnitude.
FinOps / KV Cache / Multi-Tenant Security / vLLM
9 min read
Itemising what one architectural component of a 284B model actually costs to serve - and why the published description of it was wrong by 2x.
Inference / Cost Modeling / Open Source / DeepSeek-V4
6 min read
Most infrastructure side projects are written on a laptop and benchmarked on a slide. A per-component verification column is cheaper than a bigger claim, and worth more.
Engineering Practice / Open Source / Benchmarking / Architecture Decision Records
8 min read
Capabilities
Technical focus
Broad enough to span research, infra, and runtime operations.
Specific enough to ship in production.
Languages & Runtimes
Core implementation languages for systems work, infrastructure tooling, and performance-sensitive runtime paths.
Go, Python, TypeScript, Rust, SQL, C++, CUDA.
Agent Systems
For orchestrating multi-step agent behavior, memory, tool use, and compound execution across frameworks.
Multi-agent orchestration, A2A protocol, MCP server and client development, Context engineering, Persistent and episodic memory systems, Hybrid retrieval-augmented generation (RAG), Tool-use and function-calling pipelines.
Inference & Serving
From single-GPU deployments to multi-tenant inference clusters under real production load.
VLLM, Triton Inference Server, Model routing, AWQ and GPTQ quantization, KV-cache optimization, Prompt caching, Structured output contracts, Low-Rank Adaptation (LoRA) and Quantized Low-Rank Adaptation (QLoRA) pipelines.
Scale & Reliability
For systems that need to stay observable, cost-aware, and resilient as traffic, tenants, and model complexity grow.
Multi-tenant agent platforms, Kubernetes at scale, Service Level Objective (SLO) and error budget design, Incident response, Chaos engineering, Capacity planning, OpenTelemetry, Prometheus and Grafana, Agent trace instrumentation, Real-time memory analytics, Token spend tracking, Audit trails, Evaluation loops.
Edge & Constrained Environments
When Kubernetes is too much and reliability still is not optional.
Zero-dependency binaries, Embedded storage, Offline-first agents, Single-binary MCP servers, Bubble Tea observability, WASM-compatible tooling.
Security & Governance
Enforcement that runs at runtime, not just on paper, across agent policies, compliance, and tool-call controls.
Policy-as-code, Runtime tool-call validation, Deterministic evaluation loops, Reachability analysis, Agent BOM, Desktop agent scanning, Prompt injection mitigation.
Delivery & Automation
For shipping changes repeatedly, enforcing standards, and reducing manual operational overhead.
GitHub Actions, ArgoCD, MCP integrations, CLI tooling, Pre-commit hooks, SARIF, GitOps pipelines.
Education
Academic background
Computer science, business, and applied ML training with an emphasis on systems thinking.
S21 University
B.S. in Computer Science
GPA 3.92
Buenos Aires, Argentina
Coursework
Data Structures and Algorithms, Advanced Operating Systems, Software Engineering, Database Systems, Computer Architecture, Entrepreneurial Development.
Jan. 2018 - May 2021
IAE Business School
Master of Business Administration (in progress)
Buenos Aires, Argentina / Online
Coursework
Corporate Finance, Strategy & Business Model Innovation, Financial Accounting, Operations & Logistics, Leadership & Human Behavior.
Feb. 2025 - May 2027
DeepLearning.AI / Coursera
MLOps Specialization
Online
Coursework
Introduction to Machine Learning in Production, Machine Learning Data Lifecycle in Production, Machine Learning Modeling Pipelines in Production, Deploying Machine Learning Models in Production.
Mar. 2024