Like most AI agent products, we’ve gone through our fair share of product rewrites as the foundation models have evolved. Our last rewrite (fourth? by some measure) was anchored on splitting the agent runtime from the agent definition.
The runtime is how an agent runs: the loop that calls the model and executes tools, with capabilities like receiving messages mid-turn (“steering”) or checkpointing work. The definition is what an agent is: its system prompt, its model, and which tools, skills, and subagents it has. The runtime is in code, but the definition is just data.
This split has served us well: agent iterations are much faster, and the runtime feels more solid than before. This is not strictly a new idea: any “managed agents” service, like the ones from Anthropic or Vercel, ship a runtime along with an interface to define the agent. We chose to (mostly) hand-roll ours: important for us to have minimal external parts between us and the model.
Background
Our previous stack ran the agent loop in a Cloudflare Durable Object, with external infra for heavy tools that needed headless browsers. It organically grew to expand runtime capabilities (e.g., edit messages to create new branches for message history). The agent definition lived next to it, and everything was versioned with git. That worked well until we realized we wanted two new agents:
- A code review agent that adversarially reviews the main agent’s output.
- A video analyzer subagent that reads video frames without inflating the main agent’s context window.
Both needed their own context window, system prompt, and tool set, plus integrations with the main agent. We managed to fit them in, but then realized we needed to support a lot more: more custom agents that had to run independently or as subagents, and versions of those agents we wanted to evaluate separately.
Should we outsource the runtime?
“How the agent runs” and “what the agent does” are distinct problems, and the refactor forced us to question why we were solving both in-house.
Building the runtime in-house was always a questionable decision. We had looked at LangGraph back when it was the default answer, and a few things put us off: too many building blocks that a basic agent loop does not need, docs that lagged the actual code, and a Python-first approach (we’ve always been TypeScript). Mastra and others prioritized TypeScript, but they all seemed to overcomplicate a basic loop.
The “thinking in LangGraph” approach makes agent development sound like a “graph problem”, but it is not. It is about the model’s context, when it reaches for a tool, and where its judgment breaks down. Fighting an external abstraction in the core product path feels like taking valuable time away from actually building the product.
Discovering Pi
But then I found Pi. Others have already written the love letters, so I’ll keep mine short: a minimal core, an extension system that can carry real weight, and source code that’s a pleasure to read.
We realized that we can run Pi in RPC mode, and the extensions API was enough to cover all of our runtime needs. Today our agent runtime is basically the Pi core, with 16 custom extensions around it.
Turns each configured helper agent into a callable tool, running it with its own context window and returning its output to the parent.
“Hive” is where our agents live
Pi + extensions solved our agent runtime, and our agent definitions needed a place to live. That became Hive, an internal tool that solves “versioned prompt management” for our agents, tools, and skills.
Every off-the-shelf LLM evaluation/tracing tool bundles a “prompt management” feature that probably does 50% of what you want. When we started building Empirical, we explored a few of them and then chose to keep everything in the repo. Git gives versioning and code reviews for free, right?
In 2026, you can build your own tool that does 100% of what you want. Building reviews and versioning with a sick diffs UI library is a few sessions with your favorite coding agent.
Hive is now our central registry for agents, their versions, and the tools and skills they can use. We’ve also built in support for shared “prompt blocks” that render into more than one agent’s prompt.
Here’s how Hive expresses an agent definition in TypeScript.
interface HiveAgentVersion {
version: number;
system_prompt: string;
model: string;
tools: string[];
all_tools: boolean;
skill_refs: string[];
all_skills: boolean;
subagent_refs: string[];
}Agents become data that implements the type. As an illustration, the Pirate agent runs with GLM-5.2, and will rely on its subagent to bridge the model’s lack of vision.
pirate-agent:
version: 1
model: fireworks/models/glm-5p2
system_prompt: |
Always talk like a pirate. Use subagent to read images.
tools: []
all_tools: false
skill_refs: []
all_skills: false
subagent_refs:
- look-at-agentlook-at-agent:
version: 1
model: anthropic/claude-sonnet-5
system_prompt: |
Inspect images and return an accurate description of the relevant
visual details to the parent agent.
tools:
- read
- bash
all_tools: false
skill_refs: []
all_skills: false
subagent_refs: []One runtime for a hundred agents
That interface above converts the agent into a row in a
database. Publishing a new version is a quick change in the Hive UI, or an
API call — no need to wait on PR checks or a deploy. Versions coexist,
so evaluating a prompt change
means publishing a pre-release and pointing an eval run at it, while
@latest keeps serving production sessions.
Hive and the runtime now have their own orthogonal roadmaps: the runtime grows capabilities as extensions, and Hive is becoming our own custom prompt management, tracing, and evaluation stack.
This decoupling almost treats the agent like a “user”. Users don’t live in your codebase: they’re rows in a database, they sign up without you shipping a deploy, and one product serves them all. That’s now our baseline assumption for what any “AI employee” or background-agent product needs.
The runtime belongs in your codebase. Your agents don’t.
Oh, and if working on problems like this sounds fun, we’re hiring.