Watch a coding agent work and count what it actually does. Read a file. Search for a symbol. List a directory. Run the tests. Read another file. Somewhere in between, it plans a change, debugs a failure or writes a tricky edit.

Every one of those steps is a full request to the model you picked, usually the most capable and most expensive one. The routine steps cost as much per token as the hard ones.

ThetaRelay is a proxy that sits between the agent and the model. For each request it decides whether the next step is easy. Easy steps go to a cheap lite model; the rest go to the frontier model. The agent doesn’t see the difference: each response comes from one model, unchanged.

Two tiers, one model per response

Tier	Default model
Lite	Claude Haiku 5.5
Frontier	Claude Opus 5.5

ThetaRelay never mixes two models’ output in one response, and never shows one model’s output to another for approval. Whichever model handles a request writes the whole response, and the agent gets it as is.

It works with agents that speak the Anthropic Messages API or the OpenAI Chat Completions API, such as Claude Code, OpenCode, Pi, Cursor and Aider. Pointing one at it is a base URL:

ANTHROPIC_BASE_URL=http://127.0.0.1:8788 claude

How a step gets routed

ThetaRelay applies these rules in order. The first that applies picks the tier.

1. Too big for the lite model? Frontier.
2. Did a recent tool result fail? A FAILED, a Traceback or a non-zero exit code usually means debugging, which is hard. Frontier.
3. Ask the judge. A small model running locally on Ollama answers one question: can a small, fast model do the next step? If it’s confident the answer is yes, lite.
4. Ask the frontier model. Otherwise the request goes to the frontier model with a short note: if a smaller model can do the next step correctly, reply only <EASY/>. If the frontier model does the step itself, its reply goes straight to the agent with no extra call. If it answers <EASY/>, ThetaRelay drops that reply and sends the original request to the lite model.

Two fallbacks sit on top. If the judge or the lite server fails, the request goes to the frontier model. And if the lite model’s reply would end the agent’s turn, ThetaRelay discards it and asks the frontier model instead. The frontier model decides when the work is done.

The reasoning behind the order is that the two kinds of mistake don’t cost the same. An easy step sent to the frontier model costs money. A hard step sent to the lite model costs quality. So every uncertain case goes to the frontier model.

What the judge sees

The judge reads at most 2,048 tokens, far less than a coding session. So ThetaRelay doesn’t hand it the conversation. It builds a short summary from the request, without calling a model: the user’s latest goal, one line for each of the last 12 steps, the end of the last message, and a count of failed tool results.

Because agents send the whole conversation with every request, ThetaRelay keeps no state of its own. A restart doesn’t change any decision.

On 59 hand-written coding-agent steps, 30 easy and 29 hard, the default judge sent none of the hard steps to the lite model and 29 of the 30 easy ones. Those steps were chosen to be clearly easy or clearly hard. Real sessions are murkier, so ThetaRelay writes every decision to a log you can check against your own work.

Keeping the prompt cache warm

Routing that breaks the frontier model’s prompt cache would cost more than it saves. A long agent session is mostly a repeated prefix, and cached input tokens are much cheaper than fresh ones.

So ThetaRelay changes as little as possible. It rewrites the model field. On a frontier request that goes through step 4, it adds the short note after the last message, and puts the cache boundary before it. The system prompt, the messages and the tools are untouched, so the frontier provider sees the same prefix every time, and lite steps in between only add to the end of it.

What it saves, and where it doesn’t

With the defaults, Opus 5.5 and Haiku 5.5 on OpenRouter, our test cost 32% less than Opus alone, with the same results. In that test an agent builds an app from a written spec, and hidden tests it never sees check the result.

That’s the only setup we’ve tested it to save money with, and the caveats are real:

• Other frontier models can cost more. The OpenAI models we tried didn’t hand steps to the lite model, so the triage note bought nothing. With a frontier model that’s already cheap, there’s little to save.
• Small local lite models stop too early. A model like gemma4:e4b often tries to end the turn, which sends the step back to the frontier model. That adds time and saves nothing.
• The judge adds latency. About 0.3 to 3 seconds per request on an Apple M3 Pro.
• Lite tool calls aren’t checked. If the lite model writes a wrong edit, the agent gets it. The next failed tool result moves the session back to the frontier model.

If you want different models, the repository includes the evaluation, so you can measure your setup with and without routing before you rely on it.

Try it

You need Python 3.13, uv, Ollama for the judge, and an OpenRouter key for the two tiers.

ollama pull tev1
uv tool install thetarelay

export OPENROUTER_API_KEY=sk-or-...
export THETARELAY_LITE_API_KEY=$OPENROUTER_API_KEY
thetarelay proxy --debug

With --debug, every request prints one line: which rule decided, which tier, and how long it took.

ThetaRelay is open source under Apache-2.0. The code, the routing details and the full list of known limitations are on GitHub.