Open-weight models like Qwen3.8-27B and Gemma 4 31B are strong enough for real work, including coding agents, and they run well on a single 24–80 GB GPU. Renting that GPU by the hour is cheap. Getting the model running on it well is not.

You have to choose a weight format, a context length and a KV-cache precision that together fit in the card’s memory. The model needs its own vLLM flags. Then you have to secure the endpoint, and remember to shut it down before it runs up a bill.

ThetaServe turns that into one command. You name a model and a use case; it shows the cheapest GPU options that fit and launches the one you pick.

Start with a dry run

You can see what ThetaServe would do without an account or a network connection:

uv tool install thetaserve
thetaserve up --provider runpod --model qwen3.8-27b --dry-run

For each combination of weight and KV-cache precision, it shows the cheapest GPU, a memory estimate, the exact vllm serve command and the request it would send to Runpod. It creates nothing. Without a Runpod key it uses bundled, dated sample data and tells you so.

Settings are derived, not guessed

ThetaServe calculates its options from two published sources:

• The model’s Hugging Face metadata. The config.json gives the attention layout, from which it calculates the KV cache per token. For Qwen3.8-27B that’s 64 KiB at BF16. The .safetensors sizes of quantised variants give the weight size at each precision.
• The provider’s GPU catalogue. Memory, architecture, price and stock for each GPU.

A combination fits a GPU when:

weights + KV cache at full context + 3.2 GiB overhead ≤ 92% of VRAM

The 3.2 GiB is the larger of two measured overheads, rounded up. Because everything else comes from live data, a new GPU or a new quantised variant shows up as soon as Runpod or Hugging Face lists it.

If you’ve read Will It Run?, this is the same arithmetic, done for you against real prices.

Use cases set the context

A use case (--use) sets the context length and a precision floor for the default choice. The default is coding: 128K context at 8-bit, because an agent makes long chains of exact tool calls, where small errors do more damage than in conversation. You can still see the rows below the floor and compare quality and price yourself.

What 4-bit buys you: memory, not speed

It’s tempting to assume 4-bit weights mean a cheaper GPU or faster tokens. For coding-length contexts, usually neither:

• It seldom fits a cheaper card. Qwen3.8-27B at 128K context needs 24–30 GiB even at 4-bit. A 24 GB card leaves only about 22 GiB usable, so it still doesn’t fit.
• It isn’t faster. Unpacking 4-bit weights costs about as much time as the smaller read saves. One October 2026 measurement of Qwen3.8-27B on vLLM put FP8 at 985 tok/s, BF16 at 857 and INT4 at 808.
• What it does give is free memory on the same GPU. On an A40 at 128K, FP8 weights use 39.9 of 44.2 GiB and 4-bit uses 29.3. That spare ~10 GiB becomes KV cache: more parallel agent sessions or more prefix-cache hits.

Quality at 4-bit is still an open question, so ThetaServe shows you the trade-off rather than choosing it silently.

Safe defaults

Trusted weights only. Anyone can publish a quantised model, and download counts are easy to inflate. By default ThetaServe offers only variants from the base model’s own organisation and a short list of trusted publishers. It never passes --trust-remote-code, and it pins the exact repository revision it measured, so a later push can’t change what you run.

A fresh key on every launch. Each launch generates a new API key. The endpoint requires it on every path except /health, including /metrics.

No forgotten bills. Pods stop themselves after 30 minutes idle or 10 hours total, and you can set both limits.

Then point your tools at it

thetaserve up --provider runpod --model qwen3.8-27b   # pick from the options
eval "$(thetaserve env)"                              # sets OPENAI_BASE_URL and OPENAI_API_KEY
thetaserve status                                     # endpoint, uptime, cost so far
thetaserve stop                                       # pause compute, keep the weights cache
thetaserve down                                       # destroy the pod

Any client or coding agent that speaks the OpenAI API connects straight away, with the model running on a GPU you rented, not on someone else’s shared service.

Status

ThetaServe is in beta and supports Runpod only. It’s open source under Apache-2.0; the code, tutorial and design log are on GitHub.