ThetaServe: one command from model name to running endpoint.
Name a model and a use case. thetaserve reads the model's Hugging Face metadata and the provider's GPU prices and stock, shows the cheapest options that fit, and launches the one you pick. You get a secured, OpenAI-compatible endpoint.
Try it
You need uvand a Runpod account with credit. Without a Runpod key, --dry-run uses bundled sample data and works offline.
uv tool install thetaserve
thetaserve up --provider runpod --model qwen3.8-27b --dry-runThis shows the cheapest GPU for each weight and KV-cache precision, the memory estimate, the exact vllm serve command and the Runpod request body. It creates nothing and needs no credentials.
Why it's different
Settings derived, not guessed.
The correct vLLM settings for each model and GPU take trial and error to find. thetaserve reads the model's metadata and the provider's GPU catalogue, then derives the options that fit.
Cost controlled by default.
Pods stop themselves after 30 minutes of idle time and after 10 hours of total life. You set both limits. No surprise bills from a forgotten instance.
Secured from launch.
Every launch generates a fresh API key. The endpoint requires it on every path except /health, including /metrics.
Works with your tools.
Run eval "$(thetaserve env)" to set OPENAI_BASE_URLand OPENAI_API_KEY. Any client or coding agent that speaks the OpenAI API connects immediately.
For how the settings are worked out, read Running an Open Model on a Rented GPU, Without the Trial and Error.
Usage
thetaserve list models # also: providers
thetaserve setup # store your Runpod API key
thetaserve up --provider runpod --model qwen3.8-27b # pick from the options
thetaserve status # endpoint, uptime, cost so far
eval "$(thetaserve env)" # export keys for your tools
thetaserve logs -f
thetaserve stop # pause compute, keep the weights cache
thetaserve down # destroy the podOpen source
thetaserve is released under the Apache-2.0 license. It is in beta with Runpod support. Bug reports and contributions are welcome onGitHub.
Have a workflow in mind? Let's talk it through.
A 30-minute conversation about the work you want help with, the systems it touches, and what has to stay private. No pitch deck, no pressure.