rungentBuild
Deploy
Run rungent in production. The Docker image and every environment variable, the database, secrets, the reverse proxy, replicas, and the model chain rungent falls through when one model fails.
An embedded rungent deploys with your JavaScript app; its options are on JavaScript.
Beside an API in any other language, rungent runs as its own service, rungent serve, configured from
environment variables, with its state in Postgres, behind the reverse proxy that already serves
your product.
browser ─> reverse proxy (console.example.com)
├─ /rungent/v0/*, /mcp, /skill.md, /llms.txt, /.well-known/… ─> rungent serve ─> Postgres
└─ everything else ──────────────────────────────────────> your API
rungent serve ─(the user's credential + Rungent-Assertion)─> your API
Run it#
docker run -d --name rungent -p 8080:8080 --env-file rungent.env ghcr.io/rungent/rungent-server:v0.4.0
Images are tagged with each release (v0.4.0) and latest. The image runs rungent serve as the
node user on port 8080. Without Docker, npx rungent serve runs the same server on
Node.js 24.
GET /healthz answers ok, at the root even when rungent is served below a prefix; the image's
health check calls it. On SIGTERM the server stops taking requests, lets runs in progress
finish and exits. To build on your own base image, pass --build-arg BASE_IMAGE=… --build-arg RUNGENT_VERSION=0.4.0
to the Dockerfile in packages/rungent of the rungent repository.
Configure it#
Five variables are required:
| Variable | What it is |
|---|---|
RUNGENT_URL |
The public URL rungent is served at, the assertion's iss. https://console.example.com when the proxy serves rungent at the root. |
RUNGENT_HOST_ORIGIN |
Your product's public origin, the assertion's aud. Your API verifies against it. |
RUNGENT_SIGNING_KEY |
rungent's private Ed25519 JWK with a kid, as JSON: the file rungent keys generate wrote. |
RUNGENT_SECRET |
32 bytes or more (openssl rand -base64 48). Seals the model API keys admins save. |
RUNGENT_DATABASE_URL |
postgres://…, or sqlite:<path> for a single instance on a persistent volume. |
The rest are optional:
| Variable | What it is |
|---|---|
RUNGENT_HOST_BASE_URL |
Where rungent reaches your API, when not through the public origin (http://api.internal:8000). Assertions still name the public origin. |
RUNGENT_HOST_OPENAPI |
Path or URL of your OpenAPI 3.1 document. Default /api/openapi.json. |
RUNGENT_MODELS |
The default model chain, a JSON array. Without it, runs fail until an admin saves a chain. |
RUNGENT_NAME |
The assistant's name. Default Assistant. |
RUNGENT_INSTRUCTIONS |
Product facts added to the system prompt; @/path/to/file reads a file. |
RUNGENT_PRODUCT_NAME, RUNGENT_PRODUCT_SUMMARY |
Your product's name and summary in /skill.md and /llms.txt for signed-out callers, when your OpenAPI needs a sign-in. |
RUNGENT_SKILL_NAME, RUNGENT_SKILL_GUIDE |
The name in /skill.md, and a hand-written guide placed first (or @path). |
RUNGENT_CLI_COMMAND, RUNGENT_CLI_PACKAGE, RUNGENT_CLI_INSTALL |
Your branded CLI that the documents name: its command, its npm package, or a free-form install line (which wins). The last two need the first. |
RUNGENT_MCP_INSTRUCTIONS |
What MCP clients receive on initialize, or @path. Default: the skill guide. |
RUNGENT_MCP_AUTHORIZATION_SERVERS |
Comma-separated OAuth issuers advertised to MCP clients. |
RUNGENT_KNOWLEDGE_DIR, RUNGENT_KNOWLEDGE_BASE_URL |
Markdown docs indexed at start, and the URL their pages live under. See Search and knowledge. |
RUNGENT_HOST_KEYS |
A JSON array of public JWKs your API signs Rungent-Host-Assertion with. |
RUNGENT_PREVIOUS_SECRET |
The secret before a rotation. |
RUNGENT_BASE_PATH |
A prefix for every route, when rungent is served below a path. |
RUNGENT_MAX_STEPS |
Model steps per run. Default 16. |
PORT |
The listening port. Default 8080. |
A variable that is missing, not valid JSON or out of range stops the server at start with a
message naming it, and exit code 2. Keep RUNGENT_SIGNING_KEY, RUNGENT_SECRET,
RUNGENT_PREVIOUS_SECRET, RUNGENT_DATABASE_URL and RUNGENT_MODELS (it holds API keys) in your secret
store.
Database and secrets#
Use Postgres in production. rungent keeps conversations, approvals, deferred tasks, settings and the
knowledge index there, in its own rungent_* tables, which it creates and upgrades at start. Give it
its own database or schema. SQLite cannot be shared by replicas.
The signing key and its rotation are on Acting as the user.
To rotate the settings secret, set the new value as RUNGENT_SECRET and the old one as
RUNGENT_PREVIOUS_SECRET and deploy: the new secret seals, both unseal, and saved settings are
sealed again with the new one when first read. Once any run has read them, remove
RUNGENT_PREVIOUS_SECRET. Settings sealed with a secret rungent no longer has answer
503 settings_unreadable until an admin calls DELETE /rungent/v0/settings.
Route the rungent paths#
Send these paths on your product's origin to rungent, unchanged, and everything else to your API:
| Path | What it serves |
|---|---|
/rungent/v0/* |
The assistant's HTTP API and its event streams |
/mcp |
MCP over streamable HTTP |
/skill.md, /llms.txt |
Documents generated from your catalog |
/.well-known/rungent-keys.json |
rungent's public keys |
/.well-known/oauth-protected-resource/mcp |
Only with RUNGENT_MCP_AUTHORIZATION_SERVERS |
Runs stream server-sent events, so turn off response buffering and allow long reads. With nginx:
location ~ ^/(rungent/v0/|mcp$|skill\.md$|llms\.txt$|\.well-known/(rungent-keys\.json|oauth-protected-resource/mcp)$) {
proxy_pass http://rungent:8080;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_buffering off;
proxy_read_timeout 1h;
}
Forward the Authorization and Cookie headers untouched: they are the user's credential. To
serve rungent below /assistant, set RUNGENT_BASE_PATH=/assistant and
RUNGENT_URL=https://console.example.com/assistant, proxy /assistant/, and give the web client
createClient({ baseUrl: '/assistant' }).
Run several replicas#
Several replicas can share one Postgres: an approval runs once, and one conversation is never driven by two replicas at a time. Two things stay in one replica's memory:
- MCP sessions. Route every request with the same
Mcp-Session-Idto the same replica (sticky sessions, orsessionAffinity: ClientIPon a Kubernetes Service). Elsewhere it gets404 session_not_found, and the client opens a new session. - Model rests. Each replica learns on its own that a model endpoint is failing.
On Kubernetes, run the image as a Deployment with its variables from a Secret:
containers:
- name: rungent
image: ghcr.io/rungent/rungent-server:v0.4.0
ports: [{ containerPort: 8080 }]
envFrom: [{ secretRef: { name: rungent-env } }]
readinessProbe: { httpGet: { path: /healthz, port: 8080 } }
Choose models#
rungent calls models through a chain: endpoints tried in order. When the first is out of credit or rate-limited, the same turn goes on with the next, and the user sees at most "Model busy, retrying". The chain you configure is the default; an admin can replace it at run time.
createRungent({
// ...
models: {
chain: [
{ id: 'qwen', provider: 'openai-compatible', model: 'qwen3-max',
baseURL: 'https://dashscope.aliyuncs.com/compatible-mode/v1', apiKey: process.env.DASHSCOPE_API_KEY },
{ id: 'claude', provider: 'anthropic', model: 'claude-sonnet-4-5', apiKey: process.env.ANTHROPIC_API_KEY, vision: true },
],
},
});
With rungent serve, RUNGENT_MODELS holds the same array as JSON.
| Field | What it is |
|---|---|
id |
Required, unique in the chain, 1 to 64 characters. Saved settings keep an endpoint's key by it. |
provider |
openai-compatible (any OpenAI-style chat API: DashScope, DeepSeek, vLLM, LiteLLM), openai or anthropic. |
model |
The provider's model id. |
baseURL |
Required for openai-compatible; for the others, a proxy or gateway. |
apiKey |
The provider's key. Without it, openai and anthropic read OPENAI_API_KEY and ANTHROPIC_API_KEY. |
vision |
The model reads images. Turns with images go to these endpoints first; others get a note in place of each image. |
temperature, maxOutputTokens |
Sent with every call to this endpoint. |
cooldownSeconds |
{ quota?, rateLimit?, error? }: how long a failing endpoint rests. |
providerOptions |
Passed to the provider, keyed by provider name. { 'openai-compatible': { enable_thinking: false } } keeps Qwen's reasoning models fast. |
Embedded, an entry may also be any AI SDK language model, { id, languageModel }, such as
Workers AI through agents/models/ai-sdk. It exists only in code; the settings show it read-only.
When a model fails#
A timeout, 408, 5xx, network error or unusable answer is retried on the same endpoint, up to
4 attempts. Any other failure, and the last retry, moves the step to the next endpoint, and the
failing one rests:
| Failure | Rests for | cooldownSeconds key |
|---|---|---|
Out of quota: 402, or an answer that mentions quota |
6 hours | quota |
Rate limited: 429 |
60 seconds | rateLimit |
| Anything else | 5 minutes | error |
When every endpoint rests, the run ends with model_unavailable. Rests live in one process. A
text-only model first and a vision model further down keeps ordinary turns cheap and still reads
the screenshots users attach.
Let admins change the chain#
Render ModelSettings from rungent/react on a page only admins reach. Admins add,
remove and reorder endpoints; a key is write-only, and Reset to defaults goes back to the
chain you configured. Saving needs the rungent:admin scope in the user's identity, checked on the
server. The same is GET, PUT and DELETE /rungent/v0/settings (HTTP API).
Saved keys are sealed with secret before they reach the database.
Check it#
npx rungent check --openapi https://console.example.com/api/openapi.jsonpasses.GET /rungent/v0/catalogwith a user's token lists that user's operations.- A write through the dock shows an approval card, and your API logs
actor = rungentwith the approval id. /skill.mdrenders, and a coding agent connects to/mcp.
Where to go next#
Testing measures the agent against your deployment.
For agents: deploy.mdEdit this page