Docs / Self-hosting on Frappe

Self-hosting

LinuxAir can run entirely on your own infrastructure — the web app, the routing engine and the OpenAI-compatible gateway — so prompts only ever leave your network to reach the providers you choose.

What you need

A Linux server (4 vCPU / 8 GB RAM is comfortable to start), or a VM in your own cloud
MariaDB or MySQL, and Redis
Python 3.10 or newer
Background workers and a scheduler — probing and cluster builds run asynchronously
Outbound HTTPS to your model providers and, optionally, the public price feed

Routing itself is light: the decision is a few milliseconds of vector maths. Sizing is driven by how many requests you proxy, not by the routing.

Getting the package

Self-hosting is available for teams and enterprises. Contact us and you'll get the deployment package, a step-by-step install guide for your environment and onboarding support. Installation seeds the probe set and builds an initial routing space with the offline embedder, so the system works immediately without any external keys.

What gets exposed

Path	What
`/linuxair`	The web app — sign up, dashboard, models, routing policy, billing.
`/v1/chat/completions`, `/v1/models`, `/v1/feedback`	The OpenAI-compatible gateway your applications call.
Admin settings	Platform configuration, restricted to administrators.

Configure

Open LinuxAir Settings in the admin area.

Section	Settings	Recommendation
Embeddings	Provider, base URL, model, key	Switch from Local hashing to a real embedding model (hosted or local) for much sharper clusters.
Judge	Provider type, base URL, model, key	A small, fast, capable model — it can be a local one. Enables judged probes and answer checks.
Routing	Clusters (8), soft temperature (0.05), risk aversion (1), default λ, smoothing prior, live weight, live α, retry window	Defaults are sensible; raise the cluster count only with a larger probe set.
Answer checks	Enabled, trade-down gap (0.04), error threshold (0.45), uncertainty threshold (0.10), judge threshold (0.6)
Pricing	Price feed URL	A public model-price list by default; point it at your own file to run fully offline.
Billing	Signup credits, route fee, fingerprint fee, INR / USD per 1,000 credits	Set the fees to zero for internal-only deployments.
Payments	Razorpay and Stripe keys	Leave empty to disable online top-ups.

After changing the embedding model, open Platform → Router and click Rebuild clusters, then re-probe existing models so judge-scored probes are included.

Streaming behind a reverse proxy

The gateway sends X-Accel-Buffering: no. Also disable proxy buffering for /v1/ and allow a timeout longer than your longest completion:

nginx

location /v1/ {
    proxy_pass http://linuxair_upstream;
    proxy_buffering off;
    proxy_read_timeout 300s;
    proxy_set_header Host $host;
}

Each streaming request holds a web worker for its duration, so run the application server with enough threads or workers for your expected concurrency, and set its request timeout to match.

Payments

Razorpay — add the key id and secret. Payments are verified with the HMAC signature.
Stripe — add the secret key, point a checkout.session.completed webhook at the payments endpoint, and paste its signing secret.

For an internal deployment you can skip payments entirely and set the fees to zero.

Scheduled jobs

Frequency	Job
Hourly	Recover models left mid-probe after a worker restart.
Daily	Refresh model prices, re-probe models older than the probe set, clear request previews older than 90 days.

Custom probe sets

Administrators can import probes from Platform → Probe set as JSON — each with a prompt, category, scorer (exact, contains_all, contains_any, number, regex, json_keys or judge) and expected value — then rebuild clusters. Probes drawn from your own workload make routing noticeably sharper.

json

[{"prompt": "What is 17% of 2,340?", "category": "math", "scorer": "number", "expected": "397.8"},
 {"prompt": "Return JSON with keys name and city for: Areeb, Hyderabad", "category": "json_format", "scorer": "json_keys", "expected": "[\"name\", \"city\"]"}]

Running fully offline

Point models at local endpoints such as an on-premise inference server — local models are priced as free.
Use a local embedding model, or keep the built-in offline embedder.
Use a local judge model for answer checks.
Host the price list yourself, or enter prices manually.

Upgrading

Each release ships with an upgrade note. The usual sequence is: back up the database, replace the package, run the migration step and restart the services. Fingerprints, request history and credits are preserved; if the probe set changed, models are re-probed automatically in the background.