L
LuminalLLM Routing Gateway
NEWLuminal now routes across 8 providers with budget-aware fallbacks

The cheapest model
that can actually answer
your prompt.

Luminal is a self-hosted LLM gateway. It scores every prompt, optionally pulls context from your docs, calls your tools, picks the right model for the job — and logs everything for cost & quality analytics.

See how it works
Live SSE trace8 live providersLangGraph pipelineBudget-aware
luminal · live traceSTREAMING
→POST /route {"prompt":"Explain quantum entanglement to a 10yo"}
·analyze · complexity=0.62 (heuristic + llm-judge)
·retrieve · 4 chunks matched (chroma)
·tool · none required
·route · tier=medium → mistral-medium-latest (mistral)
·generate · 312 tok in / 184 tok out · $0.00018
·critic · score=0.91 ✓ no regeneration
✓200 OK in 423ms budget left: $87.42 / $100
↳ Quantum entanglement is when two tiny particles become best friends: whatever happens to one instantly affects the other, even if they're on opposite sides of the universe…
Features

One gateway. Every model.
Every cent accounted for.

Hybrid complexity scoring

Heuristic + LLM-as-judge compute a single difficulty score per prompt. The router reuses it so the analysis and the model picked can never disagree.

Built-in RAG

Keyword + embedding retrieval over Chroma, Pinecone, or Weaviate — context and citations injected before generation.

MCP tool calling

Register external tools. Luminal decides when to call them — risky actions pause for your approval before they run.

LangGraph pipeline

analyze → retrieve → tool → approval → route → generate → critic → error_recovery. Every transition logged.

Budget-aware routing

At 80% spend Luminal downgrades one tier. At 95% it pins the cheapest model. Go over your monthly budget and requests are rejected until it resets.

Self-healing generation

A critic model scores cloud replies and regenerates if quality drops (local Ollama responses skip scoring). Errors trigger recovery with adjusted temperature or disabled RAG.

Pipeline

Every prompt walks a state machine.

Eight LangGraph nodes handle a single request. Each one logs its decision and hands off to the next — visible live in your dashboard.

analyze
Hybrid complexity scoring
retrieve
RAG over vector store
tool
MCP decision
approval
Human-in-the-loop pause
route
Pick tier + provider
generate
Cache + retry + fallback
critic
Score & regenerate
error_recovery
Adjusted retry
Retrieval

Your docs become instant context.

Drop PDFs, Markdown, or text. Luminal chunks, embeds, and indexes them into Chroma, Pinecone, or Weaviate. Top-k chunks are injected into the prompt with inline citations, so answers are grounded in your own docs.

  • Keyword + embedding hybrid search
  • Live ingestion from the dashboard
  • Citations included whenever RAG is triggered
  • Pluggable vector stores
"Why are my dashboard sessions breaking on refresh?"
Auth & sessions.md
4.2 kb · 1.1k tok
MCP spec v2.md
12.7 kb · 3.2k tok
Routing rules.md
2.1 kb · 540 tok
Q3 roadmap.pdf
89 kb · 22.4k tok
Onboarding guide.md
6.8 kb · 1.7k tok
top-k chunks
0.94Auth & sessions.md
JWT sessions are stored with `sub` claim as a string. An integer subject is rejected and breaks dashboard loading.
0.87MCP spec v2.md
Tools with `requires_approval=True` pause the run before executing. Resume via `POST /route/approve {session_id, approved}`.
0.71Routing rules.md
At 80% spend, the router downgrades one tier; at 95% it pins the cheapest model until reset.
injected into prompt with [1] [2] [3] citations
By the numbers

Built for production scale.

Providers supported
0
Pipeline stages
0
Budget downgrade threshold
0%
Cheapest-model failsafe
0%
Vector store backends
0
Critic regeneration cap
0 retries
Developer-first

Three endpoints.
Zero surprises.

Luminal speaks JSON over HTTP. Every response carries the model used, the cost, the tokens, and the trace.

POST /route

Send a prompt. Get a model-routed, budget-checked, RAG-augmented response. JSON in, JSON out.

{
  "prompt": "Explain quantum entanglement",
  "session_id": null
}
GET /route/trace/{session_id}

Server-sent events stream every LangGraph node decision live — same view as the dashboard terminal.

data: {"node":"retrieve","action":"complete",
  "data":{"used_rag":true,"chunks":4}}

data: {"node":"route","action":"complete",
  "data":{"model":"...","provider":"..."}}
Plain HTTP, no SDK required

Auth via lum_ API keys created from the dashboard. Call it from anywhere with a Bearer token — curl, Python, JS, whatever.

curl -X POST http://localhost:8000/route \
  -H "Authorization: Bearer lum_..." \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Why is JWT sub a string?"}'
Self-hosted. Runs on your own infra.

Stop overpaying
for tokens you'll waste.

Clone the repo, start the FastAPI backend and the dashboard, drop in your provider keys, and start routing every prompt through Luminal.

Read the docs
FastAPI + Next.jsSQLite → PostgresDocker readyBYO keys