OmniRoute

OmniRoute is an AI gateway. It puts many cloud LLM providers behind one OpenAI-compatible endpoint, with a dashboard to manage them. Use it for cloud providers, especially their free tiers. For models running in your cluster, agents call Ollama directly, not through OmniRoute.

OmniRoute is installed in the inference namespace, at:

http://omniroute.inference.svc.cluster.local:20128

The dashboard and the API share that port. The API is under /v1.

Open the Dashboard

Forward the port, then open http://localhost:20128:

kubectl -n inference port-forward svc/omniroute 20128

Log in with the initial admin password, which this command prints:

kubectl -n inference get secret omniroute -o jsonpath='{.data.INITIAL_PASSWORD}' | base64 -d

⚠ Important

Change the admin password right away, under Settings → Security. The initial one is the same on every installation, and the dashboard holds your providers' API keys. After you change it, the command above still prints the initial password, not your new one.

Add Providers and an API Key

1. Add your cloud providers and their API keys under Providers.
2. Create an API key under API Manager. This is the OmniRoute API key: agents use it to call OmniRoute, and every call to /v1 needs one. The key is shown only once.

To check that it works, with the port still forwarded:

curl -s http://localhost:20128/v1/chat/completions -H "Authorization: Bearer <omniroute-api-key>" -H "Content-Type: application/json" -d '{"model": "<model>", "messages": [{"role": "user", "content": "Hello"}]}' | jq -r '.choices[0].message.content'

• <omniroute-api-key> is the key you created under API Manager.
• <model> is either a model ID that OmniRoute lists, or auto. With auto, OmniRoute picks the best connected provider for each request, and tries the next one if it fails. Variants such as auto/fast and auto/cheap change what "best" means.

To list the model IDs, which also needs the API key:

curl -s http://localhost:20128/v1/models -H "Authorization: Bearer <omniroute-api-key>" | jq -r '.data[].id'

Use It in an AIFabric

To the agents, OmniRoute is an OpenAI-compatible server. Store the OmniRoute API key in a Secret in the AIFabric's namespace, trirematics, not in inference. The agents are the ones that read it, and a pod can only use Secrets from its own namespace.

kubectl -n trirematics create secret generic omniroute-api-key --from-literal=OPENAI_API_KEY=<omniroute-api-key>

⚠ Important

The key is named OPENAI_API_KEY only because the agents talk to OmniRoute as if it were OpenAI. Its value must be the OmniRoute API key you created under API Manager, not a key from OpenAI or from any other provider. Your providers' keys stay in OmniRoute's dashboard.

Then declare an LLM with provider: openai that points at OmniRoute, and have each agent reference it with llm: omniroute:

spec:
  llms:
    - name: omniroute
      provider: openai
      model: <model>
      endpoint:
        baseUrl: http://omniroute.inference.svc.cluster.local:20128/v1
      apiKeySecretRef:
        name: omniroute-api-key
        key: OPENAI_API_KEY

<model> is a model ID from /v1/models, or auto. Apply the fabric with:

brc install aifabric myfabric.yaml

Note: with model: auto, consecutive calls from the same agent can go to different providers and models, so the agent's behavior can vary from one call to the next.

Your Configuration

OmniRoute stores everything you set in the dashboard inside the cluster: the providers, the API keys and the routing settings. It's kept across restarts and upgrades.

Troubleshooting

Symptom	Check
401 on /v1	The API key is missing or wrong. In an AIFabric, check the apiKeySecretRef, and that the Secret holds the OmniRoute API key, not a provider's key.
A model ID isn't listed by /v1/models	The provider that serves it isn't connected. Check Providers in the dashboard.
The dashboard doesn't open	Check that the pod is Running: kubectl -n inference get pods -l app=omniroute.
The pod isn't Running	Contact BubbleRAN support with the output of kubectl -n inference describe pod -l app=omniroute.