Custom (OpenAI-compatible)

TL;DR

ONGRID_CUSTOM_API_KEY=...
ONGRID_CUSTOM_BASE_URL=https://your-relay/v1     # REQUIRED — no upstream default
ONGRID_CUSTOM_MODEL=your-default-model

Provider id: custom. SDK adapter: eino-openai (any OpenAI Chat Completions–compatible endpoint).

The Custom slot is the 7th provider card in the Settings UI. It exists for everything that's OpenAI-compatible at the wire level but isn't one of the six baked-in providers:

• Self-hosted vLLM / TGI / llama.cpp.
• OpenRouter, Together, Fireworks, Anyscale, Groq, Mistral.
• Internal corporate model gateways.
• New providers that ship before Ongrid adds a dedicated slot for them.

The base-URL guard

The custom slot requires an explicit base URL. Without one, the underlying SDK would silently fall back to OpenAI's default (https://api.openai.com/v1), sending the operator's custom key to the wrong host. The resolver skips the provider in that case:

// internal/manager/biz/setting/llm.go:155
if pk.id == model.LLMProviderCustom && strings.TrimSpace(baseURL) == "" {
    continue
}

Provide a BaseURL via env or Settings UI; otherwise the provider never appears in the catalog.

Env vars

Var	Default	Notes
ONGRID_CUSTOM_API_KEY	—	Empty = provider dropped
ONGRID_CUSTOM_BASE_URL	—	REQUIRED
ONGRID_CUSTOM_MODEL	—	Default model id sent on every call
ONGRID_CUSTOM_MODELS	—	Catalog list (closed set the SPA picker shows)

There's no out-of-box default catalog — fill in what your endpoint supports.

Pre-registered inner ChatModel

The graph-kernel runtime pre-registers an inner ChatModel for the custom provider id at boot, even when no key is yet configured. This is the "configured AFTER boot routes immediately, no restart" property operators expect from the Settings UI:

// cmd/ongrid/main.go:2457
for _, id := range []string{
    llm.ProviderOpenAI, llm.ProviderAnthropic, llm.ProviderZhipu,
    llm.ProviderGemini, llm.ProviderDeepSeek, llm.ProviderKimi, llm.ProviderCustom,
} {
    if _, ok := innerModels[id]; !ok {
        addInner(id, "") // model supplied per-call (picker / DefaultResolver)
    }
}

The inner exists at boot; the per-call key / baseURL is resolved dynamically via the LLM-settings resolver. Unconfigured providers never reach the picker (the /v1/aiops/models catalog gates on ResolveProviders), so the inner is harmless until it's actually needed.

Configuration examples

OpenRouter

ONGRID_CUSTOM_API_KEY=sk-or-v1-...
ONGRID_CUSTOM_BASE_URL=https://openrouter.ai/api/v1
ONGRID_CUSTOM_MODEL=anthropic/claude-sonnet-4
ONGRID_CUSTOM_MODELS=anthropic/claude-sonnet-4,anthropic/claude-opus-4,meta-llama/llama-3.3-70b

Local vLLM

ONGRID_CUSTOM_API_KEY=sk-not-checked-by-vllm
ONGRID_CUSTOM_BASE_URL=http://vllm.internal:8000/v1
ONGRID_CUSTOM_MODEL=Qwen2.5-72B-Instruct
ONGRID_CUSTOM_MODELS=Qwen2.5-72B-Instruct

vLLM doesn't validate API keys by default — set any non-empty string.

Groq

ONGRID_CUSTOM_API_KEY=gsk_...
ONGRID_CUSTOM_BASE_URL=https://api.groq.com/openai/v1
ONGRID_CUSTOM_MODEL=llama-3.3-70b-versatile

Mistral

ONGRID_CUSTOM_API_KEY=...
ONGRID_CUSTOM_BASE_URL=https://api.mistral.ai/v1
ONGRID_CUSTOM_MODEL=mistral-large-latest

Making Custom the default

ONGRID_LLM_DEFAULT_PROVIDER=custom

Watch out for the LLM-resolver bug — if default_provider is unset, the resolver falls back to the first sorted provider id (anthropic). With OpenRouter wired via the custom slot but default_provider="", chat calls go to Anthropic's model id on Anthropic's endpoint, not OpenRouter — which fails on auth. Set default_provider=custom explicitly.

Quirks

• Schema fidelity — assumes the endpoint speaks real OpenAI Chat Completions. Endpoints that strip fields (tool_calls, finish_reason) will degrade the agent's function-calling reliably.
• Streaming — many relays don't yet support streaming. The adapter handles both modes; if Stream() errors with "streaming not supported", the chat UI auto-falls-back to buffered.
• No native rate-limit — corporate gateways often impose per-token rate limits that don't show up as standard 429s. Set ONGRID_LLM_DAILY_TOKEN_LIMIT for a cross-provider budget guard — see Budget.

See also

• OpenAI (GPT) — for the real OpenAI endpoint or Azure OpenAI.
• Models overview.
• Routing.