Omni works with no configuration: the built-in providers load at startup, each with its models, and each reads its API key from the environment. This guide is for when you want something else: fewer providers, a provider Omni has no built-in for, a server of your own, a model the catalog lacks, or different model data.
Three ideas
- A model is one LLM: its id, limits, prices, and what it can do. You name it in every request.
- A provider is the service that hosts models: where it lives, how to authenticate, and which APIs it serves. Each model belongs to one provider.
- A dialect is the wire format of an API, such as OpenAI's Chat Completions or Anthropic's Messages. Many providers speak the same dialect, and a gateway may speak several. Each model names the dialect it is reached through.
All three are plain data, built once at startup from a catalog of model data, Omni's knowledge of each built-in provider, and your config. Two config keys shape the result, and they have separate jobs. config :omni, :models selects what loads. config :omni, :providers configures providers by id.
Choosing which providers load
By default every built-in loads. To load fewer, or more, list provider ids:
# Only these two
config :omni, :models, providers: [:anthropic, :openai]
# Every built-in, plus two more
config :omni, :models, providers: [:builtins, :deepseek, :gpu_box]A provider's id is its key in the models.dev catalog, in snake case: :anthropic, :fireworks_ai, :ollama_cloud. The same id is used everywhere: in this list, in config, and in model references.
Three kinds of provider
Any provider can be loaded, and Omni knows more about some than others.
Built-in providers
Common providers Omni ships code for. Each is set up for its API's quirks and run against the live API before a release. Built-in providers lists them.
Catalog providers
The rest of the models.dev catalog, over two hundred of them. Add the id to providers: and Omni takes the provider's URL, API key variable and models from catalog data:
config :omni, :models, providers: [:builtins, :deepseek]Omni.generate_text("deepseek:deepseek-v4-pro", "Hello!")A catalog provider is spoken to in its dialect with no adjustments. Many work as they are. One that deviates from the dialect needs compat flags.
Custom providers
Providers the catalog doesn't know, such as an inference server of your own. Define one wholly in config: an endpoint for the dialect it speaks, how it authenticates, and its models.
config :omni, :models, providers: [:builtins, :gpu_box]
config :omni, :providers,
gpu_box: [
dialects: %{Omni.Dialects.OpenAICompletions => [base_url: "http://gpu-box:8000/v1"]},
auth: :none,
models: [
[id: "llama-3.3-70b", dialects: %{text: Omni.Dialects.OpenAICompletions}],
[id: "qwen3-32b", context_size: 32_768, dialects: %{text: Omni.Dialects.OpenAICompletions}]
]
]Whatever kind of provider serves a model, mix omni.verify tells you whether it works.
Configuring a provider
Each entry under config :omni, :providers takes these keys. Your values are layered over the catalog's and the built-in's, so set only what you want to change.
| Key | What it sets |
|---|---|
:name | A display name |
:module | A provider module, for behaviour config can't express |
:host | Another origin for the provider's URLs, such as "http://gpu-box:11434" |
:api_key | A string, {:system, "ENV_VAR"}, or {Module, :function, args} called on each request |
:auth | How the key is sent: :bearer, {:header, "x-api-key"}, :none, or :dialect (the default: the way the dialect's own API does it) |
:dialects | Endpoints, by dialect module: a :base_url and :compat flags for each |
:headers | Extra headers for every request |
:models | Models to add or correct |
:source | Where this provider's model data comes from |
An unknown key is an error at startup. A missing API key is not: you can pass :api_key on each request. A request with no key returns {:error, :no_api_key}, or {:error, {:missing_env_var, "ANTHROPIC_API_KEY"}} when the key is read from a variable that isn't set.
URLs
A provider has an endpoint for each dialect it serves. The endpoint's base_url is the whole URL up to and including the version, and a request goes to that plus the dialect's own path:
# dialects: %{Omni.Dialects.OpenAICompletions => [base_url: "http://gpu-box:8000/v1"]}
# POST http://gpu-box:8000/v1/chat/completionsMost of the time you want the same API somewhere else: another machine, a region, a proxy. That is :host, which swaps the origin (scheme, host and port) of every URL the provider has and keeps their paths:
config :omni, :providers,
ollama: [host: "http://gpu-box:11434"],
alibaba: [host: "https://dashscope-us.aliyuncs.com"]:host is also a request option, for sending one request elsewhere. To change a path, set the endpoint's base_url instead.
Compat flags
Many providers speak a dialect with small differences from the API it is named after: another field name for the token limit, another way of switching thinking on. Compat flags describe those differences, one flag each, and they are set on the endpoint they apply to:
config :omni, :providers,
gpu_box: [
dialects: %{
Omni.Dialects.OpenAICompletions => [
base_url: "http://gpu-box:8000/v1",
compat: [max_tokens_field: "max_tokens", omit: [:cache]]
]
}
]Omni.Dialects.OpenAICompletions lists the flags and what each does. An unknown flag is an error at startup.
Some catalog providers mirror another provider's compat flags. like: copies a built-in's flags:
config :omni, :providers,
zai_coding_plan: [dialects: %{
Omni.Dialects.OpenAICompletions => [compat: [like: :zai]]
}]Models
A model reference is a provider id and a model id, as a tuple or a string. The provider id ends at the first : or /, so model ids may contain either:
{:openrouter, "typesafe/jev-1.14"}
"openrouter:typesafe/jev-1.14"
"openrouter/typesafe/jev-1.14"Use Omni.get_model/1 and Omni.list_models/1 to inspect loaded models:
{:ok, model} = Omni.get_model("anthropic:claude-sonnet-5-5")
model.context_size #=> 1000000
model.input_modalities #=> [:text, :image, :pdf]
model.input_cost #=> 2 (USD per million tokens)
{:ok, models} = Omni.list_models(:anthropic)Omni.Model describes every field.
Adding and correcting models
When a model is missing from the catalog, such as a new release or a fine-tune, add it under its provider. A new model names its dialect, because there is no catalog data to take it from:
config :omni, :providers,
openai: [models: [
[id: "ft:gpt-4o:acme::abc123", dialects: %{text: Omni.Dialects.OpenAIResponses}]
]]An entry whose id the catalog already has is merged over the catalog's model, so you can correct one field and leave the rest:
config :omni, :providers,
anthropic: [models: [
[id: "claude-sonnet-5-5", max_output_tokens: 200_000]
]]A local Ollama has no catalog at all: its models are whatever you have pulled. List them by id:
config :omni, :providers, ollama: [models: ["llama3.1:8b", "qwen3.5:4b"]]Where model data comes from
Model data comes from a source. The default reads a snapshot of the models.dev catalog that ships inside Omni, so startup needs no network. Each Omni release carries a fresh snapshot.
For newer data than the last release, fetch the catalog at startup. The result is cached on disk, and if models.dev can't be reached Omni falls back to the cache and then to the bundled snapshot:
config :omni, :models, source: {Omni.Sources.ModelsDev, live: true}Or read the catalog from the llm_db package, which you add to your own deps:
config :omni, :models, source: Omni.Sources.LLMDBChanging the source doesn't change provider ids: every source is asked for a provider by the same id, and matches it to its own catalog. What can change is which providers and models it knows. A provider the source has no data for is treated as a custom provider, so it needs its endpoints and models in config.
A source can be set for one provider with :source in its config. To write a source of your own, see Omni.Source.
Sources load the models Omni can use: text models that support tool use, and decision models. Embedding, image and speech models are left out, as are deprecated models.
Filtering and transforming models
Two more keys under config :omni, :models work on every model as it loads. Both take a {Module, :function} pair.
filter: decides which models are kept. The function gets each model and returns true to keep it:
config :omni, :models, filter: {MyApp.Models, :keep?}
defmodule MyApp.Models do
# Text models released since the start of 2026
def keep?(%Omni.Model{release_date: date} = model) do
Omni.Model.capability?(model, :text) and date != nil and Date.compare(date, ~D[2026-01-01]) != :lt
end
endtransform: changes models. The function gets each model and returns it, changed or as it was. It runs before the filter, and can change any field except the model's id and provider:
config :omni, :models, transform: {MyApp.Models, :transform}
defmodule MyApp.Models do
# We have a negotiated rate with this provider
def transform(%Omni.Model{provider: %{id: :deepseek}} = model) do
%{model | input_cost: model.input_cost * 0.8, output_cost: model.output_cost * 0.8}
end
def transform(model), do: model
endUse a provider's models: to correct one model by id, and transform: to change many by pattern.
Moving models to another dialect
Many providers serve the same models over more than one API. Groq, for example, serves Chat Completions and OpenAI's Responses API. The catalog gives each model one dialect, and you may want another.
You are on your own here
Omni's built-in providers are set up and checked for the dialect the catalog gives them. A dialect you move models to starts with no compat flags, and finding the ones it needs is up to you. Check the result with mix omni.verify.
It takes two steps. First declare the endpoint for the new dialect on the provider:
config :omni, :providers,
groq: [dialects: %{
Omni.Dialects.OpenAIResponses => [
base_url: "https://api.groq.com/openai/v1"
]
}]Then move the models to it in a transform:
config :omni, :models, transform: {MyApp.Models, :transform}
defmodule MyApp.Models do
alias Omni.Dialects.{OpenAICompletions, OpenAIResponses}
def transform(%Omni.Model{provider: %{id: :groq}, dialects: %{text: OpenAICompletions}} = model) do
put_in(model.dialects.text, OpenAIResponses)
end
def transform(model), do: model
endMatching on the model's current dialect matters for gateways, where not every model is served on every API. A transform that leaves a model on a dialect its provider has no endpoint for raises at startup.
Checking a model works
mix omni.verify runs a handful of real requests against one model, using your config, and reports which pass:
mix omni.verify deepseek:deepseek-v4-pro
It covers text, thinking, tool use and structured output by default, or the two decision scenarios for a decision model, with more on request (images, PDFs, caching, a multi-turn tool loop). The requests are real and billed to your key. Run it when you add a catalog or custom provider, change compat flags, or move models to another dialect. mix help omni.verify has the options.
Going further
- Loading at runtime.
Omni.Provider.load/1loads providers after startup, andOmni.put_model/1registers a single model. Both are for apps that learn about providers and models while running. - Provider modules. When a provider needs something flags can't describe, such as a quirk only some of its models have, a module can adjust its models and its request bodies.
Omni.Providerdocuments the callbacks. - Writing a dialect. An API that speaks none of Omni's dialects needs one of its own.
Omni.Dialectis the place to start.