A data struct describing a specific LLM.
A model carries its identity, what it can do and what it costs. Models
are data, not modules: they are loaded from the configured model source
(see Omni.Source) when their provider loads. Each embeds its
Omni.Provider and names its dialect per capability, so the struct alone
tells Omni where to send a request and how to format it.
Providers and models covers choosing, adding
and changing the models that load.
Struct fields
:id- the provider's string identifier (e.g."claude-sonnet-5-5"):name- human-readable display name (e.g."Claude Sonnet 5.5"):provider- theOmni.Providerserving the model:dialects- the dialect module per capability (:text,:decision), e.g.%{text: Omni.Dialects.AnthropicMessages}:context_size- maximum input tokens the model accepts:max_output_tokens- maximum tokens the model can generate:thinking- what thinking the model supports:false(none),true(supported, with no detail), or a map of what it accepts (see Thinking):input_modalities- supported input types (:text,:image,:pdf):output_modalities- supported output types (:text):input_cost- USD cost per million input tokens:output_cost- USD cost per million output tokens:cache_read_cost- USD cost per million cached input tokens (read):cache_write_cost- USD cost per million cached input tokens (write):release_date- date the model was released (from models.dev)
Capabilities and modalities
A capability is a kind of request the model serves: :text
(Omni.generate_text/3, Omni.stream_text/3) or :decision
(Omni.decide/3). The keys of :dialects are the model's capabilities,
each naming the dialect that speaks it; most models serve one, and a few
serve both (OpenAI's gpt-6-luna). Read them with capabilities/1,
capability?/2 and dialect/2:
{:ok, models} = Omni.list_models(:openrouter)
Enum.filter(models, &Omni.Model.capability?(&1, :decision))A modality is a kind of content going in or out: :text, :image,
:pdf. Text generation checks attachments against :input_modalities.
A decision's answers are structured text, so decision models report
output_modalities: [:text].
:thinking and tool use are text features. Every catalog text model
supports tool use (sources leave out those that don't), and decision
models have thinking: false.
Thinking
:thinking tells dialects how to translate the :thinking request option
into something the model accepts. A map carries any of:
:efforts- the effort values the model takes, as wire strings (["low", "medium", "high"])."none"means it can be switched off.:toggle-truewhen thinking is switched on and off as a whole:budget-{min, max}when the model takes a token budget; either bound may benil
For example:
%Omni.Model{thinking: %{efforts: ["none", "low", "medium", "high"]}}
%Omni.Model{thinking: %{toggle: true, budget: {0, 24_576}}}Sources fill it from catalog data. true means the model reasons but
nothing more is known; requests then fall back to each dialect's plain
encoding.
Looking up models
The top-level API takes a reference wherever it takes a model, so most code never holds the struct. Look one up to inspect it, or list what a provider has:
{:ok, model} = Omni.get_model(:anthropic, "claude-sonnet-5-5")
model.context_size #=> 1000000
{:ok, models} = Omni.list_models(:openai)
Enum.map(models, & &1.id)Model references
A reference names a model by provider id and model id, as a tuple or a string. Strings take a colon or a slash; the provider id ends at the first of either, so model ids keep any they hold:
{:openrouter, "typesafe/jev-1.14"}
"openrouter:typesafe/jev-1.14"
"openrouter/typesafe/jev-1.14"The provider may be spelled as its catalog key ("fireworks-ai:…" for
:fireworks_ai). A bare model id is not a reference: the same id can be
served by many providers.
to_ref/2 writes a model or tuple in any form, parse_ref/1 reads a string
back into a tuple, and interpolation gives the colon form:
Model.to_ref(model) #=> {:anthropic, "claude-sonnet-5-5"}
Model.to_ref(model, :slash) #=> "anthropic/claude-sonnet-5-5"
"#{model}" #=> "anthropic:claude-sonnet-5-5"
Model.parse_ref("anthropic:claude-sonnet-5-5")
#=> {:ok, {:anthropic, "claude-sonnet-5-5"}}Registering models at runtime
Models the catalog lacks (new releases, fine-tunes) are usually added in
config, as Adding and correcting models
shows. To add one while the app runs, build it with new/1 and register
it with put/1. The model names its dialect, since there is no catalog
data to take it from:
model =
Omni.Model.new(
id: "my-fine-tune",
name: "My Fine-Tune",
provider: :openai,
dialects: %{text: Omni.Dialects.OpenAIResponses},
context_size: 128_000,
max_output_tokens: 16_384,
input_cost: 2.0,
output_cost: 8.0
)
Omni.put_model(model)The model is then found by Omni.get_model/2 and Omni.list_models/1. A
model struct also works without being registered: pass it straight to
Omni.generate_text/3, Omni.stream_text/3 or Omni.decide/3.
Summary
Types
A capability a model can serve: text generation or decisions.
A model reference: {provider_id, model_id}, "provider_id:model_id" or
"provider_id/model_id".
A form to_ref/2 writes a reference in.
An LLM model descriptor.
What thinking a model supports: none, unknown detail, or what it accepts.
Functions
Returns the capabilities the model serves, in sorted order.
Returns whether the model serves a capability.
Returns the model's dialect for a capability.
Looks up a model by provider id and model id.
Lists all models for a provider.
Creates a new model struct from a keyword list or map.
Reads a string reference into a {provider_id, model_id} tuple.
Registers a model under its provider.
Writes a model, or a {provider_id, model_id} tuple, as a reference.
Types
@type capability() :: :text | :decision
A capability a model can serve: text generation or decisions.
A model reference: {provider_id, model_id}, "provider_id:model_id" or
"provider_id/model_id".
The provider id is a loaded provider's atom id (e.g. :anthropic), or in a
string that id or its catalog key ("fireworks-ai"). The model id is the
provider's string identifier for the model (e.g. "claude-sonnet-5-5"). A
string's provider id ends at its first : or /. See
Model references.
@type ref_format() :: :tuple | :colon | :slash
A form to_ref/2 writes a reference in.
@type t() :: %Omni.Model{ cache_read_cost: number(), cache_write_cost: number(), context_size: non_neg_integer(), dialects: %{optional(capability()) => module()}, id: String.t(), input_cost: number(), input_modalities: [atom()], max_output_tokens: non_neg_integer(), name: String.t(), output_cost: number(), output_modalities: [atom()], provider: Omni.Provider.t() | nil, release_date: Date.t() | nil, thinking: thinking() }
An LLM model descriptor.
:provider is the embedded provider struct and :dialects the dialect
module per capability. Cost fields are per million tokens. Modalities are
filtered to the supported set (see new/1).
@type thinking() :: boolean() | %{ optional(:efforts) => [String.t()], optional(:toggle) => true, optional(:budget) => {non_neg_integer() | nil, non_neg_integer() | nil} }
What thinking a model supports: none, unknown detail, or what it accepts.
Functions
@spec capabilities(t()) :: [capability()]
Returns the capabilities the model serves, in sorted order.
{:ok, model} = Omni.get_model(:openai, "gpt-6-luna")
Model.capabilities(model) #=> [:decision, :text]
@spec capability?(t(), capability()) :: boolean()
Returns whether the model serves a capability.
{:ok, models} = Omni.list_models(:openrouter)
Enum.filter(models, &Model.capability?(&1, :decision))
@spec dialect(t(), capability()) :: {:ok, module()} | {:error, {:unsupported_capability, capability()}}
Returns the model's dialect for a capability.
{:ok, model} = Omni.get_model(:anthropic, "claude-sonnet-5-5")
Model.dialect(model, :text) #=> {:ok, Omni.Dialects.AnthropicMessages}
Model.dialect(model, :decision) #=> {:error, {:unsupported_capability, :decision}}
Looks up a model by reference: a tuple or a string (see ref/0).
Model.get("anthropic:claude-sonnet-5-5")
Model.get({:anthropic, "claude-sonnet-5-5"})Returns the errors of parse_ref/1 for a string, then those of get/2.
Looks up a model by provider id and model id.
Returns {:ok, model} if found. {:error, {:unknown_provider, id}} means
the provider isn't loaded: it doesn't exist, or isn't selected in
config :omni, :models, providers:.
{:error, {:unknown_model, provider_id, model_id}} means the provider has
no model with that id.
Lists all models for a provider.
Returns {:error, {:unknown_provider, id}} if the provider isn't loaded.
@spec new(Enumerable.t()) :: t()
Creates a new model struct from a keyword list or map.
:provideris anOmni.Provideror the id of a loaded provider. Raises if that provider isn't loaded:dialectsmaps each capability to its dialect (%{text: Module}). It is never filled from the provider, so a model without one can't make requests:thinkingtakes a boolean, or a map or keyword list of:efforts,:toggleand:budget. An empty map meanstrue- Modalities are cut down to the supported set: unsupported values are
dropped, and an empty list becomes
[:text]
Other fields are stored as given, without validation. See Registering models at runtime for an example.
@spec parse_ref(String.t()) :: {:ok, {atom(), String.t()}} | {:error, {:invalid_model_ref, String.t()} | {:unknown_provider, String.t()}}
Reads a string reference into a {provider_id, model_id} tuple.
The provider id is everything before the first : or /, and may be
spelled as its catalog key. The provider must be loaded; the model is not
looked up (see get/1).
Model.parse_ref("openrouter:typesafe/jev-1.14") #=> {:ok, {:openrouter, "typesafe/jev-1.14"}}
Model.parse_ref("fireworks-ai/accounts/fireworks/models/kimi-k3")
#=> {:ok, {:fireworks_ai, "accounts/fireworks/models/kimi-k3"}}Returns {:error, {:invalid_model_ref, ref}} when the string has no
provider and model id to split, and {:error, {:unknown_provider, name}},
with the provider as written, when that provider isn't loaded.
@spec put(t()) :: :ok
Registers a model under its provider.
The model is then found by get/2 and list/1. A model with the same id
under that provider is replaced. Raises if the model's provider isn't
loaded or the model names no dialects. See
Registering models at runtime
for an example.
@spec to_ref(t() | {atom(), String.t()}, ref_format()) :: ref()
Writes a model, or a {provider_id, model_id} tuple, as a reference.
The format is :tuple (the default), :colon or :slash. To reformat a
string reference, read it with parse_ref/1 first.
{:ok, model} = Omni.get_model(:anthropic, "claude-sonnet-5-5")
Model.to_ref(model) #=> {:anthropic, "claude-sonnet-5-5"}
Model.to_ref(model, :colon) #=> "anthropic:claude-sonnet-5-5"
Model.to_ref(model, :slash) #=> "anthropic/claude-sonnet-5-5"