Omni.Model (Omni v2.0.0)

Copy Markdown View Source

A data struct describing a specific LLM.

A model carries its identity, what it can do and what it costs. Models are data, not modules: they are loaded from the configured model source (see Omni.Source) when their provider loads. Each embeds its Omni.Provider and names its dialect per capability, so the struct alone tells Omni where to send a request and how to format it. Providers and models covers choosing, adding and changing the models that load.

Struct fields

  • :id - the provider's string identifier (e.g. "claude-sonnet-5-5")
  • :name - human-readable display name (e.g. "Claude Sonnet 5.5")
  • :provider - the Omni.Provider serving the model
  • :dialects - the dialect module per capability (:text, :decision), e.g. %{text: Omni.Dialects.AnthropicMessages}
  • :context_size - maximum input tokens the model accepts
  • :max_output_tokens - maximum tokens the model can generate
  • :thinking - what thinking the model supports: false (none), true (supported, with no detail), or a map of what it accepts (see Thinking)
  • :input_modalities - supported input types (:text, :image, :pdf)
  • :output_modalities - supported output types (:text)
  • :input_cost - USD cost per million input tokens
  • :output_cost - USD cost per million output tokens
  • :cache_read_cost - USD cost per million cached input tokens (read)
  • :cache_write_cost - USD cost per million cached input tokens (write)
  • :release_date - date the model was released (from models.dev)

Capabilities and modalities

A capability is a kind of request the model serves: :text (Omni.generate_text/3, Omni.stream_text/3) or :decision (Omni.decide/3). The keys of :dialects are the model's capabilities, each naming the dialect that speaks it; most models serve one, and a few serve both (OpenAI's gpt-6-luna). Read them with capabilities/1, capability?/2 and dialect/2:

{:ok, models} = Omni.list_models(:openrouter)
Enum.filter(models, &Omni.Model.capability?(&1, :decision))

A modality is a kind of content going in or out: :text, :image, :pdf. Text generation checks attachments against :input_modalities. A decision's answers are structured text, so decision models report output_modalities: [:text].

:thinking and tool use are text features. Every catalog text model supports tool use (sources leave out those that don't), and decision models have thinking: false.

Thinking

:thinking tells dialects how to translate the :thinking request option into something the model accepts. A map carries any of:

  • :efforts - the effort values the model takes, as wire strings (["low", "medium", "high"]). "none" means it can be switched off.
  • :toggle - true when thinking is switched on and off as a whole
  • :budget - {min, max} when the model takes a token budget; either bound may be nil

For example:

%Omni.Model{thinking: %{efforts: ["none", "low", "medium", "high"]}}
%Omni.Model{thinking: %{toggle: true, budget: {0, 24_576}}}

Sources fill it from catalog data. true means the model reasons but nothing more is known; requests then fall back to each dialect's plain encoding.

Looking up models

The top-level API takes a reference wherever it takes a model, so most code never holds the struct. Look one up to inspect it, or list what a provider has:

{:ok, model} = Omni.get_model(:anthropic, "claude-sonnet-5-5")
model.context_size  #=> 1000000

{:ok, models} = Omni.list_models(:openai)
Enum.map(models, & &1.id)

Model references

A reference names a model by provider id and model id, as a tuple or a string. Strings take a colon or a slash; the provider id ends at the first of either, so model ids keep any they hold:

{:openrouter, "typesafe/jev-1.14"}
"openrouter:typesafe/jev-1.14"
"openrouter/typesafe/jev-1.14"

The provider may be spelled as its catalog key ("fireworks-ai:…" for :fireworks_ai). A bare model id is not a reference: the same id can be served by many providers.

to_ref/2 writes a model or tuple in any form, parse_ref/1 reads a string back into a tuple, and interpolation gives the colon form:

Model.to_ref(model)          #=> {:anthropic, "claude-sonnet-5-5"}
Model.to_ref(model, :slash)  #=> "anthropic/claude-sonnet-5-5"
"#{model}"                   #=> "anthropic:claude-sonnet-5-5"

Model.parse_ref("anthropic:claude-sonnet-5-5")
#=> {:ok, {:anthropic, "claude-sonnet-5-5"}}

Registering models at runtime

Models the catalog lacks (new releases, fine-tunes) are usually added in config, as Adding and correcting models shows. To add one while the app runs, build it with new/1 and register it with put/1. The model names its dialect, since there is no catalog data to take it from:

model =
  Omni.Model.new(
    id: "my-fine-tune",
    name: "My Fine-Tune",
    provider: :openai,
    dialects: %{text: Omni.Dialects.OpenAIResponses},
    context_size: 128_000,
    max_output_tokens: 16_384,
    input_cost: 2.0,
    output_cost: 8.0
  )

Omni.put_model(model)

The model is then found by Omni.get_model/2 and Omni.list_models/1. A model struct also works without being registered: pass it straight to Omni.generate_text/3, Omni.stream_text/3 or Omni.decide/3.

Summary

Types

A capability a model can serve: text generation or decisions.

A model reference: {provider_id, model_id}, "provider_id:model_id" or "provider_id/model_id".

A form to_ref/2 writes a reference in.

t()

An LLM model descriptor.

What thinking a model supports: none, unknown detail, or what it accepts.

Functions

Returns the capabilities the model serves, in sorted order.

Returns whether the model serves a capability.

Returns the model's dialect for a capability.

Looks up a model by reference: a tuple or a string (see ref/0).

Looks up a model by provider id and model id.

Lists all models for a provider.

Creates a new model struct from a keyword list or map.

Reads a string reference into a {provider_id, model_id} tuple.

Registers a model under its provider.

Writes a model, or a {provider_id, model_id} tuple, as a reference.

Types

capability()

@type capability() :: :text | :decision

A capability a model can serve: text generation or decisions.

ref()

@type ref() :: {atom(), String.t()} | String.t()

A model reference: {provider_id, model_id}, "provider_id:model_id" or "provider_id/model_id".

The provider id is a loaded provider's atom id (e.g. :anthropic), or in a string that id or its catalog key ("fireworks-ai"). The model id is the provider's string identifier for the model (e.g. "claude-sonnet-5-5"). A string's provider id ends at its first : or /. See Model references.

ref_format()

@type ref_format() :: :tuple | :colon | :slash

A form to_ref/2 writes a reference in.

t()

@type t() :: %Omni.Model{
  cache_read_cost: number(),
  cache_write_cost: number(),
  context_size: non_neg_integer(),
  dialects: %{optional(capability()) => module()},
  id: String.t(),
  input_cost: number(),
  input_modalities: [atom()],
  max_output_tokens: non_neg_integer(),
  name: String.t(),
  output_cost: number(),
  output_modalities: [atom()],
  provider: Omni.Provider.t() | nil,
  release_date: Date.t() | nil,
  thinking: thinking()
}

An LLM model descriptor.

:provider is the embedded provider struct and :dialects the dialect module per capability. Cost fields are per million tokens. Modalities are filtered to the supported set (see new/1).

thinking()

@type thinking() ::
  boolean()
  | %{
      optional(:efforts) => [String.t()],
      optional(:toggle) => true,
      optional(:budget) => {non_neg_integer() | nil, non_neg_integer() | nil}
    }

What thinking a model supports: none, unknown detail, or what it accepts.

Functions

capabilities(model)

@spec capabilities(t()) :: [capability()]

Returns the capabilities the model serves, in sorted order.

{:ok, model} = Omni.get_model(:openai, "gpt-6-luna")
Model.capabilities(model)  #=> [:decision, :text]

capability?(model, capability)

@spec capability?(t(), capability()) :: boolean()

Returns whether the model serves a capability.

{:ok, models} = Omni.list_models(:openrouter)
Enum.filter(models, &Model.capability?(&1, :decision))

dialect(model, capability)

@spec dialect(t(), capability()) ::
  {:ok, module()} | {:error, {:unsupported_capability, capability()}}

Returns the model's dialect for a capability.

{:ok, model} = Omni.get_model(:anthropic, "claude-sonnet-5-5")
Model.dialect(model, :text)      #=> {:ok, Omni.Dialects.AnthropicMessages}
Model.dialect(model, :decision)  #=> {:error, {:unsupported_capability, :decision}}

get(ref)

@spec get(ref()) :: {:ok, t()} | {:error, term()}

Looks up a model by reference: a tuple or a string (see ref/0).

Model.get("anthropic:claude-sonnet-5-5")
Model.get({:anthropic, "claude-sonnet-5-5"})

Returns the errors of parse_ref/1 for a string, then those of get/2.

get(provider_id, model_id)

@spec get(atom(), String.t()) :: {:ok, t()} | {:error, term()}

Looks up a model by provider id and model id.

Returns {:ok, model} if found. {:error, {:unknown_provider, id}} means the provider isn't loaded: it doesn't exist, or isn't selected in config :omni, :models, providers:. {:error, {:unknown_model, provider_id, model_id}} means the provider has no model with that id.

list(provider_id)

@spec list(atom()) :: {:ok, [t()]} | {:error, term()}

Lists all models for a provider.

Returns {:error, {:unknown_provider, id}} if the provider isn't loaded.

new(attrs)

@spec new(Enumerable.t()) :: t()

Creates a new model struct from a keyword list or map.

  • :provider is an Omni.Provider or the id of a loaded provider. Raises if that provider isn't loaded
  • :dialects maps each capability to its dialect (%{text: Module}). It is never filled from the provider, so a model without one can't make requests
  • :thinking takes a boolean, or a map or keyword list of :efforts, :toggle and :budget. An empty map means true
  • Modalities are cut down to the supported set: unsupported values are dropped, and an empty list becomes [:text]

Other fields are stored as given, without validation. See Registering models at runtime for an example.

parse_ref(ref)

@spec parse_ref(String.t()) ::
  {:ok, {atom(), String.t()}}
  | {:error, {:invalid_model_ref, String.t()} | {:unknown_provider, String.t()}}

Reads a string reference into a {provider_id, model_id} tuple.

The provider id is everything before the first : or /, and may be spelled as its catalog key. The provider must be loaded; the model is not looked up (see get/1).

Model.parse_ref("openrouter:typesafe/jev-1.14")  #=> {:ok, {:openrouter, "typesafe/jev-1.14"}}
Model.parse_ref("fireworks-ai/accounts/fireworks/models/kimi-k3")
#=> {:ok, {:fireworks_ai, "accounts/fireworks/models/kimi-k3"}}

Returns {:error, {:invalid_model_ref, ref}} when the string has no provider and model id to split, and {:error, {:unknown_provider, name}}, with the provider as written, when that provider isn't loaded.

put(model)

@spec put(t()) :: :ok

Registers a model under its provider.

The model is then found by get/2 and list/1. A model with the same id under that provider is replaced. Raises if the model's provider isn't loaded or the model names no dialects. See Registering models at runtime for an example.

to_ref(model, format \\ :tuple)

@spec to_ref(t() | {atom(), String.t()}, ref_format()) :: ref()

Writes a model, or a {provider_id, model_id} tuple, as a reference.

The format is :tuple (the default), :colon or :slash. To reformat a string reference, read it with parse_ref/1 first.

{:ok, model} = Omni.get_model(:anthropic, "claude-sonnet-5-5")
Model.to_ref(model)          #=> {:anthropic, "claude-sonnet-5-5"}
Model.to_ref(model, :colon)  #=> "anthropic:claude-sonnet-5-5"
Model.to_ref(model, :slash)  #=> "anthropic/claude-sonnet-5-5"