Universal Elixir client for LLM APIs. Text generation, tool use, structured output and decisions across many providers.
This module is the whole top-level API. New to Omni? Getting started is the place to begin.
Text generation
generate_text/3 sends a context to a model and returns the finished
response. stream_text/3 takes the same arguments and returns events as
the model writes. Tools in the context are run for you, and :output
asks for a validated map in place of prose.
{:ok, response} = Omni.generate_text("anthropic:claude-sonnet-5-5", "Hello!")
response.message
#=> %Omni.Message{role: :assistant, content: [%Omni.Content.Text{text: "Hello! How can..."}]}Text generation covers conversations, streaming, thinking, tools and structured output.
Decisions
decide/3 asks a decision model typed questions about a shared state and
returns a probability for each answer.
{:ok, decision} =
Omni.decide("openrouter:typesafe/jev-1.14", {ticket_text, [
urgent: Omni.predicate("Does this need a reply within the hour?")
]})
decision.answers[:urgent]
#=> %Omni.Answer.Predicate{probability: 0.91}Decisions covers the question types and their answers.
Models
Every request names a model, as an Omni.Model or a reference to one: a
{provider_id, model_id} tuple or a "provider_id:model_id" string.
get_model/1 and list_models/1 look models up.
{:ok, models} = Omni.list_models(:anthropic)Providers and models covers choosing and configuring providers, adding your own, and where model data comes from.
Building requests
context/1, message/1, tool/1, attachment/2 and tool_result/1
build what goes into a text request. predicate/2, choice/2 and
score/2 build the questions of a decision. Each is a shortcut for the
new function of the struct it returns.
Summary
Text generation
Generates text and returns the finished response.
Streams a text generation request.
Decisions
Creates a question with one answer from a set of choices, for decide/3.
Asks a decision model typed questions about a shared state.
Creates a yes/no question for decide/3.
Creates a question rating the state on an ordered scale, for decide/3.
Models
Looks up a model by reference.
Looks up a model by provider id and model id.
Looks up a loaded provider by id.
Lists all models for a provider.
Registers a model under its provider at runtime.
Context
Creates an attachment content block from a keyword list or map.
Creates an attachment content block from a source and media type.
Creates a context from a string, list of messages, keyword list, or map.
Creates a message from a string, keyword list, or map.
Creates a tool from a keyword list or map.
Creates a tool result content block from a keyword list or map.
Text generation
@spec generate_text(Omni.Model.ref() | Omni.Model.t(), term(), keyword()) :: {:ok, Omni.Response.t()} | {:error, term()}
Generates text and returns the finished response.
Takes the same arguments and options as stream_text/3: it is
stream_text/3 followed by Omni.StreamingResponse.complete/1.
Returns an Omni.Response with the assistant's message, token usage,
stop reason, and structured output (when :output is set).
{:ok, response} = Omni.generate_text("anthropic:claude-sonnet-5-5", "Hello!")
response.message #=> %Omni.Message{role: :assistant, content: [%Omni.Content.Text{...}]}
@spec stream_text(Omni.Model.ref() | Omni.Model.t(), term(), keyword()) :: {:ok, Omni.StreamingResponse.t()} | {:error, term()}
Streams a text generation request.
Returns an Omni.StreamingResponse: enumerate it for events as the model
writes, or call Omni.StreamingResponse.complete/1 for the finished
response.
The model is an Omni.Model or a reference to one: a
{provider_id, model_id} tuple or a "provider_id:model_id" string (see
Omni.Model.ref/0). The context is a string, a list of messages, or an
Omni.Context.
{:ok, stream} = Omni.stream_text("anthropic:claude-sonnet-5-5", "Tell me a story")
stream
|> Omni.StreamingResponse.text_stream()
|> Enum.each(&IO.write/1)When the context holds tools with handlers, Omni runs each tool the model
uses and sends the results back, until the model stops using tools. A
:tool_result event is emitted for each result along the way.
Options
:max_tokens- maximum output tokens (Anthropic defaults to 4096):temperature- sampling temperature:thinking- how much the model reasons before answering. Unset, the model's own default applies. Ignored for models that don't reason.:low,:medium,:high,:xhighor:max- an effort level, rounded to the nearest the model supports, or turned into a token budget for models that take one- a positive integer - a token budget, clamped to the model's range, or turned into an effort level for models that take levels
false- off, where the model can be switched off
:output- a JSON Schema map for structured output (see Structured output):cache- prompt caching::shortfor the provider's standard cache lifetime,:longfor its extended one where it has one:max_steps- the most requests the tool loop may make (default:infinity). Pass1to run tools yourself:tool_timeout- how long each tool may run, in milliseconds (default30_000):timeout- how long to wait for each chunk of the response, in milliseconds (default300_000). A response that keeps streaming is not cut off; use:max_tokensto bound its length:api_key- API key, overriding provider and app config:host- an origin ("http://localhost:4000") replacing the origin of the endpoint URL for this request; the path is kept (seeOmni.Provider):headers- extra request headers, merged over the provider's:metadata- a map passed to the provider:raw- whentrue, attaches raw{%Req.Request{}, %Req.Response{}}tuples to the response (one per request):plug- a Req test plug for stubbing HTTP responses in tests
Decisions
@spec choice(Omni.Question.instructions_input(), list()) :: Omni.Question.Choice.t()
Creates a question with one answer from a set of choices, for decide/3.
See Omni.Question.Choice.new/2.
Omni.choice("Which team owns this?", billing: "Payments and refunds", infra: "Outages", other: nil)
@spec decide( Omni.Model.ref() | Omni.Model.t(), Omni.Decision.t() | Omni.Decision.input(), keyword() ) :: {:ok, Omni.Decision.t()} | {:error, term()}
Asks a decision model typed questions about a shared state.
The model is an Omni.Model or a reference to one (see stream_text/3),
and must serve decisions (see Omni.Model.capability?/2). The decision is
an Omni.Decision or the shorthand {state, questions}
(Omni.Decision.input/0): the state is a string, a map, or a list of
them, and the questions a keyword list built with predicate/2,
choice/2 and score/2.
{:ok, decision} =
Omni.decide("openrouter:typesafe/jev-1.14", {"I was charged twice", [
refund: Omni.predicate("Is the customer asking for a refund?"),
tone: Omni.choice("What is the tone?", [:calm, :frustrated, :angry])
]})
decision.answers[:refund] #=> %Omni.Answer.Predicate{probability: 0.97}
decision.usage #=> %Omni.Usage{input_tokens: 61, total_cost: 2.6e-6, ...}Returns the decision with :answers, :usage and :model filled in. A
model with no decision dialect returns
{:error, {:unsupported_capability, :decision}}; invalid questions are an
error before any request is made.
Options
:api_key- API key, overriding provider and app config:host- an origin ("http://localhost:4000") replacing the origin of the endpoint URL for this request; the path is kept (seeOmni.Provider):headers- extra request headers, merged over the provider's:timeout- how long to wait for the response, in milliseconds (default300_000):raw- whentrue, attaches the raw{%Req.Request{}, %Req.Response{}}pair to the decision:plug- a Req test plug for stubbing HTTP responses in tests
A dialect may take more: Omni.Dialects.OpenAIDecisions takes
:safety_identifier and :prompt_cache_key.
@spec predicate(Omni.Question.instructions_input(), [{boolean(), String.t()}] | nil) :: Omni.Question.Predicate.t()
Creates a yes/no question for decide/3.
See Omni.Question.Predicate.new/2.
Omni.predicate("Is this spam?", true: "Unsolicited advertising", false: "A real conversation")
@spec score(Omni.Question.instructions_input(), list()) :: Omni.Question.Score.t()
Creates a question rating the state on an ordered scale, for decide/3.
See Omni.Question.Score.new/2.
Omni.score("How angry is the customer?", ["Calm", "Frustrated but civil", "Very angry"])
Models
@spec get_model(Omni.Model.ref()) :: {:ok, Omni.Model.t()} | {:error, term()}
Looks up a model by reference.
A reference is a {provider_id, model_id} tuple or a
"provider_id:model_id" string (a slash works too).
{:ok, model} = Omni.get_model("anthropic:claude-sonnet-5-5")
{:ok, model} = Omni.get_model("openrouter/typesafe/jev-1.14")A malformed string is {:error, {:invalid_model_ref, ref}}; otherwise the
errors are those of get_model/2. See Omni.Model for the reference forms.
@spec get_model(atom(), String.t()) :: {:ok, Omni.Model.t()} | {:error, term()}
Looks up a model by provider id and model id.
Returns {:ok, model} if found, {:error, {:unknown_provider, id}} if the
provider isn't loaded (it doesn't exist, or isn't selected in
config :omni, :models, providers:), or
{:error, {:unknown_model, provider_id, model_id}} if the model doesn't
exist.
{:ok, model} = Omni.get_model(:anthropic, "claude-sonnet-5-5")
model.context_size #=> 1000000
@spec get_provider(atom()) :: {:ok, Omni.Provider.t()} | {:error, term()}
Looks up a loaded provider by id.
Returns {:ok, provider}, or {:error, {:unknown_provider, id}} if no
provider with that id is loaded.
{:ok, provider} = Omni.get_provider(:anthropic)
provider.name #=> "Anthropic"
@spec list_models(atom()) :: {:ok, [Omni.Model.t()]} | {:error, term()}
Lists all models for a provider.
{:ok, models} = Omni.list_models(:anthropic)
Enum.map(models, & &1.id)
#=> ["claude-fable-5", "claude-fable-5-1", "claude-haiku-4-5", ...]
@spec put_model(Omni.Model.t()) :: :ok
Registers a model under its provider at runtime.
The model can then be found by get_model/2 and list_models/1. See
Omni.Model.put/1.
model =
Omni.Model.new(
id: "my-fine-tune",
name: "My Fine-Tune",
provider: :openai,
dialects: %{text: Omni.Dialects.OpenAIResponses}
)
Omni.put_model(model)
Context
@spec attachment(Enumerable.t()) :: Omni.Content.Attachment.t()
Creates an attachment content block from a keyword list or map.
See Omni.Content.Attachment.new/1.
Omni.attachment(source: {:url, "https://example.com/cat.png"}, media_type: "image/png", meta: %{name: "cat.png"})
@spec attachment(Omni.Content.Attachment.source(), String.t()) :: Omni.Content.Attachment.t()
Creates an attachment content block from a source and media type.
The source is {:base64, data} (base64-encoded) or {:url, url}. See
Omni.Content.Attachment.
Omni.message(role: :user, content: [
"What's in this image?",
Omni.attachment({:base64, Base.encode64(File.read!("photo.png"))}, "image/png")
])
@spec context(String.t() | [Omni.Message.t()] | Omni.Context.t() | Enumerable.t()) :: Omni.Context.t()
Creates a context from a string, list of messages, keyword list, or map.
A string is a single user message. A list of messages is the message
history. A keyword list or map sets :system, :messages and :tools.
See Omni.Context.
Omni.context("Hello!")
Omni.context(system: "You are helpful.", messages: [...], tools: [...])
@spec message(String.t() | Enumerable.t()) :: Omni.Message.t()
Creates a message from a string, keyword list, or map.
A string is a user message. A keyword list or map sets :role and
:content. See Omni.Message.new/1.
Omni.message("What is Elixir?")
Omni.message(role: :assistant, content: "Elixir is...")
@spec tool(Enumerable.t()) :: Omni.Tool.t()
Creates a tool from a keyword list or map.
For inline tools with a handler function. For reusable tool modules, see
Omni.Tool.
Omni.tool(
name: "search",
description: "Searches the web",
input_schema: Omni.Schema.object(%{query: Omni.Schema.string()}, required: [:query]),
handler: fn input -> do_search(input.query) end
)
@spec tool_result(Enumerable.t()) :: Omni.Content.ToolResult.t()
Creates a tool result content block from a keyword list or map.
For running tools yourself instead of through the loop: answer each
Omni.Content.ToolUse with a result in the next user message. String
content is wrapped in a text block. See Omni.Content.ToolResult.
Omni.message(role: :user, content: [
Omni.tool_result(tool_use_id: tool_use.id, name: tool_use.name, content: "72°F and sunny")
])