
Omni is an Elixir client for LLM APIs. One set of functions covers text generation, streaming, tool use, structured output and decisions, across Anthropic, OpenAI, Google and many other providers. Changing model means changing one argument.
Install
Add Omni to your dependencies:
def deps do
[
{:omni, "~> 2.0"}
]
endSet an API key
Each built-in provider reads its key from the environment variable that provider's own tools use: ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY and so on. If the variable is set, there is nothing to configure. Built-in providers lists every provider and its variable.
To read a key from somewhere else, configure the provider by its id:
# config/runtime.exs
config :omni, :providers,
anthropic: [api_key: {:system, "MY_ANTHROPIC_KEY"}],
openai: [api_key: {MyApp.Secrets, :fetch, ["openai"]}]You can also pass :api_key on any request.
Generate text
Name a model and give it a prompt:
{:ok, response} = Omni.generate_text("anthropic:claude-sonnet-5-5", "Why is the sky blue?")
response.message
#=> %Omni.Message{role: :assistant, content: [%Omni.Content.Text{text: "Sunlight is..."}]}
response.usage
#=> %Omni.Usage{input_tokens: 14, output_tokens: 120, total_cost: 0.0018, ...}A model is named by its provider id and model id, as a string ("anthropic:claude-sonnet-5-5") or a tuple ({:anthropic, "claude-sonnet-5-5"}). To see what a provider offers:
{:ok, models} = Omni.list_models(:anthropic)
Enum.map(models, & &1.id)
#=> ["claude-fable-5", "claude-fable-5-1", "claude-haiku-4-5", ...]For a conversation, pass a context with a system prompt and messages:
context =
Omni.context(
system: "You are a patient physics tutor.",
messages: [
Omni.message("Why is the sky blue?"),
Omni.message(role: :assistant, content: "Sunlight is scattered by the air..."),
Omni.message("So why are sunsets red?")
]
)
{:ok, response} = Omni.generate_text("anthropic:claude-sonnet-5-5", context)Stream the response
Omni.stream_text/3 takes the same arguments and returns as soon as the response starts. Handle events as they arrive, then collect the finished response:
alias Omni.StreamingResponse
{:ok, stream} = Omni.stream_text("anthropic:claude-sonnet-5-5", "Tell me a story")
{:ok, response} =
stream
|> StreamingResponse.on(:text_delta, fn %{delta: text} -> IO.write(text) end)
|> StreamingResponse.complete()Give the model tools
A tool is a name, a description, a schema for its input and a function to run. Put tools in the context, and Omni runs each one the model asks for, sends the result back, and repeats until the model answers in text:
weather_tool =
Omni.tool(
name: "get_weather",
description: "Gets the current weather for a city",
input_schema: Omni.Schema.object(%{city: Omni.Schema.string()}, required: [:city]),
handler: fn input -> "18°C and cloudy in #{input.city}" end
)
context =
Omni.context(
messages: [Omni.message("What's the weather in London?")],
tools: [weather_tool]
)
{:ok, response} = Omni.generate_text("anthropic:claude-sonnet-5-5", context)Get structured output
Pass a schema as :output and the answer comes back as a validated map:
schema =
Omni.Schema.object(
%{
name: Omni.Schema.string(description: "The capital city"),
population: Omni.Schema.integer(description: "Approximate population")
},
required: [:name, :population]
)
{:ok, response} =
Omni.generate_text("anthropic:claude-sonnet-5-5", "What is the capital of France?", output: schema)
response.output
#=> %{name: "Paris", population: 2161000}Ask a decision model
Decision models are a different kind of model. They don't write text. They answer typed questions about an input with probabilities, in a single request:
{:ok, decision} =
Omni.decide("openrouter:typesafe/jev-1.14", {ticket_text, [
urgent: Omni.predicate("Does this need a reply within the hour?"),
team: Omni.choice("Which team owns this?", [:billing, :infra, :other])
]})
decision.answers[:urgent]
#=> %Omni.Answer.Predicate{probability: 0.91}
decision.answers[:team]
#=> %Omni.Answer.Choice{choice: :infra, probabilities: [billing: 0.02, infra: 0.95, other: 0.03], confidence: 0.93}Where next
- Text generation for conversations, streaming events, thinking, attachments, tools and structured output in full.
- Decisions for the question types and how to read their answers.
- Providers and models to choose which providers load, add providers and models of your own, and change where model data comes from.
- Built-in providers for each provider's id, API key variable and quirks.
- Upgrading to v2 if you are coming from Omni 1.x.
To build agents on top of Omni, with state, sessions and long-running loops, see omni_agent.