Omni.Response (Omni v2.0.0)

Copy Markdown View Source

The result of a text generation request.

Returned by Omni.generate_text/3 and Omni.StreamingResponse.complete/1. It holds the assistant's message, why the model stopped and what the request cost. To continue the conversation, add the response to the context with Omni.Context.push/2.

{:ok, response} = Omni.generate_text("anthropic:claude-sonnet-5-5", "Hello!")

response.message.content  #=> [%Omni.Content.Text{text: "Hello! How can I help?"}]
response.stop_reason      #=> :stop
response.usage.total_cost #=> 0.000213

Struct fields

  • :model - the Omni.Model that handled the request
  • :message - the assistant's final message
  • :messages - every message this request produced. Usually just [response.message]; after a tool loop, each assistant message and each user message of tool results, in order
  • :output - the validated, decoded output when the :output option was set, else nil
  • :stop_reason - why generation ended (see stop_reason/0)
  • :error - a description of the error when :stop_reason is :error, else nil
  • :raw - a list of {%Req.Request{}, %Req.Response{}} tuples, one per request, when the :raw option was set
  • :usage - the Omni.Usage for the whole request, summed over every step of a tool loop

Summary

Types

Why generation ended.

t()

A generation response envelope.

Functions

Creates a new response struct from a keyword list or map.

Types

stop_reason()

@type stop_reason() :: :stop | :length | :tool_use | :refusal | :error | :cancelled

Why generation ended.

  • :stop - the model finished
  • :length - the token limit was reached
  • :tool_use - the model used a tool that Omni did not run
  • :refusal - the model or provider declined, on content or safety grounds
  • :error - the stream failed part-way (see the :error field)
  • :cancelled - generation was cancelled. Omni never sets this itself; it is for code built on Omni that cancels a stream and keeps the partial response

t()

@type t() :: %Omni.Response{
  error: String.t() | nil,
  message: Omni.Message.t() | nil,
  messages: [Omni.Message.t()],
  model: Omni.Model.t(),
  output: map() | list() | nil,
  raw: [{Req.Request.t(), Req.Response.t()}] | nil,
  stop_reason: stop_reason(),
  usage: Omni.Usage.t()
}

A generation response envelope.

Functions

new(attrs)

@spec new(Enumerable.t()) :: t()

Creates a new response struct from a keyword list or map.