A streaming text response, yielding events as the model writes.
Returned by Omni.stream_text/3. The stream is lazy and runs in the
process that consumes it: nothing is read from the provider until you
enumerate it. Streaming introduces the
three ways to consume one:
on/3registers a handler for one event type, andcomplete/1runs the stream to its end and returns theOmni.Responsetext_stream/1gives only the text, chunk by chunk- the struct is
Enumerable, soEnumandStreamfunctions andforsee every event
{:ok, stream} = Omni.stream_text("anthropic:claude-sonnet-5-5", "Hello!")
{:ok, response} =
stream
|> StreamingResponse.on(:text_delta, fn %{delta: text} -> IO.write(text) end)
|> StreamingResponse.complete()A stream can be consumed once. To stop early, call cancel/1.
Struct fields
Use the functions in this module; the fields are for building on top of them.
:stream- the lazy stream of events:cancel- a function that closes the HTTP response, ornil
Events
Each event is a {type, data, partial_response} tuple. The third element
is the Omni.Response as built so far, rebuilt after every event.
Every content block arrives as a _start, any number of _deltas and an
_end. :index is the block's position in the message content. Blocks
never overlap: one ends before the next starts.
{:text_start, %{index: 0}, %Response{}}
{:text_delta, %{index: 0, delta: "Hello"}, %Response{}}
{:text_end, %{index: 0, content: %Text{}}, %Response{}}
{:thinking_start, %{index: 0}, %Response{}}
{:thinking_delta, %{index: 0, delta: "..."}, %Response{}}
{:thinking_end, %{index: 0, content: %Thinking{}}, %Response{}}
{:tool_use_start, %{index: 1, id: "call_1", name: "weather"}, %Response{}}
{:tool_use_delta, %{index: 1, delta: "{\"city\":"}, %Response{}}
{:tool_use_end, %{index: 1, content: %ToolUse{}}, %Response{}}When Omni runs tools, a :tool_result event follows for each result,
before the next request starts. Its data is the result block, and its
third element is the finished response of the step that used the tool:
{:tool_result, %ToolResult{}, %Response{}}Every stream ends with exactly one of these:
{:done, %{stop_reason: :stop}, %Response{}}
{:error, reason, %Response{}}:done means the model finished, with the response's stop reason. An
:error reason is {:stream_error, reason} when the stream failed
part-way, or :incomplete_stream when it ended before the model
finished.
Summary
Types
An event: its type, its data and the response so far.
The type of an event (see Events).
A streaming response.
Functions
Cancels the request by closing the HTTP response.
Runs the stream to its end and returns the final response.
Registers a handler for events of one type.
Returns a stream of text delta binaries.
Types
@type event() :: {event_type(), map(), Omni.Response.t()} | {:error, term(), Omni.Response.t()}
An event: its type, its data and the response so far.
@type event_type() ::
:text_start
| :text_delta
| :text_end
| :thinking_start
| :thinking_delta
| :thinking_end
| :tool_use_start
| :tool_use_delta
| :tool_use_end
| :tool_result
| :error
| :done
The type of an event (see Events).
@type t() :: %Omni.StreamingResponse{cancel: (-> :ok) | nil, stream: Enumerable.t()}
A streaming response.
Functions
@spec cancel(t()) :: :ok
Cancels the request by closing the HTTP response.
Call it when you stop consuming a stream before its end, so the connection is not left open.
@spec complete(t()) :: {:ok, Omni.Response.t()} | {:error, term()}
Runs the stream to its end and returns the final response.
Handlers registered with on/3 fire along the way. Returns
{:ok, response}, or {:error, reason} if the stream ended with an
error.
@spec on(t(), event_type(), (map() -> any()) | (map(), Omni.Response.t() -> any())) :: t()
Registers a handler for events of one type.
Returns a new streaming response with the handler in its pipeline. The
handler is called with the event's data as the stream is consumed (by
complete/1 or any Enum function), and its return value is ignored.
Handlers chain, and each fires for its own events:
{:ok, response} =
stream
|> StreamingResponse.on(:text_delta, fn %{delta: d} -> IO.write(d) end)
|> StreamingResponse.on(:thinking_delta, fn %{delta: d} -> IO.write(d) end)
|> StreamingResponse.on(:done, fn %{stop_reason: r} -> IO.puts("\nStop: #{r}") end)
|> StreamingResponse.complete()A two-argument handler also receives the partial response, for example to track token usage as it grows:
StreamingResponse.on(stream, :text_delta, fn _event, partial ->
send(self(), {:tokens, partial.usage.output_tokens})
end)
@spec text_stream(t()) :: Enumerable.t()
Returns a stream of text delta binaries.
Keeps only the :text_delta events and takes the text from each, for
when the text is all you need:
stream
|> StreamingResponse.text_stream()
|> Enum.into("")