Omni.Tool.Runner (Omni v2.0.0)

Copy Markdown View Source

Runs tool uses in parallel and returns their tool results.

The tool loop uses this to run the tools the model asks for. Use it yourself when you take over the loop, with max_steps: 1, and want the same validation, timeouts and error handling (see Running tools yourself).

{:ok, response} = Omni.generate_text(model, context, max_steps: 1)

tool_uses = Enum.filter(response.message.content, &match?(%Omni.Content.ToolUse{}, &1))
tools = Map.new(context.tools, &{&1.name, &1})

results = Omni.Tool.Runner.run(tool_uses, tools)

context =
  context
  |> Omni.Context.push(response)
  |> Omni.Context.push(Omni.message(role: :user, content: results))

Summary

Functions

Runs tool uses in parallel, returning one result for each, in order.

Functions

run(tool_uses, tool_map, timeout \\ 5000)

@spec run(
  [Omni.Content.ToolUse.t()],
  %{required(String.t()) => Omni.Tool.t()},
  timeout()
) :: [
  Omni.Content.ToolResult.t()
]

Runs tool uses in parallel, returning one result for each, in order.

tool_map maps each tool's name to its Omni.Tool. A result's content is what the handler returned: a string as it is, anything else as JSON.

A tool use never fails the call. A name with no tool in the map, input that fails the schema, a handler that raises and a tool that outruns the timeout each give a result with is_error: true and a message the model can read.

timeout is in milliseconds and applies to each tool (default 5_000; the tool loop passes its :tool_timeout).