Token counts and computed costs for a request.
Found on every Omni.Response as response.usage, and on every answered
Omni.Decision as decision.usage. Costs are worked out from the token
counts and the model's pricing (see price/2).
Struct fields
:input_tokens,:output_tokens- tokens sent and tokens generated:cache_read_tokens,:cache_write_tokens- tokens read from and written to the provider's prompt cache:total_tokens- the sum of the four token counts:input_cost,:output_cost,:cache_read_cost,:cache_write_cost- the cost of each in USD, from the model's pricing:total_cost- the sum of the four costs
All fields default to 0. Use add/2 and sum/1 to total usage across
requests.
Summary
Functions
Sums corresponding fields of two usage structs.
Creates a new usage struct from a keyword list or map.
Fills totals and costs from the token counts and a model's pricing.
Reduces a list of usage structs into a single summed usage.
Types
@type t() :: %Omni.Usage{ cache_read_cost: number(), cache_read_tokens: non_neg_integer(), cache_write_cost: number(), cache_write_tokens: non_neg_integer(), input_cost: number(), input_tokens: non_neg_integer(), output_cost: number(), output_tokens: non_neg_integer(), total_cost: number(), total_tokens: non_neg_integer() }
Token usage and cost breakdown.
Functions
Sums corresponding fields of two usage structs.
@spec new(Enumerable.t()) :: t()
Creates a new usage struct from a keyword list or map.
@spec price(t(), Omni.Model.t() | nil) :: t()
Fills totals and costs from the token counts and a model's pricing.
Each cost is the token count times the model's rate per million tokens; a
missing rate (or a nil model) costs nothing. :total_tokens and
:total_cost are the sums of the four categories. Existing costs and
totals are overwritten.
usage = Usage.new(input_tokens: 1_000, output_tokens: 200)
Usage.price(usage, model)
#=> %Usage{input_tokens: 1000, output_tokens: 200, total_tokens: 1200,
# input_cost: 0.003, output_cost: 0.003, total_cost: 0.006, ...}
Reduces a list of usage structs into a single summed usage.