Omni.Usage (Omni v2.0.0)

Copy Markdown View Source

Token counts and computed costs for a request.

Found on every Omni.Response as response.usage, and on every answered Omni.Decision as decision.usage. Costs are worked out from the token counts and the model's pricing (see price/2).

Struct fields

  • :input_tokens, :output_tokens - tokens sent and tokens generated
  • :cache_read_tokens, :cache_write_tokens - tokens read from and written to the provider's prompt cache
  • :total_tokens - the sum of the four token counts
  • :input_cost, :output_cost, :cache_read_cost, :cache_write_cost - the cost of each in USD, from the model's pricing
  • :total_cost - the sum of the four costs

All fields default to 0. Use add/2 and sum/1 to total usage across requests.

Summary

Types

t()

Token usage and cost breakdown.

Functions

Sums corresponding fields of two usage structs.

Creates a new usage struct from a keyword list or map.

Fills totals and costs from the token counts and a model's pricing.

Reduces a list of usage structs into a single summed usage.

Types

t()

@type t() :: %Omni.Usage{
  cache_read_cost: number(),
  cache_read_tokens: non_neg_integer(),
  cache_write_cost: number(),
  cache_write_tokens: non_neg_integer(),
  input_cost: number(),
  input_tokens: non_neg_integer(),
  output_cost: number(),
  output_tokens: non_neg_integer(),
  total_cost: number(),
  total_tokens: non_neg_integer()
}

Token usage and cost breakdown.

Functions

add(a, b)

@spec add(t(), t()) :: t()

Sums corresponding fields of two usage structs.

new(attrs)

@spec new(Enumerable.t()) :: t()

Creates a new usage struct from a keyword list or map.

price(usage, model)

@spec price(t(), Omni.Model.t() | nil) :: t()

Fills totals and costs from the token counts and a model's pricing.

Each cost is the token count times the model's rate per million tokens; a missing rate (or a nil model) costs nothing. :total_tokens and :total_cost are the sums of the four categories. Existing costs and totals are overwritten.

usage = Usage.new(input_tokens: 1_000, output_tokens: 200)
Usage.price(usage, model)
#=> %Usage{input_tokens: 1000, output_tokens: 200, total_tokens: 1200,
#          input_cost: 0.003, output_cost: 0.003, total_cost: 0.006, ...}

sum(usages)

@spec sum([t()]) :: t()

Reduces a list of usage structs into a single summed usage.