Omni.Codec (Omni v2.0.0)

Copy Markdown View Source

Lossless serialisation of Omni structs to and from JSON-safe maps.

Use this when persisting messages, content blocks, usage records or decisions to a storage layer that speaks JSON (Ecto :map columns, document stores, the wire). encode/1 produces a plain map (or list of maps) with string keys and only JSON-compatible values; decode/2 reverses it, returning the original structs.

The encoded shape describes itself: every encoded value carries a "__type" key, so decode/2 needs nothing else to rebuild it.

iex> message = Omni.Message.new("hello")
iex> encoded = Omni.Codec.encode(message)
iex> {:ok, ^message} = Omni.Codec.decode(encoded)

Supported types

Lists of any combination of the above are also supported. Keyword lists (a decision's questions or answers) are encoded only inside a decision.

Message origin

Message.origin is encoded readably, as %{"model" => "provider:model", "dialect" => "Omni.Dialects.AnthropicMessages"}, the model a string reference (see Omni.Model). A provider that isn't loaded, or a dialect unknown to the decoding node, decodes as nil, so that message's thinking is not sent back.

Decisions

A decision is encoded whole: state, questions, and once answered its answers, usage and model.

  • :model is stored as a string reference ("openai:gpt-6-luna") and looked up when decoding, so the decoded decision holds the model as it is loaded now. A model that isn't loaded decodes as nil.
  • :raw is not encoded and decodes as nil.
  • Questions and answers are lists of %{"name" => ..., "question" => ...} (or "answer") maps in order. Choices, levels, probabilities and criteria are lists of [key, value] pairs, so order survives stores that reorder object keys.
  • Choice and level keys keep their type: a string key is a JSON string, an atom key %{"atom" => "billing"}. Question names are always atoms and are stored as plain strings.

Atoms

By default decoding never creates atoms, so it is safe on input you don't control: question names and atom keys must already exist on the decoding node ({:error, {:unknown_atom, name}} otherwise), and opaque blobs are read with :erlang.binary_to_term/2's :safe option, failing with {:error, {:invalid_etf, _}} on an unknown atom.

An atom exists once a module that mentions it is loaded. A release loads every module at boot, but in development modules load on first use, and atoms built at runtime (:"record_#{id}") don't exist on a fresh node. When decoding data your app wrote itself, pass atoms: :create to create them, for decision keys and opaque blobs alike:

Omni.Codec.decode(stored, atoms: :create)

Never pass it for untrusted input: atoms are not garbage-collected, and enough distinct ones crash the node.

Opaque fields

Message.private and Attachment.meta can hold arbitrary terms (atoms, tuples, structs from provider-specific data) that have no portable JSON representation. These are encoded as opaque base64-encoded ETF blobs (%{"__etf" => "..."}), decoded as described under Atoms.

Encoding arbitrary terms

encode_term/1 and decode_term/2 expose the same opaque-blob mechanism for any Erlang term. Use them in persistence layers that need to stash values outside the Omni struct family while keeping the same JSON-safe shape.

Summary

Types

Decode options (see decode/2).

Any value the codec can encode.

Decode error reasons.

Functions

Decodes a previously-encoded map (or list of maps) back into Omni structs.

Decodes a wrapper produced by encode_term/1 back into the original term.

Encodes an Omni struct (or a list of structs) to a JSON-safe map.

Encodes an arbitrary Erlang term as an opaque JSON-safe wrapper.

Types

decode_opts()

@type decode_opts() :: [{:atoms, :existing | :create}]

Decode options (see decode/2).

encodable()

Any value the codec can encode.

error()

@type error() ::
  :invalid_input
  | {:unknown_type, String.t()}
  | {:invalid_role, term()}
  | {:missing_field, atom()}
  | {:invalid_field, atom()}
  | {:invalid_key, term()}
  | {:unknown_atom, String.t()}
  | {:invalid_source, term()}
  | {:invalid_timestamp, term()}
  | {:invalid_etf, term()}

Decode error reasons.

Functions

decode(input, opts \\ [])

@spec decode(map() | [map()], decode_opts()) :: {:ok, term()} | {:error, error()}

Decodes a previously-encoded map (or list of maps) back into Omni structs.

Returns {:ok, struct} or {:ok, [struct, ...]} on success. Returns {:error, reason} if the input is malformed, has an unknown "__type", or fails field-level validation. For lists, decoding stops on the first failing element.

Options

  • :atoms - :existing (default) decodes only atoms that already exist on this node; :create creates them, for data your app wrote itself (see Atoms)

decode_term(wrapper, opts \\ [])

@spec decode_term(map(), decode_opts()) :: {:ok, term()} | {:error, error()}

Decodes a wrapper produced by encode_term/1 back into the original term.

By default uses :erlang.binary_to_term/2 with the :safe option, so unknown atoms are not created and code is not loaded from the binary. Takes the same :atoms option as decode/2; atoms: :create drops :safe, for data your app wrote itself.

encode(list)

@spec encode(encodable() | [encodable()]) :: map() | [map()]

Encodes an Omni struct (or a list of structs) to a JSON-safe map.

Always succeeds for valid input types. Lists are encoded element-wise and returned as a bare list (no envelope).

encode_term(term)

@spec encode_term(term()) :: %{required(String.t()) => String.t()}

Encodes an arbitrary Erlang term as an opaque JSON-safe wrapper.

The term is serialised via :erlang.term_to_binary/1 and base64-encoded inside a %{"__etf" => "..."} map. Use this for values that have no portable JSON representation (atom-keyed maps, tuples, structs) when you need to stash them in a JSON-typed storage column.

iex> wrapper = Omni.Codec.encode_term({:ok, %{a: 1}})
iex> {:ok, {:ok, %{a: 1}}} = Omni.Codec.decode_term(wrapper)