Automatic context compaction for the native agent provider.
When a conversation grows large enough to approach the model's context limit, this module summarizes older turns into a condensed message while preserving the system prompt and recent turns verbatim.
The summary is generated by an LLM call using the same model, so it captures the salient points of the conversation without losing critical context like file paths, decisions made, and code snippets discussed.
Summary
Types
Options for compaction.
A function that takes (model, messages, opts) and returns {:ok, text} or {:error, reason}.
Functions
Forces compaction regardless of token count.
Checks whether compaction is needed and performs it if so.
Types
@type compact_opts() :: [ model: String.t(), llm_client: summary_fn(), context_limit: pos_integer() | nil, threshold: float(), keep_recent: pos_integer() ]
Options for compaction.
@type summary_fn() :: (String.t(), [ReqLLM.Message.t()], keyword() -> {:ok, String.t()} | {:error, term()})
A function that takes (model, messages, opts) and returns {:ok, text} or {:error, reason}.
Functions
@spec compact(ReqLLM.Context.t(), compact_opts()) :: {:compacted, ReqLLM.Context.t(), String.t()} | {:ok, ReqLLM.Context.t()} | {:error, String.t()}
Forces compaction regardless of token count.
Useful for the /compact slash command.
@spec maybe_compact(ReqLLM.Context.t(), compact_opts()) :: {:compacted, ReqLLM.Context.t(), String.t()} | {:ok, ReqLLM.Context.t()}
Checks whether compaction is needed and performs it if so.
Returns {:compacted, new_context, summary_text} if compaction was
performed, or {:ok, context} if the context is within limits.
Options:
:model— the model string (used for context limit lookup and summary call):llm_client— the LLM client function for generating the summary:context_limit— override the model's context limit:threshold— fraction of context limit that triggers compaction (default: 0.80):keep_recent— number of recent messages to preserve verbatim (default: 6)