How to Use Summarization Transforms#
Prerequisites
This guide assumes you are familiar with the following concepts:
Overview#
Agent Spec supports Summarization Transforms that help manage long conversations and large messages by automatically summarizing content before it reaches the LLM. This prevents context window limits and improves efficiency.
There are two types of summarization transforms:
MessageSummarizationTransform: Summarizes individual messages that exceed a specified character limit
ConversationSummarizationTransform: Summarizes conversation history when the total number of messages exceeds a threshold
Note
By default, summarization transforms use an in-memory datastore for caching summarized content. You can specify your own datastore by providing a Datastore parameter. For information on how to configure different types of datastores such as Oracle or PostgreSQL, see How to Use Datastores.
This guide will walk you through:
Configuring an LLM for summarization
Creating message summarization transforms
Creating conversation summarization transforms
Using transforms in agents
Serializing agents with transforms
Basic implementation#
1. Configure an LLM for summarization#
Summarization transforms summarize messages or conversations before they are passed to the LLM to reduce the context size of the agent LLM. To perform the summarization, they require an LLM configuration.
from pyagentspec.llms import OpenAiConfig
# Configure an LLM for summarization
llm_config = OpenAiConfig(
id="summarization_llm",
name="summarization_llm",
model_id="gpt-4o-mini",
)
API Reference: OpenAiConfig
2. Create summarization transforms#
In this section, we will create both transforms with appropriate thresholds and instructions.
from pyagentspec.transforms import ConversationSummarizationTransform, MessageSummarizationTransform
# Create a message summarization transform
message_summarizer = MessageSummarizationTransform(
id="message_summarizer",
name="message_summarizer",
llm=llm_config,
max_message_size=10_000, # Summarize messages longer than 10k characters
summarization_instructions="Create a concise summary of this message focusing on key points and actionable information.",
summarized_message_template="Message summary: {{summary}}",
)
# Create a conversation summarization transform
conversation_summarizer = ConversationSummarizationTransform(
id="conversation_summarizer",
name="conversation_summarizer",
llm=llm_config,
max_num_messages=50, # Summarize when conversation exceeds 50 messages
min_num_messages=10, # Keep the last 10 messages unsummarized
summarization_instructions="Summarize this conversation thread, highlighting key decisions, action items, and important context.",
summarized_conversation_template="Conversation summary: {{summary}}",
)
API Reference: MessageSummarizationTransform, ConversationSummarizationTransform
3. Use transforms in agents#
Add summarization transforms to agents to automatically handle long content.
from pyagentspec.agent import Agent
# Create an agent that uses both summarization transforms
agent_with_summarization = Agent(
id="summarizing_agent",
name="summarizing_agent",
system_prompt="You are a helpful assistant that can handle long conversations and messages efficiently.",
llm_config=llm_config,
transforms=[message_summarizer, conversation_summarizer],
)
API Reference: Agent
4. Serializing agents with transforms#
MessageTransform configurations can be serialized to JSON or YAML for deployment.
from pyagentspec.serialization import AgentSpecSerializer
# Serialize the agent with transforms
serialized_agent = AgentSpecSerializer().to_json(agent_with_summarization)
print("\nSerialized Agent with Transforms:")
print(serialized_agent)
API Reference: AgentSpecSerializer
Here is what the serialized agent with transforms will look like ↓
Click here to see the agent configuration.
{
"component_type": "Agent",
"id": "summarizing_agent",
"name": "summarizing_agent",
"description": null,
"metadata": {},
"inputs": [],
"outputs": [],
"llm_config": {
"$component_ref": "summarization_llm"
},
"system_prompt": "You are a helpful assistant that can handle long conversations and messages efficiently.",
"tools": [],
"toolboxes": [],
"human_in_the_loop": true,
"transforms": [
{
"component_type": "MessageSummarizationTransform",
"id": "message_summarizer",
"name": "message_summarizer",
"description": null,
"metadata": {},
"llm": {
"$component_ref": "summarization_llm"
},
"max_message_size": 10000,
"summarization_instructions": "Create a concise summary of this message focusing on key points and actionable information.",
"summarized_message_template": "Message summary: {{summary}}",
"max_cache_size": 10000,
"max_cache_lifetime": 14400,
"cache_collection_name": "summarized_messages_cache",
"datastore": {
"component_type": "InMemoryCollectionDatastore",
"id": "dfffd0b3-e5a5-415f-9739-c911f68286bb",
"name": "default-inmemory-datastore",
"description": null,
"metadata": {},
"datastore_schema": {
"summarized_messages_cache": {
"properties": {
"cache_key": {
"type": "string"
},
"cache_content": {
"type": "string"
},
"created_at": {
"type": "number"
},
"last_used_at": {
"type": "number"
}
},
"type": "object"
}
}
}
},
{
"component_type": "ConversationSummarizationTransform",
"id": "conversation_summarizer",
"name": "conversation_summarizer",
"description": null,
"metadata": {},
"llm": {
"$component_ref": "summarization_llm"
},
"max_num_messages": 50,
"min_num_messages": 10,
"summarization_instructions": "Summarize this conversation thread, highlighting key decisions, action items, and important context.",
"summarized_conversation_template": "Conversation summary: {{summary}}",
"max_cache_size": 10000,
"max_cache_lifetime": 14400,
"cache_collection_name": "summarized_conversations_cache",
"datastore": {
"component_type": "InMemoryCollectionDatastore",
"id": "e6e0857c-1608-4c7d-a4eb-a65fbaadb670",
"name": "default-inmemory-datastore",
"description": null,
"metadata": {},
"datastore_schema": {
"summarized_conversations_cache": {
"properties": {
"cache_key": {
"type": "string"
},
"cache_content": {
"type": "string"
},
"prefix_size": {
"type": "integer"
},
"created_at": {
"type": "number"
},
"last_used_at": {
"type": "number"
}
},
"type": "object"
}
}
}
}
],
"$referenced_components": {
"summarization_llm": {
"component_type": "OpenAiConfig",
"id": "summarization_llm",
"name": "summarization_llm",
"description": null,
"metadata": {},
"default_generation_parameters": null,
"model_id": "gpt-4o-mini",
"api_type": "chat_completions",
"api_key": null
}
},
"agentspec_version": "26.2.0"
}
Recap#
This guide covered how to implement summarization transforms in Agent Spec to handle long conversations and messages efficiently.
Below is the complete code from this guide.
1# Copyright © 2025 Oracle and/or its affiliates.
2#
3# This software is under the Apache License 2.0
4# (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0) or Universal Permissive License
5# (UPL) 1.0 (LICENSE-UPL or https://oss.oracle.com/licenses/upl), at your option.
6
7# .. define-llm-config
8from pyagentspec.llms import OpenAiConfig
9
10# Configure an LLM for summarization
11llm_config = OpenAiConfig(
12 id="summarization_llm",
13 name="summarization_llm",
14 model_id="gpt-4o-mini",
15)
16
17# .. start-transforms
18from pyagentspec.transforms import ConversationSummarizationTransform, MessageSummarizationTransform
19
20# Create a message summarization transform
21message_summarizer = MessageSummarizationTransform(
22 id="message_summarizer",
23 name="message_summarizer",
24 llm=llm_config,
25 max_message_size=10_000, # Summarize messages longer than 10k characters
26 summarization_instructions="Create a concise summary of this message focusing on key points and actionable information.",
27 summarized_message_template="Message summary: {{summary}}",
28)
29
30# Create a conversation summarization transform
31conversation_summarizer = ConversationSummarizationTransform(
32 id="conversation_summarizer",
33 name="conversation_summarizer",
34 llm=llm_config,
35 max_num_messages=50, # Summarize when conversation exceeds 50 messages
36 min_num_messages=10, # Keep the last 10 messages unsummarized
37 summarization_instructions="Summarize this conversation thread, highlighting key decisions, action items, and important context.",
38 summarized_conversation_template="Conversation summary: {{summary}}",
39)
40# .. end-transforms
41
42# .. start-agent-with-transforms
43from pyagentspec.agent import Agent
44
45# Create an agent that uses both summarization transforms
46agent_with_summarization = Agent(
47 id="summarizing_agent",
48 name="summarizing_agent",
49 system_prompt="You are a helpful assistant that can handle long conversations and messages efficiently.",
50 llm_config=llm_config,
51 transforms=[message_summarizer, conversation_summarizer],
52)
53# .. end-agent-with-transforms
54
55# .. start-serialization
56from pyagentspec.serialization import AgentSpecSerializer
57
58# Serialize the agent with transforms
59serialized_agent = AgentSpecSerializer().to_json(agent_with_summarization)
60
61print("\nSerialized Agent with Transforms:")
62print(serialized_agent)
63# .. end-serialization