How to Use Summarization Transforms#

Prerequisites

This guide assumes you are familiar with the following concepts:

Overview#

Agent Spec supports Summarization Transforms that help manage long conversations and large messages by automatically summarizing content before it reaches the LLM. This prevents context window limits and improves efficiency.

There are two types of summarization transforms:

Note

By default, summarization transforms use an in-memory datastore for caching summarized content. You can specify your own datastore by providing a Datastore parameter. For information on how to configure different types of datastores such as Oracle or PostgreSQL, see How to Use Datastores.

This guide will walk you through:

  1. Configuring an LLM for summarization

  2. Creating message summarization transforms

  3. Creating conversation summarization transforms

  4. Using transforms in agents

  5. Serializing agents with transforms

Basic implementation#

1. Configure an LLM for summarization#

Summarization transforms summarize messages or conversations before they are passed to the LLM to reduce the context size of the agent LLM. To perform the summarization, they require an LLM configuration.

from pyagentspec.llms import OpenAiConfig

# Configure an LLM for summarization
llm_config = OpenAiConfig(
    id="summarization_llm",
    name="summarization_llm",
    model_id="gpt-4o-mini",
)

API Reference: OpenAiConfig

2. Create summarization transforms#

In this section, we will create both transforms with appropriate thresholds and instructions.

from pyagentspec.transforms import ConversationSummarizationTransform, MessageSummarizationTransform

# Create a message summarization transform
message_summarizer = MessageSummarizationTransform(
    id="message_summarizer",
    name="message_summarizer",
    llm=llm_config,
    max_message_size=10_000,  # Summarize messages longer than 10k characters
    summarization_instructions="Create a concise summary of this message focusing on key points and actionable information.",
    summarized_message_template="Message summary: {{summary}}",
)

# Create a conversation summarization transform
conversation_summarizer = ConversationSummarizationTransform(
    id="conversation_summarizer",
    name="conversation_summarizer",
    llm=llm_config,
    max_num_messages=50,  # Summarize when conversation exceeds 50 messages
    min_num_messages=10,  # Keep the last 10 messages unsummarized
    summarization_instructions="Summarize this conversation thread, highlighting key decisions, action items, and important context.",
    summarized_conversation_template="Conversation summary: {{summary}}",
)

API Reference: MessageSummarizationTransform, ConversationSummarizationTransform

If you prefer a character-based threshold for conversations, set max_num_messages=None explicitly and use max_num_characters instead. max_num_characters measures the total number of characters in the conversation, can still be combined with min_num_messages to keep the latest messages unsummarized, and cannot be used at the same time as max_num_messages:

conversation_summarizer_by_characters = ConversationSummarizationTransform(
    id="conversation_summarizer_by_characters",
    name="conversation_summarizer_by_characters",
    llm=llm_config,
    max_num_messages=None,  # Explicitly disable the message-count threshold
    max_num_characters=20_000,  # Summarize once the total conversation size gets too large in characters
    min_num_messages=10,  # Still keep the latest 10 messages unsummarized
    summarization_instructions="Summarize this conversation thread once its character footprint grows too large.",
    summarized_conversation_template="Conversation summary: {{summary}}",
)

3. Use transforms in agents#

Add summarization transforms to agents to automatically handle long content.

from pyagentspec.agent import Agent

# Create an agent that uses both summarization transforms
agent_with_summarization = Agent(
    id="summarizing_agent",
    name="summarizing_agent",
    system_prompt="You are a helpful assistant that can handle long conversations and messages efficiently.",
    llm_config=llm_config,
    transforms=[message_summarizer, conversation_summarizer],
)

API Reference: Agent

4. Serializing agents with transforms#

MessageTransform configurations can be serialized to JSON or YAML for deployment.

from pyagentspec.serialization import AgentSpecSerializer

# Serialize the agent with transforms
serialized_agent = AgentSpecSerializer().to_json(agent_with_summarization)

print("\nSerialized Agent with Transforms:")
print(serialized_agent)

API Reference: AgentSpecSerializer

Here is what the serialized agent with transforms will look like ↓

Click here to see the agent configuration.
{
    "component_type": "Agent",
    "id": "summarizing_agent",
    "name": "summarizing_agent",
    "description": null,
    "metadata": {},
    "inputs": [],
    "outputs": [],
    "llm_config": {
        "$component_ref": "summarization_llm"
    },
    "system_prompt": "You are a helpful assistant that can handle long conversations and messages efficiently.",
    "tools": [],
    "toolboxes": [],
    "human_in_the_loop": true,
    "transforms": [
        {
            "component_type": "MessageSummarizationTransform",
            "id": "message_summarizer",
            "name": "message_summarizer",
            "description": null,
            "metadata": {},
            "llm": {
                "$component_ref": "summarization_llm"
            },
            "max_message_size": 10000,
            "summarization_instructions": "Create a concise summary of this message focusing on key points and actionable information.",
            "summarized_message_template": "Message summary: {{summary}}",
            "max_cache_size": 10000,
            "max_cache_lifetime": 14400,
            "cache_collection_name": "summarized_messages_cache",
            "datastore": {
                "component_type": "InMemoryCollectionDatastore",
                "id": "dfffd0b3-e5a5-415f-9739-c911f68286bb",
                "name": "default-inmemory-datastore",
                "description": null,
                "metadata": {},
                "datastore_schema": {
                    "summarized_messages_cache": {
                        "properties": {
                            "cache_key": {
                                "type": "string"
                            },
                            "cache_content": {
                                "type": "string"
                            },
                            "created_at": {
                                "type": "number"
                            },
                            "last_used_at": {
                                "type": "number"
                            }
                        },
                        "type": "object"
                    }
                }
            }
        },
        {
            "component_type": "ConversationSummarizationTransform",
            "id": "conversation_summarizer",
            "name": "conversation_summarizer",
            "description": null,
            "metadata": {},
            "llm": {
                "$component_ref": "summarization_llm"
            },
            "max_num_messages": 50,
            "min_num_messages": 10,
            "summarization_instructions": "Summarize this conversation thread, highlighting key decisions, action items, and important context.",
            "summarized_conversation_template": "Conversation summary: {{summary}}",
            "max_cache_size": 10000,
            "max_cache_lifetime": 14400,
            "cache_collection_name": "summarized_conversations_cache",
            "datastore": {
                "component_type": "InMemoryCollectionDatastore",
                "id": "e6e0857c-1608-4c7d-a4eb-a65fbaadb670",
                "name": "default-inmemory-datastore",
                "description": null,
                "metadata": {},
                "datastore_schema": {
                    "summarized_conversations_cache": {
                        "properties": {
                            "cache_key": {
                                "type": "string"
                            },
                            "cache_content": {
                                "type": "string"
                            },
                            "prefix_size": {
                                "type": "integer"
                            },
                            "created_at": {
                                "type": "number"
                            },
                            "last_used_at": {
                                "type": "number"
                            }
                        },
                        "type": "object"
                    }
                }
            }
        }
    ],
    "$referenced_components": {
        "summarization_llm": {
            "component_type": "OpenAiConfig",
            "id": "summarization_llm",
            "name": "summarization_llm",
            "description": null,
            "metadata": {},
            "default_generation_parameters": null,
            "model_id": "gpt-4o-mini",
            "api_type": "chat_completions",
            "api_key": null
        }
    },
    "agentspec_version": "26.2.0"
}

Recap#

This guide covered how to implement summarization transforms in Agent Spec to handle long conversations and messages efficiently.

Below is the complete code from this guide.
 1# Copyright © 2025 Oracle and/or its affiliates.
 2#
 3# This software is under the Apache License 2.0
 4# (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0) or Universal Permissive License
 5# (UPL) 1.0 (LICENSE-UPL or https://oss.oracle.com/licenses/upl), at your option.
 6
 7# .. define-llm-config
 8from pyagentspec.llms import OpenAiConfig
 9
10# Configure an LLM for summarization
11llm_config = OpenAiConfig(
12    id="summarization_llm",
13    name="summarization_llm",
14    model_id="gpt-4o-mini",
15)
16
17# .. start-transforms
18from pyagentspec.transforms import ConversationSummarizationTransform, MessageSummarizationTransform
19
20# Create a message summarization transform
21message_summarizer = MessageSummarizationTransform(
22    id="message_summarizer",
23    name="message_summarizer",
24    llm=llm_config,
25    max_message_size=10_000,  # Summarize messages longer than 10k characters
26    summarization_instructions="Create a concise summary of this message focusing on key points and actionable information.",
27    summarized_message_template="Message summary: {{summary}}",
28)
29
30# Create a conversation summarization transform
31conversation_summarizer = ConversationSummarizationTransform(
32    id="conversation_summarizer",
33    name="conversation_summarizer",
34    llm=llm_config,
35    max_num_messages=50,  # Summarize when conversation exceeds 50 messages
36    min_num_messages=10,  # Keep the last 10 messages unsummarized
37    summarization_instructions="Summarize this conversation thread, highlighting key decisions, action items, and important context.",
38    summarized_conversation_template="Conversation summary: {{summary}}",
39)
40# .. end-transforms
41
42# .. start-conversation-character-threshold
43conversation_summarizer_by_characters = ConversationSummarizationTransform(
44    id="conversation_summarizer_by_characters",
45    name="conversation_summarizer_by_characters",
46    llm=llm_config,
47    max_num_messages=None,  # Explicitly disable the message-count threshold
48    max_num_characters=20_000,  # Summarize once the total conversation size gets too large in characters
49    min_num_messages=10,  # Still keep the latest 10 messages unsummarized
50    summarization_instructions="Summarize this conversation thread once its character footprint grows too large.",
51    summarized_conversation_template="Conversation summary: {{summary}}",
52)
53# .. end-conversation-character-threshold
54
55# .. start-agent-with-transforms
56from pyagentspec.agent import Agent
57
58# Create an agent that uses both summarization transforms
59agent_with_summarization = Agent(
60    id="summarizing_agent",
61    name="summarizing_agent",
62    system_prompt="You are a helpful assistant that can handle long conversations and messages efficiently.",
63    llm_config=llm_config,
64    transforms=[message_summarizer, conversation_summarizer],
65)
66# .. end-agent-with-transforms
67
68# .. start-serialization
69from pyagentspec.serialization import AgentSpecSerializer
70
71# Serialize the agent with transforms
72serialized_agent = AgentSpecSerializer().to_json(agent_with_summarization)
73
74print("\nSerialized Agent with Transforms:")
75print(serialized_agent)
76# .. end-serialization