How to Enable Agents to Handle Long Contexts#
Download Python ScriptPython script/notebook for this guide.
Prerequisites
This guide assumes familiarity with:
In agentic systems, conversations can become lengthy, building up extensive context over many interactions. Large Language Models (LLMs) have limits on the context they can process effectively, which may result in reduced performance or errors when these limits are surpassed. Long context also incur higher costs.
To address these performance issues and reduce cost, you can apply techniques that reduce the context size while retaining key information.
This guide demonstrates three methods for reducing context size using built-in transforms:
Summarizing long messages using MessageSummarizationTransform: Individual messages that exceed size limits (such as large tool outputs or long user inputs) are automatically summarized using an LLM.
Summarizing long conversations using ConversationSummarizationTransform: When conversations become too lengthy, older messages are summarized to maintain context while keeping the total size manageable.
Using both transforms together: Combining both approaches for comprehensive context management.
This guide shows how to use built-in MessageTransform objects to reduce the context size in agents.
Introduction#
Message Transforms#
A MessageTransform is a transformation applied to a list of messages before they are sent to the LLM powering the agent. You can learn more about them in the Advanced Prompting Techniques guide.
In this guide, we use built-in message transforms to adjust the agent’s chat history by passing them directly to the Agent constructor.
LLM Setup#
There are multiple ways to define an LLM in WayFlow depending on the provider. Choose the one adapted to your needs:
from wayflowcore.models import OCIGenAIModel, OCIClientConfigWithApiKey
llm = OCIGenAIModel(
model_id="provider.model-id",
compartment_id="compartment-id",
client_config=OCIClientConfigWithApiKey(
service_endpoint="https://url-to-service-endpoint.com",
),
)
from wayflowcore.models import VllmModel
llm = VllmModel(
model_id="model-id",
host_port="VLLM_HOST_PORT",
)
from wayflowcore.models import OllamaModel
llm = OllamaModel(
model_id="model-id",
)
In this guide, we will use VllmModel. We can use two different LLMs - a capable LLM as the agent and a fast LLM as the summarizer. Note that if conversations include images, the summarization LLM should support image processing:
from wayflowcore.models import VllmModel
summarization_llm = VllmModel(
model_id="SMALL_LLAMA_MODEL_ID",
host_port="SMALL_LLAMA_API_URL",
)
agent_llm = VllmModel(
model_id="LLAMA_MODEL_ID",
host_port="LLAMA_API_URL",
)
Summarizing Long Messages#
We use the built-in MessageSummarizationTransform to automatically summarize individual messages that exceed a specified size limit, including tool outputs, user messages and images.
from wayflowcore.transforms import MessageSummarizationTransform
transform = MessageSummarizationTransform(
llm=summarization_llm,
max_message_size=10000,
summarized_message_template="Summarized result:\n{{summary}}"
)
Note
When creating MessageSummarizationTransform without specifying a datastore, it will initialize a default InMemoryDatastore for caching, which is only suitable for prototyping. This will raise a user warning indicating that in-memory datastores are not recommended for production systems.
For production systems, you can use OracleDatabaseDatastore or PostgresDatabaseDatastore.
Let’s integrate the MessageSummarizationTransform into the Agent:
from wayflowcore import Agent, Message, tool
@tool
def read_logs_tool() -> str:
"""Return logs from the system"""
return (
"Starting long processing\n"
+ "Waiting for process ...\n" * 2_000
+ "Found error: Missing credentials for user kurt_andrews."
+ " Please pass the correct credentials.\n"
)
agent = Agent(
llm=agent_llm,
tools=[read_logs_tool],
transforms=[transform],
)
Before seeing an example, let’s define a token counting event listener that will help us see the impact of our summarizer on token consumption.
from wayflowcore.events.eventlistener import EventListener
from wayflowcore.events.event import LlmGenerationResponseEvent
from wayflowcore.events import register_event_listeners
from wayflowcore.tokenusage import TokenUsage
class TokenListener(EventListener):
def __init__(self):
self.usage = TokenUsage()
def __call__(self, event):
if isinstance(event, LlmGenerationResponseEvent):
self.usage += event.completion.token_usage
In this example, we run a conversation where the agent uses a tool that returns a very long log output. The MessageSummarizationTransform automatically summarizes this long message to stay within size limits. The agent then answers questions about the error, demonstrating that key information is preserved despite the summarization.
listener_with_transform = TokenListener()
with register_event_listeners([listener_with_transform]):
from wayflowcore.executors.executionstatus import UserMessageRequestStatus
conversation = agent.start_conversation()
conversation.append_user_message("Can you explain the error in the system?")
status = conversation.execute()
if isinstance(status, UserMessageRequestStatus):
print(status.message.content)
# The error is likely due to a missing credential for
# a user named kurt_andrews. Please ensure the credentials
# are provided correctly for kurt_andrews to resolve the issue.
print(f"Token usage with MessageSummarizationTransform after 1 user message: {listener_with_transform.usage.total_tokens} tokens")
# Token usage with MessageSummarizationTransform after 1 user message: 8854 tokens
conversation.append_user_message('What is the exact message repeated in a loop? Do not recall the tool')
status = conversation.execute()
if isinstance(status, UserMessageRequestStatus):
print(status.message.content)
# The tool is stuck in an infinite loop, repeatedly displaying the message "Waiting for process" until it encounters an error due to missing credentials for user "kurt_andrews".
print(f"Token usage with MessageSummarizationTransform after 2 user messages: {listener_with_transform.usage.total_tokens} tokens")# .. start-##_Create_Token_Event_Listener
# Token usage with MessageSummarizationTransform after 2 user messages: 9314 tokens
To compare, let’s run the same conversation without any transforms to see the difference in token usage.
agent_no_transform = Agent(
llm=agent_llm,
tools=[read_logs_tool],
transforms=[], # no transforms
)
After one user message, we have used the following number of tokens:
# ... (same conversation as above)
print(f"Token usage without MessageSummarizationTransform after 1 user message: {listener_no_transform.usage.total_tokens} tokens")
# Token usage without MessageSummarizationTransform after 1 user message: 12757 tokens
And after the second user message:
print(f"Token usage without MessageSummarizationTransform after 2 user messages: {listener_no_transform.usage.total_tokens} tokens")# .. start-##_Create_Token_Event_Listener
# Token usage without MessageSummarizationTransform after 2 user messages: 25224 tokens
After one user message, both approaches show high token usage:
Without MessageSummarizationTransform: the agent LLM must process the full long tool output (around 12,800 tokens)
With MessageSummarizationTransform: the summarization LLM generates a summary (around 8,900 tokens)
After a second message, the difference becomes significant:
Without the transform: token usage doubles to about 25,300 tokens as the agent LLM must reprocess the long tool output
With the transform: token usage only increases slightly to about 9,400 tokens due to summaries being cached in datastores and not regenerated.
This demonstrates that the messages were effectively summarized and cached.
Summarizing Long Conversations#
This method uses the built-in ConversationSummarizationTransform to summarize older messages when conversations become too long, preserving historical information while keeping the context manageable.
from wayflowcore.transforms import ConversationSummarizationTransform
transform = ConversationSummarizationTransform(
llm=summarization_llm,
max_num_messages=5,
min_num_messages=2
)
Let’s integrate the MessageTransform into the Agent:
agent = Agent(
llm=agent_llm,
tools=[],
transforms=[transform],
)
conversation = agent.start_conversation()
conversation.append_user_message("What is the capital of Switzerland?")
conversation.execute()
conversation.append_user_message("Tell me a fun fact about it.")
conversation.execute()
conversation.append_user_message("What is the population?")
conversation.execute()
conversation.append_user_message("What is the area?")
conversation.execute()
conversation.append_user_message("What languages are spoken there?")
conversation.execute()
# The conversation will be summarized when it exceeds 5 messages
If you want to check that the ConversationSummarizationTransform is really summarizing old messages together, you either use the same TokenLister as in the previous section to measure tokens or you can also create your own EventListener that prints the conversation.
Using Both Transforms Together#
For comprehensive context management, you can apply both transforms together. The MessageSummarizationTransform will first handle individual long messages, and then the ConversationSummarizationTransform will manage the overall conversation length.
transforms = [
MessageSummarizationTransform(
llm=summarization_llm,
max_message_size=10000,
summarized_message_template="Summarized content:\n{{summary}}"
),
ConversationSummarizationTransform(
llm=summarization_llm,
max_num_messages=5,
min_num_messages=2
)
]
Let’s integrate the MessageTransform into the Agent:
agent = Agent(
llm=summarization_llm,
tools=[read_logs_tool],
transforms=transforms,
)
conversation = agent.start_conversation()
LONG_CONVERSATION = [
{"role": "user", "content": "Hi! Can you tell me something interesting about dolphins?"},
{
"role": "assistant",
"content": "Absolutely! Dolphins are fascinating creatures, famous for their intelligence and complex behavior. For example, they have been observed using tools, such as covering their snouts with sponges to protect themselves while foraging on the seafloor. Dolphins also display strong social bonds and have been known to help injured individuals within their pods. Communication among dolphins is advanced; they use a series of clicks, whistles, and body movements to convey information, and some species even have distinctive signature whistles that function like names.",
},
{"role": "user", "content": "Wow, I didn’t know that. Do dolphins have good memory?"},
{
"role": "assistant",
"content": "Dolphins possess remarkable memories, particularly when it comes to their social groups and vocal communication. Researchers have discovered that dolphins can remember the unique signature whistles of other dolphins for over 20 years, which is the longest social memory recorded in non-human animals. This ability highlights their sophisticated cognitive abilities and the importance of long-term relationships in dolphin societies. Memory also plays a crucial role in their navigation and hunting skills, as dolphins migrate and follow paths in the oceans over great distances. In addition, their keen memory supports learning from one another, enhancing the social structure of their pods.",
},
{
"role": "user",
"content": "That’s impressive. Are there other animals with similar intelligence?",
},
{
"role": "assistant",
"content": "Indeed, there are several other animals that exhibit forms of intelligence comparable to dolphins. Great apes, such as chimpanzees and orangutans, have been observed making and using tools, solving puzzles, and displaying empathy and self-awareness. Elephants show emotional intelligence and have complex social relationships, including mourning their dead and helping injured companions. Crows and parrots, members of the bird family, demonstrate advanced problem-solving, mimicry, and even the ability to plan for future events. These examples across different branches of the animal kingdom show that intelligence has evolved in diverse and fascinating ways.",
},
{"role": "user", "content": "Can elephants recognize themselves in a mirror like dolphins?"},
{
"role": "assistant",
"content": "Yes, elephants are among the elite group of species known to pass the mirror self-recognition test, an experiment generally considered evidence of self-awareness. In these experiments, elephants explore themselves in mirrors and investigate marks on their bodies they otherwise couldn’t see. Notably, this test has only been passed by a few species, including certain great apes, dolphins, magpies, and elephants. Such ability hints at a deep cognitive capacity, suggesting they have a mental representation of themselves, which is foundational for empathy and understanding the experiences of others. This self-recognition is just one of the many amazing aspects of elephant intelligence that continue to be studied.",
},
{"role": "user", "content": "What is the mirror test?"},
{
"role": "assistant",
"content": "The mirror test, also known as the mark test, is a behavioral technique designed to assess self-awareness in animals. In the test, an animal is marked with an odorless dye in a place it cannot see without a mirror. The animal is then given access to a mirror, and researchers observe whether it investigates or tries to touch the mark on its own body, indicating recognition of its reflection as itself, not another animal. Passing the mirror test suggests a level of self-concept, which is considered an advanced cognitive trait. However, not all intelligent animals pass the mirror test, and failing it doesn’t necessarily mean an animal lacks self-awareness – it might simply not care about the mark or not rely on visual cues.",
},
{
"role": "user",
"content": "Besides animals, what’s another field where recognition is important?",
},
{
"role": "assistant",
"content": "Recognition is vitally important in many human technologies, notably in the field of artificial intelligence. Facial recognition technology is widely used for security purposes, such as unlocking smartphones or monitoring public spaces. In social media, these algorithms help automatically tag friends in photos, organizing vast amounts of visual data. Additionally, recognition systems are applied in retail, for personalized marketing, and in health care, for patient identification and monitoring. However, these powerful tools also raise important questions around privacy and data security, sparking ongoing debates about their regulation and ethical use.",
},
{"role": "user", "content": "Is facial recognition always accurate?"},
{
"role": "assistant",
"content": "Facial recognition technology has improved significantly over recent years, but it is not flawless and its accuracy can vary depending on several factors. Lighting, angle, image quality, and the diversity of data used to train the algorithms all influence performance. Notably, many studies have shown that facial recognition systems can have higher error rates when identifying people with darker skin tones or from underrepresented demographics, leading to concerns about bias and discrimination. This has prompted researchers and policymakers to push for better training data, algorithmic transparency, and stricter oversight. As the technology becomes more widespread, ensuring fairness and accuracy is an essential focus for developers and regulators alike.",
},
{"role": "user", "content": "What are some other ethical concerns in AI?"},
{
"role": "assistant",
"content": "Ethical challenges in AI are numerous and complex, reflecting the far-reaching impact of these technologies on society. Privacy concerns are paramount, as AI systems often require large amounts of personal data, raising questions about consent and data protection. Bias in algorithms is another major issue, as AI can inadvertently reinforce existing societal prejudices if not carefully monitored and tested. Transparency is crucial, because many AI systems, especially those using deep learning, can behave as 'black boxes'—making decisions in ways that aren’t easily understandable to humans. Additionally, there are fears around job displacement, as automation could replace roles faster than new opportunities are created, necessitating careful consideration and retraining initiatives.",
},
{
"role": "user",
"content": "Job displacement is interesting. Which jobs are most at risk from AI?",
},
{
"role": "assistant",
"content": "Jobs that involve repetitive, predictable tasks are currently considered the most vulnerable to automation by AI. For example, data entry clerks, assembly line workers, and certain types of customer service representatives face a high likelihood of being replaced or augmented by machines. Even within fields like transportation, we’re seeing the rise of autonomous vehicles that could impact truck drivers and delivery personnel. On the other hand, roles that require creativity, complex decision-making, emotional intelligence, or physical dexterity in unpredictable environments—such as healthcare providers, artists, and skilled trades—are less susceptible for now. To manage this transition, many experts recommend investing in retraining programs and encouraging adaptability in the workforce.",
},
{"role": "user", "content": "On a lighter note, do you know any fun space facts?"},
{
"role": "assistant",
"content": "Absolutely, space offers an endless trove of fascinating discoveries. For example, Venus has a very unusual rotation: a single day on Venus (one rotation on its axis) is actually longer than its year (one complete orbit around the Sun) due to its slow spin. There are also diamond rains on planets like Neptune and Uranus, caused by extreme pressure turning carbon into solid diamonds. You might be surprised to learn that there are more stars in the observable universe than grains of sand on all the Earth’s beaches combined. Finally, if you were to fall into a black hole, the laws of physics as we understand them break down at the singularity—a phenomenon still not fully explained by scientists.",
},
{"role": "user", "content": "That’s fascinating! How hot does it get on Venus?"},
{
"role": "assistant",
"content": "Venus is the hottest planet in our solar system, even outpacing Mercury, which is closer to the Sun. The average surface temperature on Venus can reach an astonishing 465 degrees Celsius, or about 869 degrees Fahrenheit. This is hot enough to melt lead, and it creates an extremely inhospitable environment for life as we know it. The intense heat is largely due to the planet’s dense atmosphere, composed mostly of carbon dioxide, which traps heat through a runaway greenhouse effect. In addition to being scorching, Venus’s atmosphere features thick clouds of sulfuric acid that make its surface perpetually shrouded and acidic.",
},
{"role": "user", "content": "Why is Venus hotter than Mercury?"},
{
"role": "assistant",
"content": "Despite being the second planet from the Sun, Venus is hotter than Mercury primarily because of its thick, carbon dioxide-rich atmosphere. This atmosphere acts as a blanket, trapping incoming solar radiation and preventing heat from escaping, leading to the planet's infamous runaway greenhouse effect. Meanwhile, Mercury has only a thin exosphere, which allows heat to escape into space quickly, resulting in dramatic temperature swings between day and night. The conditions on Venus are so extreme that even spacecraft sent to the surface survive only a few hours before being destroyed by the temperature and pressure. In effect, Venus offers a dramatic example of how atmospheric composition can significantly influence planetary climates.",
},
{"role": "user", "content": "Are there any missions planned to Venus soon?"},
{
"role": "assistant",
"content": "Yes, the next decade promises to be exciting for Venus exploration, with both NASA and the European Space Agency planning new missions. NASA has announced the VERITAS and DAVINCI+ missions, aiming to study Venus’s geology, atmosphere, and history, with launches expected later this decade. ESA is preparing its own mission called EnVision, which will investigate Venus's surface and atmosphere using advanced radar and spectrometry. These missions hope to resolve key questions, such as whether Venus ever had oceans, what processes led to its intense greenhouse state, and if volcanic activity is still ongoing. Renewed interest in Venus stems from its potential to teach us about planetary evolution, including factors that affect habitability.",
},
{"role": "user", "content": "Cool! Back to Earth, what's the largest living structure here?"},
{
"role": "assistant",
"content": "The largest living structure on Earth is the Great Barrier Reef, stretching more than 2,300 kilometers along Australia’s northeastern coast. This massive coral system is so large it can be seen from space, and it comprises thousands of individual reefs and hundreds of islands built by billions of tiny coral polyps. The reef supports an extraordinary diversity of marine life, hosting over 1,500 fish species, turtles, sea snakes, and countless invertebrates. It also plays a crucial ecological role, protecting coastlines from storms and supporting local economies through tourism and fishing. Unfortunately, despite its size and importance, the Great Barrier Reef faces significant threats from climate change, particularly coral bleaching and ocean acidification.",
},
{"role": "user", "content": "Is it in danger?"},
{
"role": "assistant",
"content": "Sadly, the Great Barrier Reef is indeed at risk due to multiple environmental pressures. One of the most severe challenges is coral bleaching, which occurs when higher sea temperatures stress corals, causing them to expel the symbiotic algae they rely on for food and color. Repeated bleaching events in recent years have resulted in large-scale loss of coral cover, and the recovery process can be slow or incomplete. Additional threats include poor water quality from agricultural runoff, overfishing, cyclones, and outbreaks of crown-of-thorns starfish, which prey on corals. Conservation efforts are underway, but reducing global carbon emissions is crucial to ensuring the reef’s long-term survival.",
},
{"role": "user", "content": "Can coral reefs recover from bleaching?"},
{
"role": "assistant",
"content": "Reef recovery after bleaching is possible, but it depends on the severity of the event and the time between subsequent stress episodes. Corals can regain their symbiotic algae and recover their vibrant colors if water temperatures return to normal quickly enough. However, back-to-back bleaching events—such as those in recent years—inhibit full recovery and can lead to widespread coral death. The resilience of a reef also depends on the health of its local ecosystem, water quality, and the diversity of coral species present. Active restoration efforts, like coral gardening and selective breeding for heat-tolerant corals, are being explored to help boost reef recovery in the face of a warming climate.",
},
{"role": "user", "content": "Are there artificial reefs?"},
{
"role": "assistant",
"content": "Yes, artificial reefs have been employed worldwide as a means to enhance marine habitats and promote biodiversity. These structures can be made from a variety of materials, including concrete blocks, purpose-built modules, and even deliberately sunken ships or decommissioned military vehicles. When placed on the ocean floor, artificial reefs provide surfaces for corals to attach to and develop, which in turn attract fish and other marine species seeking food and shelter. Over time, these reefs can become thriving ecosystems, sometimes rivaling their natural counterparts in productivity. Besides ecological benefits, artificial reefs are also used to create tourist attractions for diving and help reduce pressure on natural reefs by diverting fishing and recreational activity.",
},
{"role": "user", "content": "Do fish immediately move into new artificial reefs?"},
{
"role": "assistant",
"content": "Fish and other marine organisms often begin exploring and colonizing new artificial reefs quite quickly—sometimes within just a few weeks of their deployment. The structures offer instant shelter from predators and new feeding opportunities, which attract small fish and invertebrates first. Over several months, larger predatory species may start appearing as the new reef becomes established as a hunting and breeding ground. The colonization process is further accelerated if the surrounding water already hosts healthy marine life populations. Eventually, the artificial reef can develop complex food webs, supporting a community of organisms similar to that of natural reefs, though full ecosystem development takes several years.",
},
{
"role": "user",
"content": "I love learning these facts. Can you tell me one more surprising animal fact?",
},
{
"role": "assistant",
"content": "Of course! One particularly astonishing animal fact concerns the octopus: these remarkable creatures possess not one, but three hearts. Two of the hearts are responsible for pumping blood through the gills, where it picks up oxygen, while the third pumps oxygenated blood throughout the rest of the body. Octopuses also have blue blood, a consequence of their use of a copper-rich protein called hemocyanin instead of hemoglobin, which is based on iron and gives human blood its red color. The blue blood is more efficient at transporting oxygen in the cold, low-oxygen conditions of the ocean, making it especially advantageous for the octopus’s varied habitats. In addition to their unique circulatory system, octopuses are also known for their high intelligence, problem-solving abilities, and incredible camouflage skills.",
},
{"role": "user", "content": "That’s wild. Why do they have blue blood?"},
{
"role": "assistant",
"content": "The blue color of octopus blood is due to the presence of hemocyanin, a copper-based molecule that binds with oxygen for transport throughout the body. This adaptation is particularly effective in the cold, low-oxygen environments found in many parts of the oceans where octopuses live. Hemocyanin is more efficient than hemoglobin (the iron-based molecule in red blood) under these conditions, which helps octopuses thrive in diverse and sometimes extreme marine settings. The evolution of blue blood is an example of the many unique physiological characteristics that have enabled cephalopods to become such successful marine invertebrates. It’s remarkable how life develops specialized solutions to meet the challenges of different environments.",
},
]
for msg in LONG_CONVERSATION:
conversation.append_message(
Message(content=msg["content"], role=msg["role"]) # type: ignore
)
conversation.append_user_message("What tool do dolphins sometimes use when foraging on the seafloor, and why?")
conversation.execute()
Next Steps#
We have seen in this how-to how to leverage MessageSummarizationTransform and ConversationSummarizationTransform to reduce the size of the LLM context. We have also seen how to use event listeners to measure LLM token consumption. With this new knowledge, you can proceed to How to Build Assistants with Tools.
Full Code#
Click the card at the top of this page to download the complete code for this guide, or copy it below.
1# Copyright © 2025 Oracle and/or its affiliates.
2#
3# This software is under the Apache License 2.0
4# %%[markdown]
5# Code Example - How to use MessageTransform
6# ------------------------------------------
7
8# How to use:
9# Create a new Python virtual environment and install the latest WayFlow version.
10# ```bash
11# python -m venv venv-wayflowcore
12# source venv-wayflowcore/bin/activate
13# pip install --upgrade pip
14# pip install "wayflowcore==26.1"
15# ```
16
17# You can now run the script
18# 1. As a Python file:
19# ```bash
20# python howto_long_context.py
21# ```
22# 2. As a Notebook (in VSCode):
23# When viewing the file,
24# - press the keys Ctrl + Enter to run the selected cell
25# - or Shift + Enter to run the selected cell and move to the cell below# (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0) or Universal Permissive License
26# (UPL) 1.0 (LICENSE-UPL or https://oss.oracle.com/licenses/upl), at your option.
27
28
29import logging
30
31logging.basicConfig(level=logging.INFO)
32
33
34# %%[markdown]
35## Define LLMS
36
37# %%
38from wayflowcore.models import VllmModel
39
40summarization_llm = VllmModel(
41 model_id="SMALL_LLAMA_MODEL_ID",
42 host_port="SMALL_LLAMA_API_URL",
43)
44agent_llm = VllmModel(
45 model_id="LLAMA_MODEL_ID",
46 host_port="LLAMA_API_URL",
47)
48
49
50# %%[markdown]
51## Create MessageSummarizationTransform
52
53# %%
54from wayflowcore.transforms import MessageSummarizationTransform
55
56
57transform = MessageSummarizationTransform(
58 llm=summarization_llm,
59 max_message_size=10000,
60 summarized_message_template="Summarized result:\n{{summary}}"
61)
62
63
64
65# %%[markdown]
66## Create Agent
67
68# %%
69from wayflowcore import Agent, Message, tool
70
71@tool
72def read_logs_tool() -> str:
73 """Return logs from the system"""
74 return (
75 "Starting long processing\n"
76 + "Waiting for process ...\n" * 2_000
77 + "Found error: Missing credentials for user kurt_andrews."
78 + " Please pass the correct credentials.\n"
79 )
80
81agent = Agent(
82 llm=agent_llm,
83 tools=[read_logs_tool],
84 transforms=[transform],
85)
86
87
88# %%[markdown]
89## Create Token Event Listener
90
91# %%
92from wayflowcore.events.eventlistener import EventListener
93from wayflowcore.events.event import LlmGenerationResponseEvent
94from wayflowcore.events import register_event_listeners
95from wayflowcore.tokenusage import TokenUsage
96
97class TokenListener(EventListener):
98 def __init__(self):
99 self.usage = TokenUsage()
100 def __call__(self, event):
101 if isinstance(event, LlmGenerationResponseEvent):
102 self.usage += event.completion.token_usage
103
104
105# %%[markdown]
106## Run Agent With MessageSummarizationTransform
107
108# %%
109listener_with_transform = TokenListener()
110with register_event_listeners([listener_with_transform]):
111 from wayflowcore.executors.executionstatus import UserMessageRequestStatus
112
113 conversation = agent.start_conversation()
114 conversation.append_user_message("Can you explain the error in the system?")
115 status = conversation.execute()
116 if isinstance(status, UserMessageRequestStatus):
117 print(status.message.content)
118 # The error is likely due to a missing credential for
119 # a user named kurt_andrews. Please ensure the credentials
120 # are provided correctly for kurt_andrews to resolve the issue.
121 print(f"Token usage with MessageSummarizationTransform after 1 user message: {listener_with_transform.usage.total_tokens} tokens")
122 # Token usage with MessageSummarizationTransform after 1 user message: 8854 tokens
123
124 conversation.append_user_message('What is the exact message repeated in a loop? Do not recall the tool')
125 status = conversation.execute()
126 if isinstance(status, UserMessageRequestStatus):
127 print(status.message.content)
128 # The tool is stuck in an infinite loop, repeatedly displaying the message "Waiting for process" until it encounters an error due to missing credentials for user "kurt_andrews".
129 print(f"Token usage with MessageSummarizationTransform after 2 user messages: {listener_with_transform.usage.total_tokens} tokens")# .. start-##_Create_Token_Event_Listener
130 # Token usage with MessageSummarizationTransform after 2 user messages: 9314 tokens
131
132
133
134
135
136# %%[markdown]
137## Create Agent Without Transform
138
139# %%
140agent_no_transform = Agent(
141 llm=agent_llm,
142 tools=[read_logs_tool],
143 transforms=[], # no transforms
144)
145
146listener_no_transform = TokenListener()
147with register_event_listeners([listener_no_transform]):
148 conversation = agent_no_transform.start_conversation()
149 conversation.append_user_message("Can you explain the error in the system?")
150 status = conversation.execute()
151 if isinstance(status, UserMessageRequestStatus):
152 print(status.message.content)
153
154 # ... (same conversation as above)
155
156 print(f"Token usage without MessageSummarizationTransform after 1 user message: {listener_no_transform.usage.total_tokens} tokens")
157 # Token usage without MessageSummarizationTransform after 1 user message: 12757 tokens
158
159
160 conversation.append_user_message('What is the exact message repeated in a loop? Do not recall the tool')
161 status = conversation.execute()
162 if isinstance(status, UserMessageRequestStatus):
163 print(status.message.content)
164
165 print(f"Token usage without MessageSummarizationTransform after 2 user messages: {listener_no_transform.usage.total_tokens} tokens")# .. start-##_Create_Token_Event_Listener
166 # Token usage without MessageSummarizationTransform after 2 user messages: 25224 tokens
167
168
169
170
171
172# %%[markdown]
173## Create ConversationSummarizationTransform
174
175# %%
176from wayflowcore.transforms import ConversationSummarizationTransform
177
178transform = ConversationSummarizationTransform(
179 llm=summarization_llm,
180 max_num_messages=5,
181 min_num_messages=2
182)
183
184
185# %%[markdown]
186## Run Agent With ConversationSummarizationTransform
187
188# %%
189
190agent = Agent(
191 llm=agent_llm,
192 tools=[],
193 transforms=[transform],
194)
195
196conversation = agent.start_conversation()
197conversation.append_user_message("What is the capital of Switzerland?")
198conversation.execute()
199conversation.append_user_message("Tell me a fun fact about it.")
200conversation.execute()
201conversation.append_user_message("What is the population?")
202conversation.execute()
203conversation.append_user_message("What is the area?")
204conversation.execute()
205conversation.append_user_message("What languages are spoken there?")
206conversation.execute()
207# The conversation will be summarized when it exceeds 5 messages
208
209
210
211# %%[markdown]
212## Create Both Transforms
213
214# %%
215transforms = [
216 MessageSummarizationTransform(
217 llm=summarization_llm,
218 max_message_size=10000,
219 summarized_message_template="Summarized content:\n{{summary}}"
220 ),
221 ConversationSummarizationTransform(
222 llm=summarization_llm,
223 max_num_messages=5,
224 min_num_messages=2
225 )
226]
227
228
229# %%[markdown]
230## Run Agent With Both Transforms
231
232# %%
233agent = Agent(
234 llm=summarization_llm,
235 tools=[read_logs_tool],
236 transforms=transforms,
237)
238
239conversation = agent.start_conversation()
240
241LONG_CONVERSATION = [
242 {"role": "user", "content": "Hi! Can you tell me something interesting about dolphins?"},
243 {
244 "role": "assistant",
245 "content": "Absolutely! Dolphins are fascinating creatures, famous for their intelligence and complex behavior. For example, they have been observed using tools, such as covering their snouts with sponges to protect themselves while foraging on the seafloor. Dolphins also display strong social bonds and have been known to help injured individuals within their pods. Communication among dolphins is advanced; they use a series of clicks, whistles, and body movements to convey information, and some species even have distinctive signature whistles that function like names.",
246 },
247 {"role": "user", "content": "Wow, I didn’t know that. Do dolphins have good memory?"},
248 {
249 "role": "assistant",
250 "content": "Dolphins possess remarkable memories, particularly when it comes to their social groups and vocal communication. Researchers have discovered that dolphins can remember the unique signature whistles of other dolphins for over 20 years, which is the longest social memory recorded in non-human animals. This ability highlights their sophisticated cognitive abilities and the importance of long-term relationships in dolphin societies. Memory also plays a crucial role in their navigation and hunting skills, as dolphins migrate and follow paths in the oceans over great distances. In addition, their keen memory supports learning from one another, enhancing the social structure of their pods.",
251 },
252 {
253 "role": "user",
254 "content": "That’s impressive. Are there other animals with similar intelligence?",
255 },
256 {
257 "role": "assistant",
258 "content": "Indeed, there are several other animals that exhibit forms of intelligence comparable to dolphins. Great apes, such as chimpanzees and orangutans, have been observed making and using tools, solving puzzles, and displaying empathy and self-awareness. Elephants show emotional intelligence and have complex social relationships, including mourning their dead and helping injured companions. Crows and parrots, members of the bird family, demonstrate advanced problem-solving, mimicry, and even the ability to plan for future events. These examples across different branches of the animal kingdom show that intelligence has evolved in diverse and fascinating ways.",
259 },
260 {"role": "user", "content": "Can elephants recognize themselves in a mirror like dolphins?"},
261 {
262 "role": "assistant",
263 "content": "Yes, elephants are among the elite group of species known to pass the mirror self-recognition test, an experiment generally considered evidence of self-awareness. In these experiments, elephants explore themselves in mirrors and investigate marks on their bodies they otherwise couldn’t see. Notably, this test has only been passed by a few species, including certain great apes, dolphins, magpies, and elephants. Such ability hints at a deep cognitive capacity, suggesting they have a mental representation of themselves, which is foundational for empathy and understanding the experiences of others. This self-recognition is just one of the many amazing aspects of elephant intelligence that continue to be studied.",
264 },
265 {"role": "user", "content": "What is the mirror test?"},
266 {
267 "role": "assistant",
268 "content": "The mirror test, also known as the mark test, is a behavioral technique designed to assess self-awareness in animals. In the test, an animal is marked with an odorless dye in a place it cannot see without a mirror. The animal is then given access to a mirror, and researchers observe whether it investigates or tries to touch the mark on its own body, indicating recognition of its reflection as itself, not another animal. Passing the mirror test suggests a level of self-concept, which is considered an advanced cognitive trait. However, not all intelligent animals pass the mirror test, and failing it doesn’t necessarily mean an animal lacks self-awareness – it might simply not care about the mark or not rely on visual cues.",
269 },
270 {
271 "role": "user",
272 "content": "Besides animals, what’s another field where recognition is important?",
273 },
274 {
275 "role": "assistant",
276 "content": "Recognition is vitally important in many human technologies, notably in the field of artificial intelligence. Facial recognition technology is widely used for security purposes, such as unlocking smartphones or monitoring public spaces. In social media, these algorithms help automatically tag friends in photos, organizing vast amounts of visual data. Additionally, recognition systems are applied in retail, for personalized marketing, and in health care, for patient identification and monitoring. However, these powerful tools also raise important questions around privacy and data security, sparking ongoing debates about their regulation and ethical use.",
277 },
278 {"role": "user", "content": "Is facial recognition always accurate?"},
279 {
280 "role": "assistant",
281 "content": "Facial recognition technology has improved significantly over recent years, but it is not flawless and its accuracy can vary depending on several factors. Lighting, angle, image quality, and the diversity of data used to train the algorithms all influence performance. Notably, many studies have shown that facial recognition systems can have higher error rates when identifying people with darker skin tones or from underrepresented demographics, leading to concerns about bias and discrimination. This has prompted researchers and policymakers to push for better training data, algorithmic transparency, and stricter oversight. As the technology becomes more widespread, ensuring fairness and accuracy is an essential focus for developers and regulators alike.",
282 },
283 {"role": "user", "content": "What are some other ethical concerns in AI?"},
284 {
285 "role": "assistant",
286 "content": "Ethical challenges in AI are numerous and complex, reflecting the far-reaching impact of these technologies on society. Privacy concerns are paramount, as AI systems often require large amounts of personal data, raising questions about consent and data protection. Bias in algorithms is another major issue, as AI can inadvertently reinforce existing societal prejudices if not carefully monitored and tested. Transparency is crucial, because many AI systems, especially those using deep learning, can behave as 'black boxes'—making decisions in ways that aren’t easily understandable to humans. Additionally, there are fears around job displacement, as automation could replace roles faster than new opportunities are created, necessitating careful consideration and retraining initiatives.",
287 },
288 {
289 "role": "user",
290 "content": "Job displacement is interesting. Which jobs are most at risk from AI?",
291 },
292 {
293 "role": "assistant",
294 "content": "Jobs that involve repetitive, predictable tasks are currently considered the most vulnerable to automation by AI. For example, data entry clerks, assembly line workers, and certain types of customer service representatives face a high likelihood of being replaced or augmented by machines. Even within fields like transportation, we’re seeing the rise of autonomous vehicles that could impact truck drivers and delivery personnel. On the other hand, roles that require creativity, complex decision-making, emotional intelligence, or physical dexterity in unpredictable environments—such as healthcare providers, artists, and skilled trades—are less susceptible for now. To manage this transition, many experts recommend investing in retraining programs and encouraging adaptability in the workforce.",
295 },
296 {"role": "user", "content": "On a lighter note, do you know any fun space facts?"},
297 {
298 "role": "assistant",
299 "content": "Absolutely, space offers an endless trove of fascinating discoveries. For example, Venus has a very unusual rotation: a single day on Venus (one rotation on its axis) is actually longer than its year (one complete orbit around the Sun) due to its slow spin. There are also diamond rains on planets like Neptune and Uranus, caused by extreme pressure turning carbon into solid diamonds. You might be surprised to learn that there are more stars in the observable universe than grains of sand on all the Earth’s beaches combined. Finally, if you were to fall into a black hole, the laws of physics as we understand them break down at the singularity—a phenomenon still not fully explained by scientists.",
300 },
301 {"role": "user", "content": "That’s fascinating! How hot does it get on Venus?"},
302 {
303 "role": "assistant",
304 "content": "Venus is the hottest planet in our solar system, even outpacing Mercury, which is closer to the Sun. The average surface temperature on Venus can reach an astonishing 465 degrees Celsius, or about 869 degrees Fahrenheit. This is hot enough to melt lead, and it creates an extremely inhospitable environment for life as we know it. The intense heat is largely due to the planet’s dense atmosphere, composed mostly of carbon dioxide, which traps heat through a runaway greenhouse effect. In addition to being scorching, Venus’s atmosphere features thick clouds of sulfuric acid that make its surface perpetually shrouded and acidic.",
305 },
306 {"role": "user", "content": "Why is Venus hotter than Mercury?"},
307 {
308 "role": "assistant",
309 "content": "Despite being the second planet from the Sun, Venus is hotter than Mercury primarily because of its thick, carbon dioxide-rich atmosphere. This atmosphere acts as a blanket, trapping incoming solar radiation and preventing heat from escaping, leading to the planet's infamous runaway greenhouse effect. Meanwhile, Mercury has only a thin exosphere, which allows heat to escape into space quickly, resulting in dramatic temperature swings between day and night. The conditions on Venus are so extreme that even spacecraft sent to the surface survive only a few hours before being destroyed by the temperature and pressure. In effect, Venus offers a dramatic example of how atmospheric composition can significantly influence planetary climates.",
310 },
311 {"role": "user", "content": "Are there any missions planned to Venus soon?"},
312 {
313 "role": "assistant",
314 "content": "Yes, the next decade promises to be exciting for Venus exploration, with both NASA and the European Space Agency planning new missions. NASA has announced the VERITAS and DAVINCI+ missions, aiming to study Venus’s geology, atmosphere, and history, with launches expected later this decade. ESA is preparing its own mission called EnVision, which will investigate Venus's surface and atmosphere using advanced radar and spectrometry. These missions hope to resolve key questions, such as whether Venus ever had oceans, what processes led to its intense greenhouse state, and if volcanic activity is still ongoing. Renewed interest in Venus stems from its potential to teach us about planetary evolution, including factors that affect habitability.",
315 },
316 {"role": "user", "content": "Cool! Back to Earth, what's the largest living structure here?"},
317 {
318 "role": "assistant",
319 "content": "The largest living structure on Earth is the Great Barrier Reef, stretching more than 2,300 kilometers along Australia’s northeastern coast. This massive coral system is so large it can be seen from space, and it comprises thousands of individual reefs and hundreds of islands built by billions of tiny coral polyps. The reef supports an extraordinary diversity of marine life, hosting over 1,500 fish species, turtles, sea snakes, and countless invertebrates. It also plays a crucial ecological role, protecting coastlines from storms and supporting local economies through tourism and fishing. Unfortunately, despite its size and importance, the Great Barrier Reef faces significant threats from climate change, particularly coral bleaching and ocean acidification.",
320 },
321 {"role": "user", "content": "Is it in danger?"},
322 {
323 "role": "assistant",
324 "content": "Sadly, the Great Barrier Reef is indeed at risk due to multiple environmental pressures. One of the most severe challenges is coral bleaching, which occurs when higher sea temperatures stress corals, causing them to expel the symbiotic algae they rely on for food and color. Repeated bleaching events in recent years have resulted in large-scale loss of coral cover, and the recovery process can be slow or incomplete. Additional threats include poor water quality from agricultural runoff, overfishing, cyclones, and outbreaks of crown-of-thorns starfish, which prey on corals. Conservation efforts are underway, but reducing global carbon emissions is crucial to ensuring the reef’s long-term survival.",
325 },
326 {"role": "user", "content": "Can coral reefs recover from bleaching?"},
327 {
328 "role": "assistant",
329 "content": "Reef recovery after bleaching is possible, but it depends on the severity of the event and the time between subsequent stress episodes. Corals can regain their symbiotic algae and recover their vibrant colors if water temperatures return to normal quickly enough. However, back-to-back bleaching events—such as those in recent years—inhibit full recovery and can lead to widespread coral death. The resilience of a reef also depends on the health of its local ecosystem, water quality, and the diversity of coral species present. Active restoration efforts, like coral gardening and selective breeding for heat-tolerant corals, are being explored to help boost reef recovery in the face of a warming climate.",
330 },
331 {"role": "user", "content": "Are there artificial reefs?"},
332 {
333 "role": "assistant",
334 "content": "Yes, artificial reefs have been employed worldwide as a means to enhance marine habitats and promote biodiversity. These structures can be made from a variety of materials, including concrete blocks, purpose-built modules, and even deliberately sunken ships or decommissioned military vehicles. When placed on the ocean floor, artificial reefs provide surfaces for corals to attach to and develop, which in turn attract fish and other marine species seeking food and shelter. Over time, these reefs can become thriving ecosystems, sometimes rivaling their natural counterparts in productivity. Besides ecological benefits, artificial reefs are also used to create tourist attractions for diving and help reduce pressure on natural reefs by diverting fishing and recreational activity.",
335 },
336 {"role": "user", "content": "Do fish immediately move into new artificial reefs?"},
337 {
338 "role": "assistant",
339 "content": "Fish and other marine organisms often begin exploring and colonizing new artificial reefs quite quickly—sometimes within just a few weeks of their deployment. The structures offer instant shelter from predators and new feeding opportunities, which attract small fish and invertebrates first. Over several months, larger predatory species may start appearing as the new reef becomes established as a hunting and breeding ground. The colonization process is further accelerated if the surrounding water already hosts healthy marine life populations. Eventually, the artificial reef can develop complex food webs, supporting a community of organisms similar to that of natural reefs, though full ecosystem development takes several years.",
340 },
341 {
342 "role": "user",
343 "content": "I love learning these facts. Can you tell me one more surprising animal fact?",
344 },
345 {
346 "role": "assistant",
347 "content": "Of course! One particularly astonishing animal fact concerns the octopus: these remarkable creatures possess not one, but three hearts. Two of the hearts are responsible for pumping blood through the gills, where it picks up oxygen, while the third pumps oxygenated blood throughout the rest of the body. Octopuses also have blue blood, a consequence of their use of a copper-rich protein called hemocyanin instead of hemoglobin, which is based on iron and gives human blood its red color. The blue blood is more efficient at transporting oxygen in the cold, low-oxygen conditions of the ocean, making it especially advantageous for the octopus’s varied habitats. In addition to their unique circulatory system, octopuses are also known for their high intelligence, problem-solving abilities, and incredible camouflage skills.",
348 },
349 {"role": "user", "content": "That’s wild. Why do they have blue blood?"},
350 {
351 "role": "assistant",
352 "content": "The blue color of octopus blood is due to the presence of hemocyanin, a copper-based molecule that binds with oxygen for transport throughout the body. This adaptation is particularly effective in the cold, low-oxygen environments found in many parts of the oceans where octopuses live. Hemocyanin is more efficient than hemoglobin (the iron-based molecule in red blood) under these conditions, which helps octopuses thrive in diverse and sometimes extreme marine settings. The evolution of blue blood is an example of the many unique physiological characteristics that have enabled cephalopods to become such successful marine invertebrates. It’s remarkable how life develops specialized solutions to meet the challenges of different environments.",
353 },
354]
355
356for msg in LONG_CONVERSATION:
357 conversation.append_message(
358 Message(content=msg["content"], role=msg["role"]) # type: ignore
359 )
360
361conversation.append_user_message("What tool do dolphins sometimes use when foraging on the seafloor, and why?")
362conversation.execute()