How to Do Structured LLM Generation in Flows#
Prerequisites
This guide assumes familiarity with Flows.
WayFlow enables to leverage LLMs to generate text and structured outputs. This guide will show you how to:
use the PromptExecutionStep to generate text using an LLM
use the PromptExecutionStep to generate structured outputs
use the AgentExecutionStep to generate structured outputs using an agent
Basic implementation#
In this how-to guide, you will learn how to do a structured LLM generation with Flows.
WayFlow supports several LLM API providers. Select an LLM from the options below:
from wayflowcore.models import OCIGenAIModel
if __name__ == "__main__":
llm = OCIGenAIModel(
model_id="provider.model-id",
service_endpoint="https://url-to-service-endpoint.com",
compartment_id="compartment-id",
auth_type="API_KEY",
)
from wayflowcore.models import VllmModel
llm = VllmModel(
model_id="model-id",
host_port="VLLM_HOST_PORT",
)
from wayflowcore.models import OllamaModel
llm = OllamaModel(
model_id="model-id",
)
Assuming you want to summarize this article:
article = """Sea turtles are ancient reptiles that have been around for over 100 million years. They play crucial roles in marine ecosystems, such as maintaining healthy seagrass beds and coral reefs. Unfortunately, they are under threat due to poaching, habitat loss, and pollution. Conservation efforts worldwide aim to protect nesting sites and reduce bycatch in fishing gear."""
WayFlow offers the PromptExecutionStep for this type of queries. Use the code below to generate a 10-words summary:
from wayflowcore.controlconnection import ControlFlowEdge
from wayflowcore.dataconnection import DataFlowEdge
from wayflowcore.flow import Flow
from wayflowcore.property import StringProperty
from wayflowcore.steps import PromptExecutionStep, StartStep
start_step = StartStep(input_descriptors=[StringProperty("article")])
summarize_step = PromptExecutionStep(
llm=llm,
prompt_template="""Summarize this article in 10 words:\n {{article}}""",
output_mapping={PromptExecutionStep.OUTPUT: "summary"},
)
summarize_step_name = "summarize_step"
flow = Flow(
begin_step=start_step,
steps={
"start_step": start_step,
summarize_step_name: summarize_step,
},
control_flow_edges=[
ControlFlowEdge(source_step=start_step, destination_step=summarize_step),
ControlFlowEdge(source_step=summarize_step, destination_step=None),
],
data_flow_edges=[
DataFlowEdge(start_step, "article", summarize_step, "article"),
],
)
Note
In the prompt, article is a Jinja2 syntax to specify a placeholder for a variable, which will appear as an input for the step.
If you use {{var_name}}, the variable named var_name will be of type StringProperty.
If you specify anything else Jinja2 compatible (for loops, filters, and so on), it will be of type AnyProperty.
Now execute the flow:
conversation = flow.start_conversation(inputs={"article": article})
status = conversation.execute()
print(status.output_values["summary"])
# Sea turtles face threats from poaching, habitat loss, and pollution globally.
As expected, your flow has generated the article summary!
Structured generation with Flows#
In many cases, generating raw text within a flow is not very useful, as it is difficult to leverage in later steps.
Instead, you might want to generate attributes that follow a particular schema.
The PromptExecutionStep class enables this through the output_descriptors parameter.
from wayflowcore.property import ListProperty, StringProperty
from wayflowcore.steps import PromptExecutionStep, StartStep
animal_output = StringProperty(
name="animal_name",
description="name of the animal",
default_value="",
)
danger_level_output = StringProperty(
name="danger_level",
description='level of danger of the animal. Can be "HIGH", "MEDIUM" or "LOW"',
default_value="",
)
threats_output = ListProperty(
name="threats",
description="list of threats for the animal",
item_type=StringProperty("threat"),
default_value=[],
)
start_step = StartStep(input_descriptors=[StringProperty("article")])
summarize_step = PromptExecutionStep(
llm=llm,
prompt_template="""Extract from the following article the name of the animal, its danger level and the threats it's subject to. The article:\n\n {{article}}""",
output_descriptors=[animal_output, danger_level_output, threats_output],
)
summarize_step_name = "summarize_step"
flow = Flow(
begin_step=start_step,
steps={
"start_step": start_step,
summarize_step_name: summarize_step,
},
control_flow_edges=[
ControlFlowEdge(source_step=start_step, destination_step=summarize_step),
ControlFlowEdge(source_step=summarize_step, destination_step=None),
],
data_flow_edges=[
DataFlowEdge(start_step, "article", summarize_step, "article"),
],
)
conversation = flow.start_conversation(inputs={"article": article})
status = conversation.execute()
print(status.output_values)
# {'threats': ['poaching', 'habitat loss', 'pollution'], 'danger_level': 'HIGH', 'animal_name': 'Sea turtles'}
Complex JSON objects#
Sometimes, you might need to generate an object that follows a specific JSON Schema.
You can do that by using an output descriptor of type ObjectProperty, or directly converting your JSON Schema into a descriptor:
from wayflowcore.property import Property, StringProperty
from wayflowcore.steps import PromptExecutionStep, StartStep
animal_json_schema = {
"title": "animal_object",
"description": "information about the animal",
"type": "object",
"properties": {
"animal_name": {
"type": "string",
"description": "name of the animal",
"default": "",
},
"danger_level": {
"type": "string",
"description": 'level of danger of the animal. Can be "HIGH", "MEDIUM" or "LOW"',
"default": "",
},
"threats": {
"type": "array",
"description": "list of threats for the animal",
"items": {"type": "string"},
"default": [],
},
},
}
animal_descriptor = Property.from_json_schema(animal_json_schema)
start_step = StartStep(input_descriptors=[StringProperty("article")])
summarize_step = PromptExecutionStep(
llm=llm,
prompt_template="""Extract from the following article the name of the animal, its danger level and the threats it's subject to. The article:\n\n {{article}}""",
output_descriptors=[animal_descriptor],
)
summarize_step_name = "summarize_step"
flow = Flow(
begin_step=start_step,
steps={
"start_step": start_step,
summarize_step_name: summarize_step,
},
control_flow_edges=[
ControlFlowEdge(source_step=start_step, destination_step=summarize_step),
ControlFlowEdge(source_step=summarize_step, destination_step=None),
],
data_flow_edges=[
DataFlowEdge(start_step, "article", summarize_step, "article"),
],
)
conversation = flow.start_conversation(inputs={"article": article})
status = conversation.execute()
print(status.output_values)
# {'animal_object': {'animal_name': 'Sea turtles', 'danger_level': 'MEDIUM', 'threats': ['Poaching', 'Habitat loss', 'Pollution']}}
Structured generation with Agents#
In certain scenarios, you might need an agent to generate well-formatted outputs within your flow.
You can instruct the agent to generate them, and use it in the AgentExecutionStep class to perform structured generation.
from wayflowcore.agent import Agent, CallerInputMode
from wayflowcore.controlconnection import ControlFlowEdge
from wayflowcore.steps import AgentExecutionStep, StartStep
start_step = StartStep(input_descriptors=[])
agent = Agent(
llm=llm,
custom_instruction="""Extract from the article given by the user the name of the animal, its danger level and the threats it's subject to.""",
initial_message=None,
caller_input_mode=CallerInputMode.ALWAYS,
output_descriptors=[animal_output, danger_level_output, threats_output],
)
summarize_agent_step = AgentExecutionStep(agent=agent)
summarize_step_name = "summarize_step"
flow = Flow(
begin_step=start_step,
steps={
"start_step": start_step,
summarize_step_name: summarize_agent_step,
},
control_flow_edges=[
ControlFlowEdge(source_step=start_step, destination_step=summarize_agent_step),
ControlFlowEdge(source_step=summarize_agent_step, destination_step=None),
],
data_flow_edges=[],
)
conversation = flow.start_conversation()
conversation.append_user_message("Here is the article: " + article)
status = conversation.execute()
print(status.output_values)
# {'animal_name': 'Sea turtles', 'danger_level': 'HIGH', 'threats': ['poaching', 'habitat loss', 'pollution']}
Agent Spec Exporting/Loading#
You can export the assistant configuration to its Agent Spec configuration using the AgentSpecExporter.
from wayflowcore.agentspec import AgentSpecExporter
serialized_assistant = AgentSpecExporter().to_json(flow)
Here is what the Agent Spec representation will look like ↓
Click here to see the assistant configuration.
{
"component_type": "Flow",
"id": "e075072d-6aa0-4da4-84bd-dd54ce060ff4",
"name": "flow_021b3829__auto",
"description": "",
"metadata": {
"__metadata_info__": {}
},
"inputs": [],
"outputs": [
{
"description": "name of the animal",
"type": "string",
"title": "animal_name",
"default": ""
},
{
"description": "list of threats for the animal",
"type": "array",
"items": {
"type": "string",
"title": "threat"
},
"title": "threats",
"default": []
},
{
"description": "level of danger of the animal. Can be \"HIGH\", \"MEDIUM\" or \"LOW\"",
"type": "string",
"title": "danger_level",
"default": ""
}
],
"start_node": {
"$component_ref": "5c227b46-f36a-413c-8090-3be75fafbdec"
},
"nodes": [
{
"$component_ref": "5c227b46-f36a-413c-8090-3be75fafbdec"
},
{
"$component_ref": "4c691ba3-2d4f-4d30-8714-11fe71a731a0"
},
{
"$component_ref": "6bb4d577-8da4-4e05-83da-e16d10dbfa1c"
}
],
"control_flow_connections": [
{
"component_type": "ControlFlowEdge",
"id": "de3e74f8-9b09-402f-a1ec-b8d5a35d9a3d",
"name": "start_step_to_summarize_step_control_flow_edge",
"description": null,
"metadata": {
"__metadata_info__": {}
},
"from_node": {
"$component_ref": "5c227b46-f36a-413c-8090-3be75fafbdec"
},
"from_branch": null,
"to_node": {
"$component_ref": "4c691ba3-2d4f-4d30-8714-11fe71a731a0"
}
},
{
"component_type": "ControlFlowEdge",
"id": "47583cac-4599-4e3a-8cab-7edb2cba870f",
"name": "summarize_step_to_None End node_control_flow_edge",
"description": null,
"metadata": {},
"from_node": {
"$component_ref": "4c691ba3-2d4f-4d30-8714-11fe71a731a0"
},
"from_branch": null,
"to_node": {
"$component_ref": "6bb4d577-8da4-4e05-83da-e16d10dbfa1c"
}
}
],
"data_flow_connections": [
{
"component_type": "DataFlowEdge",
"id": "3620497d-5f8e-4280-a496-439cc2f32936",
"name": "summarize_step_animal_name_to_None End node_animal_name_data_flow_edge",
"description": null,
"metadata": {},
"source_node": {
"$component_ref": "4c691ba3-2d4f-4d30-8714-11fe71a731a0"
},
"source_output": "animal_name",
"destination_node": {
"$component_ref": "6bb4d577-8da4-4e05-83da-e16d10dbfa1c"
},
"destination_input": "animal_name"
},
{
"component_type": "DataFlowEdge",
"id": "137eb74d-0a75-48c5-8084-99b679567f44",
"name": "summarize_step_threats_to_None End node_threats_data_flow_edge",
"description": null,
"metadata": {},
"source_node": {
"$component_ref": "4c691ba3-2d4f-4d30-8714-11fe71a731a0"
},
"source_output": "threats",
"destination_node": {
"$component_ref": "6bb4d577-8da4-4e05-83da-e16d10dbfa1c"
},
"destination_input": "threats"
},
{
"component_type": "DataFlowEdge",
"id": "90669e70-722c-459c-9f64-cc8ba6fac091",
"name": "summarize_step_danger_level_to_None End node_danger_level_data_flow_edge",
"description": null,
"metadata": {},
"source_node": {
"$component_ref": "4c691ba3-2d4f-4d30-8714-11fe71a731a0"
},
"source_output": "danger_level",
"destination_node": {
"$component_ref": "6bb4d577-8da4-4e05-83da-e16d10dbfa1c"
},
"destination_input": "danger_level"
}
],
"$referenced_components": {
"4c691ba3-2d4f-4d30-8714-11fe71a731a0": {
"component_type": "AgentNode",
"id": "4c691ba3-2d4f-4d30-8714-11fe71a731a0",
"name": "summarize_step",
"description": "",
"metadata": {
"__metadata_info__": {}
},
"inputs": [],
"outputs": [
{
"description": "name of the animal",
"type": "string",
"title": "animal_name",
"default": ""
},
{
"description": "level of danger of the animal. Can be \"HIGH\", \"MEDIUM\" or \"LOW\"",
"type": "string",
"title": "danger_level",
"default": ""
},
{
"description": "list of threats for the animal",
"type": "array",
"items": {
"type": "string",
"title": "threat"
},
"title": "threats",
"default": []
}
],
"branches": [
"next"
],
"agent": {
"component_type": "Agent",
"id": "eb2f9b51-e5b2-44e2-91d5-69711665b550",
"name": "agent_72c8e146__auto",
"description": "",
"metadata": {
"__metadata_info__": {}
},
"inputs": [],
"outputs": [
{
"description": "name of the animal",
"type": "string",
"title": "animal_name",
"default": ""
},
{
"description": "level of danger of the animal. Can be \"HIGH\", \"MEDIUM\" or \"LOW\"",
"type": "string",
"title": "danger_level",
"default": ""
},
{
"description": "list of threats for the animal",
"type": "array",
"items": {
"type": "string",
"title": "threat"
},
"title": "threats",
"default": []
}
],
"llm_config": {
"component_type": "VllmConfig",
"id": "ce3c577c-9e03-44f8-bec2-258bda52789e",
"name": "llm_8052f2ad__auto",
"description": null,
"metadata": {
"__metadata_info__": {}
},
"default_generation_parameters": null,
"url": "LLAMA_API_URL",
"model_id": "LLAMA_MODEL_ID"
},
"system_prompt": "Extract from the article given by the user the name of the animal, its danger level and the threats it's subject to.",
"tools": []
}
},
"6bb4d577-8da4-4e05-83da-e16d10dbfa1c": {
"component_type": "EndNode",
"id": "6bb4d577-8da4-4e05-83da-e16d10dbfa1c",
"name": "None End node",
"description": "End node representing all transitions to None in the WayFlow flow",
"metadata": {},
"inputs": [
{
"description": "name of the animal",
"type": "string",
"title": "animal_name",
"default": ""
},
{
"description": "list of threats for the animal",
"type": "array",
"items": {
"type": "string",
"title": "threat"
},
"title": "threats",
"default": []
},
{
"description": "level of danger of the animal. Can be \"HIGH\", \"MEDIUM\" or \"LOW\"",
"type": "string",
"title": "danger_level",
"default": ""
}
],
"outputs": [
{
"description": "name of the animal",
"type": "string",
"title": "animal_name",
"default": ""
},
{
"description": "list of threats for the animal",
"type": "array",
"items": {
"type": "string",
"title": "threat"
},
"title": "threats",
"default": []
},
{
"description": "level of danger of the animal. Can be \"HIGH\", \"MEDIUM\" or \"LOW\"",
"type": "string",
"title": "danger_level",
"default": ""
}
],
"branches": [],
"branch_name": "next"
},
"5c227b46-f36a-413c-8090-3be75fafbdec": {
"component_type": "StartNode",
"id": "5c227b46-f36a-413c-8090-3be75fafbdec",
"name": "start_step",
"description": "",
"metadata": {
"__metadata_info__": {}
},
"inputs": [],
"outputs": [],
"branches": [
"next"
]
}
},
"agentspec_version": "25.4.1"
}
component_type: Flow
id: a7d68700-f2f6-43cb-b8e2-db5e0f7eb984
name: flow_0c07fc25__auto
description: ''
metadata:
__metadata_info__: {}
inputs: []
outputs:
- description: list of threats for the animal
type: array
items:
type: string
title: threat
title: threats
default: []
- description: level of danger of the animal. Can be "HIGH", "MEDIUM" or "LOW"
type: string
title: danger_level
default: ''
- description: name of the animal
type: string
title: animal_name
default: ''
start_node:
$component_ref: a65b2a43-c65a-4cca-947f-f76e1410ff66
nodes:
- $component_ref: a65b2a43-c65a-4cca-947f-f76e1410ff66
- $component_ref: fe378524-69cb-4fcd-9d3a-f8fcb1c28316
- $component_ref: cb466062-c8c3-4d8c-ae59-2ec21e1df9ea
control_flow_connections:
- component_type: ControlFlowEdge
id: 2c43b11c-9e07-469d-b38d-c4c2a323cc89
name: start_step_to_summarize_step_control_flow_edge
description: null
metadata:
__metadata_info__: {}
from_node:
$component_ref: a65b2a43-c65a-4cca-947f-f76e1410ff66
from_branch: null
to_node:
$component_ref: fe378524-69cb-4fcd-9d3a-f8fcb1c28316
- component_type: ControlFlowEdge
id: df47bf8f-46d3-496f-9e5e-a57c8abb9658
name: summarize_step_to_None End node_control_flow_edge
description: null
metadata: {}
from_node:
$component_ref: fe378524-69cb-4fcd-9d3a-f8fcb1c28316
from_branch: null
to_node:
$component_ref: cb466062-c8c3-4d8c-ae59-2ec21e1df9ea
data_flow_connections:
- component_type: DataFlowEdge
id: a14b42bc-c53a-4f58-a385-af6bca465872
name: summarize_step_threats_to_None End node_threats_data_flow_edge
description: null
metadata: {}
source_node:
$component_ref: fe378524-69cb-4fcd-9d3a-f8fcb1c28316
source_output: threats
destination_node:
$component_ref: cb466062-c8c3-4d8c-ae59-2ec21e1df9ea
destination_input: threats
- component_type: DataFlowEdge
id: 78113887-9b21-4924-a35a-c6042b5bbe2b
name: summarize_step_danger_level_to_None End node_danger_level_data_flow_edge
description: null
metadata: {}
source_node:
$component_ref: fe378524-69cb-4fcd-9d3a-f8fcb1c28316
source_output: danger_level
destination_node:
$component_ref: cb466062-c8c3-4d8c-ae59-2ec21e1df9ea
destination_input: danger_level
- component_type: DataFlowEdge
id: 5225f4ae-dfec-4edc-92f4-52e57a53000f
name: summarize_step_animal_name_to_None End node_animal_name_data_flow_edge
description: null
metadata: {}
source_node:
$component_ref: fe378524-69cb-4fcd-9d3a-f8fcb1c28316
source_output: animal_name
destination_node:
$component_ref: cb466062-c8c3-4d8c-ae59-2ec21e1df9ea
destination_input: animal_name
$referenced_components:
a65b2a43-c65a-4cca-947f-f76e1410ff66:
component_type: StartNode
id: a65b2a43-c65a-4cca-947f-f76e1410ff66
name: start_step
description: ''
metadata:
__metadata_info__: {}
inputs: []
outputs: []
branches:
- next
fe378524-69cb-4fcd-9d3a-f8fcb1c28316:
component_type: AgentNode
id: fe378524-69cb-4fcd-9d3a-f8fcb1c28316
name: summarize_step
description: ''
metadata:
__metadata_info__: {}
inputs: []
outputs:
- description: name of the animal
type: string
title: animal_name
default: ''
- description: level of danger of the animal. Can be "HIGH", "MEDIUM" or "LOW"
type: string
title: danger_level
default: ''
- description: list of threats for the animal
type: array
items:
type: string
title: threat
title: threats
default: []
branches:
- next
agent:
component_type: Agent
id: 3fe4580a-2bc6-4de8-85bb-cb8b0b361366
name: agent_776abfae__auto
description: ''
metadata:
__metadata_info__: {}
inputs: []
outputs:
- description: name of the animal
type: string
title: animal_name
default: ''
- description: level of danger of the animal. Can be "HIGH", "MEDIUM" or "LOW"
type: string
title: danger_level
default: ''
- description: list of threats for the animal
type: array
items:
type: string
title: threat
title: threats
default: []
llm_config:
component_type: VllmConfig
id: b32f4c62-2bd2-477d-a660-60ec1443b00c
name: llm_650f3ae8__auto
description: null
metadata:
__metadata_info__: {}
default_generation_parameters: null
url: LLAMA_API_URL
model_id: LLAMA_MODEL_ID
system_prompt: Extract from the article given by the user the name of the animal,
its danger level and the threats it's subject to.
tools: []
cb466062-c8c3-4d8c-ae59-2ec21e1df9ea:
component_type: EndNode
id: cb466062-c8c3-4d8c-ae59-2ec21e1df9ea
name: None End node
description: End node representing all transitions to None in the WayFlow flow
metadata: {}
inputs:
- description: list of threats for the animal
type: array
items:
type: string
title: threat
title: threats
default: []
- description: level of danger of the animal. Can be "HIGH", "MEDIUM" or "LOW"
type: string
title: danger_level
default: ''
- description: name of the animal
type: string
title: animal_name
default: ''
outputs:
- description: list of threats for the animal
type: array
items:
type: string
title: threat
title: threats
default: []
- description: level of danger of the animal. Can be "HIGH", "MEDIUM" or "LOW"
type: string
title: danger_level
default: ''
- description: name of the animal
type: string
title: animal_name
default: ''
branches: []
branch_name: next
agentspec_version: 25.4.1
You can then load the configuration back to an assistant using the AgentSpecLoader.
from wayflowcore.agentspec import AgentSpecLoader
new_assistant = AgentSpecLoader().load_json(serialized_assistant)
Recap#
In this guide, you learned how to incorporate LLMs into flows to:
generate raw text
produce structured output
generate structured generation using the agent and AgentExecutionStep
Next steps#
Having learned how to perform structured generation in WayFlow, you may now proceed to:
Config Generation to change LLM generation parameters.
Catching Exceptions to ensure robustness of the generated outputs.
Full code#
Click on the card at the top of this page to download the full code for this guide or copy the code below.
1# Copyright © 2025 Oracle and/or its affiliates.
2#
3# This software is under the Apache License 2.0
4# %%[markdown]
5# Wayflow Code Example - How to Do Structured LLM Generation in Flows
6# -------------------------------------------------------------------
7
8# How to use:
9# Create a new Python virtual environment and install the latest WayFlow version.
10# ```bash
11# python -m venv venv-wayflowcore
12# source venv-wayflowcore/bin/activate
13# pip install --upgrade pip
14# pip install "wayflowcore==26.1"
15# ```
16
17# You can now run the script
18# 1. As a Python file:
19# ```bash
20# python howto_promptexecutionstep.py
21# ```
22# 2. As a Notebook (in VSCode):
23# When viewing the file,
24# - press the keys Ctrl + Enter to run the selected cell
25# - or Shift + Enter to run the selected cell and move to the cell below# (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0) or Universal Permissive License
26# (UPL) 1.0 (LICENSE-UPL or https://oss.oracle.com/licenses/upl), at your option.
27
28
29# %%[markdown]
30## Define the article
31
32# %%
33article = """Sea turtles are ancient reptiles that have been around for over 100 million years. They play crucial roles in marine ecosystems, such as maintaining healthy seagrass beds and coral reefs. Unfortunately, they are under threat due to poaching, habitat loss, and pollution. Conservation efforts worldwide aim to protect nesting sites and reduce bycatch in fishing gear."""
34
35
36# %%[markdown]
37## Define the llm
38
39# %%
40from wayflowcore.models import VllmModel
41
42llm = VllmModel(
43 model_id="LLAMA_MODEL_ID",
44 host_port="LLAMA_API_URL",
45)
46
47# %%[markdown]
48## Create the flow using the prompt execution step
49
50# %%
51from wayflowcore.controlconnection import ControlFlowEdge
52from wayflowcore.dataconnection import DataFlowEdge
53from wayflowcore.flow import Flow
54from wayflowcore.property import StringProperty
55from wayflowcore.steps import PromptExecutionStep, StartStep
56
57start_step = StartStep(input_descriptors=[StringProperty("article")])
58summarize_step = PromptExecutionStep(
59 llm=llm,
60 prompt_template="""Summarize this article in 10 words:\n {{article}}""",
61 output_mapping={PromptExecutionStep.OUTPUT: "summary"},
62)
63summarize_step_name = "summarize_step"
64flow = Flow(
65 begin_step=start_step,
66 steps={
67 "start_step": start_step,
68 summarize_step_name: summarize_step,
69 },
70 control_flow_edges=[
71 ControlFlowEdge(source_step=start_step, destination_step=summarize_step),
72 ControlFlowEdge(source_step=summarize_step, destination_step=None),
73 ],
74 data_flow_edges=[
75 DataFlowEdge(start_step, "article", summarize_step, "article"),
76 ],
77)
78
79
80# %%[markdown]
81## Run the flow to get the summary
82
83# %%
84conversation = flow.start_conversation(inputs={"article": article})
85status = conversation.execute()
86print(status.output_values["summary"])
87# Sea turtles face threats from poaching, habitat loss, and pollution globally.
88
89from wayflowcore.controlconnection import ControlFlowEdge
90from wayflowcore.dataconnection import DataFlowEdge
91
92
93# %%[markdown]
94## Use structured generation to extract formatted information
95
96# %%
97from wayflowcore.property import ListProperty, StringProperty
98from wayflowcore.steps import PromptExecutionStep, StartStep
99
100animal_output = StringProperty(
101 name="animal_name",
102 description="name of the animal",
103 default_value="",
104)
105danger_level_output = StringProperty(
106 name="danger_level",
107 description='level of danger of the animal. Can be "HIGH", "MEDIUM" or "LOW"',
108 default_value="",
109)
110threats_output = ListProperty(
111 name="threats",
112 description="list of threats for the animal",
113 item_type=StringProperty("threat"),
114 default_value=[],
115)
116
117
118start_step = StartStep(input_descriptors=[StringProperty("article")])
119summarize_step = PromptExecutionStep(
120 llm=llm,
121 prompt_template="""Extract from the following article the name of the animal, its danger level and the threats it's subject to. The article:\n\n {{article}}""",
122 output_descriptors=[animal_output, danger_level_output, threats_output],
123)
124summarize_step_name = "summarize_step"
125flow = Flow(
126 begin_step=start_step,
127 steps={
128 "start_step": start_step,
129 summarize_step_name: summarize_step,
130 },
131 control_flow_edges=[
132 ControlFlowEdge(source_step=start_step, destination_step=summarize_step),
133 ControlFlowEdge(source_step=summarize_step, destination_step=None),
134 ],
135 data_flow_edges=[
136 DataFlowEdge(start_step, "article", summarize_step, "article"),
137 ],
138)
139
140conversation = flow.start_conversation(inputs={"article": article})
141status = conversation.execute()
142print(status.output_values)
143# {'threats': ['poaching', 'habitat loss', 'pollution'], 'danger_level': 'HIGH', 'animal_name': 'Sea turtles'}
144
145from wayflowcore.controlconnection import ControlFlowEdge
146from wayflowcore.dataconnection import DataFlowEdge
147
148
149# %%[markdown]
150## Use structured generation with JSON schema
151
152# %%
153from wayflowcore.property import Property, StringProperty
154from wayflowcore.steps import PromptExecutionStep, StartStep
155
156animal_json_schema = {
157 "title": "animal_object",
158 "description": "information about the animal",
159 "type": "object",
160 "properties": {
161 "animal_name": {
162 "type": "string",
163 "description": "name of the animal",
164 "default": "",
165 },
166 "danger_level": {
167 "type": "string",
168 "description": 'level of danger of the animal. Can be "HIGH", "MEDIUM" or "LOW"',
169 "default": "",
170 },
171 "threats": {
172 "type": "array",
173 "description": "list of threats for the animal",
174 "items": {"type": "string"},
175 "default": [],
176 },
177 },
178}
179animal_descriptor = Property.from_json_schema(animal_json_schema)
180
181start_step = StartStep(input_descriptors=[StringProperty("article")])
182summarize_step = PromptExecutionStep(
183 llm=llm,
184 prompt_template="""Extract from the following article the name of the animal, its danger level and the threats it's subject to. The article:\n\n {{article}}""",
185 output_descriptors=[animal_descriptor],
186)
187summarize_step_name = "summarize_step"
188flow = Flow(
189 begin_step=start_step,
190 steps={
191 "start_step": start_step,
192 summarize_step_name: summarize_step,
193 },
194 control_flow_edges=[
195 ControlFlowEdge(source_step=start_step, destination_step=summarize_step),
196 ControlFlowEdge(source_step=summarize_step, destination_step=None),
197 ],
198 data_flow_edges=[
199 DataFlowEdge(start_step, "article", summarize_step, "article"),
200 ],
201)
202
203conversation = flow.start_conversation(inputs={"article": article})
204status = conversation.execute()
205print(status.output_values)
206# {'animal_object': {'animal_name': 'Sea turtles', 'danger_level': 'MEDIUM', 'threats': ['Poaching', 'Habitat loss', 'Pollution']}}
207
208
209
210# %%[markdown]
211## Use structured generation with Agents in flows
212
213# %%
214from wayflowcore.agent import Agent, CallerInputMode
215from wayflowcore.controlconnection import ControlFlowEdge
216from wayflowcore.steps import AgentExecutionStep, StartStep
217
218start_step = StartStep(input_descriptors=[])
219agent = Agent(
220 llm=llm,
221 custom_instruction="""Extract from the article given by the user the name of the animal, its danger level and the threats it's subject to.""",
222 initial_message=None,
223 caller_input_mode=CallerInputMode.ALWAYS,
224 output_descriptors=[animal_output, danger_level_output, threats_output],
225)
226
227summarize_agent_step = AgentExecutionStep(agent=agent)
228summarize_step_name = "summarize_step"
229flow = Flow(
230 begin_step=start_step,
231 steps={
232 "start_step": start_step,
233 summarize_step_name: summarize_agent_step,
234 },
235 control_flow_edges=[
236 ControlFlowEdge(source_step=start_step, destination_step=summarize_agent_step),
237 ControlFlowEdge(source_step=summarize_agent_step, destination_step=None),
238 ],
239 data_flow_edges=[],
240)
241
242conversation = flow.start_conversation()
243conversation.append_user_message("Here is the article: " + article)
244status = conversation.execute()
245print(status.output_values)
246# {'animal_name': 'Sea turtles', 'danger_level': 'HIGH', 'threats': ['poaching', 'habitat loss', 'pollution']}
247
248
249# %%[markdown]
250## Export config to Agent Spec
251
252# %%
253from wayflowcore.agentspec import AgentSpecExporter
254serialized_assistant = AgentSpecExporter().to_json(flow)
255
256# %%[markdown]
257## Load Agent Spec config
258
259# %%
260from wayflowcore.agentspec import AgentSpecLoader
261new_assistant = AgentSpecLoader().load_json(serialized_assistant)