How to Build a Swarm of Agents

The Swarm pattern is a type of agentic pattern that takes inspiration from Swarm intelligence. It is often observed in biological systems such as ant colonies, bee hives, and bird flocks. In this agentic pattern, each agent is assigned a specific responsibility and can delegate tasks to other specialized agents to improve overall performance.

When to use the Swarm pattern?

Compared to the hierarchical pattern, the communication in Swarm pattern reduces the number of LLM calls as showcased in the diagram below.

In the hierarchical pattern, a route User → Agent K → User will require:

1. All intermediate agents to call the correct sub-agent to go down to the Agent K.

2. The Agent K to generate its answer.

3. All intermediate agents to relay the answer back to the user.

In the swarm pattern, a route User → Agent K → User will require:

1. The first agent to call or handoff the conversation the Agent K (provided that the developer allows the connection between the two agents).

2. The Agent K to generate its answer.

3. The first agent to relay the answer (only when NOT using handoff; with handoff the Agent K replaces the first agent and is thus directly communicating with the human user)

---

This guide presents an example of a simple Swarm of agents applied to a medical use case.

This guide will walk you through the following steps:

1. Define the configuration for an LLM model

2. Define tools for the agents

3. Define agents equipped with tools

4. Define a Swarm using the defined agents

It also covers how to enable handoff when building the Swarm.

Basic implementation

1. Define the LLM model

The decision mechanism of Agents is powered by a Large Language Model. Defining the agent with Agent Spec requires to pass the configuration for the LLM. There are several options such as using OCI GenAI Service or self hosting a model with vLLM.

Start by defining the LLM configuration to be shared across all agents.

OCI GenAIvLLMOllama

from pyagentspec.llms import OciGenAiConfig
from pyagentspec.llms.ociclientconfig import OciClientConfigWithApiKey

client_config = OciClientConfigWithApiKey(
    name="Oci Client Config",
    service_endpoint="https://url-to-service-endpoint.com",
    auth_profile="DEFAULT",
    auth_file_location="~/.oci/config"
)

llm_config = OciGenAiConfig(
    name="Oci GenAI Config",
    model_id="provider.model-id",
    compartment_id="compartment-id",
    client_config=client_config,
)

2. Define tools

from pyagentspec.property import ListProperty, StringProperty
from pyagentspec.tools import ServerTool

symptoms_checker = ServerTool(
    name="Symptoms checker",
    description="Checks symptoms against the medical knowledge base.",
    inputs=[ListProperty(title="symptoms", item_type=StringProperty())],
    outputs=[ListProperty(title="possible_conditions", item_type=StringProperty())],
)

get_medication_info = ServerTool(
    name="Get medication info",
    description="Provides availability and information about a medication.",
    inputs=[StringProperty(title="drug_name")],
    outputs=[StringProperty(title="medication_info")],
)

knowledge_tool = ServerTool(
    name="Knowledge tool",
    description=" Provides diagnosis and treatment information for skin conditions.",
    inputs=[StringProperty(title="condition")],
    outputs=[StringProperty(title="treatment_info")],
)

API Reference: ServerTool

Note

To require user confirmation for a tool, set requires_confirmation=True (see Tool). This signals that execution environments should require user approval before running the tool, which is useful for tools performing sensitive actions.

3. Define agents equipped with tools

General Practitioner Agent

The first agent the user interacts with is the General Practitioner Agent.

This agent is equipped with the symptoms checker tool, and can interact with the Pharmacist Agent as well as the Dermatologist Agent.

from pyagentspec.agent import Agent

general_practitioner_system_prompt = """
You are an helpful general practitioner LLM doctor responsible for handling patient consultations.
Your goal is to assess patients' symptoms, provide initial diagnoses, pescribe medication for
mild conditions or refer to other specialists as needed.

## When to Use Each Tool
- symptoms_checker: Use this to look up possible conditions based on symptoms reported by the patient.

## When to query other Agents
- Pharmacist: Every time you need to prescribe some medication, give the condition description and prescription to the Pharmacist.
- Dermatologist: If the patient has a skin condition, ask expert advice/diagnosis to the Dermatologist after the initial exchange.

# Specific instructions
## Initial exchange
When a patient first shares symptoms ask about for exactly 1 round of questions, consisting of asking about medical history and
simple questions (e.g. do they have known allergies, when did the symptoms started, did they already tried some treatment/medication, etc)).
Always ask for this one round of information to the user before prescribing medication / referring to other agents.

## Identifying condition
Use the symptoms checker to confirm the condition (for mild conditions only. For other conditions, refer to specialists).

## Mild conditions
* If the patient has a mild cold, prescribe medicationA.
    - Give the condition description and prescription to the Pharmacist.

## Skin conditions
* If the patient has a skin condition, ask for expert advice/diagnosis from the Dermatologist.
Here, you don't need to ask for the user confirmation. Directly call the Dermatologist
    - Provide the patient's symptoms/initial hypothesis to the Dermatologist and ask for a diagnosis.
    - The Dermatologist may query the Pharmacist to confirm the availability of the prescribed medication.
    - Once the treatment is confirmed, pass on the prescription to the patient and ask them to follow the instructions.
""".strip()

general_practitioner = Agent(
    name="GeneralPractitioner",
    description="General Practitioner. Primary point of contact for patients, handles general medical inquiries, provides initial diagnoses, and manages referrals.",
    llm_config=llm_config,
    tools=[symptoms_checker],
    system_prompt=general_practitioner_system_prompt,
)

API Reference: Agent

Pharmacist Agent

The Pharmacist Agent is equipped with the tool to obtain medication information. This agent cannot initiate a discussion with the other agents in the Swarm.

pharmacist_system_prompt = """
You are an helpful Pharmacist LLM Agent responsible for giving information about medication.
Your goal is to answer queries from the General Practitioner Doctor about medication information
and availabilities.

## When to Use Each Tool
- get_medication_info: Use this to look up availability and information about a specific medication.
""".strip()

pharmacist = Agent(
    name="Pharmacist",
    description="Pharmacist. Gives availability and information about specific medication.",
    llm_config=llm_config,
    tools=[get_medication_info],
    system_prompt=pharmacist_system_prompt,
)

Dermatologist Agent

The final agent in the Swarm is the Dermatologist agent which is equipped with a tool to query a skin condition knowledge base. This agent can initiate a discussion with the Pharmacist Agent.

dermatologist_system_prompt = """
You are an helpful Dermatologist LLM Agent responsible for diagnosing and treating skin conditions.
Your goal is to assess patients' symptoms, provide accurate diagnoses, and prescribe effective treatments.

## When to Use Each Tool
- knowledge_tool: Use this to look up diagnosis and treatment information for specific skin conditions.

## When to query other Agents
- Pharmacist: Every time you need to prescribe some medication, give the condition description and prescription to the Pharmacist.

# Specific instructions
## Initial exchange
When a patient's symptoms are referred to you by the General Practitioner, review the symptoms and use the knowledge tool to confirm the diagnosis.
## Prescription
Prescribe the recommended treatment for the diagnosed condition and query the Pharmacist to confirm the availability of the prescribed medication.

When answering back to the General Practitioner, describe your diagnosis and the prescription.
Tell the General Practitioner that you already checked with the pharmacist for availability.
""".strip()

dermatologist = Agent(
    name="Dermatologist",
    description="Dermatologist. Diagnoses and treats skin conditions.",
    llm_config=llm_config,
    tools=[knowledge_tool],
    system_prompt=dermatologist_system_prompt,
)

4. Define a Swarm

The Swarm has two main parameters:

• The first_agent — the initial agent the user interacts with (in this example, the General Practitioner Agent).

• A list of relationships between agents.

Additionally, the list of “relationships” between the agents must be defined.

Each relationship is defined as a tuple of Caller Agent and Recipient Agent.

In this example, the General Practitioner Doctor Agent can initiate discussions with both the Pharmacist and the Dermatologist. The Dermatologist can also initiate discussions with the Pharmacist.

When invoked, each agent can either respond to its caller (a human user or another agent) or choose to initiate a discussion with another agent if they are given the capability to do so.

from pyagentspec.swarm import Swarm

assistant = Swarm(
    name="Swarm",
    first_agent=general_practitioner,
    relationships=[
        (general_practitioner, pharmacist),
        (general_practitioner, dermatologist),
        (dermatologist, pharmacist),
    ],
)

API Reference: Swarm

5. Export the Agent Spec configuration

The Agent Spec configuration is generated in JSON format. These configurations can be loaded and executed in Agent Spec-compatible systems such as the WayFlow runtime. See, for example, How to Execute Agent Spec Configurations with WayFlow.

from pyagentspec.serialization import AgentSpecSerializer

serialized_assistant = AgentSpecSerializer().to_json(assistant)

API Reference: AgentSpecSerializer

Here is what the Agent Spec representation will look like ↓

Click here to see the assistant configuration.

JSONYAML

{
    "component_type": "Swarm",
    "id": "0625ef9d-e1a7-4f8c-852f-1077b3b9383a",
    "name": "Swarm",
    "description": null,
    "metadata": {},
    "inputs": [],
    "outputs": [],
    "first_agent": {
        "$component_ref": "55f143da-4dff-466e-bc5c-f8da5bf8c22a"
    },
    "relationships": [
        [
            {
                "$component_ref": "55f143da-4dff-466e-bc5c-f8da5bf8c22a"
            },
            {
                "$component_ref": "13265e9c-f26e-437d-8f4f-02bbf9e33463"
            }
        ],
        [
            {
                "$component_ref": "55f143da-4dff-466e-bc5c-f8da5bf8c22a"
            },
            {
                "$component_ref": "5c88130e-24f9-4ec0-8009-93a91c4ccd6e"
            }
        ],
        [
            {
                "$component_ref": "5c88130e-24f9-4ec0-8009-93a91c4ccd6e"
            },
            {
                "$component_ref": "13265e9c-f26e-437d-8f4f-02bbf9e33463"
            }
        ]
    ],
    "handoff": true,
    "$referenced_components": {
        "55f143da-4dff-466e-bc5c-f8da5bf8c22a": {
            "component_type": "Agent",
            "id": "55f143da-4dff-466e-bc5c-f8da5bf8c22a",
            "name": "GeneralPractitioner",
            "description": "General Practitioner. Primary point of contact for patients, handles general medical inquiries, provides initial diagnoses, and manages referrals.",
            "metadata": {},
            "inputs": [],
            "outputs": [],
            "llm_config": {
                "$component_ref": "5fd42c44-d0e7-484c-bf6b-1e9649d42576"
            },
            "system_prompt": "You are an helpful general practitioner LLM doctor responsible for handling patient consultations.\nYour goal is to assess patients' symptoms, provide initial diagnoses, pescribe medication for\nmild conditions or refer to other specialists as needed.\n\n## When to Use Each Tool\n- symptoms_checker: Use this to look up possible conditions based on symptoms reported by the patient.\n\n## When to query other Agents\n- Pharmacist: Every time you need to prescribe some medication, give the condition description and prescription to the Pharmacist.\n- Dermatologist: If the patient has a skin condition, ask expert advice/diagnosis to the Dermatologist after the initial exchange.\n\n# Specific instructions\n## Initial exchange\nWhen a patient first shares symptoms ask about for exactly 1 round of questions, consisting of asking about medical history and\nsimple questions (e.g. do they have known allergies, when did the symptoms started, did they already tried some treatment/medication, etc)).\nAlways ask for this one round of information to the user before prescribing medication / referring to other agents.\n\n## Identifying condition\nUse the symptoms checker to confirm the condition (for mild conditions only. For other conditions, refer to specialists).\n\n## Mild conditions\n* If the patient has a mild cold, prescribe medicationA.\n    - Give the condition description and prescription to the Pharmacist.\n\n## Skin conditions\n* If the patient has a skin condition, ask for expert advice/diagnosis from the Dermatologist.\nHere, you don't need to ask for the user confirmation. Directly call the Dermatologist\n    - Provide the patient's symptoms/initial hypothesis to the Dermatologist and ask for a diagnosis.\n    - The Dermatologist may query the Pharmacist to confirm the availability of the prescribed medication.\n    - Once the treatment is confirmed, pass on the prescription to the patient and ask them to follow the instructions.",
            "tools": [
                {
                    "component_type": "ServerTool",
                    "id": "825e06ad-512c-4052-8757-1448e462dc30",
                    "name": "Symptoms checker",
                    "description": "Checks symptoms against the medical knowledge base.",
                    "metadata": {},
                    "inputs": [
                        {
                            "title": "symptoms",
                            "items": {
                                "type": "string"
                            },
                            "type": "array"
                        }
                    ],
                    "outputs": [
                        {
                            "title": "possible_conditions",
                            "items": {
                                "type": "string"
                            },
                            "type": "array"
                        }
                    ]
                }
            ]
        },
        "5fd42c44-d0e7-484c-bf6b-1e9649d42576": {
            "component_type": "VllmConfig",
            "id": "5fd42c44-d0e7-484c-bf6b-1e9649d42576",
            "name": "meta.llama-3.3-70b-instruct",
            "description": null,
            "metadata": {},
            "default_generation_parameters": null,
            "url": "LLAMA_PLACEHOLDER_LINK",
            "model_id": "model_id"
        },
        "13265e9c-f26e-437d-8f4f-02bbf9e33463": {
            "component_type": "Agent",
            "id": "13265e9c-f26e-437d-8f4f-02bbf9e33463",
            "name": "Pharmacist",
            "description": "Pharmacist. Gives availability and information about specific medication.",
            "metadata": {},
            "inputs": [],
            "outputs": [],
            "llm_config": {
                "$component_ref": "5fd42c44-d0e7-484c-bf6b-1e9649d42576"
            },
            "system_prompt": "You are an helpful Pharmacist LLM Agent responsible for giving information about medication.\nYour goal is to answer queries from the General Practitioner Doctor about medication information\nand availabilities.\n\n## When to Use Each Tool\n- get_medication_info: Use this to look up availability and information about a specific medication.",
            "tools": [
                {
                    "component_type": "ServerTool",
                    "id": "7a81189d-e3ed-4fa5-a1ec-546b86b98cc6",
                    "name": "Get medication info",
                    "description": "Provides availability and information about a medication.",
                    "metadata": {},
                    "inputs": [
                        {
                            "title": "drug_name",
                            "type": "string"
                        }
                    ],
                    "outputs": [
                        {
                            "title": "medication_info",
                            "type": "string"
                        }
                    ]
                }
            ]
        },
        "5c88130e-24f9-4ec0-8009-93a91c4ccd6e": {
            "component_type": "Agent",
            "id": "5c88130e-24f9-4ec0-8009-93a91c4ccd6e",
            "name": "Dermatologist",
            "description": "Dermatologist. Diagnoses and treats skin conditions.",
            "metadata": {},
            "inputs": [],
            "outputs": [],
            "llm_config": {
                "$component_ref": "5fd42c44-d0e7-484c-bf6b-1e9649d42576"
            },
            "system_prompt": "You are an helpful Dermatologist LLM Agent responsible for diagnosing and treating skin conditions.\nYour goal is to assess patients' symptoms, provide accurate diagnoses, and prescribe effective treatments.\n\n## When to Use Each Tool\n- knowledge_tool: Use this to look up diagnosis and treatment information for specific skin conditions.\n\n## When to query other Agents\n- Pharmacist: Every time you need to prescribe some medication, give the condition description and prescription to the Pharmacist.\n\n# Specific instructions\n## Initial exchange\nWhen a patient's symptoms are referred to you by the General Practitioner, review the symptoms and use the knowledge tool to confirm the diagnosis.\n## Prescription\nPrescribe the recommended treatment for the diagnosed condition and query the Pharmacist to confirm the availability of the prescribed medication.\n\nWhen answering back to the General Practitioner, describe your diagnosis and the prescription.\nTell the General Practitioner that you already checked with the pharmacist for availability.",
            "tools": [
                {
                    "component_type": "ServerTool",
                    "id": "e4f83d40-be72-48d8-8f90-61d0b7416ca7",
                    "name": "Knowledge tool",
                    "description": " Provides diagnosis and treatment information for skin conditions.",
                    "metadata": {},
                    "inputs": [
                        {
                            "title": "condition",
                            "type": "string"
                        }
                    ],
                    "outputs": [
                        {
                            "title": "treatment_info",
                            "type": "string"
                        }
                    ]
                }
            ]
        }
    },
    "agentspec_version": "25.4.2"
}

Enabling handoff in the Swarm

By default, communication in the Swarm pattern is done with agent sending blocking messages/requests to each other.

The handoff mechanism provides an alternative: when enabled, agents can handoff the conversation — that is, transfer the message history between the user and one agent to another agent within the Swarm. Agents can still communicate with each other as they do when handoff=False.

A key benefit of using handoff is reduced response latency. Talking to other agents increases the “distance” between the human user and the current agent. Transferring a conversation to another agent keeps this distance unchanged (in other words, the agent interacting with the user is different, but the user is still the same).

To enable handoff in a Swarm, set the handoff parameter to True.

assistant = Swarm(
    name="Swarm",
    first_agent=general_practitioner,
    relationships=[
        (general_practitioner, pharmacist),
        (general_practitioner, dermatologist),
        (dermatologist, pharmacist),
    ],
    handoff=True,  # <-- Add this
)

Recap

This guide covered how to define a swarm of agents with and without handoff in Agent Spec.

Below is the complete code from this guide.

from pyagentspec.llms import VllmConfig

llm_config = VllmConfig(
    model_id="model_id",
    url="LLAMA_PLACEHOLDER_LINK",
    name="meta.llama-3.3-70b-instruct",
)

# .. start-tools
from pyagentspec.property import ListProperty, StringProperty
from pyagentspec.tools import ServerTool

symptoms_checker = ServerTool(
    name="Symptoms checker",
    description="Checks symptoms against the medical knowledge base.",
    inputs=[ListProperty(title="symptoms", item_type=StringProperty())],
    outputs=[ListProperty(title="possible_conditions", item_type=StringProperty())],
)

get_medication_info = ServerTool(
    name="Get medication info",
    description="Provides availability and information about a medication.",
    inputs=[StringProperty(title="drug_name")],
    outputs=[StringProperty(title="medication_info")],
)

knowledge_tool = ServerTool(
    name="Knowledge tool",
    description=" Provides diagnosis and treatment information for skin conditions.",
    inputs=[StringProperty(title="condition")],
    outputs=[StringProperty(title="treatment_info")],
)

# .. end-tools

# .. start-first-agent
from pyagentspec.agent import Agent

general_practitioner_system_prompt = """
You are an helpful general practitioner LLM doctor responsible for handling patient consultations.
Your goal is to assess patients' symptoms, provide initial diagnoses, pescribe medication for
mild conditions or refer to other specialists as needed.

## When to Use Each Tool
- symptoms_checker: Use this to look up possible conditions based on symptoms reported by the patient.

## When to query other Agents
- Pharmacist: Every time you need to prescribe some medication, give the condition description and prescription to the Pharmacist.
- Dermatologist: If the patient has a skin condition, ask expert advice/diagnosis to the Dermatologist after the initial exchange.

# Specific instructions
## Initial exchange
When a patient first shares symptoms ask about for exactly 1 round of questions, consisting of asking about medical history and
simple questions (e.g. do they have known allergies, when did the symptoms started, did they already tried some treatment/medication, etc)).
Always ask for this one round of information to the user before prescribing medication / referring to other agents.

## Identifying condition
Use the symptoms checker to confirm the condition (for mild conditions only. For other conditions, refer to specialists).

## Mild conditions
* If the patient has a mild cold, prescribe medicationA.
    - Give the condition description and prescription to the Pharmacist.

## Skin conditions
* If the patient has a skin condition, ask for expert advice/diagnosis from the Dermatologist.
Here, you don't need to ask for the user confirmation. Directly call the Dermatologist
    - Provide the patient's symptoms/initial hypothesis to the Dermatologist and ask for a diagnosis.
    - The Dermatologist may query the Pharmacist to confirm the availability of the prescribed medication.
    - Once the treatment is confirmed, pass on the prescription to the patient and ask them to follow the instructions.
""".strip()

general_practitioner = Agent(
    name="GeneralPractitioner",
    description="General Practitioner. Primary point of contact for patients, handles general medical inquiries, provides initial diagnoses, and manages referrals.",
    llm_config=llm_config,
    tools=[symptoms_checker],
    system_prompt=general_practitioner_system_prompt,
)
# .. end-first-agent


# .. start-second-agent
pharmacist_system_prompt = """
You are an helpful Pharmacist LLM Agent responsible for giving information about medication.
Your goal is to answer queries from the General Practitioner Doctor about medication information
and availabilities.

## When to Use Each Tool
- get_medication_info: Use this to look up availability and information about a specific medication.
""".strip()

pharmacist = Agent(
    name="Pharmacist",
    description="Pharmacist. Gives availability and information about specific medication.",
    llm_config=llm_config,
    tools=[get_medication_info],
    system_prompt=pharmacist_system_prompt,
)
# .. end-second-agent

# .. start-third-agent
dermatologist_system_prompt = """
You are an helpful Dermatologist LLM Agent responsible for diagnosing and treating skin conditions.
Your goal is to assess patients' symptoms, provide accurate diagnoses, and prescribe effective treatments.

## When to Use Each Tool
- knowledge_tool: Use this to look up diagnosis and treatment information for specific skin conditions.

## When to query other Agents
- Pharmacist: Every time you need to prescribe some medication, give the condition description and prescription to the Pharmacist.

# Specific instructions
## Initial exchange
When a patient's symptoms are referred to you by the General Practitioner, review the symptoms and use the knowledge tool to confirm the diagnosis.
## Prescription
Prescribe the recommended treatment for the diagnosed condition and query the Pharmacist to confirm the availability of the prescribed medication.

When answering back to the General Practitioner, describe your diagnosis and the prescription.
Tell the General Practitioner that you already checked with the pharmacist for availability.
""".strip()

dermatologist = Agent(
    name="Dermatologist",
    description="Dermatologist. Diagnoses and treats skin conditions.",
    llm_config=llm_config,
    tools=[knowledge_tool],
    system_prompt=dermatologist_system_prompt,
)
# .. end-third-agent

# .. start-swarm
from pyagentspec.swarm import Swarm

assistant = Swarm(
    name="Swarm",
    first_agent=general_practitioner,
    relationships=[
        (general_practitioner, pharmacist),
        (general_practitioner, dermatologist),
        (dermatologist, pharmacist),
    ],
)
# .. end-swarm

# .. start-serialization
from pyagentspec.serialization import AgentSpecSerializer

serialized_assistant = AgentSpecSerializer().to_json(assistant)

Next steps

Having learned how to define a swarm, you may now proceed to how to use the WayFlow runtime to execute it.