LLMs#

This page presents all APIs and classes related to LLM models.

LlmConfig#

class pyagentspec.llms.llmconfig.LlmConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, default_generation_parameters=None, retry_policy=None)#

Bases: Component

A LLM configuration defines how to connect to a LLM to do generation requests.

This class provides the base class, while concrete classes provide the required configuration to connect to specific LLM providers.

Parameters:

id (str) – A unique identifier for this Component
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM
retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.

LLM Generation Config#

Parameters for LLM generation (max_tokens, temperature, top_p).

class pyagentspec.llms.llmgenerationconfig.LlmGenerationConfig(*, max_tokens=None, temperature=None, top_p=None, **extra_data)#

Bases: BaseModel

A configuration object defining LLM generation parameters.

Parameters include number of tokens, sampling parameters, etc.

Parameters:

max_tokens (int | None) –
temperature (float | None) –
top_p (float | None) –
extra_data (Any) –

max_tokens: int | None#: Maximum number of token that should be generated

temperature: float | None#: Value of the temperature parameter to be used for generation

top_p: float | None#: Value of the top_p parameter to be used for generation

OpenAI API Type#

class pyagentspec.llms.openaicompatibleconfig.OpenAIAPIType(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)#

Bases: str, Enum

Enumeration of OpenAI API Types.

chat completions: Chat Completions API responses: Responses API

CHAT_COMPLETIONS = 'chat_completions'#

RESPONSES = 'responses'#

OCI API Type#

class pyagentspec.llms.ocigenaiconfig.OciAPIType(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)#

Bases: str, Enum

Enumeration of API Types.

OCI = 'oci'#

OPENAI_CHAT_COMPLETIONS = 'openai_chat_completions'#

OPENAI_RESPONSES = 'openai_responses'#

All models#

OpenAI Compatible Models#

class pyagentspec.llms.openaicompatibleconfig.OpenAiCompatibleConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, default_generation_parameters=None, retry_policy=None, url, model_id, api_type=OpenAIAPIType.CHAT_COMPLETIONS, api_key=None, key_file=None, cert_file=None, ca_file=None)#

Bases: LlmConfig

Class to configure a connection to an LLM that is compatible with OpenAI completions APIs.

Requires to specify the url of the APIs to contact.

Parameters:

id (str) – ID of the model to use
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM
retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.
url (str) – Url of the OpenAI compatible model deployment
model_id (str) – ID of the model to use
api_type (OpenAIAPIType) – OpenAI API protocol to use
api_key (str | None) – An optional API KEY for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.
key_file (str | None) – The path to an optional client private key file (PEM format).
cert_file (str | None) – The path to an optional client certificate chain file (PEM format).
ca_file (str | None) – The path to an optional trusted CA certificate file (PEM format) used to verify the server.

api_key: str | None#: An optional API KEY for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.

api_type: OpenAIAPIType#: OpenAI API protocol to use

ca_file: str | None#: The path to an optional trusted CA certificate file (PEM format) used to verify the server.

cert_file: str | None#: The path to an optional client certificate chain file (PEM format).

key_file: str | None#: The path to an optional client private key file (PEM format).

model_id: str#: ID of the model to use

url: str#: Url of the OpenAI compatible model deployment

VLLM Models#

class pyagentspec.llms.vllmconfig.VllmConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, default_generation_parameters=None, retry_policy=None, url, model_id, api_type=OpenAIAPIType.CHAT_COMPLETIONS, api_key=None, key_file=None, cert_file=None, ca_file=None)#

Bases: OpenAiCompatibleConfig

Class to configure a connection to a vLLM-hosted LLM.

Requires to specify the url at which the instance is running.

Parameters:

id (str) – ID of the model to use
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM
retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.
url (str) – Url of the OpenAI compatible model deployment
model_id (str) – ID of the model to use
api_type (OpenAIAPIType) – OpenAI API protocol to use
api_key (str | None) – An optional API KEY for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.
key_file (str | None) – The path to an optional client private key file (PEM format).
cert_file (str | None) – The path to an optional client certificate chain file (PEM format).
ca_file (str | None) – The path to an optional trusted CA certificate file (PEM format) used to verify the server.

api_key: str | None#: An optional API KEY for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.

api_type: OpenAIAPIType#: OpenAI API protocol to use

ca_file: str | None#: The path to an optional trusted CA certificate file (PEM format) used to verify the server.

cert_file: str | None#: The path to an optional client certificate chain file (PEM format).

key_file: str | None#: The path to an optional client private key file (PEM format).

model_id: str#: ID of the model to use

url: str#: Url of the OpenAI compatible model deployment

Ollama Models#

class pyagentspec.llms.ollamaconfig.OllamaConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, default_generation_parameters=None, retry_policy=None, url, model_id, api_type=OpenAIAPIType.CHAT_COMPLETIONS, api_key=None, key_file=None, cert_file=None, ca_file=None)#

Bases: OpenAiCompatibleConfig

Class to configure a connection to a local model ran with Ollama.

Requires to specify the url and port at which the model is running.

Parameters:

id (str) – ID of the model to use
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM
retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.
url (str) – Url of the OpenAI compatible model deployment
model_id (str) – ID of the model to use
api_type (OpenAIAPIType) – OpenAI API protocol to use
api_key (str | None) – An optional API KEY for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.
key_file (str | None) – The path to an optional client private key file (PEM format).
cert_file (str | None) – The path to an optional client certificate chain file (PEM format).
ca_file (str | None) – The path to an optional trusted CA certificate file (PEM format) used to verify the server.

api_key: str | None#: An optional API KEY for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.

api_type: OpenAIAPIType#: OpenAI API protocol to use

ca_file: str | None#: The path to an optional trusted CA certificate file (PEM format) used to verify the server.

cert_file: str | None#: The path to an optional client certificate chain file (PEM format).

key_file: str | None#: The path to an optional client private key file (PEM format).

model_id: str#: ID of the model to use

url: str#: Url of the OpenAI compatible model deployment

OpenAI Models#

class pyagentspec.llms.openaiconfig.OpenAiConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, default_generation_parameters=None, retry_policy=None, model_id, api_type=OpenAIAPIType.CHAT_COMPLETIONS, api_key=None)#

Bases: LlmConfig

Class to configure a connection to a OpenAI LLM.

Requires to specify the identity of the model to use.

Parameters:

id (str) – ID of the model to use
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM
retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.
model_id (str) – ID of the model to use
api_type (OpenAIAPIType) – OpenAI API protocol to use
api_key (str | None) – An optional API KEY for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.

api_key: str | None#: An optional API KEY for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.

api_type: OpenAIAPIType#: OpenAI API protocol to use

model_id: str#: ID of the model to use

Gemini Models#

class pyagentspec.llms.geminiauthconfig.GeminiAuthConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v26_2_0, max_agentspec_version=AgentSpecVersionEnum.v26_2_0)#

Bases: Component

Base class for Gemini authentication configuration.

Parameters:

id (str) – A unique identifier for this Component
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –

class pyagentspec.llms.geminiauthconfig.GeminiAIStudioAuthConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v26_2_0, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, api_key=None)#

Bases: GeminiAuthConfig

Authentication settings for Gemini via Google AI Studio.

Parameters:

id (str) – A unique identifier for this Component
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
api_key (str | None) – API key to use. If unset, runtimes may load it from GEMINI_API_KEY.

api_key: str | None#: API key to use. If unset, runtimes may load it from GEMINI_API_KEY.

class pyagentspec.llms.geminiauthconfig.GeminiVertexAIAuthConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v26_2_0, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, project_id=None, location='global', credentials=None)#

Bases: GeminiAuthConfig

Authentication settings for Gemini via Google Vertex AI.

Parameters:

id (str) – Optional Google Cloud project identifier. This may still need to be set explicitly when the runtime cannot infer the project from Application Default Credentials (ADC) or other local Google Cloud configuration.
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
project_id (str | None) – Optional Google Cloud project identifier. This may still need to be set explicitly when the runtime cannot infer the project from Application Default Credentials (ADC) or other local Google Cloud configuration.
location (str) – Vertex AI location/region.
credentials (str | Dict[str, Any] | None) – Optional local file path to a Google Cloud JSON credential file, such as a service-account key file, or an inline dictionary containing the parsed JSON contents of that file. When unset, runtimes may rely on Application Default Credentials (ADC), such as GOOGLE_APPLICATION_CREDENTIALS, credentials made available through the local Google Cloud environment, or an attached service account. Even then, project_id may still need to be provided separately if it cannot be resolved from the environment.

credentials: str | Dict[str, Any] | None#

Optional local file path to a Google Cloud JSON credential file, such as a service-account key file, or an inline dictionary containing the parsed JSON contents of that file.

When unset, runtimes may rely on Application Default Credentials (ADC), such as GOOGLE_APPLICATION_CREDENTIALS, credentials made available through the local Google Cloud environment, or an attached service account. Even then, project_id may still need to be provided separately if it cannot be resolved from the environment.

location: str#: Vertex AI location/region.

project_id: str | None#

Optional Google Cloud project identifier.

This may still need to be set explicitly when the runtime cannot infer the project from Application Default Credentials (ADC) or other local Google Cloud configuration.

class pyagentspec.llms.geminiconfig.GeminiConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v26_2_0, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, default_generation_parameters=None, retry_policy=None, model_id, auth)#

Bases: LlmConfig

Configure a connection to a Gemini LLM (AI Studio or Vertex AI).

Parameters:

id (str) – Identifier of the Gemini model to use.
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM
retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.
model_id (str) – Identifier of the Gemini model to use.
auth (GeminiAuthConfig) – Authentication configuration used to connect to the Gemini service.

auth: GeminiAuthConfig#: Authentication configuration used to connect to the Gemini service.

model_id: str#: Identifier of the Gemini model to use.

OciGenAi Models#

class pyagentspec.llms.ocigenaiconfig.ServingMode(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)#

Bases: str, Enum

Serving mode to use for the GenAI service

DEDICATED = 'DEDICATED'#

ON_DEMAND = 'ON_DEMAND'#

class pyagentspec.llms.ocigenaiconfig.ModelProvider(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)#

Bases: str, Enum

Provider of the model. It is used to ensure the requests to this model respect the format expected by the provider.

COHERE = 'COHERE'#

GROK = 'GROK'#

META = 'META'#

OTHER = 'OTHER'#

XAI = 'XAI'#

class pyagentspec.llms.ocigenaiconfig.OciGenAiConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, default_generation_parameters=None, retry_policy=None, model_id, compartment_id, serving_mode=ServingMode.ON_DEMAND, provider=None, client_config, api_type=OciAPIType.OCI, conversation_store_id=None)#

Bases: LlmConfig

Class to configure a connection to a OCI GenAI hosted model.

Requires to specify the model id and the client configuration to the OCI GenAI service.

Parameters:

id (str) – The identifier of the model to use.
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM
retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.
model_id (str) – The identifier of the model to use.
compartment_id (str) – The OCI compartment ID where the model is hosted.
serving_mode (ServingMode) – The serving mode for the model.
provider (ModelProvider | None) – The provider of the model. If None, it will be automatically detected by the runtime using the model ID.
client_config (OciClientConfig) – The client configuration for connecting to OCI GenAI service.
api_type (OciAPIType) – API protocol to use.
conversation_store_id (str | None) – ID of the conversation store to persist conversations when using the openai responses API.

api_type: OciAPIType#: API protocol to use.

client_config: OciClientConfig#: The client configuration for connecting to OCI GenAI service.

compartment_id: str#: The OCI compartment ID where the model is hosted.

conversation_store_id: str | None#: ID of the conversation store to persist conversations when using the openai responses API.

model_id: str#: The identifier of the model to use.

provider: ModelProvider | None#: The provider of the model. If None, it will be automatically detected by the runtime using the model ID.

serving_mode: ServingMode#: The serving mode for the model.

class pyagentspec.llms.ociclientconfig.OciClientConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, service_endpoint, auth_type)#

Bases: Component

Base abstract class for OCI client config.

Parameters:

id (str) – A unique identifier for this Component
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
service_endpoint (str) –
auth_type (Literal['SECURITY_TOKEN', 'INSTANCE_PRINCIPAL', 'RESOURCE_PRINCIPAL', 'API_KEY']) –

auth_type: Literal['SECURITY_TOKEN', 'INSTANCE_PRINCIPAL', 'RESOURCE_PRINCIPAL', 'API_KEY']#

service_endpoint: str#

class pyagentspec.llms.ociclientconfig.OciClientConfigWithApiKey(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, service_endpoint, auth_type='API_KEY', auth_profile, auth_file_location)#

Bases: OciClientConfig

OCI client config class for authentication using API_KEY and a config file.

Parameters:

id (str) – A unique identifier for this Component
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
service_endpoint (str) –
auth_type (Literal['API_KEY']) –
auth_profile (str) –
auth_file_location (str) –

auth_file_location: str#

auth_profile: str#

auth_type: Literal['API_KEY']#

class pyagentspec.llms.ociclientconfig.OciClientConfigWithSecurityToken(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, service_endpoint, auth_type='SECURITY_TOKEN', auth_profile, auth_file_location)#

Bases: OciClientConfig

OCI client config class for authentication using SECURITY_TOKEN.

Parameters:

id (str) – A unique identifier for this Component
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
service_endpoint (str) –
auth_type (Literal['SECURITY_TOKEN']) –
auth_profile (str) –
auth_file_location (str) –

auth_file_location: str#

auth_profile: str#

auth_type: Literal['SECURITY_TOKEN']#

class pyagentspec.llms.ociclientconfig.OciClientConfigWithInstancePrincipal(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, service_endpoint, auth_type='INSTANCE_PRINCIPAL')#

Bases: OciClientConfig

OCI client config class for authentication using INSTANCE_PRINCIPAL.

Parameters:

id (str) – A unique identifier for this Component
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
service_endpoint (str) –
auth_type (Literal['INSTANCE_PRINCIPAL']) –

auth_type: Literal['INSTANCE_PRINCIPAL']#

class pyagentspec.llms.ociclientconfig.OciClientConfigWithResourcePrincipal(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, service_endpoint, auth_type='RESOURCE_PRINCIPAL')#

Bases: OciClientConfig

OCI client config class for authentication using RESOURCE_PRINCIPAL.

Parameters:

id (str) – A unique identifier for this Component
name (str) – Name of this Component
description (str | None) – Optional description of this Component
metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component
min_agentspec_version (AgentSpecVersionEnum) –
max_agentspec_version (AgentSpecVersionEnum) –
service_endpoint (str) –
auth_type (Literal['RESOURCE_PRINCIPAL']) –

auth_type: Literal['RESOURCE_PRINCIPAL']#