LLMs#

This page presents all APIs and classes related to LLM models.

LlmConfig#

class pyagentspec.llms.llmconfig.LlmConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, model_id, provider=None, api_provider=None, api_type=None, url=None, api_key=None, default_generation_parameters=None, retry_policy=None)#

Bases: Component

A LLM configuration defines how to connect to a LLM to do generation requests.

This class can be used directly with the provider, api_provider, and api_type fields to describe any LLM without a dedicated subclass. Concrete subclasses provide additional configuration for specific LLM providers.

Parameters:

  • id (str) – Identifier of the model to use, as expected by the selected API provider.

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • model_id (str) – Identifier of the model to use, as expected by the selected API provider.

  • provider (str | None) – The provider of the model (e.g. ‘meta’, ‘openai’, ‘cohere’).

  • api_provider (str | None) – The API provider used to serve the model (e.g. ‘openai’, ‘oci’, ‘vllm’).

  • api_type (str | None) – The API format to use (e.g. ‘chat_completions’, ‘responses’).

  • url (str | None) – URL of the API endpoint (e.g. ‘https://api.openai.com/v1’).

  • api_key (str | None) – An optional API key for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.

  • default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM

  • retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.

LLM Generation Config#

Parameters for LLM generation (max_tokens, temperature, top_p).

class pyagentspec.llms.llmgenerationconfig.LlmGenerationConfig(*, max_tokens=None, temperature=None, top_p=None, **extra_data)#

Bases: BaseModel

A configuration object defining LLM generation parameters.

Parameters include number of tokens, sampling parameters, etc.

Parameters:

  • max_tokens (int | None) –

  • temperature (float | None) –

  • top_p (float | None) –

  • extra_data (Any) –

max_tokens: int | None#

Maximum number of token that should be generated

temperature: float | None#

Value of the temperature parameter to be used for generation

top_p: float | None#

Value of the top_p parameter to be used for generation

OpenAI API Type#

class pyagentspec.llms.openaicompatibleconfig.OpenAIAPIType(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)#

Bases: str, Enum

Enumeration of OpenAI API Types.

chat completions: Chat Completions API responses: Responses API

CHAT_COMPLETIONS = 'chat_completions'#
RESPONSES = 'responses'#

OCI API Type#

class pyagentspec.llms.ocigenaiconfig.OciAPIType(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)#

Bases: str, Enum

Enumeration of API Types.

OCI = 'oci'#
OPENAI_CHAT_COMPLETIONS = 'openai_chat_completions'#
OPENAI_RESPONSES = 'openai_responses'#

All models#

OpenAI Compatible Models#

class pyagentspec.llms.openaicompatibleconfig.OpenAiCompatibleConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, model_id, provider=None, api_provider=None, api_type=OpenAIAPIType.CHAT_COMPLETIONS, url, api_key=None, default_generation_parameters=None, retry_policy=None, key_file=None, cert_file=None, ca_file=None)#

Bases: LlmConfig

Class to configure a connection to an LLM that is compatible with OpenAI completions APIs.

Requires to specify the url of the APIs to contact.

Parameters:

  • id (str) – ID of the model to use

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • model_id (str) – ID of the model to use

  • provider (str | None) – The provider of the model (e.g. ‘meta’, ‘openai’, ‘cohere’).

  • api_provider (str | None) – The API provider used to serve the model (e.g. ‘openai’, ‘oci’, ‘vllm’).

  • api_type (OpenAIAPIType) – OpenAI API protocol to use

  • url (str) – Url of the OpenAI compatible model deployment

  • api_key (str | None) – An optional API key for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.

  • default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM

  • retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.

  • key_file (str | None) – The path to an optional client private key file (PEM format).

  • cert_file (str | None) – The path to an optional client certificate chain file (PEM format).

  • ca_file (str | None) – The path to an optional trusted CA certificate file (PEM format) used to verify the server.

ca_file: str | None#

The path to an optional trusted CA certificate file (PEM format) used to verify the server.

cert_file: str | None#

The path to an optional client certificate chain file (PEM format).

key_file: str | None#

The path to an optional client private key file (PEM format).

VLLM Models#

class pyagentspec.llms.vllmconfig.VllmConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, model_id, provider=None, api_provider='vllm', api_type=OpenAIAPIType.CHAT_COMPLETIONS, url, api_key=None, default_generation_parameters=None, retry_policy=None, key_file=None, cert_file=None, ca_file=None)#

Bases: OpenAiCompatibleConfig

Class to configure a connection to a vLLM-hosted LLM.

Requires to specify the url at which the instance is running.

Parameters:

  • id (str) – ID of the model to use

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • model_id (str) – ID of the model to use

  • provider (str | None) – The API provider used to serve the model.

  • api_provider (Literal['vllm']) – The API provider used to serve the model.

  • api_type (OpenAIAPIType) – OpenAI API protocol to use

  • url (str) – Url of the OpenAI compatible model deployment

  • api_key (str | None) – An optional API key for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.

  • default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM

  • retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.

  • key_file (str | None) – The path to an optional client private key file (PEM format).

  • cert_file (str | None) – The path to an optional client certificate chain file (PEM format).

  • ca_file (str | None) – The path to an optional trusted CA certificate file (PEM format) used to verify the server.

Ollama Models#

class pyagentspec.llms.ollamaconfig.OllamaConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, model_id, provider=None, api_provider='ollama', api_type=OpenAIAPIType.CHAT_COMPLETIONS, url, api_key=None, default_generation_parameters=None, retry_policy=None, key_file=None, cert_file=None, ca_file=None)#

Bases: OpenAiCompatibleConfig

Class to configure a connection to a local model ran with Ollama.

Requires to specify the url and port at which the model is running.

Parameters:

  • id (str) – ID of the model to use

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • model_id (str) – ID of the model to use

  • provider (str | None) – The API provider used to serve the model.

  • api_provider (Literal['ollama']) – The API provider used to serve the model.

  • api_type (OpenAIAPIType) – OpenAI API protocol to use

  • url (str) – Url of the OpenAI compatible model deployment

  • api_key (str | None) – An optional API key for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.

  • default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM

  • retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.

  • key_file (str | None) – The path to an optional client private key file (PEM format).

  • cert_file (str | None) – The path to an optional client certificate chain file (PEM format).

  • ca_file (str | None) – The path to an optional trusted CA certificate file (PEM format) used to verify the server.

OpenAI Models#

class pyagentspec.llms.openaiconfig.OpenAiConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, model_id, provider='openai', api_provider='openai', api_type=OpenAIAPIType.CHAT_COMPLETIONS, url=None, api_key=None, default_generation_parameters=None, retry_policy=None)#

Bases: LlmConfig

Class to configure a connection to a OpenAI LLM.

Requires to specify the identity of the model to use.

Parameters:

  • id (str) – Identifier of the model to use, as expected by the selected API provider.

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • model_id (str) – Identifier of the model to use, as expected by the selected API provider.

  • provider (Literal['openai']) – The provider of the model.

  • api_provider (Literal['openai']) – The API provider used to serve the model.

  • api_type (OpenAIAPIType) – OpenAI API protocol to use

  • url (str | None) – URL of the API endpoint (e.g. ‘https://api.openai.com/v1’).

  • api_key (str | None) – An optional API key for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.

  • default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM

  • retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.

Gemini Models#

class pyagentspec.llms.geminiauthconfig.GeminiAuthConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v26_1_2, max_agentspec_version=AgentSpecVersionEnum.v26_2_0)#

Bases: Component

Base class for Gemini authentication configuration.

Parameters:

  • id (str) – A unique identifier for this Component

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

class pyagentspec.llms.geminiauthconfig.GeminiAIStudioAuthConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v26_1_2, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, api_key=None)#

Bases: GeminiAuthConfig

Authentication settings for Gemini via Google AI Studio.

Parameters:

  • id (str) – A unique identifier for this Component

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • api_key (str | None) – API key to use. If unset, runtimes may load it from GEMINI_API_KEY.

class pyagentspec.llms.geminiauthconfig.GeminiVertexAIAuthConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v26_1_2, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, project_id=None, location='global', credentials=None)#

Bases: GeminiAuthConfig

Authentication settings for Gemini via Google Vertex AI.

Parameters:

  • id (str) – Optional Google Cloud project identifier. This may still need to be set explicitly when the runtime cannot infer the project from Application Default Credentials (ADC) or other local Google Cloud configuration.

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • project_id (str | None) – Optional Google Cloud project identifier. This may still need to be set explicitly when the runtime cannot infer the project from Application Default Credentials (ADC) or other local Google Cloud configuration.

  • location (str) – Vertex AI location/region.

  • credentials (str | Dict[str, Any] | None) – Optional local file path to a Google Cloud JSON credential file, such as a service-account key file, or an inline dictionary containing the parsed JSON contents of that file. When unset, runtimes may rely on Application Default Credentials (ADC), such as GOOGLE_APPLICATION_CREDENTIALS, credentials made available through the local Google Cloud environment, or an attached service account. Even then, project_id may still need to be provided separately if it cannot be resolved from the environment.

credentials: str | Dict[str, Any] | None#

Optional local file path to a Google Cloud JSON credential file, such as a service-account key file, or an inline dictionary containing the parsed JSON contents of that file.

When unset, runtimes may rely on Application Default Credentials (ADC), such as GOOGLE_APPLICATION_CREDENTIALS, credentials made available through the local Google Cloud environment, or an attached service account. Even then, project_id may still need to be provided separately if it cannot be resolved from the environment.

location: str#

Vertex AI location/region.

project_id: str | None#

Optional Google Cloud project identifier.

This may still need to be set explicitly when the runtime cannot infer the project from Application Default Credentials (ADC) or other local Google Cloud configuration.

class pyagentspec.llms.geminiconfig.GeminiConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v26_1_2, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, model_id, provider='google', api_provider=None, api_type=None, url=None, api_key=None, default_generation_parameters=None, retry_policy=None, auth)#

Bases: LlmConfig

Configure a connection to a Gemini LLM (AI Studio or Vertex AI).

Parameters:

  • id (str) – Identifier of the model to use, as expected by the selected API provider.

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • model_id (str) – Identifier of the model to use, as expected by the selected API provider.

  • provider (Literal['google']) – The provider of the model.

  • api_provider (str | None) – The API provider used to serve the model (e.g. ‘openai’, ‘oci’, ‘vllm’).

  • api_type (str | None) – The API format to use (e.g. ‘chat_completions’, ‘responses’).

  • url (str | None) – URL of the API endpoint (e.g. ‘https://api.openai.com/v1’).

  • api_key (str | None) – An optional API key for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.

  • default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM

  • retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.

  • auth (GeminiAuthConfig) – Authentication configuration used to connect to the Gemini service.

auth: GeminiAuthConfig#

Authentication configuration used to connect to the Gemini service.

OciGenAi Models#

class pyagentspec.llms.ocigenaiconfig.ServingMode(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)#

Bases: str, Enum

Serving mode to use for the GenAI service

DEDICATED = 'DEDICATED'#
ON_DEMAND = 'ON_DEMAND'#
class pyagentspec.llms.ocigenaiconfig.ModelProvider(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)#

Bases: str, Enum

Provider of the model. It is used to ensure the requests to this model respect the format expected by the provider.

COHERE = 'COHERE'#
GROK = 'GROK'#
META = 'META'#
OTHER = 'OTHER'#
XAI = 'XAI'#
class pyagentspec.llms.ocigenaiconfig.OciGenAiConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, model_id, provider=None, api_provider='oci', api_type=OciAPIType.OCI, url=None, api_key=None, default_generation_parameters=None, retry_policy=None, compartment_id, serving_mode=ServingMode.ON_DEMAND, client_config, conversation_store_id=None)#

Bases: LlmConfig

Class to configure a connection to a OCI GenAI hosted model.

Requires to specify the model id and the client configuration to the OCI GenAI service.

Parameters:

  • id (str) – The OCI compartment ID where the model is hosted.

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • model_id (str) – Identifier of the model to use, as expected by the selected API provider.

  • provider (ModelProvider | None) – The API provider used to serve the model.

  • api_provider (Literal['oci']) – The API provider used to serve the model.

  • api_type (OciAPIType) – API protocol to use.

  • url (str | None) – URL of the API endpoint (e.g. ‘https://api.openai.com/v1’).

  • api_key (str | None) – An optional API key for the remote LLM model. If specified, the value of the api_key will be excluded and replaced by a reference when exporting the configuration.

  • default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM

  • retry_policy (RetryPolicy | None) – Optional retry configuration for remote LLM calls.

  • compartment_id (str) – The OCI compartment ID where the model is hosted.

  • serving_mode (ServingMode) – The serving mode for the model.

  • client_config (OciClientConfig) – The client configuration for connecting to OCI GenAI service.

  • conversation_store_id (str | None) – ID of the conversation store to persist conversations when using the openai responses API.

client_config: OciClientConfig#

The client configuration for connecting to OCI GenAI service.

compartment_id: str#

The OCI compartment ID where the model is hosted.

conversation_store_id: str | None#

ID of the conversation store to persist conversations when using the openai responses API.

serving_mode: ServingMode#

The serving mode for the model.

class pyagentspec.llms.ociclientconfig.OciClientConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, service_endpoint, auth_type)#

Bases: Component

Base abstract class for OCI client config.

Parameters:

  • id (str) – A unique identifier for this Component

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • service_endpoint (str) –

  • auth_type (Literal['SECURITY_TOKEN', 'INSTANCE_PRINCIPAL', 'RESOURCE_PRINCIPAL', 'API_KEY']) –

auth_type: Literal['SECURITY_TOKEN', 'INSTANCE_PRINCIPAL', 'RESOURCE_PRINCIPAL', 'API_KEY']#
service_endpoint: str#
class pyagentspec.llms.ociclientconfig.OciClientConfigWithApiKey(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, service_endpoint, auth_type='API_KEY', auth_profile, auth_file_location)#

Bases: OciClientConfig

OCI client config class for authentication using API_KEY and a config file.

Parameters:

  • id (str) – A unique identifier for this Component

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • service_endpoint (str) –

  • auth_type (Literal['API_KEY']) –

  • auth_profile (str) –

  • auth_file_location (str) –

auth_file_location: str#
auth_profile: str#
auth_type: Literal['API_KEY']#
class pyagentspec.llms.ociclientconfig.OciClientConfigWithSecurityToken(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, service_endpoint, auth_type='SECURITY_TOKEN', auth_profile, auth_file_location)#

Bases: OciClientConfig

OCI client config class for authentication using SECURITY_TOKEN.

Parameters:

  • id (str) – A unique identifier for this Component

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • service_endpoint (str) –

  • auth_type (Literal['SECURITY_TOKEN']) –

  • auth_profile (str) –

  • auth_file_location (str) –

auth_file_location: str#
auth_profile: str#
auth_type: Literal['SECURITY_TOKEN']#
class pyagentspec.llms.ociclientconfig.OciClientConfigWithInstancePrincipal(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, service_endpoint, auth_type='INSTANCE_PRINCIPAL')#

Bases: OciClientConfig

OCI client config class for authentication using INSTANCE_PRINCIPAL.

Parameters:

  • id (str) – A unique identifier for this Component

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • service_endpoint (str) –

  • auth_type (Literal['INSTANCE_PRINCIPAL']) –

auth_type: Literal['INSTANCE_PRINCIPAL']#
class pyagentspec.llms.ociclientconfig.OciClientConfigWithResourcePrincipal(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v26_2_0, service_endpoint, auth_type='RESOURCE_PRINCIPAL')#

Bases: OciClientConfig

OCI client config class for authentication using RESOURCE_PRINCIPAL.

Parameters:

  • id (str) – A unique identifier for this Component

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • service_endpoint (str) –

  • auth_type (Literal['RESOURCE_PRINCIPAL']) –

auth_type: Literal['RESOURCE_PRINCIPAL']#
On this page