LLMs#

This page presents all APIs and classes related to LLM models.

LlmConfig#

class pyagentspec.llms.llmconfig.LlmConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v25_4_1, default_generation_parameters=None)#

Bases: Component

A LLM configuration defines how to connect to a LLM to do generation requests.

This class provides the base class, while concrete classes provide the required configuration to connect to specific LLM providers.

Parameters:

  • id (str) – A unique identifier for this Component

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM

LLM Generation Config#

Parameters for LLM generation (max_tokens, temperature, top_p).

class pyagentspec.llms.llmgenerationconfig.LlmGenerationConfig(*, max_tokens=None, temperature=None, top_p=None, **extra_data)#

Bases: BaseModel

A configuration object defining LLM generation parameters.

Parameters include number of tokens, sampling parameters, etc.

Parameters:

  • max_tokens (int | None) –

  • temperature (float | None) –

  • top_p (float | None) –

  • extra_data (Any) –

max_tokens: int | None#

Maximum number of token that should be generated

temperature: float | None#

Value of the temperature parameter to be used for generation

top_p: float | None#

Value of the top_p parameter to be used for generation

All models#

OpenAI Compatible Models#

class pyagentspec.llms.openaicompatibleconfig.OpenAiCompatibleConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v25_4_1, default_generation_parameters=None, url, model_id)#

Bases: LlmConfig

Class to configure a connection to an LLM that is compatible with OpenAI completions APIs.

Requires to specify the url of the APIs to contact.

Parameters:

  • id (str) – ID of the model to use

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM

  • url (str) – Url of the OpenAI compatible model deployment

  • model_id (str) – ID of the model to use

model_id: str#

ID of the model to use

url: str#

Url of the OpenAI compatible model deployment

VLLM Models#

class pyagentspec.llms.vllmconfig.VllmConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v25_4_1, default_generation_parameters=None, url, model_id)#

Bases: OpenAiCompatibleConfig

Class to configure a connection to a vLLM-hosted LLM.

Requires to specify the url at which the instance is running.

Parameters:

  • id (str) – ID of the model to use

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM

  • url (str) – Url of the OpenAI compatible model deployment

  • model_id (str) – ID of the model to use

Ollama Models#

class pyagentspec.llms.ollamaconfig.OllamaConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v25_4_1, default_generation_parameters=None, url, model_id)#

Bases: OpenAiCompatibleConfig

Class to configure a connection to a local model ran with Ollama.

Requires to specify the url and port at which the model is running.

Parameters:

  • id (str) – ID of the model to use

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM

  • url (str) – Url of the OpenAI compatible model deployment

  • model_id (str) – ID of the model to use

OpenAI Models#

class pyagentspec.llms.openaiconfig.OpenAiConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v25_4_1, default_generation_parameters=None, model_id)#

Bases: LlmConfig

Class to configure a connection to a OpenAI LLM.

Requires to specify the identity of the model to use.

Parameters:

  • id (str) – ID of the model to use

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM

  • model_id (str) – ID of the model to use

model_id: str#

ID of the model to use

OciGenAi Models#

class pyagentspec.llms.ocigenaiconfig.ServingMode(value)#

Bases: str, Enum

An enumeration.

DEDICATED = 'DEDICATED'#
ON_DEMAND = 'ON_DEMAND'#
class pyagentspec.llms.ocigenaiconfig.ModelProvider(value)#

Bases: str, Enum

Provider of the model. It is used to ensure the requests to this model respect the format expected by the provider.

COHERE = 'COHERE'#
GROK = 'GROK'#
META = 'META'#
OTHER = 'OTHER'#
class pyagentspec.llms.ocigenaiconfig.OciGenAiConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v25_4_1, default_generation_parameters=None, model_id, compartment_id, serving_mode=ServingMode.ON_DEMAND, provider=None, client_config)#

Bases: LlmConfig

Class to configure a connection to a OCI GenAI hosted model.

Requires to specify the model id and the client configuration to the OCI GenAI service.

Parameters:

  • id (str) – The identifier of the model to use.

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • default_generation_parameters (LlmGenerationConfig | None) – Parameters used for the generation call of this LLM

  • model_id (str) – The identifier of the model to use.

  • compartment_id (str) – The OCI compartment ID where the model is hosted.

  • serving_mode (ServingMode) – The serving mode for the model.

  • provider (ModelProvider | None) – The provider of the model. If None, it will be automatically detected by the runtime using the model ID

  • client_config (OciClientConfig) – The client configuration for connecting to OCI GenAI service.

client_config: OciClientConfig#

The client configuration for connecting to OCI GenAI service.

compartment_id: str#

The OCI compartment ID where the model is hosted.

model_id: str#

The identifier of the model to use.

provider: ModelProvider | None#

The provider of the model. If None, it will be automatically detected by the runtime using the model ID

serving_mode: ServingMode#

The serving mode for the model.

class pyagentspec.llms.ociclientconfig.OciClientConfig(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v25_4_1, service_endpoint, auth_type)#

Bases: Component

Base abstract class for OCI client config.

Parameters:

  • id (str) – A unique identifier for this Component

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • service_endpoint (str) –

  • auth_type (Literal['SECURITY_TOKEN', 'INSTANCE_PRINCIPAL', 'RESOURCE_PRINCIPAL', 'API_KEY']) –

auth_type: Literal['SECURITY_TOKEN', 'INSTANCE_PRINCIPAL', 'RESOURCE_PRINCIPAL', 'API_KEY']#
service_endpoint: str#
class pyagentspec.llms.ociclientconfig.OciClientConfigWithApiKey(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v25_4_1, service_endpoint, auth_type='API_KEY', auth_profile, auth_file_location)#

Bases: OciClientConfig

OCI client config class for authentication using API_KEY and a config file.

Parameters:

  • id (str) – A unique identifier for this Component

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • service_endpoint (str) –

  • auth_type (Literal['API_KEY']) –

  • auth_profile (str) –

  • auth_file_location (str) –

auth_file_location: str#
auth_profile: str#
auth_type: Literal['API_KEY']#
class pyagentspec.llms.ociclientconfig.OciClientConfigWithSecurityToken(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v25_4_1, service_endpoint, auth_type='SECURITY_TOKEN', auth_profile, auth_file_location)#

Bases: OciClientConfig

OCI client config class for authentication using SECURITY_TOKEN.

Parameters:

  • id (str) – A unique identifier for this Component

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • service_endpoint (str) –

  • auth_type (Literal['SECURITY_TOKEN']) –

  • auth_profile (str) –

  • auth_file_location (str) –

auth_file_location: str#
auth_profile: str#
auth_type: Literal['SECURITY_TOKEN']#
class pyagentspec.llms.ociclientconfig.OciClientConfigWithInstancePrincipal(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v25_4_1, service_endpoint, auth_type='INSTANCE_PRINCIPAL')#

Bases: OciClientConfig

OCI client config class for authentication using INSTANCE_PRINCIPAL.

Parameters:

  • id (str) – A unique identifier for this Component

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • service_endpoint (str) –

  • auth_type (Literal['INSTANCE_PRINCIPAL']) –

auth_type: Literal['INSTANCE_PRINCIPAL']#
class pyagentspec.llms.ociclientconfig.OciClientConfigWithResourcePrincipal(*, id=<factory>, name, description=None, metadata=<factory>, min_agentspec_version=AgentSpecVersionEnum.v25_4_1, max_agentspec_version=AgentSpecVersionEnum.v25_4_1, service_endpoint, auth_type='RESOURCE_PRINCIPAL')#

Bases: OciClientConfig

OCI client config class for authentication using RESOURCE_PRINCIPAL.

Parameters:

  • id (str) – A unique identifier for this Component

  • name (str) – Name of this Component

  • description (str | None) – Optional description of this Component

  • metadata (Dict[str, Any] | None) – Optional, additional metadata related to this Component

  • min_agentspec_version (AgentSpecVersionEnum) –

  • max_agentspec_version (AgentSpecVersionEnum) –

  • service_endpoint (str) –

  • auth_type (Literal['RESOURCE_PRINCIPAL']) –

auth_type: Literal['RESOURCE_PRINCIPAL']#