bat.chat_model_client.config

ModelProvider

ModelProvider is a type alias for the supported model providers. The currently supported providers are:

• anthropic
• deepseek
• nvidia
• ollama
• openai

ChatModelClientConfig Objects

class ChatModelClientConfig(BaseModel)

Configuration for the chat model client.

This class is used to configure the chat model client with the necessary parameters. Some model providers may require specific environment variables to be set, like OPENAI_API_KEY for OpenAI.

Attributes

model (str): The name of the model to use.
model_provider (ModelProvider): The provider of the model
    (e.g., OpenAI, Meta, etc.).
base_url (str, optional): The base URL for the model provider, required
    for non-OpenAI providers.
client_name (str, optional): Name for the client.

The class can be instantiated directly or created from environment variables using the from_env class method (usually preferred).

Examples

Direct instantiation:

config = ChatModelClientConfig(
    model="gpt-4o-mini",
    model_provider="openai",
    base_url="https://api.openai.com/v1",
    client_name="SampleClient",
)

From environment variables:

config = ChatModelClientConfig.from_env(
    client_name="SampleClient",
)

__init__

def __init__(model: str,
             model_provider: ModelProvider,
             base_url: Optional[str] = None,
             client_name: Optional[str] = None)

Initialize the ChatModelClientConfig with the provided parameters.

Arguments:

• model str - The name of the model to use.
• model_provider ModelProvider - The provider of the model (e.g., openai, nvidia, etc.).
• base_url Optional[str] - The base URL for the model provider, required for non-OpenAI providers.
• client_name Optional[str] - Name for the client.

from_env

@classmethod
def from_env(cls,
             client_name: Optional[str] = None) -> "ChatModelClientConfig"

Create a ChatModelClientConfig instance from environment variables.

This method reads the following environment variables:

• MODEL: The model name, which can be in the format <provider>:<model>.
• MODEL_PROVIDER (optional): The provider of the model (e.g., openai, nvidia, ollama, etc.).

Arguments:

• client_name Optional[str] - Name for the client.

Returns:

An instance of ChatModelClientConfig configured with values from environment variables.

Raises:

• EnvironmentError - If the required environment variables are not set or if the format is incorrect.

bat.chat_model_client.client

ChatModelClient Objects

class ChatModelClient()

Client that facilitates interaction with a chat model.

This client can be used to send user instructions to the chat model and receive responses. It supports both single and batch invocations, and can handle tool calls if tools are provided.

If stored as a property of an object deriving the AgentGraph class, UsageMetadata will be automatically collected and returned as metadata of the streaming response.

Arguments:

chat_model_config (ChatModelClientConfig, optional): Configuration for the chat model client. system_instructions (str): System instructions to be used in the chat model. tools (Sequence[Dict[str, Any] | type | ... | None], optional): LangChain-defined tools to be used by the chat model.

Examples:

config = ChatModelClientConfig.from_env(
    client_name="SampleClient",
)
client = ChatModelClient(
    chat_model_config=config,
    system_instructions="You always reply in pirate language.",
)
response = client.invoke(HumanMessage("What is the weather like today?"))

__init__

def __init__(chat_model_config: ChatModelClientConfig | None = None,
             system_instructions: str = "You are a helpful assistant.",
             tools: Sequence[Dict[str, Any] | type | Callable | BaseTool
                             | None] = None,
             output_schema: Optional[type[BaseModel]] = None)

Initialize the ChatModelClient with the given configuration, system instructions, and tools.

Arguments:

chat_model_config (ChatModelClientConfig, optional): Configuration for the chat model client. If None, it will be loaded from environment variables. system_instructions (str): System instructions to be used by the chat model. tools (Sequence[Dict[str, Any] | ... | BaseTool | None], optional): LangChain-defined tools to be used by the chat model. output_schema (type[BaseModel], optional): If provided, the chat model response will be parsed according to this Pydantic schema. The raw response from the chat model will be included in the output as well.

Raises:

• EnvironmentError - If the chat model configuration is not provided and cannot be loaded from environment variables.

chat_model

@property
def chat_model() -> BaseChatModel

The chat model instance configured with the provided model and tools.

invoke

def invoke(
        input: str | HumanMessage | List[ToolMessage],
        history: Optional[List[BaseMessage]] = None) -> Union[AIMessage, Any]

Invoke the chat model with user instructions or tool call results.

If the history is provided, it will be prepended to the input message. This method modifies the history in-place to include the input and output messages.

Arguments:

• input str | HumanMessage | List[ToolMessage] - The user input or tool call results to process.
• history Optional[List[BaseMessage]] - Optional history of messages.

Returns:

AIMessage | Any: The response from the chat model.

Raises:

• ValueError - If the input/output type is invalid or if there is an error parsing the response according to the output schema.
• KeyError - If the expected keys are not found in the response when the output schema is used.
• ValidationError - If the parsed response does not conform to the output schema.

batch

def batch(inputs: List[HumanMessage],
          history: Optional[List[BaseMessage]] = None) -> List[AIMessage]

Batch process multiple human messages in batch.

If the history is provided, it will be prepended to each input message. This method does NOT modify the history in-place.

Arguments:

• inputs List[HumanMessage] - List of user inputs to process.
• history Optional[List[BaseMessage]] - Optional history of messages.

Returns:

• List[AIMessage] - List of responses from the chat model for each input.

Raises:

• ValueError - If the input type is invalid or if the response from the chat model is not an AIMessage.

get_usage_metadata

def get_usage_metadata(
        from_timestamp: Optional[float] = None) -> UsageMetadata

Get the aggregated usage metadata from the chat model client.

Arguments:

• from_timestamp Optional[float] - If provided, only usage metadata after this timestamp will be considered. If None, all usage metadata will be considered.

Returns:

• UsageMetadata - The aggregated usage metadata.