bat.chat_model_client.client

ChatModelClient Objects

class ChatModelClient()

Client that facilitates interaction with a chat model.

This client can be used to send user instructions to the chat model and receive responses. It supports both single and batch invocations, and can handle tool calls if tools are provided.

Token usage and timing are captured through OpenTelemetry spans (the OpenInference auto-instrumentation of the underlying chat model), not by this client; see bat.telemetry.

Arguments:

chat_model_config (ChatModelClientConfig, optional): Configuration for the chat model client. system_instructions (str): System instructions to be used in the chat model. tools (Sequence[Dict[str, Any] | type | ... | None], optional): LangChain-defined tools to be used by the chat model.

Examples:

config = ChatModelClientConfig.load(
    client_name="SampleClient",
)
client = ChatModelClient(
    chat_model_config=config,
    system_instructions="You always reply in pirate language.",
)
response = client.invoke(HumanMessage("What is the weather like today?"))

__init__

def __init__(chat_model_config: ChatModelClientConfig | None = None,
             system_instructions: str = "You are a helpful assistant.",
             tools: Sequence[Dict[str, Any] | type | Callable | BaseTool
                             | None] = None,
             output_schema: Optional[type[BaseModel]] = None)

Initialize the ChatModelClient with the given configuration, system instructions, and tools.

Arguments:

chat_model_config (ChatModelClientConfig, optional): Configuration for the chat model client. If None, it will be loaded from environment variables. system_instructions (str): System instructions to be used by the chat model. tools (Sequence[Dict[str, Any] | ... | BaseTool | None], optional): LangChain-defined tools to be used by the chat model. output_schema (type[BaseModel], optional): If provided, the chat model response will be parsed according to this Pydantic schema. The raw response from the chat model will be included in the output as well.

Raises:

• EnvironmentError - If the chat model configuration is not provided and cannot be loaded from environment variables.

chat_model

@property
def chat_model() -> BaseChatModel

The chat model instance configured with the provided model and tools.

invoke

def invoke(
        input: str | HumanMessage | List[ToolMessage],
        history: Optional[List[BaseMessage]] = None) -> Union[AIMessage, Any]

Invoke the chat model with user instructions or tool call results.

If the history is provided, it will be prepended to the input message. This method modifies the history in-place to include the input and output messages.

Arguments:

• input str | HumanMessage | List[ToolMessage] - The user input or tool call results to process.
• history Optional[List[BaseMessage]] - Optional history of messages.

Returns:

AIMessage | Any: The response from the chat model.

Raises:

• ValueError - If the input/output type is invalid, if the output was truncated, or if there is an error parsing the response according to the output schema.
• KeyError - If the expected keys are not found in the response when the output schema is used.
• ValidationError - If the parsed response does not conform to the output schema.

ainvoke

async def ainvoke(
        input: str | HumanMessage | List[ToolMessage],
        history: Optional[List[BaseMessage]] = None) -> Union[AIMessage, Any]

Asynchronous invoke: same arguments, result, errors and history update, but it awaits the chat model instead of blocking the event loop.

batch

def batch(inputs: List[HumanMessage],
          history: Optional[List[BaseMessage]] = None) -> List[AIMessage]

Batch process multiple human messages in batch.

If the history is provided, it will be prepended to each input message. This method does NOT modify the history in-place.

Arguments:

• inputs List[HumanMessage] - List of user inputs to process.
• history Optional[List[BaseMessage]] - Optional history of messages.

Returns:

• List[AIMessage] - List of responses from the chat model for each input.

Raises:

• ValueError - If the input type is invalid, if the response from the chat model is not an AIMessage, or if an output was truncated.

bat.chat_model_client.config

ModelProvider

ModelProvider = Literal[
    "anthropic",
    "deepseek",
    "nvidia",
    "ollama",
    "openai",
]

ModelProvider is a type alias for the supported model providers. The currently supported providers are:

• anthropic
• deepseek
• nvidia
• ollama
• openai

ChatModelClientConfig Objects

class ChatModelClientConfig(BaseModel)

Configuration for the chat model client.

This class is used to configure the chat model client with the necessary parameters. Some model providers may require specific environment variables to be set, like OPENAI_API_KEY for OpenAI.

Attributes

model (str): The name of the model to use.
model_provider (ModelProvider): The provider of the model
    (e.g., OpenAI, Meta, etc.).
base_url (str, optional): The base URL for the model provider, required
    for non-OpenAI providers.
client_name (str, optional): Name for the client.
reasoning_effort (str, optional): How much the model reasons before
    answering, e.g. "low". Sent only to providers that support it.
service_tier (str, optional): Processing tier for the requests, e.g.
    "flex". Sent only to providers that support it.

The class can be instantiated directly or created from environment variables using the load class method (usually preferred).

Examples

Direct instantiation:

config = ChatModelClientConfig(
    model="gpt-4o-mini",
    model_provider="openai",
    base_url="https://api.openai.com/v1",
    client_name="SampleClient",
)

From environment variables:

config = ChatModelClientConfig.load(
    client_name="SampleClient",
)

__init__

def __init__(model: str,
             model_provider: ModelProvider,
             base_url: Optional[str] = None,
             client_name: Optional[str] = None,
             reasoning_effort: Optional[str] = None,
             service_tier: Optional[str] = None)

Initialize the ChatModelClientConfig with the provided parameters.

Arguments:

• model str - The name of the model to use.
• model_provider ModelProvider - The provider of the model (e.g., openai, nvidia, etc.).
• base_url Optional[str] - The base URL for the model provider, required for non-OpenAI providers.
• client_name Optional[str] - Name for the client.
• reasoning_effort Optional[str] - How much the model reasons before answering, e.g. "low".
• service_tier Optional[str] - Processing tier for the requests, e.g. "flex".

load

@classmethod
def load(cls, client_name: Optional[str] = None) -> "ChatModelClientConfig"

Create a ChatModelClientConfig from environment variables, falling back to the config.yaml.

Resolution follows the precedence env > config.yaml:

• MODEL: model name (or <provider>:<model>); falls back to model.name from config.yaml.
• MODEL_PROVIDER: provider (e.g. openai, ollama); falls back to the provider in MODEL or model.provider from config.yaml.
• BASE_URL: optional base URL; falls back to model.base_url.

model.reasoning_effort and model.service_tier come from config.yaml only.

Arguments:

• client_name Optional[str] - Name for the client.

Returns:

An instance of ChatModelClientConfig.

Raises:

• EnvironmentError - If neither the environment nor config.yaml provide the model name or the provider.

from_env

@classmethod
def from_env(cls,
             client_name: Optional[str] = None) -> "ChatModelClientConfig"

Backward-compatible alias for :meth:load.

from_env was the original name of this constructor; it is retained so existing callers keep working. New code should call :meth:load, which this delegates to verbatim (same environment/config.yaml precedence and the same raised errors).

Arguments:

• client_name Optional[str] - Name for the client.

Returns:

An instance of ChatModelClientConfig.