Code
app.agents.agent_system
¶
Agent system utilities for orchestrating multi-agent workflows.
This module provides functions and helpers to create, configure, and run agent systems using Pydantic AI. It supports delegation of tasks to research, analysis, and synthesis agents, and manages agent configuration, environment setup, and execution. Args: provider (str): The name of the provider. provider_config (ProviderConfig): Configuration settings for the provider. api_key (str): API key for authentication with the provider. prompts (dict[str, str]): Configuration for prompts. include_researcher (bool): Flag to include the researcher agent. include_analyst (bool): Flag to include the analyst agent. include_synthesiser (bool): Flag to include the synthesiser agent. query (str | list[dict[str, str]]): The query or messages for the agent. chat_config (ChatConfig): The configuration object for agents and providers. usage_limits (UsageLimits): Usage limits for agent execution. pydantic_ai_stream (bool): Whether to use Pydantic AI streaming.
Functions:
Name | Description |
---|---|
get_manager |
Initializes and returns a manager agent with the specified configuration. |
run_manager |
Asynchronously runs the manager agent with the given query and provider. |
setup_agent_env |
Sets up the environment for an agent by configuring provider settings, prompts, API key, and usage limits. |
Classes¶
Functions¶
get_manager(provider, provider_config, api_key, prompts, include_researcher=False, include_analyst=False, include_synthesiser=False)
¶
Initializes and returns a Agent manager with the specified configuration. Args: provider (str): The name of the provider. provider_config (ProviderConfig): Configuration settings for the provider. api_key (str): API key for authentication with the provider. prompts (PromptsConfig): Configuration for prompts. include_researcher (bool, optional): Flag to include analyst model. Defaults to False. include_analyst (bool, optional): Flag to include analyst model. Defaults to False. include_synthesiser (bool, optional): Flag to include synthesiser model. Defaults to False. Returns: Agent: The initialized Agent manager.
Source code in src/app/agents/agent_system.py
run_manager(manager, query, provider, usage_limits, pydantic_ai_stream=False)
async
¶
Asynchronously runs the manager with the given query and provider, handling errors and printing results. Args: manager (Agent): The system agent responsible for running the query. query (str): The query to be processed by the manager. provider (str): The provider to be used for the query. usage_limits (UsageLimits): The usage limits to be applied during the query execution. pydantic_ai_stream (bool, optional): Flag to enable or disable Pydantic AI stream. Defaults to False. Returns: None
Source code in src/app/agents/agent_system.py
setup_agent_env(provider, query, chat_config, chat_env_config)
¶
Sets up the environment for an agent by configuring provider settings, prompts, API key, and usage limits.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
provider
|
str
|
The name of the provider. |
required |
query
|
UserPromptType
|
The messages or queries to be sent to the agent. |
required |
chat_config
|
ChatConfig | BaseModel
|
The configuration object containing provider and prompt settings. |
required |
chat_env_config
|
AppEnv
|
The application environment configuration containing API keys. |
required |
Returns:
Name | Type | Description |
---|---|---|
EndpointConfig |
EndpointConfig
|
The configuration object for the agent. |
Source code in src/app/agents/agent_system.py
app.agents.llm_model_funs
¶
Utility functions and classes for managing and instantiating LLM models and providers.
This module provides functions to retrieve API keys, provider configurations, and to create model instances for supported LLM providers such as Gemini and OpenAI. It also includes logic for assembling model dictionaries for system agents.
Classes¶
Functions¶
get_api_key(provider, chat_env_config)
¶
Retrieve API key from chat env config variable.
Source code in src/app/agents/llm_model_funs.py
get_models(endpoint_config, include_researcher=False, include_analyst=False, include_synthesiser=False)
¶
Get the models for the system agents. Args: endpoint_config (EndpointConfig): Configuration for the model. include_analyist (Optional[bool]): Whether to include the analyst model. Defaults to False. include_synthesiser (Optional[bool]): Whether to include the synthesiser model. Defaults to False. Returns: Dict[str, GeminiModel | OpenAIModel]: A dictionary containing the models for the system agents.
Source code in src/app/agents/llm_model_funs.py
get_provider_config(provider, providers)
¶
Retrieve configuration settings for the specified provider.
Source code in src/app/agents/llm_model_funs.py
app.config.config_app
¶
Configuration constants for the application.
app.config.data_models
¶
Data models for agent system configuration and results.
This module defines Pydantic models for representing research and analysis results, summaries, provider and agent configurations, and model dictionaries used throughout the application. These models ensure type safety and validation for data exchanged between agents and system components.
Classes¶
AgentConfig
¶
Bases: BaseModel
Configuration for an agent
Source code in src/app/config/data_models.py
AnalysisResult
¶
AppEnv
¶
Bases: BaseSettings
Application environment settings loaded from environment variables or .env file.
This class uses Pydantic’s BaseSettings to manage API keys and configuration for various inference endpoints, tools, and logging/monitoring services. Environment variables are loaded from a .env file by default.
Source code in src/app/config/data_models.py
ChatConfig
¶
Bases: BaseModel
Configuration settings for agents and model providers
Source code in src/app/config/data_models.py
EndpointConfig
¶
Bases: BaseModel
Configuration for an agent
Source code in src/app/config/data_models.py
ModelDict
¶
Bases: BaseModel
Dictionary of models used to create agent systems
Source code in src/app/config/data_models.py
ProviderConfig
¶
ResearchResult
¶
Bases: BaseModel
Research results from the research agent.
Source code in src/app/config/data_models.py
ResearchSummary
¶
Bases: BaseModel
Expected model response of research on a topic
Source code in src/app/config/data_models.py
app.evals.metrics
¶
Functions¶
output_similarity(agent_output, expected_answer)
¶
Determine to what degree the agent’s output matches the expected answer.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
agent_output
|
str
|
The output produced by the agent. |
required |
expected_answer
|
str
|
The correct or expected answer. |
required |
Returns:
Name | Type | Description |
---|---|---|
bool |
bool
|
True if the output matches the expected answer, False otherwise. |
Source code in src/app/evals/metrics.py
time_taken(start_time, end_time)
¶
Calculate duration between start and end timestamps
Parameters:
Name | Type | Description | Default |
---|---|---|---|
start_time
|
float
|
Timestamp when execution started |
required |
end_time
|
float
|
Timestamp when execution completed |
required |
Returns:
Type | Description |
---|---|
float
|
Duration in seconds with microsecond precision |
Source code in src/app/evals/metrics.py
app.main
¶
Main entry point for the Agents-eval application.
This module initializes the agentic system, loads configuration files, handles user input, and orchestrates the multi-agent workflow using asynchronous execution. It integrates logging, tracing, and authentication, and supports both CLI and programmatic execution.
Classes¶
Functions¶
main(chat_provider=CHAT_DEFAULT_PROVIDER, query='', include_researcher=False, include_analyst=False, include_synthesiser=False, pydantic_ai_stream=False, chat_config_file=CHAT_CONFIG_FILE)
async
¶
Main entry point for the application.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
chat_provider
|
str
|
The inference chat_provider to be used. |
CHAT_DEFAULT_PROVIDER
|
query
|
str
|
The query to be processed by the agent. |
''
|
include_researcher
|
bool
|
Whether to include the researcher in the process. |
False
|
include_analyst
|
bool
|
Whether to include the analyst in the process. |
False
|
include_synthesiser
|
bool
|
Whether to include the synthesiser in the process. |
False
|
pydantic_ai_stream
|
bool
|
Whether to use Pydantic AI streaming. |
False
|
chat_config_file
|
str
|
Full path to the configuration file. |
CHAT_CONFIG_FILE
|
Returns:
Type | Description |
---|---|
None
|
None |
Source code in src/app/main.py
app.utils.error_messages
¶
Error message utilities for the Agents-eval application.
This module provides concise helper functions for generating standardized error messages related to configuration loading and validation.
Functions¶
api_connection_error(error)
¶
failed_to_load_config(error)
¶
file_not_found(file_path)
¶
generic_exception(error)
¶
get_key_error(error)
¶
invalid_data_model_format(error)
¶
Generate an error message for invalid pydantic data model format.
invalid_json(error)
¶
invalid_type(expected_type, actual_type)
¶
app.utils.load_configs
¶
Configuration loading utilities.
Provides a generic function for loading and validating JSON configuration files against Pydantic models, with error handling and logging support.
Functions¶
load_config(config_path, data_model)
¶
Generic configuration loader that validates against any Pydantic model.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config_path
|
str | Path
|
Path to the JSON configuration file |
required |
model
|
Pydantic model class for validation |
required |
Returns:
Type | Description |
---|---|
BaseModel
|
Validated configuration instance |
Source code in src/app/utils/load_configs.py
app.utils.load_settings
¶
Utility functions and classes for loading application settings and configuration.
This module defines the AppEnv class for managing environment variables using Pydantic, and provides a function to load and validate application configuration from a JSON file.
Classes¶
AppEnv
¶
Bases: BaseSettings
Application environment settings loaded from environment variables or .env file.
This class uses Pydantic’s BaseSettings to manage API keys and configuration for various inference endpoints, tools, and logging/monitoring services. Environment variables are loaded from a .env file by default.
Source code in src/app/utils/load_settings.py
Functions¶
load_config(config_path)
¶
Load and validate application configuration from a JSON file.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config_path
|
str
|
Path to the JSON configuration file. |
required |
Returns:
Name | Type | Description |
---|---|---|
ChatConfig |
ChatConfig
|
An instance of ChatConfig with validated configuration data. |
Raises:
Type | Description |
---|---|
FileNotFoundError
|
If the configuration file does not exist. |
JSONDecodeError
|
If the file contains invalid JSON. |
Exception
|
For any other unexpected errors during loading or validation. |
Source code in src/app/utils/load_settings.py
app.utils.log
¶
Set up the logger with custom settings. Logs are written to a file with automatic rotation.
app.utils.login
¶
This module provides utility functions for managing login state and initializing the environment for a given project. It includes functionality to load and save login state, perform a one-time login, and check if the user is logged in.
Classes¶
Functions¶
login(project_name, chat_env_config)
¶
Logs in to the workspace and initializes the environment for the given project. Args: project_name (str): The name of the project to initialize. chat_env_config (AppEnv): The application environment configuration containing the API keys. Returns: None
Source code in src/app/utils/login.py
app.utils.utils
¶
This module provides utility functions and context managers for handling configurations, error handling, and setting up agent environments.
Functions:
Name | Description |
---|---|
load_config |
str) -> Config: Load and validate configuration from a JSON file. |
print_research_Result |
Dict, usage: Usage) -> None: Output structured summary of the research topic. |
error_handling_context |
str, console: Console = None): Context manager for handling errors during operations. |
setup_agent_env |
Config, console: Console = None) -> AgentConfig: Set up the agent environment based on the provided configuration. |
Classes¶
Functions¶
log_research_result(summary, usage)
¶
Prints the research summary and usage details in a formatted manner.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
summary
|
Dict
|
A dictionary containing the research summary with keys ‘topic’, ‘key_points’, ‘key_points_explanation’, and ‘conclusion’. |
required |
usage
|
Usage
|
An object containing usage details to be printed. |
required |
Source code in src/app/utils/utils.py
parse_args(argv)
¶
Parse command line arguments into a dictionary.
This function processes a list of command-line arguments,
extracting recognized options and their values.
Supported arguments include flags (e.g., –help, –include-researcher
and key-value pairs (e.g., --chat-provider=ollama
).
If the --help
flag is present, a list of available commands and their
descriptions is printed, and an empty dictionary is returned.
Recognized arguments as list[str]
--help Display help information and exit.
--version Display version information.
--chat-provider=<str> Specify the chat provider to use.
--query=<str> Specify the query to process.
--include-researcher Include the researcher agent.
--include-analyst Include the analyst agent.
--include-synthesiser Include the synthesiser agent.
--no-stream Disable streaming output.
--chat-config-file=<str> Specify the path to the chat configuration file.
Returns:
Type | Description |
---|---|
dict[str, str | bool]
|
|
dict[str, str | bool]
|
(with leading ‘–’ removed and hyphens replaced by underscores) |
dict[str, str | bool]
|
to their values ( |
dict[str, str | bool]
|
Returns an empty dict if |
Example
parse_args(['--chat-provider=ollama', '--include-researcher'])
returns{'chat_provider': 'ollama', 'include_researcher': True}
Source code in src/app/utils/utils.py
examples.run_simple_agent_no_tools
¶
A simple example of using a Pydantic AI agent to generate a structured summary of a research topic.
Functions¶
main()
¶
Main function to run the research agent.
Source code in src/examples/run_simple_agent_no_tools.py
examples.run_simple_agent_system
¶
This example demonstrates how to run a simple agent system that consists of a manager agent, a research agent, and an analysis agent. The manager agent delegates research and analysis tasks to the corresponding agents and combines the results to provide a comprehensive answer to the user query. https://ai.pydantic.dev/multi-agent-applications/#agent-delegation
Classes¶
Functions¶
get_manager(model_manager, model_researcher, model_analyst, prompts)
¶
Get the agents for the system.
Source code in src/examples/run_simple_agent_system.py
get_models(model_config)
¶
Get the models for the system agents.
Source code in src/examples/run_simple_agent_system.py
main()
async
¶
Main function to run the research system.
Source code in src/examples/run_simple_agent_system.py
examples.run_simple_agent_tools
¶
Run the dice game agent using simple tools.
Functions¶
main()
¶
Run the dice game agent.
Source code in src/examples/run_simple_agent_tools.py
examples.utils.agent_simple_no_tools
¶
This module contains a function to create a research agent with the specified model, result type, and system prompt.
Classes¶
Functions¶
get_research(topic, prompts, provider, provider_config, api_key)
¶
Run the research agent to generate a structured summary of a research topic.
Source code in src/examples/utils/agent_simple_no_tools.py
examples.utils.agent_simple_system
¶
This module contains a simple system of agents that can be used to research and analyze data.
Classes¶
SystemAgent
¶
Bases: Agent
A generic system agent that can be used to research and analyze data.
Source code in src/examples/utils/agent_simple_system.py
Functions¶
add_tools_to_manager_agent(manager_agent, research_agent, analysis_agent)
¶
Create and configure the joke generation agent.
Source code in src/examples/utils/agent_simple_system.py
examples.utils.agent_simple_tools
¶
Simple agent for the dice game example.
Functions¶
get_dice(player_name, guess, system_prompt, provider, api_key, config)
¶
Run the dice game agent.
Source code in src/examples/utils/agent_simple_tools.py
examples.utils.data_models
¶
Example of a module with data models
Classes¶
AnalysisResult
¶
Config
¶
Bases: BaseModel
Configuration settings for the research agent and model providers
Source code in src/examples/utils/data_models.py
ProviderConfig
¶
ResearchResult
¶
ResearchSummary
¶
Bases: BaseModel
Expected model response of research on a topic
Source code in src/examples/utils/data_models.py
examples.utils.utils
¶
Utility functions for running the research agent example.
Classes¶
Functions¶
create_model(base_url, model_name, api_key=None, provider=None)
¶
Create a model that uses base_url as inference API
Source code in src/examples/utils/utils.py
get_api_key(provider)
¶
Retrieve API key from environment variable.
Source code in src/examples/utils/utils.py
get_provider_config(provider, config)
¶
Retrieve configuration settings for the specified provider.
Source code in src/examples/utils/utils.py
load_config(config_path)
¶
Load and validate configuration from a JSON file.
Source code in src/examples/utils/utils.py
print_research_Result(summary, usage)
¶
Output structured summary of the research topic.
Source code in src/examples/utils/utils.py
gui.components.footer
¶
gui.components.header
¶
gui.components.output
¶
Functions¶
render_output(result=None, info_str=None, type=None)
¶
Renders the output in a Streamlit app based on the provided type.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
result
|
Any
|
The content to be displayed. Can be JSON, code markdown, or plain text. |
None
|
info
|
str
|
The information message to be displayed if result is None. |
required |
type
|
str
|
The type of the result content. Can be ‘json’, ‘code’, ‘md’, or other for plain text. |
None
|
Returns:
Name | Type | Description |
---|---|---|
Out |
None |
Source code in src/gui/components/output.py
gui.components.prompts
¶
gui.components.sidebar
¶
gui.config.config
¶
gui.config.styling
¶
gui.config.text
¶
gui.pages.home
¶
gui.pages.prompts
¶
Streamlit component for editing agent system prompts.
This module provides a function to render and edit prompt configurations for agent roles using a Streamlit-based UI. It validates the input configuration, displays warnings if prompts are missing, and allows interactive editing of each prompt.
Classes¶
Functions¶
render_prompts(chat_config)
¶
Render and edit the prompt configuration for agent roles in the Streamlit UI.
Source code in src/gui/pages/prompts.py
gui.pages.run_app
¶
Streamlit interface for running the agentic system interactively.
This module defines the render_app function, which provides a Streamlit-based UI for users to select a provider, enter a query, and execute the main agent workflow. Results and errors are displayed in real time, supporting asynchronous execution.
Functions¶
render_app(provider=None)
async
¶
Render the main app interface for running agentic queries via Streamlit.
Displays input fields for provider and query, a button to trigger execution, and an area for output or error messages. Handles async invocation of the main agent workflow and logs any exceptions.
Source code in src/gui/pages/run_app.py
gui.pages.settings
¶
Streamlit settings UI for provider and agent configuration.
This module provides a function to render and edit agent system settings, including provider selection and related options, within the Streamlit GUI. It validates the input configuration and ensures correct typing before rendering.
Classes¶
Functions¶
render_settings(chat_config)
¶
Render and edit agent system settings in the Streamlit UI.
Displays a header and a selectbox for choosing the inference provider. Validates that the input is a ChatConfig instance and displays an error if not.
Source code in src/gui/pages/settings.py
run_gui
¶
This module sets up and runs a Streamlit application for a Multi-Agent System.
The application includes the following components: - Header - Sidebar for configuration options - Main content area for prompts - Footer
The main function loads the configuration, renders the UI components, and handles the execution of the Multi-Agent System based on user input.
Functions: - run_app(): Placeholder function to run the main application logic. - main(): Main function to set up and run the Streamlit application.