Architecture Overview

This document provides an architectural overview of the AI Controller system, explaining its components, data flow, and integration points.

High-Level Architecture

AI Controller provides a centralized gateway between your applications and various LLM providers:

flowchart TD
    %% Define Core Components
    Client[Client Applications] -->|Request| AIC[1. AIC Service]
    AIC -->|Validates & Routes| Rules[2. Rules Engine]
    Rules -->|Enforces Policies| Provider[3. Provider Management]
    Provider -->|External Calls| LLM[LLM Providers]
    LLM -->|Responses| Provider
    Provider -->|Returns Result| AIC
    AIC -->|Response| Client

    AIC <-->|Query/Store| DB[4. Database System]
    AIC <-->|Cache Lookup| Cache[5. Caching System]
    WebUI[6. Web Interface] <-->|Administration| AIC

Architecture diagram showing the six core components of the AI Controller system

Core Components

AI Controller consists of several key components that work together to provide a complete AI gateway solution. Let's explore each one:

1. AI Controller Service

The AI Controller Service is the central component that handles all incoming requests and routes them to appropriate handlers. It provides:

RESTful API endpoints for all interactions
Request validation and sanitization to ensure data quality
Authentication and authorization to secure your AI resources
Request routing to the right LLM providers
Response formatting and delivery back to applications

The AI Controller Service is the main entry point for all communication with AI Controller, providing a consistent interface regardless of which underlying AI provider is being used.

2. Rules Engine

The Rules Engine evaluates each request against configured rules to determine processing logic. When a request arrives, the Rules Engine checks:

If the user has permission to access the requested model
Which provider should handle the request based on your policies

All requests must match at least one enabled rule to be processed. This gives you fine-grained control over how AI resources are used across your organization.

For an introduction into rule configuration and policy enforcement, see the Rules Engine documentation, which introduces how to implement departmental AI policies, cost-tiered access, and risk management controls.

3. Provider Management

The Provider Management subsystem handles all communication with external LLM services. This component:

Maintains provider configurations and securely stores credentials
Handles provider-specific request formatting requirements
Manages connections to external LLM APIs
Monitors provider availability and performance metrics

By centralizing provider management, AI Controller simplifies the addition of new AI services and helps maintain consistent security practices across all AI integrations.

The API Key Management feature explains how AI Controller securely handles provider credentials, including encryption, access controls, and key rotation policies.

4. Database System

AI Controller uses a relational database (MySQL) to store configuration and operational data. The database stores:

User accounts and their associated permissions
Provider configurations and encrypted API keys
Rules and access policies for controlling model usage
Logging and audit information for compliance
Usage statistics and metrics for reporting

The database provides persistent storage for all configuration data and creates a reliable audit trail of system activities.

5. Caching System

The caching system improves performance and reduces costs by storing previous responses. It identifies identical requests and serves cached responses when appropriate, avoiding redundant LLM API calls.

The Response Caching documentation provides details on how caching works, including performance benchmarks and cost savings analysis. Implementing effective caching strategies can significantly reduce operational costs, especially for high-volume applications.

6. Web Interface

The administrative web interface provides user-friendly tools for managing AI Controller. Administrators can use it for:

Configuration management through intuitive controls
Dashboards for monitoring usage across providers
Log viewing and filtering for troubleshooting
Rule management for access control
User and group administration

The web interface makes complex management tasks simpler and more accessible for administrators who may not have deep technical expertise in API configuration.

Integration Points

AI Controller provides several ways for applications to connect to its services:

REST API

The primary integration method is the REST API, which includes:

/work - Main endpoint for processing LLM requests
/api - API key management for application access
/providers - Provider configuration management
/users - User account management
/rules - Rules configuration for access control

Each endpoint follows REST principles and returns standardized responses, making integration straightforward for developers. For detailed API documentation, refer to the API documentation.

OpenAI-Compatible Interface

The /work endpoint offers OpenAI-compatible request and response formats, allowing:

Drop-in replacement for existing OpenAI API calls
Compatibility with popular OpenAI SDKs and libraries
Similar integration patterns across multiple providers

This compatibility layer makes it easy to migrate existing applications to AI Controller with minimal code changes. For guidance on working with LLM prompts through AI Controller, see the Prompt an LLM documentation.

Web Interface

The browser-based interface provides interactive tools for:

Manual testing of prompts and responses
System administration and configuration
Monitoring and reporting
Log investigation and troubleshooting

The web interface is particularly useful for administrators who need to manage the system without writing code. For details on monitoring system activity, see the Logging and Monitoring documentation.

Next Steps

Now that you understand AI Controller's architecture, consider exploring these related topics:

Updated: 2025-05-15