AI Controller Data Flow

This document explains how data moves through the AI Controller system, from when your application sends a request to when it receives the final response. Understanding this flow will help you optimize your deployments, troubleshoot issues, and build more efficient integrations.

How Data Flows Through AI Controller

AI Controller acts as an intelligent proxy between your applications and various LLM providers. When your application sends a request, it follows a clear path through the system:

flowchart TD
    A["Application"] -->|Request| B["AIC"]
    B -->|Request| C["LLM Provider"]
    C -->|Response| B
    B -->|Response| A

    subgraph B[AIC]
        D["Authentication"] --> E["Rules"]
        E --> F["Routing"]
        F --> G["Caching"]
    end

    classDef application fill:#5D8AA8,stroke:#333,stroke-width:1px,color:#fff
    classDef controller fill:#6A5ACD,stroke:#333,stroke-width:1px,color:#fff
    classDef provider fill:#3CB371,stroke:#333,stroke-width:1px,color:#fff
    classDef component fill:#FF7F50,stroke:#333,stroke-width:1px,color:#fff

    class A application
    class B controller
    class C provider
    class D,E,F,G component

Each stage of this process plays an important role in handling and securing your LLM requests. The following sections explain each step in detail.

Request Processing

Authentication

Before processing the request content, AI Controller needs to know who's making the request and what they're allowed to do:

1. AI Controller extracts the API key from your request header
2. It checks if this key is valid
3. It retrieves the user account and permissions linked to this key
4. It verifies you haven't exceeded rate limits or usage quotas

API keys are the main authentication method, with each key linked to specific user and service. For more details on how AI Controller manages and secures these keys, see the API Key Management documentation.

If authentication fails at any point, AI Controller stops processing and returns an appropriate error code.

Request Initialization

The process begins when your application sends a request to AI Controller's /work endpoint. This request typically looks like this:

POST /work HTTP/1.1
Host: your-AIC-server:9090
Authorization: Bearer your_api_key
Content-Type: application/json

{
  "model": "gpt-4",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain quantum computing in simple terms."}
  ]
}

When AI Controller receives this request, it first:

• Checks that the JSON structure is valid
• Verifies all required fields are present on the incoming request based on your configuration
• Processes any custom headers from your application

Rules Engine

After confirming who you are, AI Controller evaluates your request against the rules you've configured:

1. The Rules Engine gets all currently enabled rules
2. It compares each rule against your request:
   • Does the user or group match?
   • Does the model name match the pattern?
   • Does the provider match?
3. The first matching rule allows your request to proceed
4. Logs the interaction
5. If no rules match, your request is denied

This rules-based approach gives you fine-grained control over who can use which AI models and how. For complete details on setting up and managing rules, check out the Rules Engine documentation.

Provider Selection

Once your request passes the Rules Engine, AI Controller determines which LLM provider should handle it:

1. Direct Association: If your API key links to a specific provider, AI Controller uses that provider
2. Model Name Matching: AI Controller matches the requested model name to configured providers
3. No Match Found: If neither method succeeds, AI Controller returns a 400 response

After selecting a provider, AI Controller prepares your request for that specific service and adds the necessary authentication headers.

This provider mapping system enables interaction with multiple AI services through a single, consistent interface. You can learn more about managing providers in the Models and Providers documentation.

Cache Verification

Before sending your request to an external provider, AI Controller checks if it already has the answer:

1. It creates a unique cache key based on:

   • Your normalized request content
   • The model name and provider
   • Any cache-affecting parameters

2. It looks for a valid cached response with this key

If AI Controller finds a matching cache entry that hasn't expired, it:

1. Retrieves the cached response
2. Records a cache hit in its metrics
3. Returns the response directly to your application

This caching system can significantly improve performance and reduce costs by avoiding redundant API calls. For strategies to optimize your cache usage, see the Response Caching documentation.

External Provider Communication

If there's no valid cache entry, AI Controller forwards your request to the selected provider:

1. It adds the provider's authentication credentials
2. It sends the request to the provider's API endpoint
3. It monitors the connection for timeouts or errors

All communication with external providers uses:

• TLS encryption for security (if the chosen Provider is specified with an https endpoint)
• Secure credential handling
• Connection pooling for efficiency

Response Processing

Processing the Provider's Response

After receiving a response from the provider, AI Controller:

1. Verifies the response is properly formatted
2. Checks for any error conditions from the provider
3. Extracts the response content and metadata

Updating the Cache for Future Requests

For successful responses, AI Controller updates its cache:

1. It creates a cache entry with:

   • Your unique request hash as the key
   • The provider's response
   • The User that created the entry
   • Metadata (timestamp, provider, model)

2. It stores this entry in the MySQL database

Delivering the Final Response

Finally, AI Controller delivers the response to your application:

HTTP/1.1 200 OK
Content-Type: application/json
X-AIC-Request-ID: req_12345
X-AIC-Cache: HIT

{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677858242,
  "model": "gpt-4",
  "usage": {
    "prompt_tokens": 50,
    "completion_tokens": 120,
    "total_tokens": 170
  },
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Quantum computing is like a super powerful version of regular computing..."
      },
      "finish_reason": "stop",
      "index": 0
    }
  ]
}

As a final step, AI Controller Logs the completed request details.

Complete Data Flow Visualization

This diagram shows the entire data flow through AI Controller:

flowchart TD
    A["Application"] --> B["API Gateway"]
    B -->|"1. Receive and validate request"| C["Authentication"]
    C -->|"2. Validate API key and permissions"| D["Rules Engine"]
    D -->|"3. Evaluate access rules and policies"| E["Routing"]
    E -->|"4. Select appropriate provider"| F["Cache Lookup"]
    F -->|"5. Check for cached response"| G[(MySQL)]
    F --> H["Provider Call"]
    H -->|"6. Forward request to provider"| I["External LLM API"]
    I --> H
    H --> J["Response Processing"]
    J -->|"7. Process and validate response"| K["Cache Update"]
    K -->|"8. Store response in cache"| G
    K --> L["Response Delivery"]
    L -->|"9. Return formatted response to client"| A

    classDef application fill:#5D8AA8,stroke:#333,stroke-width:1px,color:#fff
    classDef component fill:#6A5ACD,stroke:#333,stroke-width:1px,color:#fff
    classDef external fill:#3CB371,stroke:#333,stroke-width:1px,color:#fff
    classDef database fill:#FF7F50,stroke:#333,stroke-width:1px,color:#fff

    class A application
    class B,C,D,E,F,H,J,K,L component
    class I external
    class G database

Special Flow Patterns

Beyond the standard flow, AI Controller supports several specialized patterns:

Error Handling

When something goes wrong during processing, AI Controller follows a clear error handling flow:

1. It captures error details and context
2. It logs the error with appropriate severity
3. It generates a helpful error response
4. It returns this error to your application

This systematic approach to errors makes troubleshooting easier and helps maintain application reliability. For more details on resolving common issues, see the Troubleshooting Guide.

Streaming Responses

For streaming responses (where results are delivered incrementally rather than all at once), AI Controller:

1. Maintains an open connection to your application
2. Forwards the provider's streaming responses in real-time
3. Implements specialized caching mechanisms (using chunk-based caching)
4. Terminates the connection when the provider indicates completion

Streaming improves user experience for longer responses and is particularly effective for chat applications. The Performance Optimization guide covers strategies for working effectively with streaming responses.

Related Documentation

For practical implementation of the concepts discussed in this document, see these feature guides:

• Architecture Overview
• Models and Providers
• Rules Engine
• Response Caching
• API Key Management
• Performance Optimization
• Logging and Monitoring
• Troubleshooting Guide

---

Updated: 2025-05-15