Skip to content

Models and Providers

This document explains how AI models and providers work in AI Controller. Understanding these core concepts helps you set up provider connections, implement access controls, and get the most from your LLM strategy.

Core Architecture

Models and providers form the foundation of AI Controller's architecture:

flowchart TD
    A[Applications] -->|Requests| B[AIC]
    B -->|Requests| C[Provider A]
    B -->|Requests| D[Provider B]
    C -->|Responses| B
    D -->|Responses| B
    B -->|Responses| A

    subgraph B[AIC]
        E[Provider & Model Management System]
    end

    subgraph C[Provider A]
        F[Model X]
        G[Model Y]
    end

    subgraph D[Provider B]
        H[Model Z]
    end

    classDef application fill:#5D8AA8,stroke:#333,stroke-width:1px,color:#fff
    classDef controller fill:#6A5ACD,stroke:#333,stroke-width:1px,color:#fff
    classDef provider fill:#3CB371,stroke:#333,stroke-width:1px,color:#fff
    classDef model fill:#FF7F50,stroke:#333,stroke-width:1px,color:#fff

    class A application
    class B,E controller
    class C,D provider
    class F,G,H model

AI Controller works as a bridge between your applications and AI models, providing:

  • A single interface for using different models
  • Central security and management
  • Controls for who can use which models
  • Tracking and measuring how models perform
  • Managing costs across different providers and models

What Are Providers?

In AI Controller, a "provider" is a Large Language Model (LLM) service that AI Controller connects to when processing requests. Each provider:

  • Offers one or more AI models (like GPT-4 or Claude-3)
  • Has its own API endpoints and ways to authenticate
  • Needs specific credentials to access
  • Uses its own request and response formats

Examples include OpenAI, Anthropic, Google/Gemini, and Azure OpenAI.

What Are Models in AI Controller?

A "model" is a specific AI capability offered by a provider. Models are the actual machine learning systems that generate responses, create embeddings, or perform other AI tasks.

Each model in AI Controller:

  • Connects to a specific provider
  • Has a unique name/identifier
  • Provides particular capabilities
  • Has its own cost considerations
  • May have specific usage rules

3. Access Controls

Settings that determine who can use the model. AI Controller's Rules Engine provides flexible, pattern-based access controls that let you manage which users, groups, and applications can access specific models and providers.

Provider Communication

AI Controller handles all aspects of talking to providers and also caches responses to improve performance and reduce costs. Learn more in the Response Caching documentation.

Authentication Methods

AI Controller supports various ways to authenticate with providers:

Auth Type Header Format Common Providers Example
X-API-Key X-API-Key: key Anthropic, many others X-API-Key: sk-ant-xxxx
Bearer Authorization: Bearer key OpenAI, Google Authorization: Bearer sk-xxxx
Basic Authorization: Basic base64(user:pass) Some enterprise APIs Authorization: Basic dXNlcjpwYXNz
Custom Various formats Custom or proprietary APIs Configured per provider

Provider URLs: Each provider needs the correct base URL. Using the wrong URL is a common mistake.

Provider Typical URL Notes
OpenAI https://api.openai.com/v1 Standard OpenAI API
Azure OpenAI https://{resource}.openai.azure.com/openai/deployments/{deployment} Azure-specific format
Anthropic https://api.anthropic.com/v1 Anthropic Claude API
Google/Gemini https://generativelanguage.googleapis.com/v1beta Google's Gemini API
Custom Varies by provider Any compatible API endpoint

Model Types

AI Controller supports different types of models for various needs:

Text Generation Models are the most common type. They generate text responses based on prompts. These include chat models like GPT-4 and Claude-3, instruction-following models, and older completion models. Most LLM applications use these as their main component.

Embedding Models create vector representations of text. These vectors help with searching, finding similar content, and understanding relationships between pieces of text. They power applications like document search and content recommendations.

Vision Models work with images. They can analyze pictures, describe what's in them, and understand content that combines both text and images.

Specialized Models focus on specific tasks or fields. These include code-focused models for software development, moderation models to filter content, and models trained for specific industries like legal, medical, or financial work.

Provider Request Formats

Different providers use different formats for requests. AI Controller supports all these formats:

OpenAI Format (Default)

{
  "model": "gpt-4",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Tell me about quantum computing."}
  ],
  "temperature": 0.7,
  "max_tokens": 150
}

Anthropic Format

{
  "model": "claude-3-opus-20240229",
  "messages": [
    {"role": "user", "content": "Tell me about quantum computing."}
  ],
  "temperature": 0.7,
  "max_tokens": 150
}

Google/Gemini Format

{
  "contents": [
    {"role": "user", "parts": [{"text": "Tell me about quantum computing."}]}
  ],
  "generationConfig": {
    "temperature": 0.7,
    "maxOutputTokens": 150
  }
}

For more details on how to prompt models effectively, see the Prompt an LLM guide.

Model Parameters

You can control how models behave using various settings:

Common Parameters

Parameter Description Typical Values Effect
temperature Controls randomness 0.0 - 1.0 Higher values make outputs more creative and varied
max_tokens Maximum response length 1 - model limit Limits token usage and response size
top_p Controls diversity via nucleus sampling 0.0 - 1.0 Alternative to temperature
presence_penalty Penalizes repetition of topics -2.0 - 2.0 Higher values reduce topic repetition
frequency_penalty Penalizes repetition of tokens -2.0 - 2.0 Higher values reduce exact repetition

Provider-Specific Parameters: Each provider also has unique options that AI Controller preserves when sending requests. For example, OpenAI lets you specify response_format to control output structure (like JSON), Anthropic uses system for system prompts, and Google offers custom stop_sequences. Understanding these provider-specific parameters helps you get the most from each model.

Provider Parameter Description
OpenAI response_format Control structure of response (JSON, etc.)
Anthropic system System prompt parameter
Google stop_sequences Custom stop sequences

Model Comparison

Models differ in their abilities, performance, and costs:

Performance Characteristics and Capabilities

When choosing a model, consider these factors to find the right fit for your needs.

Performance Table:

Model Provider Token Context Response Quality Latency Cost
GPT-4 OpenAI 128K Very High Medium-High High
GPT-3.5-Turbo OpenAI 16K Good Low Low
Claude-3-Opus Anthropic 200K Very High High High
Claude-3-Sonnet Anthropic 200K High Medium Medium
Gemini-Pro Google 32K High Medium Medium
Llama-3-70B Meta 8K High Medium Variable

Capability Comparison:

Model Code Generation Complex Reasoning Creative Writing Multilingual Vision
GPT-4 ★★★★★ ★★★★★ ★★★★☆ ★★★★☆ Yes*
GPT-3.5-Turbo ★★★★☆ ★★★☆☆ ★★★☆☆ ★★★☆☆ No
Claude-3-Opus ★★★★★ ★★★★★ ★★★★★ ★★★★☆ Yes
Claude-3-Sonnet ★★★★☆ ★★★★☆ ★★★★☆ ★★★★☆ Yes
Gemini-Pro ★★★★☆ ★★★★☆ ★★★☆☆ ★★★★☆ Yes

*GPT-4 Vision variant required

When picking models, think about both performance and specific capabilities. For example, if your application needs to work with images and text together, make sure you select a model with vision capabilities.

Understanding the cost implications of model selection is important for managing your AI budget. See the Cost Management documentation for strategies to optimize your spending.

Management and Configuration

AI Controller provides tools to manage providers and models through the Administrative panel. The Admin -> Providers page is where administrators can add, change, and remove provider connections, set up models, test connections, and see usage statistics.

Setting up a provider follows these steps:

  1. Get credentials from the LLM service provider (usually an API key)
  2. Add a new provider in AI Controller with a clear name
  3. Set up connection details including the right URL and authentication method
  4. Add models that belong to the provider, using exact model names (optional)
  5. Test the connection to make sure everything works
  6. Set up access rules to control who can use the provider and models

Following these steps helps ensure proper setup and appropriate access controls.

Access Control

AI Controller offers comprehensive access control for providers and models. The system combines user-based controls, API key restrictions, and a powerful Rules Engine to give you complete control over who can access which AI capabilities.

For detailed information on implementing access controls, see the Access Control and Rules Engine documentation.

AI Controller's access controls let you:

  • Restrict expensive models to specific users or groups
  • Create department-specific model access policies
  • Implement compliance controls for sensitive operations
  • Manage access through a combination of user and API key permissions

Monitoring and Metrics

AI Controller includes robust monitoring and logging capabilities to help you understand how your AI resources are being used. The Logging and Monitoring system tracks all requests, responses, and system events, giving you visibility into:

  • Which users and applications are using which models
  • How models are performing in terms of response time and quality
  • Where costs are being incurred and how they can be optimized
  • Potential security or usage issues that need attention

These insights help you make informed decisions about your AI strategy and resource allocation.

Integration Examples

Organizations typically set up AI Controller in strategic ways:

Multi-Provider Architecture: Many teams use multiple providers to get the best of each:

  • OpenAI for general text generation
  • Anthropic for longer responses that need more context
  • Mistral for basic requests at lower cost
  • Azure OpenAI for enterprise workloads with compliance requirements

Environment Separation: Suggested practices include having different setups for:

  • Development environments with cheaper models
  • Testing environments that mirror production but with separate credentials
  • Production environments with high-reliability settings and strict monitoring
  • Experimental setups for trying new models before wider use

Future Considerations

AI technology changes rapidly, and AI Controller is built to adapt to these changes. Several upcoming developments will change how teams work with AI models and providers.

Multimodal expansion will extend beyond today's text and basic image capabilities. Future AI systems will handle audio, video, and other media types. AI Controller is a central point for all these formats, maintaining consistent rules and management regardless of the content type.

As teams develop specialized knowledge, fine-tuned models will become valuable tools. These customized models, adapted for specific tasks or industry knowledge, can provide real advantages. AI Controller helps manage, track versions, and deploy these specialized models with the same controls used for public models.

Many organizations need on-premises models for security, compliance, or performance reasons. AI Controller is improving to support models running locally on dedicated hardware, in private clouds, or in secure networks, while keeping the same familiar interface and access controls.

Model composition offers new possibilities for complex tasks. Instead of using one model for everything, teams can link several specialized models together, with each handling what it does best. AI Controller will support these connected systems, letting administrators set up, manage, and monitor these model chains as complete units.

Cross-provider orchestration will help teams get better results by using multiple AI providers. Different providers have different strengths, and smart applications will use the best tool for each job. AI Controller will add features to send requests to the right provider based on what's needed, how much it costs, and how well it performs, creating a smart connection between applications and various AI services.

Updated: 2025-05-27