Feature Request: Multi-LLM Model Support with Dynamic Switching

[AhmadiRamin]

Summary
I would like to implement the ability to define multiple LLM models within Chat Copilot and allow users to dynamically switch between them during their chat sessions, similar to the interface shown in popular AI chat applications.

Currently, Chat Copilot is configured to work with a single LLM model (typically Azure OpenAI). However, different models excel at different tasks, and users would benefit from the ability to:

• Compare responses from different models (GPT-4o, Claude, Gemini, etc.)
• Choose the most appropriate model for specific use cases
• Switch between models mid-conversation without losing context
• Leverage cost-effective models for simpler tasks and more powerful models for complex reasoning

I propose implementing multi-LLM support using Semantic Kernel's built-in multi-model capabilities as the foundation. This approach would leverage existing, battle-tested infrastructure rather than building custom model management. I've already implemented this feature and it's live in our production environment.

Technical Approach
Based on Semantic Kernel's multi-model support documentation, the implementation would involve:

1. Backend Changes

• Service Registration: Register multiple AI services with unique serviceId and modelId identifiers
• Dynamic Model Selection: Implement custom IAIServiceSelector for runtime model switching
• Configuration Management: Extend app settings to support multiple model configurations
• Chat Context Preservation: Ensure conversation history is maintained when switching models

2. Frontend Changes

• Model Selection UI: Add a dropdown/selector component similar to the reference image
• Model Status Indicators: Show current active model and availability status
• Seamless Switching: Allow mid-conversation model changes without losing context
• Model-Specific Features: Display model capabilities and limitations

3. API Enhancements

• Extend chat endpoints to accept optional modelId parameter
• Add model management endpoints (list available models, get model info)
• Update response schema to include which model generated the response

User Experience

• Model Selection: Users can select from a dropdown of configured models
• Contextual Switching: Models can be changed mid-conversation with context preservation
• Model Information: Display model names, capabilities, and current status
• Fallback Support: Graceful handling when a selected model is unavailable

Configuration Example

{
  "SemanticKernel": {
    "Models": [
      {
        "Id": "azure:gpt-4o",
        "Name": "Azure GPT-4o",
        "Provider": "AzureOpenAI",
        "ModelId": "gpt-4o",
        "DeploymentName": "gpt-4o-deployment",
        "Endpoint": "https://your-resource.openai.azure.com/",
        "ApiKey": "your-api-key",
        "IsDefault": true
      },
      {
        "Id": "openai:gpt-4o-mini",
        "Name": "OpenAI GPT-4o Mini",
        "Provider": "OpenAI",
        "ModelId": "gpt-4o-mini",
        "ApiKey": "your-openai-key",
        "IsDefault": false
      }
    ]
  }
}

I'm committed to seeing this feature through to completion, including:

• Writing comprehensive tests
• Following project coding standards
• Updating documentation
• Providing ongoing maintenance support

Would the maintainers be interested in this contribution? I'm happy to discuss the approach and make any adjustments based on your feedback before beginning implementation.