LLM Providers Module Overview

This document describes the overall architecture of the LLM provider subsystem in Elftia. The subsystem is responsible for managing multi-provider configurations, API key pools, model discovery, request routing, and the Completion call pipeline.

File Locations

Module	Path
CompletionService	`packages/desktop/app/main/services/capabilities/llm/completion/CompletionService.ts`
ApiKeyPoolService	`packages/desktop/app/main/services/capabilities/llm/completion/ApiKeyPoolService.ts`
LLMConfigService	`packages/desktop/app/main/services/capabilities/llm/config-service/LLMConfigService.ts`
ProviderManager	`packages/desktop/app/main/services/capabilities/llm/config-service/ProviderManager.ts`
ModelDiscoveryManager	`packages/desktop/app/main/services/capabilities/llm/config-service/ModelDiscoveryManager.ts`
ConfigIOManager	`packages/desktop/app/main/services/capabilities/llm/config-service/ConfigIOManager.ts`
AgentModelsManager	`packages/desktop/app/main/services/capabilities/llm/config-service/AgentModelsManager.ts`
ThinkingResolver	`packages/desktop/app/main/services/capabilities/llm/completion/ThinkingResolver.ts`
StreamHandler	`packages/desktop/app/main/services/capabilities/llm/completion/StreamHandler.ts`
DirectApiHandler	`packages/desktop/app/main/services/capabilities/llm/completion/DirectApiHandler.ts`
ToolHandler	`packages/desktop/app/main/services/capabilities/llm/completion/ToolHandler.ts`
TransformerHandler	`packages/desktop/app/main/services/capabilities/llm/completion/TransformerHandler.ts`
ProviderSearchInjector	`packages/desktop/app/main/services/capabilities/llm/completion/ProviderSearchInjector.ts`
NativeSearchInjector	`packages/desktop/app/main/services/capabilities/llm/completion/NativeSearchInjector.ts`
URL Builder	`packages/desktop/app/main/services/capabilities/llm/completion/url-builder.ts`
Header Builder	`packages/desktop/app/main/services/capabilities/llm/completion/header-builder.ts`
Message Converter	`packages/desktop/app/main/services/capabilities/llm/completion/message-converter.ts`
Provider Presets	`packages/desktop/app/shared/provider-presets.ts`
IPC Routers	`packages/desktop/app/main/services/routers/llm/`

Architectural Context

graph TB
    subgraph Frontend ["Frontend (Renderer)"]
        UI[Provider Settings UI]
        ChatUI[Chat Interface]
    end

    subgraph IPCLayer ["IPC Layer"]
        PR[ProviderRouter]
        AKR[ApiKeyRouter]
        CR[CompletionRouter]
        RCR[RouterConfigRouter]
        TR[TransformerRouter]
        MPR[ModelParametersRouter]
    end

    subgraph LLMConfigService ["LLMConfigService (Delegate Pattern)"]
        PM[ProviderManager]
        MDM[ModelDiscoveryManager]
        CIOM[ConfigIOManager]
        AMM[AgentModelsManager]
    end

    subgraph CompletionService ["CompletionService (Facade Pattern)"]
        DAH[DirectApiHandler]
        SH[StreamHandler]
        TH[ToolHandler]
        THR[TransformerHandler]
        TR2[ThinkingResolver]
    end

    AKPS[ApiKeyPoolService]

    subgraph ExternalAPIs ["External APIs"]
        OpenAI[OpenAI API]
        Anthropic[Anthropic API]
        Gemini[Gemini API]
        Azure[Azure OpenAI API]
        Others[Other Providers...]
    end

    subgraph Storage ["Storage"]
        SQLite[(SQLite)]
        JSON[(llms-config.json)]
    end

    UI --> PR
    UI --> AKR
    ChatUI --> CR

    PR --> PM
    AKR --> AKPS
    CR --> CompletionService

    PM --> SQLite
    PM --> JSON
    MDM --> SQLite
    MDM --> JSON

    CompletionService --> AKPS
    CompletionService --> LLMConfigService
    DAH --> OpenAI
    DAH --> Anthropic
    DAH --> Gemini
    DAH --> Azure
    SH --> OpenAI
    SH --> Anthropic
    SH --> Gemini
    SH --> Azure

Data Structures

Core Types

// API format enum
type ApiFormat = 'openai' | 'anthropic' | 'google' | 'azure-openai' | 'openai-response';

// LLM provider
interface LLMProvider {
  id: string;                       // Unique identifier
  name: string;                     // Display name
  apiFormat?: ApiFormat;            // API format (preferred field)
  chatApiFormat?: ApiFormat;        // Legacy API format field
  apiType?: string;                 // Legacy type field
  api_base_url?: string;           // API base URL
  api_key: string;                 // Single key (legacy)
  models: string[];                // Model ID list
  modelConfigs?: ModelConfig[];    // Per-model detailed config
  modelGroups?: ModelGroup[];      // Model groups
  modelsEndpoint?: string;         // Model discovery endpoint
  enabled: boolean;                // Whether enabled
  transformer?: TransformerConfig; // Transformer chain config
  icon?: string;                   // Icon
  website?: string;                // Official website
  docsUrl?: string;                // Documentation link
  defaultSettings?: CompletionSettings; // Default completion parameters
  codingPlan?: CodingPlanConfig;   // Coding Plan config
  isSystem?: boolean;              // System built-in provider
  presetId?: string;               // Preset template ID
  createdAt: string;
  updatedAt: string;
}

// Completion request options
interface CompletionOptions {
  providerId: string;              // Provider ID
  model: string;                   // Model ID
  messages: SimpleChatMessage[];   // Message list
  maxTokens?: number;              // Max token count
  temperature?: number;            // Temperature
  stream?: boolean;                // Whether streaming
  thinkLevel?: ThinkLevel;         // Thinking level
  nativeSearchAugmentation?: NativeSearchAugmentation; // Native search augmentation
  sessionId?: string;              // Session ID (for API Key Pool affinity)
}

// API key pool entry
interface ApiKeyEntry {
  id: string;                      // Unique ID
  providerId: string;              // Owning provider
  label?: string;                  // Display label
  apiKey: string;                  // Key value
  enabled: boolean;                // Whether enabled
  weight: number;                  // Weight (1-100)
}

Data Flow Overview

Completion Request Pipeline

sequenceDiagram
    participant Frontend as Frontend
    participant CS as CompletionService
    participant LLC as LLMConfigService
    participant AKP as ApiKeyPoolService
    participant Handler as API Handler
    participant API as Provider API

    Frontend->>CS: complete() / completeStream()
    CS->>LLC: resolveRoutedModel(providerId, model)
    LLC-->>CS: actualProviderId + actualModel
    CS->>LLC: getProvider(actualProviderId)
    LLC-->>CS: provider config
    CS->>CS: resolveApiKeyForRequest()
    Note right of CS: Priority: codingPlan > pool > legacy
    CS->>AKP: getKeyForSession(providerId, sessionId)
    AKP-->>CS: resolved API key
    CS->>CS: resolveApiFormat(provider)
    CS->>Handler: callDirectHandler / callStreamHandler
    Handler->>API: HTTP request
    API-->>Handler: response / SSE stream
    Handler-->>CS: CompletionResult
    alt 429/529 error
        CS->>AKP: reportError(providerId, sessionId, status)
        AKP-->>CS: new key
        CS->>Handler: retry with new key
    end
    CS->>AKP: reportSuccess(sessionId)
    CS-->>Frontend: result

Module Boundaries and Responsibilities

Module	Responsibilities	Not Responsible For
CompletionService	Request pipeline orchestration, format dispatch, retry, vision fallback	Provider CRUD, key storage
ApiKeyPoolService	Multi-key load balancing, session affinity, cooldown backoff	Key persistence (delegated to DB)
LLMConfigService	Provider config management, model discovery, routing, Transformer chain	API calls
ProviderManager	Provider CRUD, template/preset system, SQLite persistence	Model discovery, routing config
ModelDiscoveryManager	Model list discovery, caching (SQLite + file)	Provider CRUD
ConfigIOManager	Config import/export, JSON file I/O	Database operations
AgentModelsManager	Routing config, Transformer management, global parameters	Provider CRUD
ThinkingResolver	max_tokens resolution, thinking budget calculation	Request sending
StreamHandler	SSE stream processing (OpenAI/Anthropic/Gemini)	Non-streaming requests
DirectApiHandler	Non-streaming API calls	Streaming requests
ToolHandler	Agent tool call loop	Regular Completion
TransformerHandler	Transformer chain Completion	Direct API calls

IPC Integration Table

IPC Channel	Direction	Router	Description
`llmConfig:getProviders`	Renderer -> Main	ProviderRouter	Get all providers
`llmConfig:getProvider`	Renderer -> Main	ProviderRouter	Get a single provider
`llmConfig:addProvider`	Renderer -> Main	ProviderRouter	Add a provider
`llmConfig:updateProvider`	Renderer -> Main	ProviderRouter	Update a provider
`llmConfig:deleteProvider`	Renderer -> Main	ProviderRouter	Delete a provider
`llmConfig:toggleProvider`	Renderer -> Main	ProviderRouter	Enable/disable a provider
`llmConfig:discoverModels`	Renderer -> Main	ProviderRouter	Discover model list
`llmConfig:getProviderPresets`	Renderer -> Main	ProviderRouter	Get preset templates
`llmConfig:addFromPreset`	Renderer -> Main	ProviderRouter	Add provider from preset
`llmConfig:exportConfig`	Renderer -> Main	ProviderRouter	Export config
`llmConfig:importConfig`	Renderer -> Main	ProviderRouter	Import config
`llmConfig:getApiKeys`	Renderer -> Main	ApiKeyRouter	Get key list
`llmConfig:addApiKey`	Renderer -> Main	ApiKeyRouter	Add a key
`llmConfig:updateApiKey`	Renderer -> Main	ApiKeyRouter	Update a key
`llmConfig:deleteApiKey`	Renderer -> Main	ApiKeyRouter	Delete a key
`llmConfig:toggleApiKey`	Renderer -> Main	ApiKeyRouter	Enable/disable a key

Extension Points

Adding a new API format: Add a URL builder function in url-builder.ts, add handler functions in DirectApiHandler.ts / StreamHandler.ts, and register in the switch in CompletionService
Adding a new provider preset: Add a template to the PROVIDER_PRESETS array in provider-presets.ts
Adding a new search config: Add an entry to PROVIDER_SEARCH_CONFIGS in provider-presets.ts
Custom Transformer: Register a new Transformer chain via AgentModelsManager

File	Purpose
`packages/desktop/app/shared/llm-config.ts`	LLM config shared type definitions
`packages/desktop/app/shared/completion-types.ts`	Completion type definitions
`packages/desktop/app/shared/thinking-config.ts`	Thinking config and budget calculation
`packages/desktop/app/shared/provider-presets.ts`	Provider preset templates and search configs
`packages/desktop/app/main/services/routers/llm/schemas.ts`	IPC request Zod validation schemas
`packages/desktop/app/main/services/capabilities/llm/config-service/schemas.ts`	Config service internal Zod schemas
`packages/desktop/app/main/services/infra/utils/sse-parser.ts`	SSE stream parsing utilities
`packages/desktop/app/main/workers/db/apiKeys.ts`	API key database operations
`packages/desktop/app/main/workers/DbClient.ts`	Database client
`packages/desktop/app/preload/index.ts`	Preload API exposure

File Locations​

Architectural Context​

Data Structures​

Core Types​

Data Flow Overview​

Completion Request Pipeline​

Module Boundaries and Responsibilities​

IPC Integration Table​

Extension Points​

Related Files​