Model Discovery and Caching

ModelDiscoveryManager dynamically discovers available model lists from provider APIs, manages the cache for discovery results, and works with AgentModelsManager to implement model routing and the Follow-Provider feature.

File Locations

File	Path
ModelDiscoveryManager	`packages/desktop/app/main/services/capabilities/llm/config-service/ModelDiscoveryManager.ts`
AgentModelsManager	`packages/desktop/app/main/services/capabilities/llm/config-service/AgentModelsManager.ts`
LLMConfigService	`packages/desktop/app/main/services/capabilities/llm/config-service/LLMConfigService.ts`
Provider Presets	`packages/desktop/app/shared/provider-presets.ts`
Utility functions	`packages/desktop/app/main/services/capabilities/llm/config-service/utils.ts`
Config merging	`packages/desktop/app/main/services/capabilities/llm/config/` (mergeModelLists)

Architectural Context

graph TB
    subgraph LLMConfigService
        MDM[ModelDiscoveryManager]
        AMM[AgentModelsManager]
    end

    subgraph CacheLayer ["Cache Layer"]
        SQLiteCache[(SQLite<br/>llm_model_cache)]
        FileCache[(File cache<br/>model-cache/*.json)]
        ChainCache["Transformer Chain Cache<br/>Map (10min TTL)"]
    end

    subgraph ProviderAPI ["Provider API"]
        OpenAIModels["/v1/models"]
        AnthropicModels["/v1/models"]
        GeminiModels["/v1beta/models"]
    end

    subgraph DataSources ["Data Sources"]
        ProviderConfig[Provider config<br/>modelConfigs / models]
        Templates[PROVIDER_TEMPLATES<br/>built-in templates]
        Presets[Provider Presets<br/>preset configs]
        Mappings[PROVIDER_MODEL_MAPPINGS<br/>model mappings]
    end

    MDM -->|read/write| SQLiteCache
    MDM -->|fallback| FileCache
    MDM -->|API calls| ProviderAPI
    MDM -->|fallback models| ProviderConfig
    MDM -->|fallback models| Templates

    AMM --> ChainCache
    AMM --> Mappings

Data Structures

Model Discovery Results

// Single model discovery entry
interface ProviderModelDiscoveryEntry {
  id: string;                       // Model ID (e.g. 'gpt-4o')
  name: string;                     // Display name
  description?: string;             // Description
  contextLength?: number;           // Context window length
  maxTokens?: number;               // Max output tokens
  category?: string;                // Category: 'chat' | 'reasoning' | 'image' | ...
  capabilities?: string[];          // Capability tags: 'vision' | 'function_call' | 'reasoning'
}

// Discovery result
interface ProviderModelDiscoveryResult {
  success: boolean;
  source: string;                   // 'api' | 'cache' | 'fallback'
  endpoint: string;                 // Actual API endpoint called
  models: ProviderModelDiscoveryEntry[];
  raw?: unknown;                    // Raw API response
  fetchedAt?: string;               // ISO timestamp
  error?: string;
}

// Available model list (merged)
interface AvailableModelsResult {
  models: ProviderModelDiscoveryEntry[];
  source: string;
  lastUpdated?: string;
}

Transformer Chain Cache

// Chain cache entry in LLMConfigService
interface ChainCacheEntry {
  chain: ResolvedTransformerChain;
  cachedAt: number;                 // Cache timestamp
}

// Cache TTL
const CHAIN_CACHE_TTL_MS = 10 * 60 * 1000; // 10 minutes

Model Routing Configuration

// Router configuration
interface RouterConfig {
  default?: string;                 // Default model ("providerId,modelId")
  background?: string;             // Background model
  think?: string;                   // Reasoning model
  longContext?: string;            // Long-context model
  longContextThreshold?: number;   // Long-context trigger threshold
  webSearch?: string;              // Search model
  image?: string;                  // Image model
  vision?: string;                 // Vision model
  followProviderBackground?: boolean; // Background model follows primary model's provider
  followProviderVision?: boolean;     // Vision model follows primary model's provider
}

// Code ↔ Chat routing
interface CodeToChatRouterConfig {
  providerId: string;
  modelId: string;
  actualProviderId: string;
  actualModelId: string;
}

interface ChatToCodeRouterConfig {
  providerId: string;
  modelId: string;
  actualProviderId: string;
  actualModelId: string;
}

Algorithms and Logic

Model Discovery Flow

flowchart TD
    Start[discoverModels] --> GetProvider[Get provider config]
    GetProvider --> LoadPreset[Load preset models<br/>getFallbackModels]

    LoadPreset --> DeriveEndpoint[Derive API endpoint<br/>deriveModelsEndpoint]
    DeriveEndpoint --> HasEndpoint{Valid endpoint?}

    HasEndpoint -->|No| CheckCache[Check cache]
    HasEndpoint -->|Yes| CheckForce{Force refresh?}

    CheckForce -->|Yes| CallAPI[Call API]
    CheckForce -->|No| CheckCacheAge[Check cache freshness]
    CheckCacheAge --> HasFreshCache{Cache valid?}
    HasFreshCache -->|Yes| ReturnCache[Return cached result]
    HasFreshCache -->|No| CallAPI

    CallAPI --> APISuccess{API succeeded?}
    APISuccess -->|Yes| ParseModels[Parse model list]
    APISuccess -->|No| FallbackCache[Use cache/preset fallback]

    ParseModels --> MergeModels[Merge API results + preset models]
    MergeModels --> WriteCache[Write to cache]
    WriteCache --> ReturnResult[Return result]

    CheckCache --> HasCacheAtAll{Cache exists?}
    HasCacheAtAll -->|Yes| ReturnCache
    HasCacheAtAll -->|No| ReturnFallback[Return preset models]

    FallbackCache --> HasCacheAtAll

Detailed steps:

discoverModels(providerId, options?):
  1. provider = delegate.getProvider(providerId)
     If not found → return { success: false, error: "not found" }

  2. presetModels = getFallbackModels(provider)
     // Look up in order: modelConfigs → models → PROVIDER_TEMPLATES

  3. endpoint = deriveModelsEndpoint(provider)
     // Derive from modelsEndpoint or api_base_url

  4. If no valid endpoint:
     Try to return from cache; otherwise return preset models (source: 'fallback')

  5. If not a forced refresh:
     Check cache → if valid → return cached result

  6. Call provider API:
     - OpenAI compatible: GET /v1/models
     - Anthropic: GET /v1/models
     - Gemini: GET /v1beta/models
     Request is made via Electron's net module

  7. Parse response, extract model list
     Handle response format differences across providers

  8. Merge: API-discovered models + preset models (deduplicated)

  9. Write to cache (SQLite or file)

  10. Return { success: true, source: 'api', models, ... }

Fallback Model Resolution (getFallbackModels)

When API discovery is unavailable, fall back in the following priority order:

getFallbackModels(provider):
  fallbackModels = []

  // Priority 1: Provider's own modelConfigs
  if provider.modelConfigs?.length:
    convertModelConfigs(provider.modelConfigs)
    // Extract id, name, contextLength, maxTokens, capabilities

  // Priority 2: Provider's own models array
  else if provider.models?.length:
    convertModelsArray(provider.models)
    // Only id, no detailed info

  // Priority 3: Matching built-in template
  if fallbackModels.length === 0:
    template = PROVIDER_TEMPLATES.find(matching provider.id or provider.name)
    if template:
      // Use template's modelConfigs or models

Model category mapping:

Category	Description
`chat`	General conversation model (default)
`reasoning`	Reasoning model (o1, Claude thinking)
`image`	Image generation model
`video`	Video generation model
`embedding`	Embedding model
`code`	Code generation model

Caching Strategy

Dual-Layer Cache (SQLite + File)

flowchart LR
    subgraph ReadPath ["Read Path"]
        Read[readModelCache] --> CheckSQLite{useSQLite?}
        CheckSQLite -->|Yes| SQLiteRead[db.llmModelCacheGet]
        CheckSQLite -->|No| FileRead[fs.readFile]
        SQLiteRead -->|failure| FileRead
    end

    subgraph WritePath ["Write Path"]
        Write[writeModelCache] --> CheckSQLite2{useSQLite?}
        CheckSQLite2 -->|Yes| SQLiteWrite[db.llmModelCacheSet]
        CheckSQLite2 -->|No| FileWrite[fs.writeFile]
        SQLiteWrite -->|failure| FileWrite
    end

SQLite cache table:

Field	Type	Description
providerId	TEXT	Primary key
endpoint	TEXT	API endpoint
source	TEXT	Source ('api' / 'cache')
models	JSON	Serialized model list
raw	JSON	Raw API response
fetchedAt	INTEGER	Fetch timestamp
expiresAt	INTEGER	Expiry timestamp (24 hours)

File cache:

Directory: userData/model-cache/
Filename: {providerId}.json (special characters replaced with underscores)
No expiry mechanism (relies on forced refresh)

Transformer Chain Cache

LLMConfigService maintains an in-memory cache for Transformer chains to avoid repeated parsing:

chainCache: Map<string, { chain: ResolvedTransformerChain, cachedAt: number }>
CHAIN_CACHE_TTL_MS = 10 * 60 * 1000  // 10 minutes

getTransformerChain(key):
  cached = chainCache.get(key)
  if cached && (Date.now() - cached.cachedAt < TTL):
    return cached.chain
  // Otherwise re-parse
  chain = transformerService.resolve(...)
  chainCache.set(key, { chain, cachedAt: Date.now() })
  return chain

Model Routing

Chat → Code / Code → Chat Routing

resolveRoutedModel(providerId, modelId):
  config = loadConfig()

  // Check Code → Chat routing
  for route in config.routers.codeToChat:
    if route.providerId === providerId && route.modelId === modelId:
      return { actualProviderId: route.actualProviderId, actualModelId: route.actualModelId }

  // Check Chat → Code routing
  for route in config.routers.chatToCode:
    if route.providerId === providerId && route.modelId === modelId:
      return { actualProviderId: route.actualProviderId, actualModelId: route.actualModelId }

  // No matching route
  return null

Follow-Provider Model Mapping

resolveEffectiveModels():
  router = config.router
  agentDefaults = config.agentDefaultModels

  background = agentDefaults.background
  vision = router.vision

  if (followProviderBackground || followProviderVision) && router.default:
    defaultProviderId = router.default.split(',')[0]
    provider = getProvider(defaultProviderId)
    if provider:
      mapping = resolveFollowProviderModel(provider)
      // Use PROVIDER_MODEL_MAPPINGS to find the corresponding model
      if followProviderBackground: background = mapping.background
      if followProviderVision: vision = mapping.vision

  return { background, vision }

Provider Search Config (PROVIDER_SEARCH_CONFIGS)

Defines how each provider implements web search:

Search type	Description	Example providers
`model-param`	Enable search via a request parameter	DashScope, Baidu
`builtin-tool`	Inject a built-in tool definition	Kimi, Volcengine
`mcp`	Provide search tools via MCP server	Custom MCP
`sdk-native`	SDK native support (server-side tools)	Anthropic
`none`	Search not supported	Ollama

Provider Model Mappings (PROVIDER_MODEL_MAPPINGS)

Used by the Follow-Provider feature to automatically select the same provider's background/vision models:

Provider	Primary model	Background model	Vision model
`zhipu`	glm-5	glm-4.5-air	glm-4.6v
`volcengine`	ark-code-latest	doubao-seed-2.0-lite	doubao-seed-2.0-code
`kimi`	kimi-k2.5	kimi-k2-0905-preview	kimi-k2.5
...	...	...	...

IPC Integration Table

IPC Channel	Direction	Description
`llmConfig:discoverModels`	R → M	Trigger model discovery (supports forceRefresh parameter)
`llmConfig:getProviders`	R → M	Return the model list included in each provider

Extension Points

Adding Model Discovery Support for a New Provider

Ensure the provider template has modelsEndpoint set (e.g. /v1/models)
If the provider uses a non-standard models API format, add parsing logic in ModelDiscoveryManager.discoverModels()
Add URL derivation rules in utils.ts's deriveModelsEndpoint()

Adding a Follow-Provider Model Mapping

Add to PROVIDER_MODEL_MAPPINGS in packages/desktop/app/shared/provider-presets.ts:

// Pseudocode
PROVIDER_MODEL_MAPPINGS['newProvider'] = {
  primary: 'main-model-id',
  background: 'lightweight-model-id',
  vision: 'vision-model-id',       // null if no vision model
};

Customizing Cache TTL

Model discovery cache: controlled via the expiresAt field in SQLite (currently 24 hours)
Transformer chain cache: modify LLMConfigService.CHAIN_CACHE_TTL_MS (currently 10 minutes)

File	Relationship
`capabilities/llm/config-service/LLMConfigService.ts`	Host service that initializes ModelDiscoveryManager
`capabilities/llm/config-service/utils.ts`	Utilities: deriveModelsEndpoint, resolveApiKey, etc.
`capabilities/llm/config-service/AgentModelsManager.ts`	Model routing, Follow-Provider resolution
`shared/llm-config.ts`	PROVIDER_TEMPLATES, type definitions
`shared/provider-presets.ts`	PROVIDER_MODEL_MAPPINGS, PROVIDER_SEARCH_CONFIGS
`capabilities/llm/config/model-discovery.ts`	Model list merge/dedup utility (`mergeModelLists`)
`workers/DbClient.ts`	SQLite cache read/write
`routers/llm/ProviderRouter.ts`	IPC layer discoverModels
`capabilities/llm/completion/ThinkingResolver.ts`	Consumes discoveredModelMaxTokens

File Locations​

Architectural Context​

Data Structures​

Model Discovery Results​

Transformer Chain Cache​

Model Routing Configuration​

Algorithms and Logic​

Model Discovery Flow​

Fallback Model Resolution (getFallbackModels)​

Caching Strategy​

Dual-Layer Cache (SQLite + File)​

Transformer Chain Cache​

Model Routing​

Chat → Code / Code → Chat Routing​

Follow-Provider Model Mapping​

Provider Search Config (PROVIDER_SEARCH_CONFIGS)​

Provider Model Mappings (PROVIDER_MODEL_MAPPINGS)​

IPC Integration Table​

Extension Points​

Adding Model Discovery Support for a New Provider​

Adding a Follow-Provider Model Mapping​

Customizing Cache TTL​

Related Files​

File Locations

Architectural Context

Data Structures

Model Discovery Results

Transformer Chain Cache

Model Routing Configuration

Algorithms and Logic

Model Discovery Flow

Fallback Model Resolution (getFallbackModels)

Caching Strategy

Dual-Layer Cache (SQLite + File)

Transformer Chain Cache

Model Routing

Chat → Code / Code → Chat Routing

Follow-Provider Model Mapping

Provider Search Config (PROVIDER_SEARCH_CONFIGS)

Provider Model Mappings (PROVIDER_MODEL_MAPPINGS)

IPC Integration Table

Extension Points

Adding Model Discovery Support for a New Provider

Adding a Follow-Provider Model Mapping

Customizing Cache TTL

Related Files