Skip to main content

API Key Pools

API key pooling lets you configure multiple API keys for the same provider. Requests are distributed using weighted round-robin, and the system automatically switches to another available key when rate limits or failures are encountered.

When to Use​

  • High-concurrency scenarios: A single key's rate limit is insufficient; multiple keys share the traffic load
  • Team sharing: Each team member uses their own key quota to split costs
  • Stacking free tiers: Keys from multiple free accounts are used in rotation
  • High availability: When one key fails, the system automatically switches to another key to prevent interruptions
  • Test/production separation: Use different weights to control the proportion of requests going to production keys vs. test keys

How-To Guide​

Adding an API Key​

  1. Open Settings → Provider Management
  2. Select the target provider (e.g. OpenAI)
  3. Locate the API Key Pool section
  1. Click the Add Key button
  2. Fill in the information:
FieldRequiredDescription
API KeyYesKey value or environment variable reference (prefixed with $)
LabelNoA name for easy identification, e.g. "Production Key #1" or "Test Key"
WeightNo1–100, defaults to 1; higher values increase the probability of selection
  1. Click Confirm to add

Setting the Weight​

  1. Find the target key in the API key list
  2. Modify the Weight value (1–100)
  3. The weight determines the relative probability of that key being selected

Enabling / Disabling an Individual Key​

  1. Find the target key in the API key list
  2. Toggle the Enabled switch
  3. Disabled keys are excluded from the rotation but their configuration is preserved

Deleting a Key​

  1. Find the target key in the API key list
  2. Click the Delete button
  3. Confirm the deletion (this action cannot be undone)

Configuration Reference​

SettingTypeDefaultRangeDescription
API KeyString(empty)--Key value or $ENV_VAR reference
LabelString(empty)--Note name for distinguishing keys
WeightInteger11–100Weight for weighted round-robin
EnabledBooleantrue--Whether this key participates in rotation
OrderInteger0--Display order in the list

Behavior Notes​

Weighted Round-Robin​

Each key in the pool occupies a number of "slots" proportional to its weight. Elftia cycles through these slots in a fixed sequence, distributing requests according to the weight ratio.

Weight distribution example:

Assume 3 keys with weights 3, 2, and 1 (total weight = 6):

KeyWeightShareSelections per 6 requests
Key A350%3
Key B233%2
Key C117%1

Rotation order: A → A → A → B → B → C → A → A → A → B → ...

Equal-weight example:

3 keys each with weight 1 (total weight = 3), so they rotate strictly: A → B → C → A → B → C → ...

Session Affinity​

All requests within the same chat session are bound to the same API key. This design has two important reasons:

  1. Prompt caching: Some providers (e.g. Anthropic) support prompt caching — consecutive requests using the same key can hit the cache, significantly reducing latency and cost
  2. Consistency: Prevents rate-limit counter resets caused by frequent key switches within the same conversation

Session binding is reassigned in the following situations:

  • The currently bound key is disabled or deleted
  • The currently bound key enters a cooldown period due to errors
  • The session ends (closed or deleted)

Automatic Failover​

When a request encounters an error, the key pool applies different strategies based on the error type:

Rate Limiting (429/529)​

When an HTTP 429 (rate limit) or 529 (service overloaded) response is received:

  1. The current key enters a cooldown period
  2. The system automatically switches to the next available key in the pool
  3. The request is retried with the new key

Cooldown mechanism:

Consecutive failuresCooldown durationFormula
1st60 seconds60s × 2^0
2nd120 seconds60s × 2^1
3rd240 seconds60s × 2^2
4th480 seconds60s × 2^3
5th and beyond900 seconds (cap)min(60s × 2^(n-1), 900s)

The key is excluded from rotation during the cooldown period and automatically becomes available again once it expires. A successful request resets the consecutive-failure counter for that key.

Authentication Failure (401/403)​

When an HTTP 401 (Unauthorized) or 403 (Forbidden) response is received:

  1. The key is permanently disabled (marked as disabled in the database)
  2. The system automatically switches to the next available key
  3. In the settings UI, the key is shown as disabled

This is because authentication errors usually indicate the key itself has become invalid (expired, revoked, or quota exhausted) and is unlikely to recover by waiting.

Failover Flow Chart​

Request sent
    |
    v
Use session-bound key
    |
    +---> Success ---> Reset cooldown counter ---> Return response
    |
    +---> 429/529 (rate limited)
    |         |
    |         v
    |     Current key enters cooldown
    |     (starts at 60s, exponential backoff, cap at 15 minutes)
    |         |
    |         v
    |     Any other available keys in the pool?
    |         |           |
    |        Yes          No
    |         |           |
    |         v           v
    |     Switch to new key    Request fails, return error
    |     Rebind session        (all keys unavailable)
    |         |
    |         v
    |     Retry with new key
    |
    +---> 401/403 (authentication failure)
              |
              v
          Permanently disable this key
              |
              v
          Any other available keys in the pool?
              |           |
             Yes          No
              |           |
              v           v
          Switch to new key    Request fails, return error
          Rebind session

Relationship Between Key Pool and Provider-Level API Key​

  • When a provider has both a provider-level API key (the api_key field in the Provider configuration) and a key pool configured, the key pool takes priority
  • If the key pool is empty (no entries), the provider-level API key is used
  • Using the key pool instead of a provider-level API key is recommended for load balancing and failover capabilities

Troubleshooting​

IssuePossible causeSolution
All keys are in cooldown; requests failAll keys hit rate limits simultaneouslyWait for the cooldown to expire (up to 15 minutes), or add more keys to expand pool capacity
A key is never selectedWeight is 0, or other keys have much higher weightsCheck weight settings; ensure the weight is at least 1
A key is automatically disabledA 401/403 authentication error was receivedCheck whether the key is expired or revoked; verify the key's status on the provider's website, then manually re-enable it after fixing the issue
Pool is configured correctly but requests still use the old keySession affinity is bound to the old keyStart a new chat session, or wait for the old key to enter a cooldown period so the system automatically rebinds
Added key not taking effect immediatelyKey cache not refreshedSave the configuration and retry; the system refreshes the cache automatically
Environment variable key cannot be resolvedVariable name is wrong or not setConfirm the variable name after $ exactly matches the one set in the system, including case
Cooldown is too longExponential backoff from multiple consecutive rate-limit hitsReduce request frequency, or add more keys to spread the load. The cooldown cap is 15 minutes
Key switched within the same sessionOriginal bound key entered cooldown or was disabledThis is expected behavior; the system automatically selects the best available key