Provider routing, abstraction & failure

Provider abstraction model

Every provider adapter implements one contract, BaseProvider.audit(task, instructions, input_text) → ProviderResult: and returns the same stable result shape, so callers treat mock and live providers identically.

Adapter contract, enforced by tests: never raise (return a safe fallback instead), never log raw prompt/input, never place secrets in diagnostics. selection.get_provider(name, config) resolves a name to an adapter; unknown or unimplemented names return MockProvider, so the app never crashes on an unknown provider. The mock adapter is deterministic, offline, and always available, the universal safe fallback and the default provider.

Provider routing

ProviderRouter.route(...) walks a candidate chain. The primary is the explicit requested provider, else the configured default, else mock. The chain is [primary, *fallbacks, "mock"]: deduplicated, with mock guaranteed last so routing can never fail to produce a result.

flowchart TD
  A["route(task, instructions, input, requested?)"] --> B["primary = requested or default or 'mock'"]
  B --> C["_build_chain: [primary, *fallbacks, 'mock'] deduped, mock last"]
  C --> D["for name in chain:"]
  D --> E{"_availability(name)"}
  E -->|"mock → always available"| G["attempt name"]
  E -->|"enabled AND key present"| G
  E -->|"disabled → skip(reason=disabled)"| F["record skip · fire on_provider_fallback · continue"]
  E -->|"no key → skip(reason=missing_key)"| F
  F --> D
  G --> H["provider.audit() → ProviderResult (adapter self-falls-back)"]
  H --> I["ACCEPT first attempted result"]
  I --> J["RoutingResult(fallback_used = name != primary)"]

Availability, not success, drives fallback. mock is always available; a live provider is available iff it is enabled and its key is present (presence only, the key value is never read here). Unavailable candidates are skipped with a reason (disabled or missing_key) and an on_provider_fallback hook fires. The first available candidate is attempted and its result is accepted: because every adapter already self-falls-back to a safe payload, an attempted provider is never retried elsewhere.

route_forced(name) is the legacy single-provider entry: it attempts exactly one provider with no skipping, no chain and no mock substitution (the adapter still self-falls-back). The audit endpoint's backward-compatible path uses it. The router emits safe routing_metadata: requested/selected providers, attempted list, skipped list with reasons, fallback_used, fallback_reason, router_mode (explicit/default/forced), and mirrors it into diagnostics.

Failure handling

Failure is handled at three nested levels, so the request path never raises:

1. Adapter. Each audit() catches its own errors and returns a safe ProviderResult. The Gemini adapter is a small state machine, every branch returns a valid result:

stateDiagram-v2
  [*] --> disabled: not enabled by flag
  [*] --> missing_key: enabled, no key (mock mode)
  [*] --> mock_ok: enabled + key, provider_mode=mock
  [*] --> live_ok: enabled + key + live, upstream ok
  [*] --> error: live attempt failed / upstream unavailable
  disabled --> [*]: mode=disabled, status=fallback
  missing_key --> [*]: mode=disabled, status=fallback
  mock_ok --> [*]: mode=mock, status=ok
  live_ok --> [*]: mode=live, status=ok
  error --> [*]: mode=error, status=error (safe payload)

2. Router. Unavailable candidates are skipped, not attempted; if the chain were ever exhausted, mock backstops with fallback_reason="chain_exhausted". The router never raises on well-formed input.
3. Hooks. The Evidence Engine hook seams (before_provider_call, after_provider_result, on_provider_fallback) default to no-ops and are additionally wrapped in try/except so a custom hook can never break routing.

The failure surfaces to the caller as data: status="error"/mode="error" on the provider block, fallback_used=true, and the skipped_providers reasons, all inside a normal 200 response and a valid evidence envelope.

Retry behavior