Skip to Content
ArchitectureData Flow

Data Flow

This page describes the primary data flows in OpenModels — how data moves from community contributions through validation into the registry, how users discover models via the API, and how telemetry is collected from providers.

Registry Contribution Flow

When a contributor adds or updates a model, provider, or mapping in the registry, the following sequence occurs:

Validation Steps

The validation pipeline performs the following checks on every pull request:

StepCheckFailure Behavior
1YAML syntax parsingRejects malformed YAML files
2JSON Schema validationRejects files that don’t match schema definitions
3Referential integrityRejects mappings referencing non-existent models or providers
4Duplicate detectionRejects duplicate model or provider IDs

Model Discovery Flow

When a user searches for models or retrieves model details through the API:

API Endpoints

Endpoint PatternDescription
/api/modelsList and search models with filtering
/api/models/{id}Get full model details
/api/models/{id}/providersList providers for a model with pricing
/api/models/{id}/compareCompare providers side-by-side
/api/models/popularPopular models ranked by relevance
/api/providersList all providers
/api/searchUnified search across models and providers
/api/providers/{id}/health, /api/providers/{id}/latencyProvider health and latency data

Telemetry Collection Flow

The telemetry system continuously monitors provider health and latency:

Telemetry Metrics

MetricCollection IntervalRetention
Health statusHourly at minute 00 UTC30 days
Time to first token (TTFT)At 06:15 and 18:15 UTC30 days
Total response timeAt 06:15 and 18:15 UTC30 days
Availability (uptime %)Computed from health recordsRolling 7 days
FreshnessHealth older than 90 minutes is unknownMissing metrics are null

Probe Policies

Each provider can have a custom probe policy controlling:

  • Enabled/disabled — Whether probes run for this provider
  • Schedule — Celery beat controls the global cadence above; the policy’s interval_minutes setting does not reschedule the beat task
  • Daily limit — Maximum probes per day to control costs (default: 96)

Policies are tracked in Redis with daily counters that auto-expire after 24 hours.

Provider Ranking Algorithm

When a user requests ranked providers for a model via GET /api/models/{id}/providers/ranked, the API computes a composite score:

FactorWeightSource
Uptime percentage (24-hour rolling)40%Usable health probe records
Mean latency (TTFT)35%Provider-wide valid samples in the last 24 hours
Mean input/output price per million tokens25%Registry mapping data

Unknown health/latency earns no credit for that factor but stays null in the response. Model provider lists and comparisons use model-specific latency; this ranked endpoint currently uses provider-wide latency. Local TLS trust failures are excluded from uptime rather than counted as provider outages.

End-to-End Data Lifecycle

The complete lifecycle of data in OpenModels from contribution to user consumption:

  • Architecture Overview — High-level system architecture and component descriptions
  • Schemas — YAML schema definitions for models, providers, and mappings