System Architecture Overview

OpenModels is a hybrid infrastructure layer for discovering, validating, and comparing LLM models and inference providers. The system follows a registry-first, model-centric architecture where:

• Open Registry — YAML files in a public GitHub repository serve as the single source of truth for models, providers, and mappings
• Proprietary Platform — Internal services provide telemetry, analytics, search, and a web interface
• Model-Centric Design — Users search for canonical models (e.g., deepseek-v3, qwen3-coder) and discover which providers offer them

High-Level Architecture

Key Design Principles

Technology Stack

System Components

1. Public Registry

The registry is a public GitHub repository containing YAML definitions for all models, providers, and mappings. It serves as the canonical source of truth for the entire platform.

openmodels/
├── models/           # Model definitions (e.g., deepseek-v3.yaml)
├── providers/        # Provider definitions (e.g., openai.yaml)
├── mappings/         # Model-to-provider mappings with pricing
│   ├── openai/
│   ├── anthropic/
│   └── ...
└── schemas/          # JSON Schema definitions for validation

2. Validation Pipeline

A GitHub Actions workflow that runs on every pull request to the registry:

1. YAML Syntax — Validates that all files are parseable YAML
2. Schema Validation — Checks files against JSON Schema definitions
3. Referential Integrity — Ensures mappings reference existing models and providers
4. Duplicate Detection — Prevents duplicate model or provider IDs

3. Ingestion Script

A Python script triggered on merge to main that loads registry data into PostgreSQL:

• Uses upsert logic (INSERT ... ON CONFLICT UPDATE) for idempotent operations
• Maintains transaction boundaries for data consistency
• Invalidates affected Redis cache keys after updates

4. Platform API

A NestJS REST API providing model/provider discovery and telemetry endpoints:

5. Telemetry Worker

A Python Celery worker that monitors provider health and latency:

• Health probes — Every 5 minutes, checks provider API availability
• Latency probes — Every 15 minutes, measures time-to-first-token and total response time
• Results are stored in PostgreSQL and exposed via the Platform API

6. Web Interface

A Next.js 16 application providing a user-facing interface for:

• Searching and browsing models
• Comparing providers side-by-side (pricing, latency, uptime)
• Viewing real-time telemetry dashboards

7. Caching Layer

Redis provides high-performance caching with key-based invalidation:

• Model/Provider queries — 5 minute TTL
• Telemetry data — 1 minute TTL
• Pattern-based invalidation — When data changes, affected cache keys are cleared

System Boundaries

In Scope

• Registry management (YAML files, schemas, validation)
• Data ingestion pipeline (YAML → PostgreSQL)
• REST API for model/provider discovery and comparison
• Telemetry collection (health probes, latency monitoring)
• Web interface for browsing and comparing models
• Caching layer for performance optimization

Out of Scope

• Actual LLM inference (delegated to providers)
• User authentication and authorization (future phase)
• Billing and payment processing
• Model fine-tuning or training

Related Pages

• Data Flow — Detailed sequence diagrams for registry contributions and model discovery
• Schemas — YAML schema definitions for models, providers, and mappings