A config-driven router that picks the right AI video model for every request, fails over when a provider goes down, and never charges a user for a failure.
Generative video products don’t run on one model. They run on a portfolio: a cheap self-hosted model for the free tier, a lip-sync model for dialogue scenes, a premium model for hero shots. Each one is sold by several vendors, and all of them have outages, rate limits and content filters.
This repo is a small, runnable version of the routing pattern I designed while building a generative video platform in production. No API keys needed: providers are simulated so you can see every behavior in a few seconds.
python3 demo.py # 6 scenarios, printed as a readable trace
python3 -m unittest -v # 10 tests
No dependencies. Python 3.8+.
New here? Read OVERVIEW.md for the full explanation in plain language, or try the live interactive demo in your browser.
flowchart LR
R[Request<br/>duration, plan,<br/>lip-sync, audio] --> F{Hard constraints<br/>filter}
C[(models.json<br/>catalog)] --> F
F -->|excluded + reason| X[Audit trail]
F --> S[Soft preference<br/>sort: cost, priority]
S --> T[Try candidate 1]
T -->|OK| B[Bill quoted price]
T -->|5xx / 429 / timeout| N[Try next candidate]
N --> B
T -->|400 / content policy| P[Stop, bill $0,<br/>tell the user]
3. Failover: primary is down, backup serves it, user never notices
try order kling-omni-official ($1.44) -> kling-omni-aggregator ($1.68)
attempt kling-omni-official retryable 503 service unavailable
attempt kling-omni-aggregator ok
RESULT served by kling-omni-aggregator, user billed $1.44, we absorbed $0.24
The code is short. The decisions are the point.
| Decision | Why |
|---|---|
| Users pay the quoted price, even when failover costs more. | An outage is our problem, not the customer’s. Tracking vendor_cost_usd separately makes the margin hit visible, so you can see which vendor’s reliability is costing you money. |
| Permanent errors stop immediately. | Retrying a content-policy block across three vendors triples cost and latency and still fails. Fail fast, bill nothing, explain why. |
| Every exclusion carries a human-readable reason. | Support, sales and PMs can answer “why didn’t it use Veo?” without asking an engineer. It also powers upgrade prompts. |
| “No model qualifies” is a product moment, not an error. | A free user asking for audio (scenario 6) is the best time to show an upgrade, because the trace already says which plan unlocks it. |
| A cheap self-hosted model anchors the free tier. | Near-zero marginal cost is what makes a real free tier affordable. Paid tiers route to vendor models. |
models.json the catalog (illustrative costs, not real vendor pricing)
router/catalog.py loads the catalog
router/resolver.py hard-constraint filter + soft-preference sort
router/runner.py tries candidates, classifies errors, bills
router/providers.py simulated vendors (swap in real adapters here)
demo.py six scenarios
tests/ unit tests
playground/ interactive visual demo (open index.html)
OVERVIEW.md full plain-language explanation
LEARN.md 30-minute hands-on walkthrough
This is the core pattern, deliberately small. A production version also needs: async job polling (video generation takes minutes), per-vendor circuit breakers, health-weighted ordering, idempotency keys so a retry never double-generates, and cost/latency/quality telemetry feeding back into the catalog.
Built by Prem Dutta, AI product leader. I build generative AI products and care about the unit economics behind every generated second.
MIT licensed.