Building a Python Model Router After the September 3 Outage

Claude, OpenAI, and Grok went down together on September 3, 2026. A Python fallback layer that logs cost, latency, and provider would have kept requests moving.

What Happened on September 3

On September 3, 2026, around 8:00 AM Pacific, Claude, OpenAI's ChatGPT and Codex, Grok, and Gemini all went offline within the same hour. User reports spiked past 12,000 for OpenAI alone. The outage lasted over an hour for most services, and some users reported intermittent 404s and 503 errors well into the evening.

For anyone running production workloads—agent platforms, coding assistants, customer-facing chatbots—the simultaneous failure exposed a blunt fact: betting on a single provider, no matter how reliable, is a single point of failure. One agent platform operator mentioned their system handled the disruption through automatic retries and task suspension, resuming from checkpoints once the upstream API recovered. The design avoided putting availability "on a single point."

That approach—fallback routing with observability—is something any developer can build in an afternoon.

A Router That Tracks What Actually Happened

The practical response is a lightweight Python script that tries one model, falls back to another if the first fails, and logs which provider answered, how long it took, and what it cost. You want this information in a database so you can answer basic operational questions later: Which model handled the most requests last week?

What was the average latency by provider?

How much did fallback cost compared to the primary?

VicroCode lets you run Python online, store request logs in SQLite, and expose the results through a simple HTML dashboard you can run HTML online and share as a live link. The entire setup stays in one project.

Here's the routing logic:

import time
import sqlite3
import requests
from datetime import datetime

def call_model(provider, model_name, prompt, api_key, base_url):
    start = time.time()
    try:
        response = requests.post(
            f"{base_url}/chat/completions",
            headers={"Authorization": f"Bearer {api_key}"},
            json={
                "model": model_name,
                "messages": [{"role": "user", "content": prompt}]
            },
            timeout=10
        )
        response.raise_for_status()
        latency = time.time() - start
        data = response.json()
        tokens = data.get("usage", {})
        return {
            "provider": provider,
            "model": model_name,
            "latency": latency,
            "tokens_in": tokens.get("prompt_tokens", 0),
            "tokens_out": tokens.get("completion_tokens", 0),
            "success": True,
            "error": None
        }
    except Exception as e:
        return {
            "provider": provider,
            "model": model_name,
            "latency": time.time() - start,
            "success": False,
            "error": str(e)
        }

def route_with_fallback(prompt, providers):
    for provider_config in providers:
        result = call_model(
            provider_config["name"],
            provider_config["model"],
            prompt,
            provider_config["api_key"],
            provider_config["base_url"]
        )
        log_request(result, prompt)
        if result["success"]:
            return result
    return {"error": "All providers failed"}

def log_request(result, prompt):
    conn = sqlite3.connect("router_logs.db")
    cursor = conn.cursor()
    cursor.execute("""
        INSERT INTO logs (timestamp, provider, model, prompt, latency, tokens_in, tokens_out, success, error)
        VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
    """, (
        datetime.now().isoformat(),
        result["provider"],
        result.get("model"),
        prompt[:100],
        result["latency"],
        result.get("tokens_in", 0),
        result.get("tokens_out", 0),
        result["success"],
        result.get("error")
    ))
    conn.commit()
    conn.close()

You define your provider list—primary, fallback, and optional third option—and the script walks through them in order until one returns a valid response. Each attempt gets logged with timestamp, latency, token usage, and success or failure.

Estimating Cost From Token Counts

Most model providers charge per million tokens, with separate rates for input and output. Once you have token counts in the database, cost calculation is straightforward:

PRICING = {
    "deepseek-v4-flash": {"input": 0.0825, "output": 0.33},
    "glm-5.3-flash": {"input": 0.044, "output": 0.176},
    "gpt-4o-mini": {"input": 0.15, "output": 0.60}
}

def calculate_cost(model, tokens_in, tokens_out):
    if model not in PRICING:
        return None
    rates = PRICING[model]
    return (tokens_in / 1_000_000 * rates["input"]) + (tokens_out / 1_000_000 * rates["output"])

Add this as a query-time function or a scheduled batch update that writes cost back into the logs table. Either way, you end up with per-request cost visibility.

The Dashboard

The dashboard is a single HTML file that queries the SQLite database and renders charts using a lightweight library like Chart.js. VicroCode's HTML execution environment supports inline JavaScript, so the entire dashboard can live in one file:

<!DOCTYPE html>
<html>
<head>
    <title>Model Router Dashboard</title>
    <script src="/ai-market-guide/market/664/</script>
</head>
<body>
    <h1>Model Router Metrics</h1>
    <canvas id="latencyChart"></canvas>
    <canvas id="costChart"></canvas>
    <script>
        // Fetch data from Python backend (API endpoint hosted on VicroCode)
        fetch('/api/metrics')
            .then(r => r.json())
            .then(data => {
                new Chart(document.getElementById('latencyChart'), {
                    type: 'bar',
                    data: {
                        labels: data.providers,
                        datasets: [{
                            label: 'Avg Latency (s)',
                            data: data.latencies
                        }]
                    }
                });
                new Chart(document.getElementById('costChart'), {
                    type: 'bar',
                    data: {
                        labels: data.providers,
                        datasets: [{
                            label: 'Total Cost ($)',
                            data: data.costs
                        }]
                    }
                });
            });
    </script>
</body>
</html>

The `/api/metrics` endpoint is a Python script that reads from SQLite and returns JSON. Host both the API and the HTML file on VicroCode, and you have a live, shareable dashboard that updates as new requests flow through the router.

What This Solves

This setup does not prevent outages, but it reduces their impact. When your primary model fails, the router switches to the fallback without manual intervention. The logs tell you which provider carried the load, what the latency penalty was, and whether the fallback cost more than the primary would have.

After an event like September 3, you can query the database to see how many requests hit the fallback, how long they took, and whether the quality difference was acceptable. That gives you the data to adjust your provider priority list or negotiate better rates with the fallback provider if you end up using it more than expected.

For anyone building with AI coding tools or running agent-based workflows, this router is a practical safeguard. It is not about predicting the next outage—it is about having a system that keeps working when the next one happens.

Implementation Notes

VicroCode's Model Center APIs give you access to models already available on the platform. If you are routing between external providers like OpenAI and Anthropic, you will use their API endpoints and keys. The routing logic stays the same either way.

The SQLite database lives in your project's file storage. For higher traffic, you could move to a hosted database, but SQLite handles tens of thousands of requests without issue, and it keeps the architecture simple.

If you want to test failover behavior, simulate an outage by setting an invalid API key or base URL for your primary provider. The router should fall through to the backup within one timeout period (10 seconds in the example above). Adjust the timeout based on your latency tolerance.

Cost and Latency Tradeoffs

Fallback providers are often cheaper but slower, or faster but more expensive. The router logs both, so you can make an informed decision about which tradeoff fits your use case. For batch processing, a slower, cheaper fallback makes sense. For real-time chat, you might prefer a faster, pricier option even if it costs 2x.

The logs also reveal patterns you would not see otherwise. If your primary provider has intermittent slowdowns at certain times of day, the latency column will show spikes even when requests succeed. That might push you to route through the fallback during peak hours, not just during outages.

Unverified Claims

The specific failure rates, cost savings, and latency improvements from this approach depend on your traffic patterns, provider mix, and tolerance for quality variation between models. Real-world performance is unverified without running this setup against your actual workload.

Whether the fallback model produces output of equivalent quality to your primary is also workload-dependent and remains unverified without direct comparison on your tasks.