How We Check AI API Provider Reliability (and What "Up" Really Means)

"Uptime 99.9%" is a number you can't trust from a marketing page, and an "all systems operational" banner is a snapshot that says nothing about the last 30 days. Reliability is the thing that most determines whether an AI API is usable in production, so it's worth being precise about what we can actually measure — and what we can't.

This is a plain-language look at how reliability gets assessed, and how to read the ratings in the directory.

What "up" can and can't tell you

At minimum, an endpoint being up means: you can reach it, it authenticates, and it returns a well-formed response in a reasonable time. That's a floor, not a ceiling. It does not tell you:

• Whether the output is correct. An endpoint can return a 200 and a confident, wrong answer.
• Latency distribution. "Responds" and "responds in 800ms vs. 8 seconds" are very different for a user-facing app.
• How it behaves under load. A provider can be fine for a single probe and degraded when its region is busy.
• The shape of its failures. Intermittent 5xx, silent timeouts, and long stalls all look like "down" in different ways.

So a status probe answers one narrow question — can I get a response right now? — and nothing more.

How status probes actually work

A probe is a small, repeatable request sent to the endpoint on a schedule:

1. Send a minimal, cheap request (often a one-token completion or a health/keys call).
2. Record whether it succeeded, and the latency it took.
3. Store the result with a timestamp.

Repeating this over time gives you a history — and the history is the valuable part. A single "up" is noise; a series of probes showing one provider's p95 latency creeping up, or sporadic failures clustering at certain hours, is a signal you can act on.

Two important caveats about any probe:

• It's from one network location. A probe from our servers reflects our path to the provider. Your users in another region may see different numbers. Probes are best-effort, not a guarantee about your experience.
• It's on a cadence, not continuous. Between probes, short incidents can happen. A probe every few minutes will miss brief blips.

That's why we label probe results as best-effort and never present them as a certification of the provider's own SLA. For production, you should watch the provider's own status page and run your own monitoring.

Latency is a first-class signal, not an afterthought

An endpoint can be "up" and still be too slow to be usable. We track latency alongside availability because the two are independent:

• Low latency, stable — good for real-time and interactive use.
• High latency, stable — fine for batch jobs, wrong for chat.
• Variable latency — the worst for user experience, even if average looks okay.

For a user-facing product, the tail (p95/p99) matters more than the mean. A provider that's usually fast but occasionally takes 10 seconds will feel broken to the users who hit the tail.

Reading a reliability rating

In the directory, reliability is shown as a combination of:

• Live status — the most recent probe (up / degraded / down / unknown).
• Latency — the most recent measured round-trip, in ms.
• Last checked — how stale the data is. Fresh data is useful; a week-old probe is just a memory.
• Risk tier — an editorial assessment that weighs more than a single probe: operator track record, terms transparency, reliability history, and data-handling clarity.

Keep the distinction clear: live status is a measurement, the risk tier is an assessment. The measurement can flip any second; the assessment is a slower, evidence-based judgment. Both are useful; neither is a guarantee.

What you should do for your own production

Treat any third-party reliability data (ours included) as one input, and the freshest one available — not the source of truth:

1. Run your own end-to-end monitoring against the exact model and region your users use.
2. Read the provider's status page and subscribe to incident alerts.
3. Design for failure anyway — because no probe predicts every outage. See building failover and redundancy.
4. Re-check before you commit. Reliability changes; a provider that's been rock-solid can degrade, and the reverse is true.

The bottom line

"Up" is a floor: reachable, authenticated, responsive. It doesn't cover correctness, latency, load behavior, or failure shape. Probes give you a best-effort, time-based picture from one vantage point; the risk tier layers in the slower editorial judgment. Use the freshest data you have, run your own monitoring, and build redundancy so you're not betting your product on a single provider's next incident.

Related reading

• The Real Risks of Third-Party AI API Routers
• Building Resilience: Failover & Redundancy
• How to Choose an AI API Router in 2026

FAQ

Does an "up" status guarantee the responses are correct? No. "Up" only means the endpoint is reachable and returns a well-formed response. It says nothing about the accuracy or quality of the model's output.

Why is probe latency "best-effort"? Probes come from one network location on a fixed cadence, so they reflect our path to the provider and can miss brief incidents between checks. They're a signal, not a measurement of your users' experience — run your own monitoring for production.

What's the difference between live status and a risk rating? Live status is a recent measurement (the last probe). A risk rating is a slower, evidence-based editorial judgment that weighs track record, terms, reliability history, and data handling. The measurement can change anytime; the rating changes less often.

How current should reliability data be before I trust it? The fresher, the better. A probe from hours ago is useful context; a probe from days or weeks ago is just a memory. Always cross-check with the provider's own status page before a production decision.