Ask most sports data APIs for a "confidence score" on a matchup and you will get a number back. Every time. Whether or not anything real was measured to produce it. That's the default failure mode in this industry: a plausible-looking score is easier to ship than an honest "we don't know," and nobody downstream can tell the difference just by looking at the response.
PlayCaller's internal engineering rule is the opposite of that default: a score doesn't ship as intelligence unless it's backed by a real measurement, an AUC-ROC, a calibration curve, an out-of-sample hit-rate. If a signal can't show that, it's either labeled explicitly as an unvalidated heuristic, or it doesn't ship at all. Internally we call this the Hollow Capability rule. It governs every model and confidence field across every sport in the API.
What you actually get back when a model isn't there
Concretely: when you call an endpoint for a sport or matchup that doesn't have a validated model behind it yet, the confidence/heat field in that response is null. Not a neutral-looking 50%. Not a "we'll guess conservatively" fallback number. null.
A null is a true statement about what we don't know yet. A fabricated number is a false statement wearing a measurement's clothes, and it's worse than an obvious error, because it looks authoritative right up until someone builds a real product decision on top of it.
The models that do ship, and what backs them
Seven models are currently ACTIVE in production, each validated with real walk-forward testing (trains on data up to a point in time, tests only on data after it, rolls the window forward, repeats. No feature at observation time T is allowed to use data from T or later):
- NFL Confidence: AUC-ROC 0.697, 20,585 observations across 5 seasons (2021-2025), 4.61x decile lift.
- NFL Heat: AUC-ROC 0.623, shipped as a directional gradient rather than a percentage, because that's what the validation actually supports, not because a gradient looks better.
- NBA Confidence: AUC-ROC 0.847 on the 2025 holdout (0.855 averaged across 3 walk-forward folds).
- NBA Heat: AUC-ROC 0.74, 2 of 2 walk-forward folds passed.
- MLB Pitcher Form: AUC-ROC 0.636, shipped as a qualitative read (Trending Up / Steady / Trending Down) rather than a percentage, because the underlying score wasn't built to be displayed as one.
- MLB Batter Confidence: AUC-ROC 0.625, 26.98 percentage-point quartile lift. Earlier iterations capped around AUC-ROC 0.58 before a self-relative label and park-factor features got it over the bar. That earlier ceiling is part of the documented process, not quietly discarded.
- NHL Confidence: AUC-ROC 0.669, 101,973 observations.
Every sport runs its own model on its own data. None of these numbers are a single model relabeled across sports.
The rule applies backward too
This isn't just a rule for new models going forward. An older function in PlayCaller's own codebase, computeKalshiConfidence(), computed a score without a validated model behind it: a heuristic lookup, not a measurement. It was labeled as such in the source code rather than passed off as intelligence, and in October 2026 it was retired: the props pages now show the market's own prices and nothing else. The standard has to apply to code written before the standard existed, or it isn't a real standard.
Why this matters if you're building on top of an API
If you're building a fantasy tool, a DFS product, or anything that surfaces a "confidence" or "edge" number to a real user, the failure mode that actually costs you is silent: a vendor's hollow score looks identical to a real one in your UI, right up until it's wrong in a way a real model wouldn't have been. A null is annoying to build around. It is also the only honest answer for a case the underlying science doesn't cover yet, and every non-null number you do get comes with a published AUC behind it, not just our word.
Check it yourself
GET /api/model/status is public, requires no API key, and returns the live AUC-ROC, lift, and version for every ACTIVE model in the registry:
curl https://playcallerapp.com/api/model/status
If a model isn't in that response, any API field depending on it returns null instead of a fabricated number. Check the response yourself rather than taking our word for it. The full methodology behind how each of these models gets validated before it ships is covered in how we validate every ML model before it ships.
The Developer Sandbox is free, no credit card, 200 requests a day for two weeks: start a free API key.