Skip to content

The Oracle Problem

By Bri Stanback 14 min read

My daughter asks me why things are. Constantly. Why is the sky blue? Why do dogs bark? Why can't I have ice cream for dinner? She's three, so the questions are relentless and sincere, and I answer them because that's the deal — she can't look it up herself, so she asks someone she trusts, and she believes what I say. That's the whole arrangement.

She's using me as an oracle.

The word gets thrown around loosely in tech — oracle databases, blockchain oracles, test oracles — but the core concept is older than software and simpler than we make it. An oracle is anything you go to for answers you can't verify yourself. The pharmacist in a pre-modern town. The priest interpreting scripture. Consumer Reports telling you which dishwasher to buy. Google's first page of results. Your friend who "knows wine."

The value of an oracle is precisely that you don't do the work yourself. That's the point. You trust the oracle because the alternative is learning immunology to evaluate a supplement claim, or reading 600 mattress reviews to find the one that isn't astroturfed. Oracles exist because verification is expensive and life is short.

But here's the thing I keep coming back to: the same asymmetry that makes an oracle valuable is what makes it dangerous. You can't check the oracle's work — that's why you need it. Which means if the oracle is wrong, or compromised, or optimizing for something other than your question, you have no way to know. Not until the crop fails. Not until the code doesn't work at checkout. Not until 2008.


#The accidental oracle

Most oracles don't set out to be oracles. That's the first problem.

Google didn't intend to become the arbiter of truth for the internet. It built a good search engine, people started trusting the first page of results as the answer, and suddenly a ranking algorithm was functioning as an epistemological authority. No one voted on this. No one audited the methodology. It just happened — adoption conferred oracle status whether Google wanted it or not.

Reddit is the same story, playing out right now in commerce. People append "reddit" to every product search because they've lost faith in the professional review ecosystem. Wirecutter got bought by the New York Times and now recommends the same $300 headphones in every roundup. Amazon reviews are flooded with incentivized fakes. So people route around the corrupted oracles to the one place that still feels like real humans with real opinions.

But Reddit never built the structural defenses that oracle status demands. Astroturfing is rampant. Brand accounts pose as regular users. Entire subreddits are quietly moderated by company employees. The oracle is already degrading, and most people using it don't know yet — because the whole point of an oracle is that you trust it without checking.

This is the pattern: adoption confers oracle status. Oracle status demands structural integrity. But the structural integrity is never built, because nobody planned to be an oracle in the first place.


#What actually makes an oracle trustworthy

I've been thinking about this as a taxonomy — not of oracle types, but of the structural conditions that separate the oracles that stayed trustworthy from the ones that didn't.

1. Incentive separation — the oracle can't profit from the answer going one way.

This is the big one. Consumer Reports has been trusted since 1936 for one reason: they don't accept advertising from the products they review. The incentive to be right and the incentive to make money point in the same direction. Compare that to Moody's and S&P, who were trusted credit rating agencies until the entities being rated started paying for the ratings. Nobody at Moody's decided to lie about mortgage-backed securities. The incentive structure did it for them. The AAA ratings that fueled 2008 weren't fraud in the mustache-twirling sense — they were the predictable output of a system where the oracle's revenue depended on pleasing the subject of the oracle's judgment.

Grokipedia is an interesting case — not a stealth capture but a declared corrective, launched explicitly to counter what its creators see as left-leaning bias in Wikipedia. That transparency is worth something. But the structural problem remains: the oracle's editorial direction is inseparable from its owner's worldview, with no independent veto points — no crowd of editors pushing back, no published methodology that outsiders can audit against (as of this writing, no public editorial guidelines, no article version history, and corrections reviewed by Grok itself). Wikipedia has its own well-documented biases — editor demographics, coverage gaps, activist editing on contested topics — so Grokipedia isn't wrong that the incumbent oracle has problems. The question is whether correcting one set of biases without structural defenses against over-correction just produces a mirror-image failure. Declared intent is better than hidden drift. But it's not structural integrity.

There's a deeper version of this problem, and it connects to something I've been thinking about with defaults. Grokipedia's "maximum truth-seeking" ethos frames objectivity as the removal of context — strip the editorializing, get to the ground truth. That impulse isn't wrong. Nobody wants an encyclopedia that substitutes ideology for evidence. But context isn't the same thing as feelings. The history of how data has been used, the systemic patterns that shape who a finding affects and how, the documented impacts on real populations — these are facts about facts, not editorial commentary. Presenting findings in a vacuum, especially on complex human questions where base rates, individual variation, and real-world consequences matter, can invite the very misinterpretation that a truth-seeking oracle should prevent. At the same time, once you start choosing which context to include and how to weight it, you're making editorial decisions — and that's exactly where consensus oracles have slid into motivated synthesis. The line between necessary context and stealth editorializing is thin. But the decision to draw that line somewhere — or to pretend you haven't drawn it at all — is a specific editorial position either way. The best oracles navigate this boundary honestly rather than claiming they've transcended it.

2. Track record against reality — the receipts.

The farmer's almanac worked not because anyone understood its methodology, but because the predictions were testable. The corn grew or it didn't. The frost came when the almanac said it would, or it didn't. You don't need to understand the oracle's process if you can verify its outputs over time.

This is what made early Google trustworthy, actually. You searched, you found what you needed, you came back. The track record was the trust. And it's what makes the degradation so insidious — the track record erodes slowly, one SEO-gamed result at a time, until one day you're adding "reddit" to every query and you can't remember when you started.

In commerce, this maps directly to something like code success rate — the percentage of promo codes that actually work at checkout. Not "we have a code for this store" but "this code worked when someone tried it." That's an almanac-style oracle. The output is testable. The trust is earned one correct prediction at a time, and lost the same way.

3. Methodology transparency — showing enough work to be auditable, but not so much that you're commoditized.

This is where it gets interesting, because oracles exist on a transparency spectrum, and the right position on that spectrum depends on what kind of oracle you are.

Fully opaque: the Delphic Oracle. "A great empire will be destroyed." Whose? Didn't say. Trust me.

Fully transparent: Wikipedia. Anyone can check the sources. But then you're a commons, not an oracle — and you're vulnerable to the people who care most about editing the page being the people with the most to gain from it saying a specific thing.

The interesting oracles live in the middle. You can see the methodology. You can audit the process. You probably can't replicate the work yourself — because that's the whole reason you need the oracle — but you can evaluate whether the approach is sound. Credit rating agencies (in theory) publish their criteria. UL publishes its testing standards. The question isn't "show me everything" — it's "show me enough that I can decide whether to trust your process."

4. Acknowledged uncertainty — confidence, not conviction.

The worst oracles speak in absolutes. "This supplement cures brain fog." "This is the best mattress." "AAA-rated." Good oracles speak in probabilities and conditions. "73% confidence this code works, last verified 4 hours ago." "Strong evidence at 5% concentration; this product contains 0.3%." "Likely safe for most adults; contraindicated with X."

This is counterintuitive because certainty feels more trustworthy. If I'm asking an oracle for an answer, I want the answer, not a confidence interval. But certainty is what sycophantic AI gives you, and it's what every corrupted oracle eventually sells. The hedge is the tell. An oracle that says "I'm not sure" is an oracle that's optimizing for accuracy over comfort. That's the one you want.


#The four species

I keep sorting oracles into four buckets, and everything I look at seems to fit:

Prophetic oracles predict what hasn't happened yet. Stock pickers, weather forecasters, the Delphic priestess. Can't show their work because the work is intuition or modeling of inherently uncertain systems. Trustworthy only via track record. Most aren't.

Computational oracles process more data than you can. Credit rating agencies, truth refineries, diagnostic AI. They can show methodology, and the good ones do. Trustworthy when incentive-separated and auditable. Dangerous when the entity being scored pays for the scoring.

Consensus oracles aggregate signal from many sources. Reddit, Wikipedia, blockchain oracles (Chainlink), Rotten Tomatoes. Trustworthy when the consensus is hard to game. Fragile when motivated actors can flood the input.

Captured oracles started in one of the first three categories and slid into corruption. Not through malice — through incentive drift. Moody's was a computational oracle that got captured by its revenue model. Google was a consensus oracle that got captured by advertising. Grokipedia is a computational oracle that declared its corrective intent upfront — more transparent about its biases than most, but with fewer structural guardrails against over-correction. The slide from trusted to captured is usually gradual, rarely intentional, and almost never reversed.

The scary insight is that every oracle trends toward capture unless something structural prevents it. Incentive drift is the default. Consumer Reports has held for 90 years because the no-advertising rule is constitutional, not policy — it's in their charter, not a best practice someone can waive in a tough quarter. UL has held because the testing standards are published and the manufacturers don't pick their own testers. The oracles that survive are the ones where the integrity mechanism is load-bearing architecture, not good intentions.


#The oracle's oracle

Here's where it gets recursive — and where I think the interesting future is.

If AI agents start mediating decisions (and they already are — shopping, booking, researching, recommending), those agents need oracles too. An AI agent deciding which flight to book for you needs a source of truth about prices, timing, seat quality. An agent evaluating a supplement claim needs a source of truth about clinical evidence. An agent checking whether a promo code works needs an oracle for that — and even that question isn't as binary as it sounds. The code exists, sure, but does it work for this user, with this cart, at this moment? The truth is conditional. And an agent deciding whether a $48 vitamin C serum is meaningfully different from a $12 one needs something deeper still — an oracle that's deconstructed the physics of the category itself. What does vitamin C stability actually require? What concentration matters? What delivery mechanisms have clinical evidence? The code question is conditional. The product question is structural.

But AI agents don't have dopamine. They can't be emotionally manipulated by "only 2 left!" or influenced by a clean brand aesthetic. They're going to route to whichever oracle gives them the most reliable signal. Which means the oracle game is about to get a lot more structural and a lot less vibes-based.

I wrote about the dopamine layer a few weeks ago — the three layers of noise between a person and a good decision. The attention layer selects what you see. The dopamine layer makes it feel urgent. The rationalization layer helps you construct a case for the emotional decision you've already half-made. Oracles sit upstream of all of that. The oracle is the thing telling you what's true before the noise layers get involved. Which is why a compromised oracle is worse than no oracle — it poisons the input, and everything downstream inherits the distortion.

In the agent era, the question isn't "how do we build AI that's less manipulable?" It's "which oracles will the AI trust, and what makes those oracles earn it?" Agents will do what humans can't: actually audit methodology, compare track records across providers, and switch to a better oracle the moment accuracy drops. The oracle market is about to get competitive in a way it never was when humans were the consumers, because humans are loyal and lazy and biased toward the familiar. Agents are none of those things.

That's either terrifying or exciting depending on whether your oracle is trustworthy or just popular.


#Where the oracle stops

Not all claims live in the same epistemological neighborhood.

A promo code works or it doesn't. A serum contains retinol at 0.5% or it doesn't. A sunscreen's SPF passes an independent lab test or it doesn't. These are the oracle's home turf — testable, falsifiable, receipt-shaped. The corn grew or it didn't.

Move one step out and the ground is still solid but the shape changes. A beauty brand redesigns its packaging to mono-material tubes and publishes the data: 46% reduction in carbon emissions, 20% less waste. A company's 990 filings show exactly where its charitable donations went. A retailer pulls a product from shelves before regulators require it. These are verifiable actions — not opinions, not claims of virtue, just documented behavior. But they imply something about values without the oracle having to say what.

Certifications occupy a similar zone, and they're worth pausing on because they're oracles in their own right. COSMOS processes ingredient data against published standards and outputs a verdict. Credo's Clean Standard gates which brands get shelf space. COSMOS maintains a public enforcement log that documents decertifications — a brand that lost its stamp for inadequate supply chain traceability is generating a different kind of receipt than one that's held certification for a decade. An oracle can verify all of this: the certification exists, the enforcement history is clean, the standard was met. Receipt about a receipt. Solid ground.

But notice how quickly the ground softens. "How meaningfully does this brand exceed the standard?" is a different question than "does this brand hold the certification?" A brand that barely clears the threshold and a brand that proactively reformulates ahead of standard updates are both "certified." The oracle can surface both facts. It cannot tell you which one should matter more to you — because that depends on what you're optimizing for, and that's yours.

And then there are claims that aren't really claims at all — they're identities. "This brand is ethical." "This company is a good ally." "This product is clean." These depend on definitions that shift by person, by community, by year. "Clean beauty" has no legal definition. Two people can look at the same ingredient list — same formulation, same certifications — and draw opposite conclusions about what they consider safe. The moment an oracle renders a verdict here, it's stopped being an information engine and become something else. A priest, maybe. A marketer, definitely.

The line between these layers matters because crossing it is almost always invisible and almost always gradual. Nobody announces "today we start rendering moral judgments." It happens one feature at a time — a "clean" badge here, a "sustainable" score there — until the oracle is quietly doing synthesis it never meant to do. And the problem with that drift is structural, not moral: the consumer can't tell the difference between an oracle that says "this brand is ethical" because it genuinely is and one that says it because the brand is a paying partner. That's not cynicism. That's the oracle problem in its purest form — the same asymmetry that makes the oracle valuable makes its motivations uncheckable.

The oracle that lasts is the one that knows where it stops. Receipts, not verdicts. Evidence, not synthesis. The meaning is yours.


#The question I'm sitting with

I work at a company building what we call a truth layer for commerce — two surfaces of the same commitment. One verifies whether the promo code works at checkout — not just "does this code exist" but "does it work for this user, with this cart, right now." Conditional, testable, receipt-shaped. The other deconstructs the physics of product categories themselves — the ingredients, the formulations, the claims, the clinical evidence, the gap between what a brand says and what the chemistry supports. SimplyCodes is the code oracle. Product.ai is the product oracle. Same architecture underneath: verify the claim, score the confidence, show the receipts, and don't let the merchant's ad spend influence the truth score.

I believe in it — not as marketing language, but as an architectural commitment.

But I also know the Moody's people believed in their methodology. I know early Google believed in "don't be evil." I know every captured oracle started with good intentions and drifted through incentive pressure so gradual that nobody noticed until the trust was already gone.

So the question isn't whether we want to build a trustworthy oracle. Everybody wants that. The question is whether we've built the structural conditions that make trustworthiness survivable — the kind that holds up in a bad quarter, under pressure from a big client, when the easy path is to let a thumb on the scale go unremarked.

The farmer didn't trust the almanac because the almanac had good values. The farmer trusted the almanac because last year's frost prediction was right, and the year before that, and the year before that. The receipts were the trust. Everything else was just words.

My daughter will stop using me as an oracle someday. She'll learn to look things up, evaluate sources, form her own judgments. The three-year-old's unconditional trust will be replaced by something more provisional and more earned. That's healthy. That's what growing up means.

The question is whether our oracles are growing up too — or whether we're still in the phase where we believe them just because they sound confident. And the sign of a grown-up oracle isn't that it has all the answers. It's that it knows which questions aren't its to answer.


Previously: The Dopamine Layer explored the three layers of noise between consumers and good decisions. This piece is about what sits upstream — the sources we trust to tell us what's true in the first place.

Tagged

  • ai
  • systems
  • epistemology
  • architecture