We live in an age of information overload. Every day, we’re bombarded with claims, facts, studies, and opinions. Some are rock-solid truth. Others are speculation dressed up as certainty. The problem? They often look identical.

So how do we tell the difference? First, let’s look at how certainty actually works – first in mathematics, then in science. This will give us an understanding of how “certain” things could actually be. Once we understand that, we’ll explore how we could apply similar principles to just every day life and the kinds of information we get constantly exposed to.

Table of Contents

Mathematical Certainty

Pure Math: The Gold Standard

In pure mathematics, certainty is absolute. When we say 2+2=4, there’s no wiggle room. Mathematical proofs are either valid or they aren’t. This certainty comes from logical necessity within a defined system of rules.

Applied Math: The Real World

The moment math touches reality, uncertainty creeps in. That’s not a failure, it’s honest measurement. We express it through:

• Error bounds: “This bridge can hold 50 tons, plus or minus 2 tons”
• Confidence intervals: “We’re 95% sure the actual number falls between 47 and 53”
• Significant figures: Writing 3.14 versus 3.14159 tells you how much precision we actually trust

Here’s the key: If you see a real-world measurement without any error bounds or confidence level, someone either hasn’t done their homework or is hiding the uncertainty.

How Science Handles Certainty

Science uses the same mathematical tools, but adds important layers of thinking.

The Science Confidence Ladder

• Laws (highest confidence): Mathematical relationships we’ve verified thousands of times. Things like gravity, thermodynamics, motion. Could we be wrong? Technically yes, but betting against them is betting against everything we know about reality.
• Theories (very high confidence): Big explanatory frameworks backed by mountains of evidence. Evolution. Relativity. Germ theory. Despite the word “theory,” this is as certain as science gets for complex phenomena.
• Hypotheses (testable uncertainty): Educated guesses waiting for validation. Scientists are explicit that these need more work.
• Speculation (acknowledged uncertainty): Ideas without enough testing yet. Honest scientists label these clearly.

The Critical Difference

Unlike pure math, science never proves anything absolutely true. Instead, evidence either piles up in support of an idea or contradicts it. Scientific certainty means “strength of evidence,” not “logical certainty.”

This isn’t a weakness, it’s what allows science to self-correct when better evidence appears.

How AI Evaluates Truth (And Why You Should Be Skeptical)

The Statistical Pattern Problem

An AI doesn’t “know” things the way you do. Its a sophisticated pattern-matcher trained on billions of text examples. AI confidence in a claim comes from:

1. How often it appeared in training data
2. How consistently sources stated it
3. How it connects to other patterns

This creates real problems:

• Popularity bias: If 10,000 websites repeat a myth, AI will think it’s probably true
• Knowledge cutoff: AI doesn’t know what happened after its training ended
• No reality check: AI can’t verify claims against the actual world, only against other text

What AI Can Do

Modern AI systems add some logical safeguards:

• Checking for internal contradictions
• Applying formal logic to math problems
• Recognizing patterns of known fallacies

But there’s no formula that says “Claim X is 73.4% certain.” AI predicts what text patterns should come next based on what its seen before.

Bottom Line

When AI says “evidence suggests,” its recognizing a pattern where sources expressed uncertainty, not independently judging the evidence. Its useful for synthesis and analysis, but its not a truth oracle.

By the Way

If you’ve ever been in a technical discussion with an AI and it sounds 100% confident while constantly making mistakes, this is why. Because AIs do not really have any such thing as “confidence”. They just string together words in popular patterns. THAT’S IT!

Your Practical Framework: The STEM Method

Here’s a system anyone can use to assess any claim, from news articles to social media to scientific studies.

S: Source Quality (Who Says So?)

Quick Rule: Drop your confidence level one notch for each tier you move down.

T: Testing & Replication (How Do We Know?)

Ask three questions:

• Can this claim be tested independently and objectively?
• Has it been tested multiple times?
• Have different independent researchers gotten the same results?

Quick Rule: If there’s zero testing or only one source, treat it as a hypothesis, not a fact, no matter how confidently it’s stated.

E: Error Bounds (How Precise Is This?)

Look for honesty about uncertainty:

• Are there ranges provided (plus or minus values)?
• Does the source acknowledge limitations?
• Is the claimed precision realistic for the method used?

Quick Rule: Watch out for false precision (claiming exact numbers when approximations are all we have) and missing error bars (treating estimates as absolute facts).

M: Motivation Analysis (Why Are They Telling Me This?)

Consider the incentives:

• Does the source benefit if you believe this?
• Is there financial, political, or ideological stake?
• Are opposing viewpoints mentioned, or only ignored?

Quick Rule: Higher stakes don’t mean the claim is false, but they do mean you need stronger evidence before accepting it.

Red Flags: Automatic Trust Reduction

Knock 20-40 points off your confidence level if you see:

• No sources provided or “just trust me” language
• Cherry-picked data that ignores contradictory evidence
• Appeal to authority instead of actual evidence (“Famous Person says so”)
• Mismatched confidence: Definitive language about preliminary findings
• No limitations acknowledged: Every study has weaknesses; honest researchers admit them

The 10-Level Certainty Scale

• >99.9% – Logical Certainty: Mathematical proofs, definitions, formal logic
  • Example: 2+2=4, a circle has no corners
• 99-99.9% – Empirical Laws: Physical relationships verified through countless experiments
  • Example: Gravity, thermodynamics, speed of light
• 90-99% – Robust Theories: Comprehensive frameworks with massive supporting evidence
  • Example: Evolution, germ theory, plate tectonics
• 80-90% – Well-Established Findings: Phenomena consistently replicated across multiple studies
  • Example: Smoking causes cancer, exercise improves health
• 60-80% – Moderate Evidence: Decent replication but notable limitations or conflicts
  • Example: Specific drug effectiveness claims, some dietary studies
• 40-60% – Preliminary Findings: Initial research showing patterns, lacking replication
  • Example: Single news article citing one study, FBI crime statistics (methodology limitations), corporate press releases
• 20-40% – Expert Speculation: Informed guesses by relevant experts without direct testing
  • Example: News with anonymous sources, government agency predictions, your own eyewitness account of a complex event
• 10-20% – Informed Conjecture: Hypotheses based on principles, extending beyond data
  • Example: Breaking news reports, social media posts from journalists, claims in partisan media
• 1-10% – Wild Speculation: Ideas with minimal basis in evidence or principles
  • Example: Viral social media claims, sensational headlines, “sources say” without attribution
• <1% – Contradicted/Disproven: Claims directly contradicted by strong evidence
  • Example: Flat Earth, vaccine-autism link, demonstrably false statements

Key Reality Check: Most news articles belong in the 20-60% range, not 80-99% where people mentally file them. Government agency claims without supporting data? 40-60% at best. Your eyewitness memory of an event? 20-40%.

The Real Takeaway

Mathematics and science give us tools to quantify uncertainty honestly. The STEM framework gives you a method to audit whether someone’s confidence matches their actual evidence.

Most mistakes we make don’t come from lack of information. They come from certainty inflation – treating moderate-confidence claims as near-certain facts, or treating speculation as established knowledge.

The next time someone presents information with absolute confidence, ask yourself: Does their certainty match their evidence? Or are they just hoping you won’t check?

In an age of information overload, the ability to evaluate certainty might be the most valuable skill you can develop. Not cynicism, careful calibration. Not blind trust, earned confidence.