Mathematics
How to Interpret p-Values Correctly
Quick fact
A p-value of 0.03 does NOT mean there's a 3% chance the results are due to chance; that's a common misconception.
Why this is interesting
You've seen p-values in every study: p < 0.05 means 'significant.' But what does that number actually tell you? It might not be what you think.
Read the full explanation
Understanding How to Interpret p-Values Correctly
Imagine you're testing a coin to see if it's fair (the null hypothesis). You flip it 100 times and get 60 heads. The p-value is the probability of getting 60 or more heads IF the coin is actually fair. If that probability is low (say 0.03), it means such an extreme result is unlikely under fairness. But it doesn't tell you the probability the coin is unfair—it only measures how surprising your data would be if fairness were true. The p-value is a measure of surprise, not a verdict on the hypothesis.
A deeper explanation
The p-value is defined as P(data or more extreme | null hypothesis is true). It quantifies the compatibility of the observed data with the null hypothesis. A small p-value suggests that, under the null, such an outcome is rare—so the data is inconsistent with the null. However, the p-value does NOT tell you the probability that the null is false, nor the probability of a Type I error. It also doesn't reflect the magnitude of an effect—a tiny effect can be statistically significant with a large sample. Correct interpretation requires understanding this conditional probability and avoiding the common pitfalls: equating p with the probability that the hypothesis is wrong, or treating p<0.05 as a magic threshold.