Technology
Algorithmic Bias in Criminal Justice Risk Assessment Tools
Quick fact
A 2016 ProPublica investigation found that COMPAS, a widely used risk assessment tool, falsely flags Black defendants as high-risk for reoffending almost twice as often as white defendants, while white defendants are more often mislabeled as low-risk.
Why this is interesting
Imagine a computer program helping a judge decide whether someone goes to jail before trial—and that program judging people differently because of their race. How can a machine, built on numbers, become biased?
Read the full explanation
Understanding Algorithmic Bias in Criminal Justice Risk Assessment Tools
Criminal justice risk assessment tools like COMPAS use historical data—such as prior arrests, age, and sometimes even survey questions—to predict a person's likelihood of committing future crimes. The algorithm analyzes patterns in this data and outputs a score (e.g., 1 to 10) that influences decisions like pretrial release, bail amounts, and sentence lengths. But the data used to train these models is not neutral: it reflects historical policing practices, which in the U.S. have disproportionately targeted minority communities. If the data shows that more Black individuals were arrested in the past, the algorithm learns to associate race or correlated factors (like neighborhood or prior record) with higher risk. The result is a system that can perpetuate and even amplify existing racial disparities, even though the software itself has no conscious prejudice.
A deeper explanation
The root of algorithmic bias lies in both data and model design. First, the training data may be biased—for example, it records arrests rather than actual criminal behavior, and arrests are influenced by biased policing. Second, the choice of features (like race, or proxies like ZIP code) and the prediction target (e.g., violent vs. non-violent crime) can introduce bias. In 2016, researchers compared COMPAS's predictions with actual reoffending over two years and found that the algorithm was equally accurate across races, but the pattern of errors differed: Black defendants were more often false positives (predicted high risk but didn't reoffend), while white defendants were more often false negatives (predicted low risk but did reoffend). This highlights a trade-off: you cannot simultaneously be fair in equalizing false positives and false negatives across groups, a problem known as the "fairness paradox." The algorithm's bias is not a simple bug—it emerges from the interaction of imperfect data, model choices, and social context. This matters because these tools are used to restrict liberty, and biased predictions can lead to longer pretrial detention or harsher sentences, reinforcing cycles of disadvantage. Addressing this requires careful feature selection, fairness audits, and transparent evaluation of real-world impacts.