Algorithmic Fairness and Bias

Algorithmic fairness concerns whether and how automated decision systems treat individuals and groups equitably, and the ways in which data and models can encode or amplify bias.

Definition

The study of equity and discrimination in automated decision systems, including the measurement of bias and the formal and ethical conceptions of fair treatment.

Scope

This topic covers sources of bias in data and machine-learning systems, competing formal definitions of fairness (such as demographic parity, equalized odds, and calibration), the impossibility results showing these can conflict, the relation between statistical fairness and substantive justice, and the social consequences of automated decision-making in areas like hiring, lending, and criminal justice. It describes the technical and ethical debates without prescribing which fairness criterion any system should adopt.

Core questions

• How do bias and discrimination enter data-driven decision systems?
• What does it mean for an algorithm to be 'fair', and can rival definitions be satisfied at once?
• How do statistical notions of fairness relate to legal and moral conceptions of justice?
• Who is accountable for discriminatory outcomes produced by automated systems?

Key theories

Disparate impact in data-driven systems

Barocas and Selbst analyse how data mining can produce discriminatory outcomes even without discriminatory intent, through biased training data, proxies, and feature selection.

Incompatibility of fairness criteria

Formal work shows that distinct statistical definitions of fairness—such as calibration and balanced error rates across groups—generally cannot all be satisfied simultaneously except in special cases, forcing value-laden choices.

History

Attention to algorithmic fairness grew in the mid-2010s as machine-learning systems were deployed in consequential settings. Barocas and Selbst's 2016 analysis of disparate impact, formal fairness definitions from the computer-science community, and popular critiques such as O'Neil's established the field's core questions.

Debates

Which fairness definition to use

Because formal fairness criteria can conflict, debate focuses on whether any single definition is appropriate, how to choose among them in context, and whether formal metrics can capture substantive justice at all.

Key figures

• Solon Barocas
• Andrew Selbst
• Cynthia Dwork
• Cathy O'Neil

Seminal works

• barocas2016
• oneil2016

Frequently asked questions

Can an algorithm be biased even if it ignores protected attributes?

Yes. Removing attributes such as race or gender does not guarantee fairness, because other features can serve as proxies for them, a phenomenon central to discussions of disparate impact.

Is there a single correct definition of algorithmic fairness?

No consensus exists. Several formal definitions have been proposed, and results show they can be mutually incompatible, so selecting one involves contested ethical and political judgments.