AI bias: AI bias is a systematic tendency for an AI system to produce outputs that are unfair or skewed toward certain groups, usually because the data it learned from reflected historical patterns, gaps, or human prejudice.
Models learn patterns from data, and data comes from the world as it was. If a company's past hires skewed one way, a model trained on those records will learn to prefer the same profile. If medical studies underrepresented a population, a model may perform worse for that group. The model is not malicious; it is faithfully reproducing what it saw.
Bias shows up in language models too. They can associate professions with genders, shift tone for names that signal different backgrounds, and give less careful answers in some languages than in English. The best-known workplace case is real: Reuters reported in 2018 that Amazon scrapped an experimental recruiting tool after it learned to downgrade résumés that included the word "women's," as in a women's chess club.
You cannot remove bias with a prompt, but you can manage it. Test the tool on cases that differ only by a protected characteristic and compare the results. Keep a human accountable for any decision about hiring, credit, housing, healthcare, or discipline. Under US employment law, using an automated tool does not shift responsibility for a discriminatory outcome away from the employer.
Example at work
A recruiter uses an assistant to rank fifty applicants for a field technician role. Before trusting the ranking, she swaps the names and addresses on ten résumés and reruns it. Three candidates move several places with nothing else changed. She drops the AI ranking and uses the tool only to summarize each résumé against the written job criteria.
Why it matters
If you use AI for anything that touches people's opportunities, bias is a legal and reputational risk that lands on you, not the vendor. Knowing how to spot it, and keeping a human decision-maker in charge, is part of using these tools responsibly.
Related terms
- Training dataTraining data is the collection of examples an AI model learns from, such as web pages, books, code, images, or labeled records. What is in that data, and what is missing, shapes everything the model knows, how it writes, and which mistakes it makes.
- Responsible AIResponsible AI is an approach to building and using AI that deliberately addresses fairness, transparency, privacy, safety, and accountability, so the technology's benefits do not come at the expense of the people affected by it.
- ExplainabilityExplainability is the degree to which you can understand why an AI system produced a particular output. It matters most when a decision affects a person and someone has to justify it.
- Human-in-the-loopHuman-in-the-loop is a design approach in which a person reviews, approves, or corrects an AI system's output at defined points before it takes effect, combining the speed of automation with human judgment and accountability.
- AI governanceAI governance is the set of policies, roles, controls, and oversight processes an organization uses to decide how AI is adopted, used, monitored, and held accountable across the business.