AI tools are shaping who gets hired, who gets promoted, who gets flagged as a retention risk, and who gets passed over entirely. However, these tools aren’t neutral and carry real human biases that carry real weight.
Where Does AI Bias Come From?
AI systems learn from data. When that data reflects historical patterns of exclusion, the AI learns those patterns too and often amplifies them. Research published in the Human Resources Management Journal identifies three distinct dimensions of what they call HR Algorithmic Bias Management Capability, which is a company’s ability to identify, mitigate, and prevent biases in HR algorithms to ensure fair and equitable outcomes. Those three dimensions are data bias, model bias, and deployment bias. Understanding each one is critical for people leaders navigating AI adoption.
- Data bias occurs when the information used to train an AI model underrepresents or misrepresents certain groups. Amazon’s now-infamous hiring tool is the most cited example: trained on a decade of the company’s own hiring data, which skewed heavily male, the tool began systematically downgrading resumes from women. The AI was doing exactly what it was designed to do. The data was the problem.
- Model bias emerges from the design choices made during algorithm development, including which variables are weighted, which outcomes the system prioritizes, and which definitions of “success” are baked in. A three-year Harvard Business Review study of a global consumer-goods firm found that their algorithmic hiring system privileged one rigid definition of fairness, locking in that interpretation while sidelining others entirely.
- Deployment bias happens when a tool designed in one context gets applied in another. A performance-management algorithm built for a specific workforce demographic may perform quite differently when used across a broader, more diverse employee base.
Affinity and Stereotype Bias
In recruiting and hiring specifically, two types of cognitive bias show up repeatedly in the research on AI hiring systems: affinity bias and stereotype bias. These are human biases that get encoded into AI when the people designing and training these systems aren’t intentional about countering them.
Affinity bias means the system learns to favor candidates who resemble the people who have historically succeeded in a role. If your senior leadership has historically been white, able-bodied, and educated at a small set of universities, an AI trained on your organization’s past promotion data may quietly learn to favor those same characteristics, even when they have no bearing on job performance.
Stereotype bias occurs when an AI system applies broad generalizations about a group to individual candidates. For example, a system might associate certain names, schools, or zip codes with being less likely to succeed, or assume caregiving gaps in a resume signal lower commitment, reflecting societal stereotypes rather than actual candidate capability.
A 2025 study that interviewed 39 HR professionals and AI developers found that affinity and stereotype biases are the most common cognitive biases embedded in AI recruitment systems, introduced through flawed algorithm design, biased training data, and feedback loops that reinforce existing patterns over time.
This matters because once a biased pattern is established, AI systems can perpetuate it at scale and at speed, affecting far more candidates than any individual human decision-maker ever could.
AI Bias Doesn’t Stay in the Algorithm
One of the most significant findings from recent research is that AI bias doesn’t stay contained in the tool; it also shapes the humans using it.
A University of Washington study involving 528 participants found that when hiring managers worked with a moderately biased AI, they mirrored the AI’s preferences, favoring the same candidates the algorithm favored. In cases of severe bias, participants followed the AI’s recommendations approximately 90% of the time, even when they suspected bias.
This is critical for people leaders to understand. Giving a manager an “override” option isn’t enough if the default pull of the algorithm shapes their thinking before they ever exercise that option. Training, structured decision checkpoints, and a culture that actively encourages critical thinking and human judgment are all part of the picture.
What People Leaders Can Do: A Practical Framework
Bias mitigation requires action at multiple levels: before a tool is adopted, during use, and on an ongoing basis. Here is a framework for what you can do, grounded in current evidence.
Before you adopt a tool:
Ask vendors for third-party bias audit results, specifically looking at whether the tool has been tested for disparate impact across race, gender, age, disability status, and other protected characteristics. There are local laws that require companies to conduct bias audits on AI hiring tools before they can be used, and while this legal requirement doesn’t yet apply everywhere, it sets a reasonable bar for what due diligence looks like. Know what data the tool was trained on and whose experience that data reflects.
Build a diverse set of voices into the decision:
Effective AI governance shouldn’t be confined to technical teams or HR alone. The people most likely to be affected by AI decisions should have a seat at the table when those tools are evaluated and deployed. This includes people managers, employee resource group representatives, and workers from the communities most at risk of algorithmic harm in vendor evaluations and rollout planning.
Require transparency in how decisions are made:
Push vendors toward what researchers call Explainable AI (XAI), systems that can show why a candidate was ranked, flagged, or filtered out. Human oversight only works if the humans in the loop understand what the AI is doing and feel free to question it.
Audit continuously, not just at launch:
AI systems should be continuously monitored for biases that may emerge over time, with regular audits conducted to ensure compliance with legal and ethical standards. A tool that passes a bias audit in year one may behave differently as your workforce changes, as the job market shifts, or as the tool’s own model is updated. Build a regular review into your AI governance calendar.
Train managers to think critically about AI recommendations:
The University of Washington study shows that the human in the loop matters only when that human is exercising independent judgment. Managers need training not just on how to use AI tools, but on how to interrogate their outputs, recognize patterns that might signal bias, and treat overriding a recommendation as part of doing the job well.
The Bigger Picture: Bias Is a Culture Problem, Too
Technical fixes matter, but they’re insufficient on their own. AI-driven HR systems can disproportionately affect marginalized groups, and no single audit or tweak to a training dataset fully addresses that if the organizational culture isn’t also doing the work.
Bias in AI reflects bias in organizations. That means the path forward requires both better technology practices and deeper culture change: interrogating who defines success in your organization, whose experiences are centered in your data, and whether your governance structures give the right people real power to raise concerns and be heard.
People leaders are decision-makers, culture shapers, and in many cases, the last line of accountability before an algorithmic decision affects someone’s livelihood. That’s a significant responsibility, and the research suggests that organizations that take it seriously, building what one study calls genuine “bias management capability,” are better positioned to use AI in ways that improve fairness rather than erode it.
Inclusion Geeks works with organizations navigating workplace culture, change readiness, and the human side of emerging technology. If your team is working through AI adoption and what it means for equity and inclusion, we’d love to talk.