A Poorly Framed Debate

"AI reproduces discrimination." "AI is more objective than humans." These two statements circulate in public debate, often presented as contradictory. In reality, they're both partially true — and that's precisely what makes this topic complex.

Since the Amazon case in 2018 (a resume screening algorithm that disadvantaged women), the question of algorithmic bias in recruitment has become a recurring media topic. But between alarmist headlines and reassuring vendor pitches, it's hard to know what to believe.

This article offers a factual analysis: where biases come from, how to detect them, and what guarantees to demand from an AI recruitment tool.

Where Do Algorithmic Biases Come From?

An algorithm isn't biased "by nature." It becomes biased through three main mechanisms.

1. Bias in Training Data

This is the most documented case. If you train a model on 10 years of hiring history from a company that predominantly hired men, the model will learn that "being male" is a predictive factor for success.

The Amazon case explained: the algorithm wasn't programmed to discriminate. It simply learned patterns from historical data — data that reflected the company's past biases. The model penalized resumes containing the word "women's" (like "women's chess club") because, statistically, these resumes had been selected less often in the past.

The lesson: an algorithm trained on biased data will produce biased results. Training data quality is fundamental.

2. Bias in Model Design

Even with perfect data, design choices can introduce bias.

Example: if you define recruitment "success" as "the candidate is still employed after 2 years," you implicitly favor profiles that stay long — which can disadvantage women (maternity leave) or young people (more frequent job changes).

Another example: if your algorithm uses postal code as a variable, it may learn to indirectly discriminate based on social or ethnic origin, even without ever seeing this information explicitly.

3. Interpretation Bias

An algorithm can be technically neutral, but its use can create bias.

Example: a compatibility score of 72% doesn't mean the same thing for a senior developer position (where qualified candidates are rare) and an administrative assistant role (where they're plentiful). Interpreting this score without context can lead to biased decisions.

What AI Does Better Than Humans

Before listing risks, let's acknowledge what AI objectively brings.

Elimination of Surface Biases

A well-designed algorithm doesn't see:

The candidate's name (and its ethnic connotations)
The photo (and appearance judgments)
The address (and neighborhood prejudices)
Age (when irrelevant to the position)

A study by the Discrimination Observatory (2024) showed that a resume with a North African-sounding name had 25% less chance of being called for an interview, with equal qualifications. AI, if designed to ignore these variables, eliminates this bias.

Consistency in Evaluation

Human recruiters get tired. At 5 PM, after 15 interviews, their attention and rigor diminish. The 16th candidate is evaluated differently from the 3rd, even with equal profiles.

AI applies the same criteria, in the same order, with the same rigor, whether it's the 1st or 500th resume.

Decision Traceability

When a human rejects a resume, they're often unable to explain precisely why. "They didn't fit" isn't an auditable justification.

AI can produce a trace: "Score 45/100 — insufficient technical skills (Python: not mentioned, required), no sector experience (0 years in SaaS, 3 required)."

What AI Does Less Well Than Humans

Context Understanding

AI excels at pattern analysis but struggles to understand meaningful exceptions.

Example: a 2-year gap in a resume might be a warning sign — or indicate a candidate who cared for a sick relative, demonstrating precious human qualities. AI sees the gap; it doesn't understand the story.

Subtle Soft Skills Evaluation

Leadership, creativity, emotional intelligence can't be read from a resume. AI can detect clues (career progression, experience diversity), but nuanced evaluation remains human.

Detecting Atypical Potential

Non-linear paths, career changes, brilliant autodidacts often escape standard criteria. A human can see potential behind the atypical; AI, trained on "normal" patterns, risks penalizing it.

Guarantees to Demand

If you use or are considering using AI in recruitment, here are the questions to ask your vendor.

1. What Data Is the Model Trained On?

Red flag: "Our proprietary data from millions of recruitments." Without knowing where this data comes from and how it was cleaned, you can't evaluate historical bias risk.

Best practice: use general language models (like GPT or Claude) that aren't specifically trained on biased recruitment data, then configure them with explicit fairness instructions.

2. What Variables Are Used?

Does the algorithm use discriminatory proxies?

Postal code → proxy for social/ethnic origin
University → proxy for social origin
Years of experience → proxy for age

Best practice: a transparent list of variables used, with justification for their relevance to the position.

3. How Are Results Audited?

Red flag: "Our algorithm is fair by design." No system is fair without empirical verification.

Best practice: regular audits measuring score gaps between demographic groups, with published results.

4. What Is the Human's Role in the Decision?

AI should remain a decision-support tool, not an autonomous decision-maker. The final decision must rest with a human who can contextualize, question, and if necessary, contradict the algorithmic recommendation.

The Regulatory Framework: AI Act

The European Union has adopted the AI Act, which classifies AI systems by risk level. Recruitment tools are classified as "high risk," which requires:

Transparency: candidates must be informed that AI is used
Human oversight: a human must be able to intervene and correct
Technical documentation: the vendor must document how the system works
Risk assessment: potential biases must be identified and mitigated
Traceability: decisions must be recorded and auditable

These obligations will fully apply in 2026. Companies using non-compliant tools face significant penalties.

Our Approach at Candidalyze

We've made specific technical and ethical choices to minimize bias risks.

No Training on Recruitment Data

We use general language models (Claude from Anthropic), configured with explicit instructions. We don't learn from our clients' past decisions — which avoids reproducing their historical biases.

Analysis on Skills, Not Identity

Our system analyzes:

Technical skills mentioned
Experiences and their durations
Career coherence
Match with job requirements

It doesn't analyze: name, gender, age, address, photo, university (unless it's an explicit job criterion).

Scoring Transparency

Each score comes with an explanation. You know why a candidate scores 75/100: which criteria are met, which are missing. No "black box."

Humans Decide

Candidalyze provides analysis, not decisions. The recruiter remains in control and can (must) question recommendations when context warrants it.

Conclusion: Neither Demonization Nor Idealization

AI in recruitment is neither the problem nor the solution. It's a tool, with its strengths and limitations.

Used carelessly, it can amplify existing biases and industrialize discrimination.

Used rigorously, it can reduce human biases, objectify decisions, and make recruitment more equitable.

The difference lies in technical design, vendor transparency, and user vigilance. Ask the right questions. Demand clear answers. And always keep a human in the loop.

Candidalyze is committed to ethical and transparent recruitment. Our system is designed to help recruiters, not replace them. Discover our approach

Is AI Recruitment Biased? Myths and Realities