AI face analysis used to predict MBA pay, researchers claim • The Register

A picture is worth a thousand words or, perhaps, a hundred thousand dollars in extra salary. Academics claim that personality traits inferred using AI photo analysis can predict how depicted individuals will fare in the labor market.

In a paper titled "AI Personality Extraction from Faces: Labor Market Implications," researchers from Wharton, Reichman University, Indiana University, and Yale University describe how they used LinkedIn facial images of over 96,000 MBA graduates to extract subjects' Big Five personality traits – Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism.

The machine learning algorithm used was originally described in a 2020 Scientific Reports paper titled "Assessing the Big Five personality traits using real-life static facial images," one of about two dozen papers cited as "ML-laundered junk science" in a 2024 paper titled "The reanimation of pseudoscience in machine learning and its ethical repercussions."

The algorithm "uses facial features to predict self-reported personality, rather than others' perceptions of personality based on visual appearance," according to the authors.

By applying this algorithm, the researchers found "that personality traits inferred from facial features provide substantial incremental predictive power for labor market outcomes."

The researchers determined that applying machine learning to infer personality traits from facial images produced accurate predictions for the rank of undergraduate and MBA programs attended by the depicted individual, initial compensation, salary trajectory, and job transitions.

So, based on these results, were an HR company to use a similar technique to assess the personality of managerial applicants, the result could serve as a forecast of the job applicant's future performance in the labor market – biased though it may be.

And that appears to be happening. Co-author Marina Niessner, assistant professor of finance at Indiana University, told The Register in a phone interview that companies like banks already use personality surveys in hiring and promotion decisions and that AI hiring companies are starting to use technology like Big Five personality trait analysis on video interviews.

"The regulatory environment, as you probably know, is very uncertain," said Niessner. "And so we don't think this is necessarily a valid way to do it [or] that companies should be doing it. But I think it's really important to have an academic evaluation of these methodologies if there's even going to be a regulatory discussion around this."

The paper argues that the AI-based screening needs to be considered in conjunction with the alternative, which is human decisions based on physical appearance that may also be inconsistent or biased.

The authors emphasize that they don't advocate using this technology because personality extraction from facial images is fundamentally discriminatory. Even so, they say, personality screening is already commonplace among admissions and HR committees, and AI tools that offer personality assessment are seeing rapid adoption.

This research raises serious questions about the intersection of AI technology, hiring practices, and discrimination. While the authors present their findings as a cautionary tale rather than a recommendation, the fact that such technology is already being deployed in real-world hiring scenarios suggests that regulatory frameworks may need to evolve quickly to address these emerging challenges.

The use of AI to analyze facial features for personality assessment represents a troubling trend in hiring technology. Unlike traditional personality tests that rely on self-reported answers, these systems make inferences based on physical characteristics – a practice that has been widely criticized as pseudoscience with roots in discredited physiognomy.

For MBA graduates and job seekers, this technology could mean that their LinkedIn profile photo might influence not just initial impressions but actual hiring decisions and salary offers. The researchers found that these AI-derived personality traits could predict salary trajectories and job transitions, suggesting that the impact could extend throughout an individual's career.

The timing of this research is particularly relevant given the current uncertainty around AI regulation. As Niessner noted, the regulatory environment is "very uncertain," and this technology is already being deployed by companies. The paper's authors position their work as providing the academic foundation necessary for informed regulatory discussions.

This research also highlights the broader challenge of regulating AI systems that may be biased or discriminatory. The authors argue that AI-based screening should be evaluated alongside traditional human decision-making, which can also be biased. However, the concern is that AI systems might scale and institutionalize these biases in ways that are harder to detect and challenge.

For the tech industry, this research serves as a reminder that the deployment of AI systems, particularly in sensitive areas like hiring, requires careful consideration of both technical accuracy and ethical implications. The fact that this technology is already in use despite significant questions about its validity and fairness suggests that the gap between technological capability and responsible deployment remains a critical challenge.