Organizations are rapidly adopting the use of artificial intelligence (AI) for the discovery, screening, interviewing, and hiring of candidates. It can reduce time and work needed to find job candidates and it can more accurately match applicant skills to a job opening.

But legislators and other lawmakers are concerned that using AI-based tools to discover and vet talent could intrude on job seekers' privacy and may introduce racial- and gender-based biases already baked into the software.

"We have seen a substantial groundswell over the past two to three years with regard to legislation and regulatory rule-making as it relates to the use of AI in various facets of the workplace," said Samantha Grant, a partner with the law firm of Reed Smith. 

States, including California, Maryland, and Washington, have enacted or are considering legislation to put rules around using AI for talent acquisition. The European Union's EU AI Act is also aimed at addressing issues surrounding automated hiring software.

On April 1, Italy became the first Western nation to ban further development of ChatGPT over privacy concerns after the natural language processing app experienced a data breach involving user conversations and payment information. ChatGPT is the popular chatbot created by OpenAI and backed by billions of dollars from Microsoft.

The Italian data protection authority is also investigating whether OpenAI's chatbot already violated the European Union's General Data Protection Regulation rules created to protect personal data inside and outside the EU. OpenAI has complied with the new law, according to a report by the BBC.

Congress is considering the federal Algorithmic Accountability Act, which, if passed, would require employers to perform an impact assessment of any automated decision-making system that has a significant effect on an individual's access to, terms, or availability of employment. 

In addition, the US Equal Employment Opportunity Commission (EEOC) recently announced that it intends to increase oversight and scrutiny of AI tools used to screen and hire workers. As part of that effort, the EEOC held a public hearing Jan. 31 to explore the potential benefits and harms of AI in hiring situations, according to Grant.

"The current swell of laws and regulations related to AI in HR is like a wave under the water - building, gaining momentum, and getting ready to come ashore," said Cliff Jurkiewicz, vice president of Global Strategy at Phenom, an AI-enabled hiring platform provider. "The new laws are necessary and welcomed as technology has outpaced existing regulations for protecting underrepresented groups."

New York City makes a move

One of the attempts to wrangle AI-based automated employment-decision tools is New York City's Local Law 144, slated to go into effect in April. The law, originally passed in 2021, was postponed due to the "high volume of public comments" during the rule-making process. It prohibits employers from using automated employment selection tools unless an organization institutes specific bias auditing and makes the resulting data publicly available.

A company must also disclose its use of AI to job candidates who live in New York City.

The New York City law could be a catalyst for other states to adopt similar legislation - since so many companies do business in the city and it is an epicenter of finance and commerce, Jurkiewicz said. "Implementing such a law will undoubtedly influence similar laws throughout the US and potentially other regions," he said.

While the city ordinance implies employers must conduct an audit, vendors are preemptively doing them to help companies they work with, "both as security for existing clients, as well as a way to differentiate/appeal to potential prospective clients," said Ben Eubanks, chief research officer with Lighthouse Research & Advisory.

"I think everybody's holding their breath and watching to see what's going to happen in New York, partially because the rules around these tools [require them] to be audited [and] evaluated and the vendor has to prove they've passed some approved checklist," Eubanks said. "At this point, it's hard to know what it's going to look like rolling out. I have lots of companies within the vendor community that have been watching this closely."

Companies offering AI-based recruitment software include Paradox, HireVue, iCIMS, Textio, Phenom, Jobvite, XOR.ai, Upwork, Bullhorn and Eightfold AI.

For example, HireVue's service includes a chatbot that can hold text-based conversations with job seekers to guide them to jobs that best fit their skills. Phenom's deep-learning algorithm chatbot sends tailored job recommendations and content based on skills, position fit, location, and experience to candidates so employers can "find and choose you faster." Not only does it screen applicants, but it can schedule job interviews.

AI talent acquisition software uses numerical scores based on a candidate's background, skills, and video interview to deliver an overall competency-based score and rankings that can be used in employer decision-making.

This week, Beamery, a multinational AI-based talent management software provider, announced the launch of TalentGPT, a chatbot based on GPT-4 and other large language models (LLMs). The chatbot is aimed at assisting hiring managers, recruiters, candidates and employees in talent acquisition and job searches. The company claims its AI automates rules compliance and mitigates bias risks associated with LLMs, which are the algorithms behind chatbots.

Talent acquisition software and services have touted their AI-based platforms as offering greater diversity, inclusion and equality (DEI) because the computer software can be programmed to be gender and ethnicity neutral; the goal is to eliminate as much human bias as possible.

The problem: humans program the software.

The challenges inherent in AI

As with any "disruptive technology," Jurkiewicz said AI brings challenges that should be considered and planned for by hiring organizations. They include:

• Algorithmic bias.
• Lack of transparency.
• Legal and ethical concerns.
• Over-reliance on AI.
• Privacy and data security.
• Dehumanization in the hiring process.
• Misalignment with organizational culture and values.

A US and EU joint report released this year on the potential economic impact of AI on the future of workforces found that while it can bolster workforce efficiency and innovation, it can exacerbate inequality.

"There is substantial evidence...AI has introduced and perpetuated racial or other forms of bias, both through issues with the underlying datasets used to make decisions, and by unintentional or seemingly benign decisions made by algorithm designers," the report said. "The challenge for policymakers is to foster progress and innovation in AI while shielding workers and consumers from potential types of harm that could arise."

The challenges are only expected to grow. From 35% to 45% of companies are expected to use AI-based talent acquisition software and services to help select and interview job prospects in the coming year, according to two recent studies.

Although there are few AI-related employment laws on the books at the moment, employers should expect that to change as the use of AI expands beyond just hiring and into performance evaluations, career projections, and promotion/termination decisions, according to Paul Starkman, an attorney with the Chicago-based law firm Clark Hill.

"And [that] may ultimately morph into consumer information protection laws, such as the European Union's GDPR and California's CCPA/CPRA," Starkman said.

Hiring algorithms aren't new

While the use of computer algorithms for screening prospective job candidates is not new - simple text searches have been used to parse resumes for decades - the sophistication of the applications and the breadth of their use has rapidly grown.

Nearly three in four organizations boosted their purchase of talent acquisition technology in 2022 and 70% plan to continue investing this year - even if a recession arrives - according to a survey by online enterprise hiring platform Modern Hire. Modern Hire's fifth annual Hiring Report found that 45% of companies worldwide are using AI to improve recruiting and human resource functions. 

Experts caution that AI recruiting systems are only as good as the programmers who "feed the machine." If an AI tool ingests data from  resumes of people previously hired by a company - and the recruiting departments that made those decisions harbored subconscious biases and preferences - those biases could be inherited by the AI tool.

For example, Amazon spent a decade training its applicant screening algorithm using its own hiring data. But once it went live, it reportedly showed bias against women. Just the word "woman" would cause the algorithm to rank female applicants lower than men.

Conversely, there are also top tips from online job search sites instructing applicants on how to write resumes that will pass automated screening software, making sure job candidates get seen.

While resume matching to job descriptions is the most common use of AI, tools are also being used to analyze patterns of potential candidates, including segmentation of candidates based on experience, education, skills, and their potential for retention once hired, according to Bret Greenstein, a PricewaterhouseCoopers (PwC) partner and Data Analytics and AI researcher.

To perform more detailed searches for potential candidates, AI platforms must collect massive amounts of data on prospective candidates without their expressed permission, according to Eubanks, author of the bookTalent Scarcity: How to Hire and Retain a Shrinking Workforce. That information can include facial recognition software and video interviews companies may keep, share, and filter with AI to determine favorable candidates.

In 2019, the Illinois Artificial Intelligence Video Interview Act ("AIVI Act") was signed into law, making the state the first to regulate  automated "interview bots" and other forms of AI to analyze applicants' facial expressions, body language, word choices, and vocal tones during video interviews. And in 2020, Maryland enacted a similar law prohibiting employers from using facial recognition algorithms in hiring unless the applicant agreed to it.

In a 2019 blog, Starkman wrote that AI has been used on video interviews to determine whether an applicant shows characteristics of "successful" candidates. "Multi-state employers should take note of the AIVI Act, as it explicitly applies 'when considering applicants for positions based in Illinois,'" he wrote.

The built-in bias problem

In an email response toComputerworld, Starkman said that without proper development and use, "AI systems can be biased, either because of bias in the data itself or in how the algorithm processes the data, and that may result in the unintended elimination of certain disabled candidates, foreign-born candidates, and others in discriminatory ways, if no safeguards are in place."

For example, he said, AI-driven chatbots that communicate with job candidates should be monitored to limit the inadvertent receipt of information about disabilities and other personal characteristics that could lead to discrimination claims. "Another algorithm-assisted hiring and performance evaluation system ultimately had to be scrapped because it was based on past hiring practices and could not be trained to unlearn its programmer's bias," he said.

In another illustration, Starkman said software designed to disregard candidates with gaps in their resumes may have unduly impacted women candidates because they were statistically more likely to leave the workforce than men.

"I understand where their hearts are at - they want to make things safer and more equitable for the candidate population out there," Eubanks said. "But the challenge is the rules they're making [aren't] always aligned with how companies hire."

For example, Eubanks said, the Illinois law requires companies to delete any video interviews after 30 days. Many employees quit a short time after being hired. So, by the time the company goes back to look at a second-choice candidate, their video interview is deleted.

"Some of those nuances they put into laws..., [they're] put in place by people who don't always understand how hiring works," Eubanks said. "They're not doing the day to day [work]. And because of that, it creates some complexities, challenges, and headaches."

To address the challenges, organizations should adopt a balanced approach that combines the strengths of AI with human judgment or a human in the hiring process loop that has expertise, Jurkiewicz said. "It is vital to ensure that AI-driven employment tools are explainable, unbiased, tested and compliant with applicable laws and ethical guidelines," he said.

When developed, tested, monitored and implemented responsibility, AI-powered tools can significantly increase diversity and inclusiveness in the workplace.

Studies have shown that many underrepresented communities either lack the skills or do not understand the impact of not promoting their favorable attributes (skills, behaviors, competencies and experiences) as other, systemically well-trained communities can and have [done] historically, according to Jurkiewicz.

"AI can surface individuals' positive attributes and encourage underrepresented groups to compete for work they may not have thought possible," he said.