A New York City law restricting the use of artificial intelligence tools in the hiring process will take effect early next year. The law is seen as a pioneer in protecting job seekers from algorithmic bias, but as of now, there is still a lack of clear guidance on how employers and vendors can comply, raising concerns about whether the law can effectively address bias issues in hiring algorithms.

The lawcontains two core requirements: employers must audit any automated decision-making tool before using it for hiring or promotion; and they must notify job seekers or employees at least 10 business days before use. Penalties for violations are $500 for the first offense and $1,500 for each subsequent violation.

Although Illinois has regulated AI analysis in video interviews since 2020, New York City's law is the first in the nation to cover the entire hiring process. The law aims to respond to concerns from the U.S. Equal Employment Opportunity Commission and the Department of Justice that "blind reliance" on AI tools in hiring processes could cause companies to violate the Americans with Disabilities Act.

"New York City is taking a holistic view of how automated decision-making systems are changing hiring practices," Dr. Julia Stoyanovich, a computer science professor at New York University and a member of the city's Automated Decision Systems Working Group, told HR Dive. "This is about ensuring people have fair access to economic opportunity. What if people can't get jobs for reasons they don't understand?"

Beyond scrutiny of the "model population"

AI hiring tools are designed to support HR teams throughout the recruitment process, from posting ads on job boards and screening resumes to determining compensation packages. The goal is naturally to help companies find candidates whose backgrounds and skills match.

Unfortunately, every step of this process canintroduce bias, especially when employers compare the "model population" of potential candidates against their existing workforce. Notably, Amazonwas forced to abandona recruiting tool—trained on resumes submitted over a decade to evaluate applicants—because the algorithm learned to penalize resumes containing the word "women."

"You're trying to identify who you think will be successful, treating the past as a prelude to the present," said David J. Walton, a partner at the law firm Fisher & Phillips LLP. "When you look back and use data, if the model population is predominantly white, male, and under 40, the algorithm will inevitably look for those characteristics. How do you redesign the model population to avoid biased outputs?"

AI tools used to evaluate candidates in interviews or tests can also be problematic. For example, measuring speech patterns in video interviews may screen out candidates with speech impairments, while tracking keyboard input could exclude those with arthritis or other conditions affecting dexterity.

"Many workers with disabilities will be disadvantaged by how these tools assess them," said Matt Scherer, senior policy advisor for workers' privacy at the Center for Democracy & Technology. "Many tools operate by making assumptions about people."

Walton noted that these tools are similar to the "pull-up test" commonly seen in firefighter recruitment: "It's not discriminatory on its face, but it can have a disproportionate impact on ADA-protected classes."

There is also a category of AI tools designed to help identify candidates with suitable personalities. Stoyanovich said these tools are equally problematic, and she recentlypublished an auditof two commonly used tools.

The issues are both technical—the same resume submitted in plain text versus PDF format generates different scores—and philosophical. "What is a 'team player'?" she asked. "AI is not magic. If you don't tell it what to look for and don't validate it using scientific methods, the predictions are no better than random guesses."

Legislation—or stricter regulation?

New York City's law is part of a larger trend at the state and federal levels. The federal American Data Privacy and Protection Act, proposed earlier this year,contains similar provisions, while the Algorithmic Accountability Actwould require"impact assessments" of automated decision systems for various use cases, including employment. Additionally, California isplanning to bring the use of AI hiring toolswithin the scope of the state's anti-discrimination laws.

However, some worry that legislation is not the right approach to addressing AI issues in hiring. "New York City's law doesn't impose any new requirements," Scherer said. "The disclosure requirements are minimal, and the audit requirements are just a small fraction of what federal law already requires."

Given the limited guidance issued by New York City officials before the law takes effect on January 1, 2023, the specific form or implementation of technical audits remains unclear. Walton said employers may need to work with individuals who have expertise in data and business analysis.

At a higher level, Stoyanovich said AI hiring tools would benefit from standards-based audit processes. She believes standards should be publicly discussed, and certification should be conducted by independent bodies—whether nonprofit organizations, government agencies, or other entities that do not profit from them. Given these needs, Scherer said he believes regulatory action is preferable to legislation.

For those committed to strengthening oversight of such tools, the challenge lies in pushing policymakers to lead the conversation.

"The tools already exist, and policy has failed to keep pace with technological change," Scherer said. "We're working to make policymakers aware that there need to be real audit requirements for these tools, and meaningful disclosure and accountability when tools lead to discrimination. We have a long way to go."