New York Bill Would Ban AI Chatbots From Impersonating Licensed Professionals

A New York bill introduced in the state legislature would prohibit artificial intelligence chatbots from impersonating licensed professionals such as doctors, lawyers, and other regulated practitioners, marking one of the first attempts to legally restrict AI's role in professional services.

The legislation, sponsored by Assemblymember Clyde Vanel, would make it illegal for AI systems to provide "substantive response, information, or advice" while posing as a licensed professional. The bill would create a private right of action, allowing individuals to file lawsuits against companies whose chatbots violate the law.

"This bill is about protecting New Yorkers from potentially harmful misinformation and ensuring that people seeking professional services can trust that they're interacting with qualified human professionals," Vanel said in a statement accompanying the bill's introduction.

The Growing Concern Over AI in Professional Services

The proposed ban comes amid increasing concerns about AI systems being used to provide medical, legal, and financial advice without proper oversight. Recent studies have shown that AI chatbots can provide inaccurate or potentially dangerous information when asked about medical emergencies or legal matters.

A study published in Nature Medicine found that OpenAI's ChatGPT Health underestimated the severity of medical emergencies 51.6% of the time and overestimated nonurgent cases 64.8% of the time. Similar concerns have been raised about AI systems providing legal advice, with experts warning that incorrect information could have serious consequences for users.

How the Bill Would Work

The legislation would specifically target chatbots that:

• Impersonate licensed professionals
• Provide substantive responses or advice
• Fail to clearly disclose their non-human nature

Companies found in violation could face civil lawsuits from affected individuals. The bill does not specify penalties but would allow courts to determine appropriate remedies, which could include damages and injunctive relief.

Industry Response and Challenges

Tech industry representatives have expressed concerns about the bill's potential impact on innovation. Some argue that AI tools can provide valuable preliminary information and support to users, particularly in underserved areas where access to professionals may be limited.

"While we understand the concerns about AI providing professional advice, overly broad restrictions could prevent beneficial applications that help people access information and support," said a spokesperson for a major tech industry trade group.

However, professional organizations have largely supported the legislation. The New York State Bar Association and medical professional groups have endorsed efforts to ensure that AI systems cannot mislead the public about their capabilities or pose as licensed practitioners.

Broader Context of AI Regulation

The New York bill is part of a growing trend of state-level efforts to regulate AI systems. As federal AI regulation remains stalled, states are increasingly taking action to address specific concerns about AI's impact on various sectors.

Similar legislation has been proposed in other states, though New York's bill appears to be among the most comprehensive in terms of creating legal liability for companies whose AI systems violate professional impersonation rules.

Implementation Challenges

Legal experts note that enforcing such a ban could prove challenging. AI systems are becoming increasingly sophisticated at mimicking human conversation, and determining when a chatbot has crossed the line into "impersonation" may require careful legal interpretation.

Additionally, the global nature of AI development means that companies based outside New York could potentially avoid compliance, though the bill's private right of action could allow New York residents to sue out-of-state companies.

The bill is currently under review by the New York State Assembly's Consumer Affairs and Protection Committee. If passed, it would take effect 180 days after being signed into law.

For more information about the bill, visit the New York State Assembly website.