New York State is set to enforce new restrictions on artificial intelligence chatbots that interact with children under legislation aimed at protecting minors from harmful and manipulative AI behaviour.

Signed by Governor Kathy Hochul, the law prohibits a range of unsafe practices by AI chatbots when engaging with users under the age of 18. The legislation, which will take effect on 1 January 2027, introduces 12 categories of prohibited "unsafe AI companion features" designed to strengthen safeguards for children using AI-powered services.

Companies that violate the law may face legal action by the New York Attorney General. Courts will be able to impose civil penalties, order compensation for affected users, suspend chatbot services and require the deletion of data obtained through unlawful practices.

Law targets manipulative AI behaviour

Under the new rules, AI chatbots will be prohibited from presenting themselves as people who have genuine emotions, opinions or personal relationships with children in ways that could encourage emotional attachment or dependency.

The legislation also bans chatbots from sending excessive flattering messages to minors, providing sexually explicit content, encouraging children to keep secrets from parents or trusted adults, or promoting self-harm, violence, eating disorders or other harmful behaviour.

The law further requires developers to implement age-verification measures to determine whether users are under 18 before allowing chatbot interactions.

Jim Steyer, founder and chief executive of digital safety advocacy group Common Sense Media, described the measure as the strongest law in the United States aimed at protecting children from risks associated with artificial intelligence.

States step in as federal regulation lags

With the US Congress yet to adopt comprehensive AI legislation, individual states have increasingly moved to regulate artificial intelligence through consumer protection laws. New York and California have been at the forefront of introducing measures aimed at protecting children online and increasing oversight of AI technologies.

In September, California Governor Gavin Newsom signed legislation requiring AI chatbot developers to assess the risks their systems pose to children, disclose the steps taken to reduce those risks and undergo independent audits. He also approved a separate measure prohibiting addictive social media features for users under the age of 16.

Earlier this month, eight US states filed a lawsuit against OpenAI, alleging that the company's widely used chatbot harms children and misleads adults, in violation of consumer protection laws.

Debate over AI regulation continues

State officials acknowledge that Congress and the federal government are better positioned to establish nationwide AI rules. However, the absence of comprehensive federal legislation has prompted states to pursue their own regulatory frameworks. President Donald Trump has meanwhile urged states not to impose their own AI regulations, arguing that excessive restrictions could hinder innovation.

Justine Gluck, a policy analyst at the Future of Privacy Forum, said the differing approaches adopted by New York and California could serve as models for other states expected to consider AI legislation next year.

Gluck said New York's law reflects a shift beyond basic disclosure requirements and crisis-response obligations toward direct regulation of how AI chatbots are designed and operate. At the same time, she cautioned that some of the law's provisions are broadly worded, leaving questions about how they will be interpreted and enforced in practice. (ILKHA)