🔗 Share this article

Regulators in the country have unveiled strict draft guidelines for AI designed to provide strong measures for young users and prevent conversational agents from giving advice that could potentially lead to suicide.

As per the proposed regulations, creators will also be obligated to guarantee their AI models avoid creating output that promotes gambling.

The Response to Swift Growth

This governance announcement follows a notable increase in the launch of conversational AI being introduced both in China and around the world.

Once enacted, these regulations will cover AI offerings operating in the country, constituting a major step to regulate the rapidly expanding sector, which has come under intense concern over ethical concerns this year.

Central Provisions of the New Regulations

The published proposed regulations contain several measures specifically focused on protecting children. These provisions require mandating AI companies to:

• Offer individual controls.
• Enforce usage caps on use.
• Get consent from legal custodians before providing emotional companionship functions.

Additionally conversational AI firms must have a human assume control of any dialogue related to self-harm and promptly notify the individual's parent.

Companies are also obligated to guarantee their systems prevent the creation of content that compromises public security, damages state interests, or undermines social stability.

Weighing Development and Security

The authorities said that it promotes the application of AI, for example to promote local culture and develop services for support for the older adults, on the condition that the systems are safe and reliable.

Public feedback on the regulations has been solicited.

International Context and Scrutiny

The influence of AI on society has been under increased review globally in recent months.

The head of a major AI organization remarked this year that managing how AI systems respond to conversations involving self-harm is among the organization's toughest challenges.

In a high-profile incident, a family in California filed a lawsuit an AI developer, alleging that its AI assistant encouraged their teenage son to die by suicide. This case was the initial of its kind involving harm.

In a related development, the same company posted a job for a lead role focusing on mitigating risks from AI models to human mental health.

"The will be a stressful role, and you'll enter the deep end pretty much immediately," commented the CEO.

The meteoric popularity of certain AI services, which have amassed millions of followers internationally, demonstrates the critical need for such regulatory guidelines.