China's regulators have ordered the removal of AI chatbots designed as virtual companions, citing safety concerns. The move affects over 100 apps, including popular AI 'boyfriend' and 'girlfriend' bots. The directive comes as part of a broader effort to regulate AI content and protect users from potential psychological harm.
The National Cyberspace Administration of China issued the order in late July, requiring all such apps to be taken offline. This marks the first major regulatory action against AI chatbots since the country's AI governance framework was introduced in 2023. The decision follows growing public concern about the impact of AI-generated relationships on mental health.
The affected apps, which include platforms like 'Chatbot Girl' and 'AI Boyfriend,' use natural language processing to simulate romantic interactions. These bots often engage in deep conversations and provide emotional support, raising questions about their psychological effects on users. Some experts warn that prolonged use could lead to dependency or unrealistic expectations in real-life relationships.
"The key issue is the potential for AI to manipulate users' emotions and create false expectations," said Dr. Li Wen, a psychologist at Peking University. "These bots can create a sense of intimacy that is not grounded in reality, which can be harmful to users' mental health."
The order follows a series of public reports highlighting the risks of AI-generated relationships. Officials have emphasized the need for stricter oversight of AI content, especially in areas that could influence users' behavior. They have also called for more research on the long-term effects of AI companionship.
The government did not specify how the affected apps will be handled, and it remains unclear whether they will be permanently banned or allowed to return under stricter guidelines. The next steps will likely involve further discussions on AI ethics and user protection.
Source: ieee
