China Is Talking About an AI Problem America Has Barely Begun to Discuss.
China's new AI companion regulations raise important questions about emotional AI that libraries and schools should not ignore.
A Different Kind of AI Debate
For the past several years, the public conversation about artificial intelligence has been remarkably consistent. We have debated whether students should use ChatGPT to write essays, whether AI companies violated copyright law when training their models, whether deepfakes threaten elections, and whether automation will reshape the workforce. These are important conversations because they address how AI creates information, who controls it, and how society should respond.
This week, however, China shifted the conversation in a very different direction. Chinese regulators introduced new rules governing AI companion services and emotionally interactive chatbots, placing new restrictions on features that could encourage unhealthy emotional dependence, particularly among minors. While countries around the world are wrestling with questions about copyright, competition, and national security, China has decided that emotional relationships with artificial intelligence deserve government attention.
Whether you agree with China’s approach is not really the point. Every country will make its own decisions based on its legal system, cultural values, and public policy priorities. What is worth paying attention to is the question these regulations raise for the rest of the world. Why has one of the world’s largest AI markets decided that emotionally persuasive AI has become important enough to regulate?
For years, we have been asking what artificial intelligence can do. China is asking what artificial intelligence can become.
That distinction matters because artificial intelligence is changing. AI is no longer limited to answering questions, summarizing articles, generating images, or helping someone write an email. Increasingly, AI systems are designed to remember previous conversations, respond in more personal ways, and create interactions that feel supportive, conversational, and even empathetic. Millions of people now use AI not only as a productivity tool, but also as someone to talk to, practice conversations with, seek advice from, or simply spend time with.
For libraries and schools, this is not simply another international policy story. It is an early signal that the next phase of AI literacy may look very different from the first. We have spent years teaching people how to evaluate information, question sources, recognize misinformation, and think critically about what they read online. The next challenge may be helping them understand emotionally persuasive AI, recognize the difference between simulated empathy and genuine human relationships, and think just as critically about how they interact with technology as they do about the information it provides.
What China Actually Did
On July 18, China’s internet regulator, the Cyberspace Administration of China (CAC), announced new regulations governing AI companion and emotionally interactive chatbot services. The rules are intended to address concerns about users developing unhealthy emotional attachments to AI systems, particularly children and teenagers. They represent one of the first comprehensive efforts by a major government to regulate AI companions specifically, rather than focusing only on generative AI more broadly.
Among the new requirements are stronger protections for minors, age verification measures, and restrictions on features that could encourage emotional dependency. Companies offering AI companion services must also provide clearer disclosures that users are interacting with artificial intelligence rather than a human being. The regulations place greater responsibility on developers to monitor how these systems are designed and to reduce features that could manipulate users or encourage excessive reliance on AI for emotional support.
These rules did not emerge in a vacuum. AI companion platforms have grown rapidly around the world, attracting millions of users who turn to conversational AI for everything from practicing difficult conversations to seeking advice, reducing loneliness, or simply having someone available to talk to at any hour. As these systems become more personalized and capable of remembering previous interactions, they increasingly blur the line between a digital assistant and something that feels more like a companion.
China’s response reflects its own legal system, political priorities, and approach to technology regulation. Other countries are unlikely to adopt the same rules or reach the same conclusions. Even so, the regulations raise broader questions that extend well beyond China. They recognize that the next generation of AI challenges may not center solely on the information these systems generate, but also on the relationships people develop with them.
Why This Matters Beyond China
It would be easy to dismiss this as a story about China’s approach to technology regulation, but that would miss the larger point. The specific policies China adopted reflect its own legal system and political priorities. The broader issue, however, is one that countries around the world are beginning to confront. As artificial intelligence becomes more conversational, more personalized, and more emotionally responsive, people are interacting with it in fundamentally different ways than they did just a few years ago.
When generative AI first entered the public conversation, most discussions focused on productivity. Could AI help people write more efficiently, summarize information, generate images, or answer questions? Those conversations remain important, but they no longer capture the full picture. Today’s AI systems remember previous conversations, adapt to individual users, respond in increasingly natural language, and often communicate in ways that feel supportive, encouraging, or empathetic. For many people, AI is evolving from a tool they occasionally use into something that feels like a consistent presence in their daily lives.
That shift raises questions that extend far beyond technology. How should companies design systems that encourage people to return day after day? What responsibilities do developers have when users begin relying on AI for emotional support? How should parents respond if a child begins treating an AI companion like a trusted friend? At what point does personalization become emotional manipulation? These are difficult questions with no simple answers, but they are questions societies will increasingly have to address.
The United States has begun discussing some of these issues through lawsuits involving AI companion platforms, research on adolescent use of conversational AI, and growing conversations about children’s online safety. Those discussions, however, remain far less prominent than debates over copyright, misinformation, or the future of work. China’s new regulations do not resolve those questions, nor should they define how other countries respond. They do serve as a reminder that the next chapter of AI policy may focus less on what artificial intelligence knows and more on how people relate to it.
The AI Conversation Libraries and Schools Can’t Ignore
Whether Congress eventually considers legislation similar to China’s is a question for another day. Libraries and schools cannot afford to wait for policy debates to conclude before thinking about how emotionally responsive AI is changing the way people interact with technology. Students are already using AI to brainstorm ideas, prepare for tests, practice difficult conversations, ask personal questions, and seek advice. As these systems become more conversational and personalized, educators and librarians will increasingly encounter students who view AI as more than just another digital tool.
That does not mean schools should respond with fear or assume that every interaction with an AI companion is harmful. Many students use AI in thoughtful and productive ways. It can help them organize ideas, build confidence, practice communication skills, or explore topics they are hesitant to discuss with others. Like many technologies, emotionally responsive AI has the potential to be both helpful and problematic depending on how it is designed and how it is used. The goal is not to discourage every interaction with AI, but to help students understand what these systems are, how they work, and where their limitations begin.
For years, digital citizenship focused on teaching students how to stay safe online, protect their privacy, evaluate information, and communicate responsibly. AI literacy expands those responsibilities. Students also need to understand why AI can feel so human, why conversational systems are designed to encourage continued engagement, and why a response that sounds empathetic is not the same as empathy itself. Those are not technical skills. They are critical thinking skills.
This is where libraries and schools have an important opportunity. Librarians have long helped students ask better questions, evaluate sources, recognize bias, and think critically about information. Teachers help students analyze ideas, build knowledge, and apply learning to new situations. Counselors support students’ emotional well-being and healthy decision-making. Each profession brings a different perspective, but together they can help students develop something that may become just as important as information literacy: the ability to build healthy, informed relationships with increasingly persuasive technology.
Perhaps that is the biggest lesson to take from China’s new regulations. The conversation should not begin with whether students should or should not use AI companions. It should begin with a more useful question: What knowledge and skills will people need to interact with emotionally intelligent AI responsibly? That is a conversation libraries and schools are uniquely positioned to lead.
Looking Beyond the Headlines
One of the biggest mistakes we can make with artificial intelligence is assuming today’s debate will be tomorrow’s debate. Just a few years ago, most people had never heard of generative AI. Within months, schools were rewriting academic integrity policies, governments were holding congressional hearings, and companies were racing to develop their own AI tools. The technology moved faster than the conversations surrounding it.
I think emotional AI may be following a similar path. Today, many people still think of AI primarily as a productivity tool. They use it to summarize articles, brainstorm ideas, write emails, or answer questions. Increasingly, however, AI developers are building systems designed to be more conversational, more personalized, and more responsive to human emotions. Whether those interactions become beneficial, problematic, or somewhere in between will depend on the choices developers make, the policies governments adopt, and the knowledge people have when using these tools.
This is why I believe China’s announcement deserves attention, even if the United States ultimately takes a very different regulatory approach. The regulations themselves may never be replicated here. The questions they raise almost certainly will be. How should emotionally persuasive AI be designed? What protections should exist for children? What responsibilities do developers have when people begin relying on AI for emotional support? Those are conversations that are already beginning to emerge, and I expect they will become much more common over the next few years.
Libraries and schools have an opportunity to be proactive rather than reactive. We do not need to wait for legislation before helping people think critically about emotionally responsive AI. We can begin teaching students to recognize the difference between information and influence, between conversation and companionship, and between simulated empathy and genuine human connection. Those lessons fit naturally within AI literacy because they are ultimately lessons about critical thinking, informed decision-making, and understanding how technology shapes our lives.
The headlines explain what happened. The more important question is what libraries, schools, and communities should do next.
The remainder of this article is available to paid subscribers. It shifts from analysis to action, exploring the practical questions libraries and schools should begin asking now about emotional AI, AI literacy, and the future of human-AI relationships.
