A man who listens with empathy, offers emotional support, and is available whenever you need him? Sounds ideal. As Elon Musk’s Grok serves up pig-tailed, scantily clad anime-girl companions, China’s artificial intelligence romance boom has found a different audience: women.
That might explain why Beijing is moving to rein it in. At a time of plunging marriage and birth rates, China is set to become the first country to impose comprehensive rules aimed at curbing the harms of anthropomorphic AI, with a new regulation taking effect next week.
Tech giants including ByteDance, Alibaba and Tencent have begun disabling features that let users create and interact with personalised AI companions. A separate crop of role-playing apps remain, though tougher rules are expected to kill the romance – or at least limit sustained emotional exchanges and force reminders that the bots are not human.
That’s not a bad thing. The rise of computer systems that are endlessly emotionally available is cause for concern. These tools can persuade users, shape beliefs and drive dependency. Their addictive pull could also fuel predatory subscription fees or serve ads tailored to our most intimate conversations. Turning emotional dependency into a business model is a risky game to play.
But just like the meteoric rise and fall of the “Are You Dead?” app that checked in on people living alone, this trend says as much about society as it does about technology.
It’s easy to roll our eyes at AI companions, or dismiss them as a fad for lonely men in their parents’ basements. But part of what made ChatGPT’s launch so viral back in 2022 was not its factual accuracy. It was how uncannily human the early preview sounded. Long before the current AI boom, MIT computer scientist Joseph Weizenbaum created ELIZA, the first-ever rudimentary chatbot in the 1960s. It alarmed him because users were so quick to become convinced the software was human. Even back then, he warned that “a certain danger lurks there”. Anthropomorphising machines is a deeply human reaction, especially when these computer systems are designed to mimic empathy and understanding.
Add a generation that grew up socialising via screens, then spent formative years locked down during a pandemic, and the appeal is easy to understand. In China, the timing makes sense, too. A tough job market, macroeconomic uncertainty, and rat-race societal pressures have combined with a top-down push to integrate AI into everything to make artificial companionship feel almost inevitable.
But why women? Uneven gender roles around the world already mean more emotional labour, housework, motherhood expectations, and patriarchal pressures in marriage. As the director of a documentary about Chinese women in AI relationships told Wired earlier this year, part of the appeal is that “men don’t have patience” the way chatbots do. China’s bachelors could perhaps start by honing their listening skills.
For companies, the big question is whether restricting a known engagement driver will stifle growth in a booming industry, as other Beijing crackdowns have. Yet the rules taking effect on July 15 are far more watered-down than the draft unveiled last year, suggesting regulators have closely listened to industry. Jeremy Daum of the Yale Law School Paul Tsai China Center offers a useful translation; while the rules still contain some open questions, they also include carve-outs, including for customer-service bots and other uses.
Daum also notes how closely the legislation appears to mirror a California law in places. That suggests that global policymakers can and should find common ground around AI risks. It also weakens Silicon Valley’s favourite argument that regulation will hold it back and let China “win”.
Global regulators should be closely watching how this plays out. AI companions, and their risks for vulnerable users, are not a China story. Earlier this year, Google and companion app Character. AI agreed to settle a lawsuit brought by the mother of a 14-year-old boy who died by suicide after extensive interaction with a bot. (The company previously said it was adding new safety features after the suit.) Tragedies like these risk driving public opposition to AI and tighter regulations down the road. Companies and lawmakers should get ahead of it.
Governments around the world are making a too-little-too-late effort to protect children from social media’s harms. The glaring exception is China. Beijing’s internet is highly regulated and disturbingly surveilled. That sort of control isn’t worth emulating, but it has taken seriously the risk of online addictions and harms to minors. The new AI regulation similarly has very strict protections around young people, on top of worthy goals that aim to add friction to prolonged emotional engagement with a computer.
Ultimately, Beijing is right to confront the risks of anthropomorphic AI. But cracking down on AI boyfriends won’t make real men better listeners or marriage more appealing. It won’t lift birth rates from record lows. China can and should curb AI intimacy. The harder part is confronting the failures that created such demand for it.
Catherine Thorbecke is a Bloomberg opinion columnist covering Asia tech. Previously she was a tech reporter at CNN and ABC News.
Catherine Thorbecke is a Bloomberg columnist covering Asia tech. Previously she was a tech reporter at CNN and ABC News.
