ChatGPT user in China detained for creating and spreading fake news, police say

Chinese police said they recently arrested one ChatGPT Users for allegedly using the AI-powered chatbot to create a fake message about a non-existent train wreck. It is one of the first enforcement actions under a recently enacted Chinese law regulating AI-generated “deepfakes” — seemingly realistic but fabricated digital images, videos or other media.

According to a police report from northwest China’s province gansu, a man identified only by his last name Hong, used ChatGPT to create a fake news article about an accident that allegedly resulted in the deaths of nine construction workers in Gansu. Twenty-one accounts on a popular social platform, all owned by a southern-based media company Chinaspread the fake story within a short time.

Like most foreign websites and applications, ChatGPT is not technically available in China, thanks to the country “Great firewall” Censoring the internet for residents. But determined individuals can gain access through widely available “virtual private network” software that bypasses the firewall. The police report did not describe how Hong managed to use ChatGPT.

By the time Gansu security officials realized the article was fake, it had received 15,000 views, according to the report. The police then ransacked Hong’s home to collect evidence, and then took “criminal coercive measures” against Hong himself. The police use this term to describe temporary measures to restrict a suspect’s freedom.

The new Chinese deepfake law came into force on January 10th. It bans several categories of fake media produced by “deep synthesis” technologies such as machine learning and virtual reality, but offers only vague definitions for many of these banned classes.

According to a translation of the law provided by crowdsourcing website China Law Translate, it prohibits the use of deepfakes for activities that threaten national security, damage the country’s image or the public interest of society, or that “economically or socially order”. It expressly prohibits the use of such technologies to produce, publish or transmit fake news.

__

AP News assistant Caroline Chen in Beijing contributed to this report.