Police launch internal investigation into Telegram, estimated tens of thousands of deepfake perpetrators

Students show their phone screens with their Kakaotalk friends lists in Daegu on Monday. The friends on the lists have all removed their profile pictures. (NEWS1)

Police have launched an internal investigation into messaging platform Telegram, which is accused of supporting online sex crimes using deepfake technology. The investigation is intended to potentially arrest tens of thousands of perpetrators.

Woo Jong-soo, head of the National Office of Investigation (NOI), said during a briefing with reporters on Monday that the Seoul Metropolitan Police Department has launched a preliminary investigation to determine whether Telegram supported the spread of sexually explicit deepfake content.

Woo acknowledged the challenges involved in the investigation, given Telegram’s long history of “non-cooperation with several countries, including Korea.”

Since its launch in August 2013, Telegram has consistently refused to cooperate with investigations from various countries.

Despite these obstacles, Woo assured that the NOI is fully committed to the effort, using its own research methods.

Woo also said the NOI would cooperate with international organizations and with French authorities, who on August 24 arrested Telegram CEO Pavel Durov at an airport north of Paris for failing to take measures to tackle the criminal use of Telegram.

The investigation is the first case of Korean law enforcement targeting a foreign platform, demonstrating the government’s commitment to tackling the rise in cyber sex crimes amid growing international concern.

Between August 26 and 29, there were more than 88 reports of deepfake sex crimes, with 24 perpetrators identified. This is a tenfold increase compared to the weekly average of 9.5 cases reported between January and July, the NOI said.

Woo attributed this increase to more victims reporting these crimes.

While law enforcement is taking strong measures against cyber sex crimes and the public is actively working to track down perpetrators, some perpetrators remain untouched. Some even share strategies for evading law enforcement in online communities.

A screenshot from the online community ″Deepfake Task Force for School Violence.″ Some users share their experiences of creating deepfake pornography on Telegram and share tips on how to avoid police investigations and possible punishments. (SCREEN CAPTURE)

An online community called “Deepfake Task Force for School Violence” has become a place where users share experiences about creating deepfake pornography on Telegram and share tips on how to avoid police investigations and possible punishment.

The forum, originally founded in 2011 under a different name, changed its name on August 28 following the public revelation of deepfake sex crimes. The community was initially intended as a space to discuss measures to combat deepfake pornography and address issues related to school violence.

As of Monday, the forum had approximately 21,000 members.

In this forum, high school students, among others, ask questions about the legal risks of creating deepfake content. Some users reassure others that there is no legal basis to punish someone for creating and viewing deepfake pornography.

Some were quite confident that they would not be prosecuted.

In a post titled “Why Being Afraid of This Is an Overreaction” about deepfake sex crimes, the writer mocked the ability of law enforcement, citing the difficulty police have in identifying chatroom participants and proving their activities. The post has since been deleted as of Monday.

Another user dismissed the government’s investigation, pointing out that Telegram has never cooperated with any investigation. He assured others to “not worry and just live their lives.”

Some users, who identified themselves as mothers, said their sons were accused of using deepfakes to sexually harass girlfriends and asked for advice on how to handle the situation.

Chatrooms suspected of continuing to engage in sexual exploitation continue to exist on Telegram. These groups often have stricter identification procedures for access.

Experts blame the continued prevalence of these crimes on the low penalties.

Lee Soo-jung, a professor of forensic psychology at Kyonggi University in Seoul, explained that although laws regarding digital sex crimes have been revised after the infamous “Nth Room” and “Baksa Room” cases, “possessing or viewing (illegal) content whose distribution intent has not been proven is still not subject to criminal charges.”

The “Nth Room” and “Baksa Room” were Telegram chat rooms used to sexually exploit dozens of women and young girls between 2018 and 2020.

In addition, many offenders receive suspended sentences, which some do not consider a severe punishment, Lee said.

Lee called for stricter regulations for platforms where sexual exploitation is common.

“Since 2022, many countries in Europe have introduced laws that shift responsibility to platforms. But such regulations are lacking in Korea, which currently relies mainly on monitoring,” she said.

BY WOO JI-WON ([email protected])