On April 12, CCTV's Financial Investigation exposed an AI 'pornography production' industry chain: prompts, tutorials and image-to-video tools were being packaged and sold, compressing what once required a certain technical threshold to produce pornographic content into a standard process of buying a tutorial, copying a prompt and clicking generate.
In June, police in Jiaozhou, Shandong Province, solved a case involving the use of AI to bypass real-name verification. After purchasing citizens' personal information such as names, ID card numbers and ID photos, the criminal group used AI to generate dynamic facial videos showing blinking, head shaking and similar movements, and then used a 'virtual camera' to complete platform facial recognition verification. Police seized more than 50,000 AI-synthesized dynamic facial videos; the group involved had already sold more than 80,000 real-name accounts and made more than RMB 600,000 in illegal profits.
On June 29, the Nanshan Branch of the Shenzhen Public Security Bureau announced that Luo, a self-media practitioner, had used AI tools to fabricate and widely disseminate false information about Dongpeng Beverage in order to attract attention and traffic for profit. Luo has been criminally detained in accordance with the law, and the case is still under further investigation.
By July, figures released by the Ministry of Public Security showed that in the first half of 2026, public security organs investigated and handled more than 170 cases involving the use of AI tools to generate online rumors and, in accordance with the law, investigated and dealt with more than 190 people.
Taken together, a phenomenon is becoming increasingly evident: while many legitimate AI products are still searching for a stable business model, the black and gray markets have rapidly embedded AI into mature industry chains such as fraud, pornographic content, account trading, false marketing and online rumors.
The reason is not that AI has created many entirely new forms of crime, but that it has first changed the cost structure of the existing black and gray market business.
AI did not invent new crimes, but it has driven down the cost of many crimes
Fraud, rumor-mongering, illegal acquisition of personal information, and the production and dissemination of obscene content all existed before the advent of generative AI.
What AI has truly changed is the cost a person must bear to accomplish these things.
In the past, impersonating a specific identity might have required a real person to appear on camera, imitate a voice, produce documents and maintain a long-term script; now, a publicly available video may become material for voice cloning, a few photos can generate a dynamic face, generative models can help construct a person's background and polish dialogue, and translation tools have further lowered the threshold for cross-language communication.
This industrialization trend has already emerged in scam compounds in Southeast Asia. In research published in 2026, C4ADS found that some fraud networks are embedding AI chat, translation, persona-setting and employee-management tools into their fraud workflows; a previous Reuters investigation of people who had escaped scam compounds in Myanmar also documented the use of generative AI to shape false identities, write more natural English dialogue and answer questions from potential victims.
Therefore, the most direct effect of AI on the black and gray markets can be summarized in four respects: lowering the cost of identity forgery, lowering the cost of content production, lowering communication costs, and lowering the cost of operating at scale.
In the past, one person could maintain only a few fake identities; now more accounts can be managed simultaneously. In the past, producing a fake video required professional post-production; now templates and models can generate one quickly. In the past, fraud groups needed to specially train operators speaking different languages; now machine translation and generative AI can directly step into communications.
The criminal model has not changed completely, but the efficiency of production has changed.
This is also why much of the AI abuse we see today is not a 'never-before-seen new crime,' but rather old illegal and criminal methods acquiring a new set of production tools.
Those who are truly likely to generate stable income are often the people 'selling shovels' to the industrial chain
If one looks only at the news, the most conspicuous figures are often those who directly carry out fraud, produce pornographic content, or publish fake videos. But from cases already disclosed, an increasingly obvious division of upstream and downstream labor is emerging in the AI black and gray industry.
Some are responsible for purchasing personal information, some for creating dynamic human faces, and some for registering real-name accounts; some develop or package AI tools, and some sell prompts and tutorials; further downstream are those who use these resources to carry out fraud, drive traffic, produce pornographic content, or engage in other illegal activities.
The case solved by the Jiaozhou police is a typical example. The group did not operate every fraudulent account itself; instead, it purchased personal information, generated dynamic human faces, bypassed real-name verification, and then sold the already real-name-verified accounts downstream. More than 80,000 accounts formed a kind of 'black and gray industry raw material' that could be delivered repeatedly.
A similar structure has also emerged in the 'AI undressing' industrial chain.
In a case disclosed by the procuratorial authorities of Jiangsu, Sun did not merely produce the relevant images himself; he packaged the method of using AI to generate pornographic content into software, tutorials, and paid groups. The relevant communities he operated had a cumulative total of 27,000 members, including more than 1,500 members in private paid groups, and he made more than 100,000 yuan in illegal profits. The court ultimately sentenced him to one year and six months in prison and imposed a fine for the crime of illegally using information networks.
This type of case deserves particular attention from AI entrepreneurs, because it illustrates a very important issue:
The fact that one has not personally carried out the downstream crime does not mean that providers of technology, accounts, tutorials, and services are naturally outside criminal risk.
But conversely, it also cannot be said that merely because a developer has developed an AI tool that was later used illegally, the developer necessarily commits a crime.
What truly needs to be determined is what the provider actually knew, for whom it provided services, how the product was customized, and what role it played in the overall illegal chain.
"I only do technology" is not a ground for exemption, but the misuse of technology does not in itself mean the technology provider is guilty.
This is the issue on which AI companies are most prone to veer toward two extremes.
One extreme is the view that: I am a technology company and am only responsible for models, interfaces, digital humans, voice cloning or automation tools, and what the client does with them is none of my concern.
The other extreme is the view that: as long as the client subsequently engages in illegal activity, the technology provider must bear criminal liability.
Neither of these understandings is accurate.
Criminal liability must ultimately still be assessed by reference to specific conduct and subjective knowledge. For example, with the same digital human system, one client uses it for corporate customer service, while another client explicitly requests batch impersonation of specific real persons, removal of AI labels, circumvention of platform review and registration of large numbers of fake accounts; the risks faced by the technology provider are obviously different.
In judicial assessment, what truly merits attention is often not the statement "I did not know," but whether a series of objective facts can support that explanation.
Did the client explicitly disclose the illegal use? Was the product specially modified to circumvent real-name authentication, platform risk controls or content review? Did the technology provider continue to provide after-sales service after discovering anomalies? Did the settlement method, order volume, pricing and client behavior significantly deviate from normal commercial scenarios? These facts together determine whether "technological neutrality" can still be established.
The currently effective Cybersecurity Law also expressly provides that no individual or organization may provide programs or tools specifically used for illegal intrusion into networks, theft of network data or other activities endangering cybersecurity; and that where a person knows another is engaged in activities endangering cybersecurity, that person may not continue to provide technical support, advertising promotion, payment settlement or other assistance. The amended Cybersecurity Law takes effect on January 1, 2026, and for the first time incorporates AI development and security into the basic legal framework for cybersecurity through dedicated provisions.
Therefore, for AI companies, the real question is not whether "technology is inherently guilty," but rather:
When the client's use has already shown obvious anomalies, what else did you do?
The most dangerous aspect of the AI black and gray market is that "authenticity" is becoming cheap
The Jiaozhou case has another particularly noteworthy aspect: AI is no longer merely used to generate content; it is being used to attack the most fundamental "authenticity verification" of internet platforms.
In the past, platforms used ID cards, facial recognition, and liveness detection to confirm that a real person actually existed behind the screen.
But when an ID photo can be processed into a video of blinking and turning the head, and then combined with a virtual camera input system, the "real person" the platform sees may simply be generated content.
The raw material behind this is facial information.
Guiding Case No. 192 of the Supreme People's Court has made clear that facial information is highly identifiable and constitutes citizens' personal information protected by criminal law. Without the individual's consent and without any other lawful basis for processing, illegally obtaining such information and reaching statutory thresholds such as "serious circumstances" may constitute the crime of infringing on citizens' personal information.
It should be emphasized that the fact that facial information constitutes citizens' personal information does not mean that merely obtaining a certain number of facial photographs will automatically constitute a crime. Whether the specific conduct meets the criminal incrimination standard still requires a comprehensive assessment based on the type and quantity of information, the manner of acquisition, the purpose of use, illegal gains, and the consequences caused.
But with the advent of AI, the risk value of facial data has indeed changed.
A photograph that in the past might only have been used for targeted marketing can now also be used to train face-swapping models, generate dynamic identities, bypass real-name verification, or even construct a complete digital identity.
For AI companies that hold large amounts of user photographs, voices, and videos, whether the data sources are lawful and whether the authorization covers the actual purposes of use is therefore no longer merely a formal issue in a privacy policy.
There is another business: making money from the very claim that "AI can make money"
Compared with criminal black-market schemes, another type of model is closer to ordinary users.
It does not necessarily involve the direct use of AI to commit a crime; rather, it packages "AI automatic money-making" as a product that can itself be sold.
In the "AI automatic product-selling" projects investigated by CNR in March this year, some operators advertised effects such as "zero followers, zero experience," "AI automatic publishing," and "earn RMB 200,000 in half a year," and offered various cooperation packages priced at RMB 3,760, RMB 5,210, RMB 9,510, and so on. After paying, some purchasers found that the so-called automatically generated videos received almost no traffic, and that actual income was far removed from the advertising. The report also mentioned that the Guangzhou Panyu police had opened a case against one of the companies involved on suspicion of fraud.
The legal assessment of such business cannot simply be classified as "fraud."
AI training, software subscriptions, and product-selling services can themselves all be normal business models. What truly needs to be distinguished is whether what the enterprise sells is a product or service with genuine functions, or whether it induces users to pay through fabricated cases, forged earnings, or promises of returns that cannot be realized.
Where promotional content is exaggerated or misleading, advertising, market regulation, and contractual liability may be implicated first; only where the conditions for consumer fraud are met may the rules on punitive damages under the Law on the Protection of Consumer Rights and Interests further apply; and if it can be proven that the operator had the purpose of illegal possession at the outset of collecting payment and obtained property through fabricating facts and concealing the truth, the matter may further enter the criminal assessment of the crime of fraud.
Therefore, "AI courses cutting leeks" sounds like an internet meme, but in legal terms, the distance between different projects may be very great.
What truly separates AI companies from these cases is a few specific actions
As of September 1, 2025, the Measures for Labeling AI-Generated Synthetic Content and the supporting mandatory national standards officially took effect.
Among them is a provision that deserves considerable attention from AI companies: no organization or individual may maliciously delete, tamper with, forge, or conceal AI-generated synthetic content labels established in accordance with the law, nor provide tools or services for others to carry out the above acts.
What is referred to here is not all ordinary image watermarks, but the AI-generated synthetic content labels required to be established by the rules.
This means that when customers make requests such as "remove the AI label," "make it impossible for the platform to tell it is generated," or "can it bypass review," these demands should themselves be treated as risk signals, and must not be understood merely as ordinary product feature requests.
For a normally operating AI company, the compliance system truly worth establishing is in fact not as abstract as one might imagine.
When using real-person likenesses, voices and facial data, can the authorization chain explain the source, purpose and term of use; when receiving abnormal orders such as bulk face-swapping, bulk account registration or circumvention of real-name verification, are there manual review and refusal mechanisms; is content generated by the model labeled in accordance with the rules; after complaints and clearly unlawful uses arise, can the enterprise stop providing services and retain the relevant records.
If the enterprise is a network operator, the Cybersecurity Law also requires it to adopt technical measures to monitor and record network operation status and cybersecurity incidents in accordance with the multi-level protection scheme for cybersecurity, and to retain the relevant network logs for no less than six months as required.
Ordinarily, these things look like mere process.
But when a case actually arises, what they determine is whether the enterprise can prove: when this customer came, what requests it made, what the system generated, what the company did after discovering the anomaly, and whether the enterprise was genuinely providing ordinary technical services or had already become deeply involved in another business chain.
Lawyer's Observations
Why do black and gray markets often monetize AI earlier than lawful enterprises? Not because AI is particularly suited to crime, but because black and gray markets bear almost none of the costs that ordinary business models must bear.
They do not need to obtain authorization for materials, do not need to consider data sources, and do not need to pay licensing fees for likenesses and voices; they do not need to worry about platform rules, nor do they need to spend time verifying whether a business model is sustainable in the long term. As long as AI can make a false identity more realistic, make a piece of content generate faster, and enable one person to face more targets at the same time, the efficiency brought by the technology can be immediately converted into profit.
Legitimate enterprises are the exact opposite.
Beyond technical efficiency, they must also contend with products, users, supply chains, compliance and long-term reputation. But looking only at cases that have already occurred reveals that the so-called "huge profits of the AI black and gray market" are also strongly illusory. In the Jiaozhou case, the unlawful gains behind more than 80,000 real-name accounts amounted to more than RMB 600,000; Sun operated communities with tens of thousands of members and made unlawful gains of more than RMB 100,000, in exchange for a sentence of one year and six months' imprisonment.
So what AI practitioners should really study is not which "black and gray market track" is most profitable, but another matter:
AI is turning high-risk conduct that previously required professional skills, staffing scale and time costs into standard actions that can be replicated, performed in bulk or even automated.
And the distance between enterprises and these cases is often not a statement that "we would never do anything bad," but more specific product choices: whether to circumvent authentication on customers' behalf, whether to process facial data and voices of unclear origin, whether to provide features that conceal traces of AI generation, and, upon discovering abnormal uses, whether to continue delivery or stop promptly.
Technology itself is neither black nor white.
But when a product begins to rely on concealing true identities, circumventing platform risk controls, generating false content in bulk, or evading AI labeling in order to earn revenue, what it faces is no longer merely a question of product ethics, but is gradually approaching the risk boundary that is the common concern of administrative regulation and criminal law.
Reference sources:
- China Media Group, "Financial Investigation," an investigation into the AI "pornography creation" industry chain, April 12, 2026
- Xinhua News Agency, "Xinhua Investigation | Exposing the Black Industry Chain of 'AI Face-Swapping' Fraud," June 22, 2026
- Police Situation Report of the Nanshan Branch of the Shenzhen Municipal Public Security Bureau, June 29, 2026; related reports by Securities Times and 21st Century Business Herald
- Notification by the Ministry of Public Security on the progress of the special action against "AI face-swapping" fraud, July 2026
- Jiangsu Procuratorate Network, "Sentenced for Peddling 'AI Undressing Tutorials'!", public prosecution initiated by the Haidian District People's Procuratorate of Beijing Municipality, first-instance judgment by the court on May 9, 2026
- National Business Daily, report on Zhang being sentenced for selling "AI One-Click Undressing" software, July 2026
- Hangzhou Internet Court (2023) Zhe 0192 Min Chu No. 4563 Civil Judgment, Supreme People's Court Case Database Entry No. 2025-07-2-525-001
- Guiding Case No. 192 of the Supreme People's Court, Li Kaixiang Case of Infringement of Citizens' Personal Information, Criminal Incidental Civil Public Interest Litigation
- Reuters interview with a person who escaped from the KK Park in Myanmar; Lianhe Zaobao Chinese-language report, September 15, 2025
- C4ADS, a Washington-based security research institution, analysis of internet connectivity records of AI software platforms and devices in Myanmar telecom fraud compounds, July 2026
- TMTPost, "Exposing Southeast Asia's AI Legions: Dedicated to Compounds, Telecom Fraud Efficiency Soars Tenfold"
- Cybersecurity Protection Bureau of the Ministry of Public Security, notice on administrative detention of Xing Mou in Datong, Shanxi, for using AI to impersonate a celebrity in livestream selling, May 7, 2026
- CCTV.com "News+", report on the incident of AI face-swapping involving Zhang Wenhong to promote protein bars, December 18, 2024
- Xinhuanet, "Regulating the Chaos of AI Celebrity Endorsement in Livestream Selling: E-Commerce Livestreaming Launches a Portrait 'Defense Battle'", May 14, 2026
- CNR, "Woodpecker Consumer Complaints | Investigation into the Chaos of 'AI Automated Sales' Training," March 15, 2026
- Southern Weekly, "One Yuan for Traffic, Exorbitant Prices for Courses: 'AI Financial Quotient Courses' Target Rural Elderly", April 8, 2026
- Trend Micro, "2026 Fraud Trend Forecast"
- Cyberspace Administration of China and three other departments, Measures for Labeling AI-Generated Synthetic Content, CAC Notice No. [2025] 2
- Supreme People's Court and Supreme People's Procuratorate, Interpretation on Several Issues Concerning the Application of Law in Handling Criminal Cases of Infringing on Citizens' Personal Information
/ END.
*This article is an original work of Mankun Law Firm and represents only the personal views of the author. It does not constitute legal consultation or legal advice on any specific matter. For reprinting and legal consultation, please add our customer service:mankunlawyer。
