AI grifters are creating fake Black people to sell Shein junk

Aliyah, a light-skinned Black woman dressed in country-western gear, is struggling to sell metal buckles she handmade on TikTok. In a video for the social media platform from March, she cries to the camera and pleads for views: “Even as a black woman, I have more faith that white women will stay 13 seconds [on this video] to save my belt buckle business,” the onscreen text reads. She wipes a tear off her cheek.But Aliyah isn’t real, and neither are her supposedly handmade products — she’s one of many AI-generated influencers created to sell mass-produced products via dropshipping on TikTok, Facebook, and Instagram. Identical belt buckles — sunflower design, detachable knife inlay, and all — are sold on the fast-fashion site Shein, and for a quarter of the price.Aliyahsbuckles on TikTok (left), and Amaya’s buckles on TikTok (right)There are some clues to spot to determine that this video is AI-generated. Aliyah’s voice is robotic and emotionless, which doesn’t match the crying face on the screen. In one clip, she is sewing a leather belt where there wouldn’t usually be sewing at all. When she wipes a tear off her face, the stream of liquid below where she wipes also disappears. And lastly, there are dozens of uncannily comparable videos, but with different AI-generated characters, circulating on TikTok. One, a profile for an account called “Aliyahsbuckles,” features an identical background, tabletop, and spool of twine.The Verge found dozens of accounts on TikTok with similar narratives and a variety of dropshipping products, including belt buckles, mugs shaped like cowboy boots, crochet bags, and cardigans. Some of these videos are labelled as AI-generated. Similar accounts are also active on Instagram and Facebook. Nearly all aspects of the accounts appear to be AI-generated — from the “person” in the video to automated responses to comments, which in some cases attempt to mimic African American vernacular — and experts warn scams like this are growing every day.“It’s massive,” Jeremy Carrasco, a researcher of AI-generated media and director of Riddance.ai, an organization that focuses on AI video detection, told The Verge of AI-generated videos connected to e-commerce stores. “Most of them aren’t coordinated. Some of them are coordinated. A lot of the time they’ll run a single [AI-generated] actor, or a couple actors will run all sorts of shops,” he explained. Those AI-generated avatars pretend to make the items, go to fairs to display their products, and “respond” to comments through automation. “What we’re seeing right now are these retail scams where they’ll link to the Shopify websites.”Carrasco estimates his research team is finding up to 100 accounts that attempt to sell products via AI-generated avatars every day. Most of the accounts found by The Verge were created in the last two months and contain videos about small businesses owned by marginalized individuals struggling to make a sale; these videos are incredibly similar, with only slight variations in their scripts. While we also found Native American, Hispanic, and white women characters, the most viewed and engaged-with AI-generated characters found by The Verge are Black women. Aliyah’s account alone has 40,000 followers.1/4A screenshot from Amaya’s buckles on TikTok“What we’re seeing here is empathy bait,” said Carrasco. “If there is a popular dropship item that could be sold to some sort of niche community, they will find it and they will try [to use] some personality to do it.” Carrasco explains that these trends are usually random opportunities to make money. “It’s just an arbitrary opportunity, which is what a lot of AI content out there is — the platforms don’t really care and people don’t notice.”Aliyah’s most popular video, the one described in the introduction of this article, has 814,000 likes, 6.5 million views, and almost 30,000 comments. Some comments identify the content as AI-generated, but many express the desire to help Aliyah’s business, commenting on the video to increase her visibility on the platform.India Cater-Campbell was one of those commenters, expressing a desire to buy Aliyah’s belt buckles. “I was trying to be supportive to an independent Black businesswoman,” said Cater-Campbell, an actual Black business owner working on opening a café in Seattle. “[I felt] solidarity as I am trying to start a business myself.”Despite a lack of studies on the latest gen AI video models, Carrasco estimates that these videos are “realistic enough to trick” most people. Users of short-form video content platforms have been trained to scroll mindlessly and not look further into the content they’re creating. Ironically, this is what might have saved Cater-Campbell from actually buying a belt buckle: She couldn’t immediately find a store link, so she scrolled away and promptly forgot about Aliyah.“I was trying to be supportive to an independent Black businesswoman.”But people are falling for these scams, and their prominence grows: Two weeks ago, Gizelle Bryant from The Real Housewives of Potomac admitted to buying two crocheted bags after seeing a video in which an AI-generated Black boy said he was being bullied by white boys for crocheting. “I was like, I want to help this little Black boy make his goal,” Bryant said on her podcast Reasonably Shady, adding that other celebrities had also been in the comment section. “How did I get tricked? Viola Davis was on there, too.”The trend is a kind of digital blackface, according to communications researcher Cienna Davis at the University of Pennsylvania. “Digital blackface is a phenomenon where non-Black individuals are able to use the internet and digital technologies to mimic Black cultural expression for personal, economic, or political gain,” Davis explained over a video call. Davis has written previously about digital blackface, pointing to the use of gifs of Black people as well as the impersonation of Black people for political goals. “It’s rooted in blackface minstrelsy, which is tied to the legacy of slavery,” she explained.Based on the idea that Blackness is “inherently exploitable” and “up for grabs,” digital Blackface is used to “extract value from Black bodies in whatever way [non-Black people] see fit,” Davis said. In this case, the videos are mimicking “a recognizable idea of Black struggle,” she added.Even without confirmation that the people behind the videos aren’t Black, Tempest M. Henning, an assistant professor of philosophy at Fisk University, confirms the videos are digital blackface. “Blackface is any kind of caricature-like portrayal of Black people, which can include Black people dressing up in a caricature-like fashion of Black people,” Henning says. She cites historical precedents of Black people occasionally being forced to perform in minstrel shows and the more recent example of Zoe Saldaña darkening her skin tone and wearing prosthetics to play the role of Nina Simone in a biopic.There’s an inherent falseness to the Blackness these avatars claim. As Henning explained, “The names of the avatar are coded Black, like Aliyah or Amaya, but there’s nothing else [that signals authentic Blackness] other than the avatar itself.” That’s only reinforced by the replication of content across racial identities, which, Henning said, results in the flattening of those identities.The Verge reached out to Aliyahsbuckles, as well as other stores selling similar products, but received no response.The videos posted on Aliyah’s account are replicated scene by scene across many similar accounts selling dropshipped items, with slight modifications to fit the identity of the character or the item being sold. In one video featuring another Black woman character, Amaya, a white woman mockingly throws coffee on belt buckles being showcased at a fair. A mildly frustrated Amaya — the emotions are somewhat robotic and never look quite right — sighs and goes back to the honest work of making her buckles. A similar scene appears in an account called ChubbyKnots; this time the avatar is a Black girl and the product is a crocheted butterfly cardigan.Henning said there’s an appeal to virtue signaling in these videos, where the viewer is called to perform kindness or even racial or class solidarity. “It’s virtue signaling in the sense of ‘Oh, that white lady did that, but I am not that kind of white lady, so where do I buy this belt buckle?’” Henning said. “It’s like they’re saying, ‘I don’t stand with these people.’”However, Henning pointed out the superficiality of race- or class-based support when users are not pausing to research who it is they are supporting in the first place.It’s perhaps a larger social outcome of short-form video content and the consumption of slop: Solidarity is often being performed at “surface level,” she explained, rather than through a coherent and multifaceted political lens. “I might be very invested in a Black-owned business, but that doesn’t necessarily mean that all Black businesses align with my political views and what I stand for,” Henning said.This appeal to virtue signaling goes beyond the category of race, also drawing on working-class markers and a simulation of people working hard to earn their keep in a failing economy. “[It’s not just] Black struggle because on the other videos, we can observe working class struggle, small-business struggle,” Davis said. “But it’s obviously using that [narrative] to sell mass-produced goods. It’s not creative at all. It’s really just AI-generated templates and creation of characters that get optimized for consumer identification and investment in the products. It’s broader than ‘white guilt’ altogether, because it can be used in whatever way feels necessary or feels applicable to the product.”Despite this flattening of racial identity, or maybe because of it, algorithms — which have been shown to perpetuate conscious and unconscious racial bias from the people who build them — are delivering AI-generated content that fits the interests they’ve inferred for each user, Carrasco said. These interests may include content from specific demographics that AI personas are able to mimic. “It targets the demographic that [the algorithm] can see,” he said. “Usually a Black woman will see AI-generated Black people on their feed, an Asian person will get AI-generated Asians.”Looking closer at the content, it’s easy to spot inconsistencies and AI tells. We never see more than a few seconds of the character onscreen because AI apps like Seedance 2.0, Midjourney, and OpenAI’s recently discontinued Sora 2 are not able to create clips longer than 15 seconds. When the products are showcased, the lighting is brighter rather than sepia-toned, and the hands holding the buckles are white. The emotions on the characters’ faces don’t quite match the tone of their voices.But this recognition requires AI media literacy that is not widely available, especially at the scale that these videos are being created and the technology is advancing toward faster automation and more realistic avatars. “People who aren’t trained in media literacy or critical media literacy are just going to take it in unquestionably,” Davis said.1/2A screenshot of Amayasbuckles selling a rose blade belt buckle for $40The very structures of short-form video platforms, which are major channels for misinformation, encourage a lack of discernment from users, who have been trained to want to scroll to the next video rather than think about what they are watching. “[These platforms] are being exploited by AI content partially because in order to figure out something’s AI, it usually takes a second or two,” Carrasco said. “By that time, you’ve already registered some engagement.”People who profit from these kinds of videos know this, and the tools to create scalable AI-generated short-form commercials are widely available online. In reporting this story, The Verge found YouTube channels and forums dedicated to tutorials on how to create ads like this without product samples, influencer fees, or original scripts.These tutorials teach users how to copy viral videos with multiple AI tools and replace real influencers with AI-generated characters. ChatGPT and Gemini are used to extract and copy scripts from real videos by real influencers, with the option to create multiple variants of the same script. These tools are also used to generate photos of fake people and fake backgrounds based on real influencers and real backgrounds, which are then imported into apps like Kling 2.0 and Maxfusion. With some AI video generator models, videos by real influencers can be fully imported into the app and the user can replace the real person with an AI character they created, effectively stealing the script and background of the original video.While media literacy might help individual users not fall for AI-generated video scams, social media platforms are currently failing to moderate and, at minimum, label this content. “Platforms should have stronger AI detection and labeling structures,” Davis said. “Bias checks on these platforms would also be welcome, as well as clear reporting pathways for reporting AI misuse on platforms.”In addition to moderation, Carrasco suggested that account transparency would also help users know whether they’re watching AI-generated content, with clear labels that are easy to see and don’t require extra steps.But short-form video content consumers already have years of experience scrolling their feeds and superficially engaging with whatever comes up. Reversing this cognitive training feels almost impossible to do at a mass scale without platforms themselves taking charge. With no real incentive for short-form video platforms to do so and profit at stake, that seems unlikely — but until then, AI scammers will continue to take advantage of whoever takes the bait.Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.Nicole Froio