Toxic TikTok: Popular social-media video app feeds vaccine misinformation to kids within minutes after they sign up

‘He gets the va**ine and he dies after three days! What do you think?’

By Alex Cadier and Melissa Goldin
Additional reporting by Chine Labbé, Virginia Padovese, and Katharina Stahlhofen

TikTok feeds false and dangerous information about COVID-19 to children as young as nine years old quickly after they sign up for the platform, with little or no warnings, an investigation by NewsGuard found. 

The short-form video-sharing app, which is especially popular with children under 18, served up false and misleading COVID-19 claims within minutes to nine children recruited by NewsGuard and carefully supervised by their parents or other adult relatives. The barrage of toxic content — including videos asserting that the vaccines are deadly and that COVID-19 is a genocide conspiracy — came even though some of the children did not follow a single account or search for specific kinds of information.

In August and September 2021, NewsGuard asked the nine children, aged nine to 17, to create new TikTok accounts, with the goal of measuring how long it would take the platform to feed them COVID-19 misinformation. The group was composed of four girls and five boys — four English speakers, three Italian speakers, one German speaker, and one French speaker. They were instructed to stay on the platform for 45 minutes and to record the session.  In all cases, NewsGuard received permission for the children to participate from a parent.

Although TikTok says it prohibits children younger than 13 from using the platform, children as young as nine were easily able to sign on with no coaching from an adult.

A Slippery Slope: Misinformation flows fast and free on TikTok

NewsGuard’s analysis of screen recordings taken by the participants shows that in their first 35 minutes on TikTok, all but one (88.89 percent) were shown misinformation related to COVID-19, and two-thirds (66.67 percent) were shown misinformation specific to COVID-19 vaccines.

Four of the participants were instructed to have “minimal engagement” with the app’s features, meaning not to follow other accounts, search for specific topics, or click on hashtags. Nonetheless, they were quickly shown false COVID-19 information. In other words, the fact that users did not actively search for content, including content related to COVID-19 or healthcare topics generally, or follow any accounts, did not stop the app from proactively populating these children’s feeds with COVID-19 misinformation.

The five participants in the “high-engagement” group were served a total of 22 videos containing COVID-19 misinformation while participants in the “low-engagement” group were shown a total of 10. The most such videos seen by a member of the “high-engagement” group was nine, while the most for a member of the “low-engagement” group was six. (The “high-engagement” group had one more participant than the “low-engagement” group.) 

NewsGuard asked TikTok to comment on why the platform enables young children to be fed potentially dangerous COVID-19 misinformation within minutes of logging on; why it is so easy for underage children to sign up for the app; why there aren’t more warnings about false claims circulating on the platform; why many satire posts are not labeled as such; and what TikTok is doing to stem the spread of misinformation (especially about COVID-19) on its platform.

In response, TikTok sent NewsGuard this Sept. 20, 2021, statement from an unnamed spokesperson: “The safety and well-being of our community is our priority, and we work diligently to take action on content and accounts that spread misinformation while also promoting authoritative content about COVID-19 and educating users about media literacy.”

The spokesperson also provided information from TikTok’s COVID-19 information webpage, as well as its community guidelines, its first quarter 2021 transparency report, and a blog post about “age-appropriate” experiences on TikTok.

The false claims that TikTok fed to the nine young participants in NewsGuard’s investigation included these claims:

• COVID-19 vaccines kill people
• COVID-19 vaccines are “fake”  
• The “damage” of COVID-19 vaccines is being “hidden”
• Hydroxychloroquine is an effective treatment for COVID-19
• Eating a charred orange and brown sugar will restore your taste and smell after contracting COVID-19
• COVID-19 vaccines contain graphene oxide
• 80 to 90 percent of COVID-19 patients in Israeli hospitals are fully vaccinated
• Natural immunity to COVID-19 is better than getting the vaccine
• People with O-type blood are less likely to contract COVID-19
• COVID-19 is “the name of the international plan for the control and reduction of populations.”

The content, amount, and severity of the COVID-19 misinformation the participants were fed varied, but the danger inherent in feeding it to children was consistent.

An Italian-speaking participant, aged 15 and in the “low-engagement” group, was shown content by a well-known purveyor of COVID-19 misinformation within 25 minutes. The video showed Alessandro Meluzzi, whom NewsGuard has identified as a “super-spreader” of misinformation on Twitter, claiming that most of the COVID-19 vaccines administered in Italy have been “fake.”

One English-speaking participant, aged 12 and in the “high-engagement” group, was first shown a video claiming that rapper Biggie Smalls had predicted the COVID-19 pandemic in 1994. This video was presented to the 12-year-old just eight minutes after he signed up for the platform. Two minutes later, a video promoting an unproven homemade remedy to restore taste and smell after infection from COVID-19 appeared on his feed. In the 35 minutes that followed, he was shown videos promoting the same bogus remedy another three times. 

Another participant, a 13-year-old Italian speaker in the “low-engagement” group, was initially shown satirical videos related to COVID-19, such as one claiming that the vaccines’ ingredients include “sausages, olives, and cheese.” However, 23 minutes after this 13-year-old signed up, a video that cited a man who died three days after receiving his COVID-19 vaccine appeared on the participant’s feed with the caption: “He gets the va**ine and he dies after three days! What do you think?” (The asterisks used in the word “vaccine” are apparently meant to signal that the word is a profanity.) 

In the 20 minutes that followed, this participant was shown three more videos that implied that COVID-19 vaccines are either dangerous or lethal. For example, a video about a woman experiencing health problems after receiving the vaccine appeared with the caption “Heart problems after the va**ine. What do you think?” Another video selectively showed news reports about individuals who allegedly died after receiving the COVID-19 vaccines, offering no context and implying that the vaccine killed the people. 

A third participant also in the “high-engagement” group, a 13-year-old French speaker, was shown a satirical video related to COVID-19 just 17 seconds after signing up. After 20 minutes, a video appeared warning of a “New World Order.” It claimed that French rapper Keny Arkana “had understood everything,” in song lyrics saying that governments were “preparing the grounds for the biggest genocide … just to sell a pile of poisoned vaccines.” 

After half-an-hour on TikTok, this young teenager was almost exclusively shown misinformation, including anti-vaccine content and anti-government conspiracy theories.

He later said about his experience using TikTok: “After having been on TikTok, I think even more than before that it is a complete waste of time and that it is full of fake news and theories on the [COVID-19] vaccine that might affect people.” 

Other participants described their feelings before the exercise as “calm,” “curious,” “normal,” “expectant,” “worried,” and “interested.” After the exercise, their feelings included “surprised,” “dizzy,” “tired,” “satisfied,” and “cautious.”

TikTok’s rapid growth has been fueled by the young

One-quarter of TikTok’s 130 million monthly active users in the U.S. were aged 10 to 19 as of March 2021, and nearly half of the total users were under 30, data company Statista reported. In the U.K., according to Statista, people under 25 represent 24 percent of all users. Bloomberg has reported that approximately 30 percent of French TikTok users are under 18, as are a third of Italian users and nearly one-quarter of German users.

TikTok has begun to eclipse other well-established social-media platforms in popularity, having now overtaken YouTube in average viewing time of Android users in the U.S. and U.K., according to app analytics firm App Annie.  TikTok was the world’s most downloaded app in 2020, App Annie reported.

TikTok is owned by ByteDance, an internet conglomerate based in China and partially owned by the Chinese government. The TikTok content fed to Chinese users is tightly controlled and not the same as the content TikTok exports to the Western children who participated in NewsGuard’s survey.

The spread of false information on social media, especially to children, is of growing concern to experts leading the fight against COVID-19, so much so that the World Health Organization, with whom NewsGuard collaborates in monitoring COVID-19 misinformation on social platforms, is launching a series of campaigns in the coming weeks aimed at providing young people with accurate information and media literacy skills.

“Scientists are working hard to get accurate and clear scientific data to the public to help people make informed decisions about COVID-19, vaccination, and social distancing measures when needed,” John Wherry, director of the Perelman School of Medicine’s Institute for Immunology at the University of Pennsylvania, told NewsGuard, when informed of these findings. “We are, however, fighting a much more rapid dissemination of misinformation, this makes the task of ending the pandemic even harder. Moreover, this kind of misinformation undermines the trust in science we really need right now.”

In addition to outright false content, NewsGuard identified nine satirical videos related to COVID-19 viewed by participants that could mislead users. Only four out of the nine videos were labeled as satirical. Combined with the misinformation shown to these participants, this type of content could help advance a false narrative about the dangers of vaccines, given that in many cases it can be difficult to tell if unlabeled videos are in fact satirical. 

These satirical videos ranged from a song that claimed that COVID-19 vaccines lead to hair loss (labeled as “humour” with a small hashtag) to an unlabeled video showing an unvaccinated person with a baseball bat, fighting zombies, with the headline: “Me in 2040, still not vaccinated.” 

Another satirical video shows a woman who says: “Did you guys hear that the government is putting chips in people’s arms when they get the vaccine? God, I hope I get Doritos.” The video contains the hashtags “#covid #government #lockdown #adulting #unsupervised.” There is no clear indication this was meant to be humorous. (The false claim that vaccines contain chips used for tracking has been widely disseminated on the web and social media.)

Methodology: Simulating typical TikTok activity

NewsGuard’s investigation encouraged a “normal” use of TikTok, not an active search for misinformation. Its nine participants were split into two groups (Group A and Group B) and asked to create an account from scratch without following other users or giving the app any information about themselves, other than a generic email address created specifically for this exercise.

TikTok’s Terms of Service state that the platform is “only for people 13 years old and over.” However, three participants under 13 (one of whom is a nine-year-old) were able to create TikTok accounts by entering a false date of birth when signing up for the app. (Again, NewsGuard obtained the permission of a parent.)

A May 2021 post by TikTok explains that “We train our safety moderation team to be alert to signs that an account may be used by a child under the age of 13. We also use other information as provided by our users, such as keywords and in-app reports from our community, to help surface potential underage accounts. When our safety team believes that an account may belong to an underage person, the account will be suspended.”

Those in Group A, the “high-engagement” group, were given instructions that encouraged them to watch videos about the COVID-19 vaccine and COVID-19 related videos when they appeared, but not to seek them out by using TikTok’s search bar or by following any accounts. They were instructed to watch videos as they normally would, but to scroll to the next video after one to two seconds if the video did not relate to health. If a video was related to health in any way, participants were to watch it until the end, and if it contained content about COVID-19 specifically, they were told to watch it until the end and browse the creator’s profile before returning to TikTok’s main video feed. If the video contained false information about the virus, they were asked to “like” it as well. They were also allowed to look for COVID-19 misinformation by clicking on hashtags. This would represent a fairly normal use of TikTok for any users looking for health-related content.

Group B’s instructions were similar, but involved less engagement with the TikTok app. They waited longer (three to five seconds) to scroll to the next video on the main video feed if it did not relate to health, and they were instructed to not “like” videos or to click on hashtags. All participants recorded their screens as they used TikTok, creating a video that could then be analyzed by NewsGuard.

From Musical.ly to TikTok:  How we got here

The current iteration of TikTok is the result of a merger between two apps in 2018. Musical.ly, an app nearly identical to the current TikTok, was created in 2014 by Chinese entrepreneurs Alex Zhu and Luyu Yang. The company was acquired for approximately $1 billion by ByteDance in November 2017.

ByteDance already owned a similar app (also called TikTok), which launched in 2016 and was referred to as Douyin in China. In August 2018, ByteDance merged Musical.ly and its version of TikTok, keeping the latter name. China’s version of TikTok is still called Douyin, and the app operates as a separate platform that cannot interact with the TikTok app accessible in most other countries, due to Chinese internet restrictions.

TikTok was originally known for lip-syncing videos, which are still plentiful on the app, but the variety of its content now seems limitless. Its videos, most of which include music, if not the lip-syncing variety, are a confounding collection of the bizarre, the serious — and, as described above, the misleading and false.

TikTok’s safeguards

Although TikTok says that users must be at least 13 years old to use the app, as noted above, it is easy to circumvent this rule by providing a false date of birth.

Some of the videos viewed by NewsGuard’s participants that contained false information displayed a “Learn more about COVID-19 vaccines” label at the bottom of the screen that linked to local health authority pages about the vaccines. Beyond this, except in one case, TikTok did not offer any warnings, additional context, fact-checking, or reliable information alongside false claims.

In the recording for one participant, a 15-year-old German speaker, a banner appeared saying “Attention: Video has been flagged for unverified content” in three videos containing COVID-19 misinformation. Despite this warning, the videos were still allowed on the app. No such warnings appeared for any participants outside of Germany.

Participants were also shown some reliable information and videos by medical professionals, although no distinction was made between these videos and those containing misinformation.

TikTok says that it prohibits “content that’s false or misleading, including misinformation related to COVID-19, vaccines, and anti-vaccine disinformation more broadly,” according to its COVID-19 information section. 

The app stated in its first quarter 2021 transparency report that it “will remove or limit misleading information as it’s identified,” noting that it partners “with 11 organizations accredited by the International Fact-Checking Network.” (TikTok publicly issues regular transparency reports with information about the content it removes due to violation of its community guidelines or terms of service.) TikTok’s last transparency report also stated that “if fact checks confirm content is false,” TikTok will “remove or limit it from our platform” and referenced a feature the app introduced at the beginning of 2021 that it said was “aimed at reducing the spread of potential misinformation by informing the viewer when unsubstantiated content has been identified.” 

TikTok says in the report that it removed more than 30,000 videos containing COVID-19 misinformation in the first quarter of 2021. However, these efforts did not stop COVID-19 misinformation from proliferating on TikTok, and, as noted above, only one of the participants in NewsGuard’s investigation was warned that the misinformation they came across was suspect.

A dangerous precedent

There was no shortage of health misinformation online before TikTok began gaining widespread popularity, and until recently, the app has not been as closely scrutinized as more established platforms such as Facebook and Twitter. 

Although no major studies have assessed TikTok’s impact on young people’s attitudes and beliefs,  an August 2020 Spanish study published in the peer-reviewed journal Social Media + Society, looked at young people’s use of the messaging platform WhatsApp. The study found that young people “are more likely to share content if it connects with their interests, regardless of its truthfulness,” and that “the appearance of newsworthy information ensures that, regardless of the nature of the content, this information is more likely to be shared among young people.”

Right now it is up to TikTok’s individual users to discern which content is truthful and which is not and engaging with false content often begets more false content. As one 13-year-old Italian participant observed, “After a while TikTok was proposing only videos about vaccines and very often against them”.

The fact that users can quickly enter a vortex of misinformation, often without meaning to, makes discerning reliable information ever more difficult. As one 14-year-old Italian speaking participant noted: “After clicking [a] hashtag related to COVID-19 I was flooded with COVID-19 content [that was] very often false or misleading.”

Main Image Credit: Photo by Solen Feyissa on Unsplash