In late 2019, Facebook quietly removed a network of pages and accounts that had been meddling in Libya’s chaotic political scene. The operation was linked to Yevgeny Prigozhin, the Russian businessman with a long history of running online influence campaigns. A year later, in December 2020, Facebook did it again: another Libya-focused operation, again linked to Prigozhin, was taken down. But something had changed. The 2019 version relied on pages that pretended to be independent news outlets and posted political cartoons. The 2020 version was more polished and more embedded: one of its fake media outlets had an actual on-the-ground presence, branded merchandise, and a daily podcast. That small comparison tells a larger story. Between 2018 and 2020, Facebook and Twitter announced that they had dismantled 147 influence operations in total. Researchers examined those public announcements to understand how online disinformation was evolving. What they found was a familiar arms race. Some tactics stayed the same, but the players adapted, often in direct response to the platforms’ own detection strategies. Political actors are increasingly outsourcing their dirty work to public relations and marketing firms. Fake accounts now use computer-generated faces instead of stolen photos. And platforms, for their part, are getting bolder about naming exactly who is behind an operation. The result is a constantly shifting landscape of deception, where every defense produces a new offense, and every offensive move forces platforms to rethink their tools.
The first and most encouraging shift is that Facebook and Twitter are becoming much more specific when they attribute operations. In 2018, only about 47 percent of takedowns were tied to a specific actor. In 2019, that number rose to 62 percent. By 2020, it was roughly 76 percent. That might sound like a technical detail, but it matters enormously. When platforms simply say a network originated in a particular country, it’s easy for governments to shrug and deny involvement. When they name the Iran Broadcasting Company, the Royal Thai Military, or a specific IT company in Tehran, they are publicly shaming the perpetrators and giving citizens and the international community something concrete to respond to. This kind of “name and shame” approach doesn’t always change behavior, but it is a starting point. It also reflects a serious investment in investigative work. Disinformants go to great lengths to cover their tracks, so the ability to trace an operation not just to a country but to a specific institution or individual shows how much resources and collaboration are going into these efforts. Reporters, civil society groups, and platforms are working together in ways that simply didn’t exist a few years ago. At the same time, platforms pay a price for being transparent. Sometimes the response is just mockery. After Facebook attributed a takedown to him, Prigozhin dismissed the company with a sarcastic statement predicting that its era would end and that states would close their borders to “political perverts.” Other times, the response is more menacing. After Twitter removed 7,000 accounts linked to the youth wing of Turkey’s ruling party, a senior government communications official reminded Twitter of the “eventual fate” of organizations that had tried similar steps in the past. That threat came as Turkey was tightening restrictions on online platforms. Still, the trend toward specific attribution is a meaningful development. It creates a public record, shifts blame away from vague nations and onto actual decision-makers, and makes it harder for bad actors to hide behind ambiguity.
But as defenses improve, disinformants innovate. The cat-and-mouse game between those running influence operations and those trying to expose them produced some striking new tactics in 2020. Fake accounts are the backbone of most influence operations, and a convincing fake account needs a profile picture. For years, researchers caught fake accounts by using reverse image searches to find the real person whose photo had been stolen. So propagandists adapted. They started using AI-generated profile pictures, computer-created faces that do not belong to any real human. These images are freely available, and because they are not photographs of actual people, reverse image searches turn up nothing. The first major takedown involving AI-generated profile pictures deployed at scale happened in December 2019. In 2020, at least seven takedowns included networks using these synthetic faces. They ranged from domestic operations, like a student group linked to the Communist Party of Cuba targeting Cuban audiences, to cross-border operations, like a Russia-linked network aimed at the United States. Another new tactic is handle switching. Previously, researchers could identify suspicious accounts by looking them up on archiving sites. An account might have been involved in an operation targeting Sudan, then delete its Sudan-related tweets when it shifted to a new operation targeting Morocco. The archived history would reveal the inconsistency. In 2020, several disinformation actors started changing not just their usernames and bios, but their handles too. An account could go from twitter.com/john123 to twitter.com/jen123 while keeping all of its followers. That makes it nearly impossible for researchers to discover what the account was doing before. One striking example involved accounts linked to the Saudi government. They posed as a Qatari interim government and dissident Qatari royals living in Saudi Arabia, spreading quickly discredited rumors of a coup in Qatar. Three of those accounts had more than 100,000 followers each, but they had only recently assumed those identities. It’s unclear how they built such large audiences, but they may have spent their earlier lives engaging in spammy follow-back activity. By deleting old tweets and changing their handles, they could start fresh while keeping their reach. Platforms are starting to fight back. Facebook’s Page Transparency feature, for example, shows the history of a page’s name, making it harder for bad actors to switch identities on that platform. Twitter could implement a similar feature in the future. But for now, the attackers often seem one step ahead.
A third trend is the outsourcing of disinformation to private firms. In 2020, Facebook and Twitter attributed at least 15 takedowns to marketing or public relations companies. This practice first became notorious in 2016, when Russia used the Internet Research Agency, an ostensibly private company run by Prigozhin, to interfere in the U.S. election. Since then, third-party actors have popped up all over the world: troll farms in the Philippines, strategic communication firms in the United States, PR companies in Ukraine. On the surface, outsourcing seems risky. If you want to keep an operation secret, why bring in a third party who might leak or mess up? But there are real benefits, especially deniability. When a platform discovers an inauthentic network run by a marketing firm, technical indicators may show that the firm is behind it, but digital evidence rarely reveals who paid for it. Politically, even if everyone can guess the likely sponsor, the PR firm caught red-handed gives the political player someone to blame. Yet outsourcing also creates problems for governments. They face a principal-agent problem: the firm may have different incentives than the government that hired it. A marketing company might be able to hit certain quotas, like generating a certain number of tweets or trending hashtags, without actually achieving the political impact the client thinks it purchased. In one suspended Twitter network linked to a Saudi digital marketing firm, researchers observed accounts using something called “hashtag training.” The accounts would tweet a nonsensical series of hashtags, including long, convoluted ones that advertised the firm’s bug exterminator client, along with political hashtags criticizing Qatar. The apparent goal was to get these hashtags to trend on Twitter. That might impress the marketing firm’s client, but it’s unlikely to produce the kind of organic political movement the government hoped for. In other words, outsourcing may make disinformation easier to deny, but it also makes it harder to control and less effective in practice.
Not all countries are targeted equally, and that is another important lesson from 2020. Among the operations publicly announced by Facebook and Twitter, the countries most frequently targeted by foreign disinformation campaigns were the United States, the United Kingdom, and Egypt. Those same countries topped the list in 2019, though that year also saw several operations aimed at Qatar and Ukraine. Why these countries? Geopolitical realities probably play a role. Ukraine and Qatar have strategic importance for Russia and Saudi Arabia, respectively, and both of those countries have strong disinformation capabilities. The United States and the United Kingdom may be attractive targets because of their open media environments, which make it easier for outsiders to inject false or divisive content. But there are real challenges in interpreting these numbers. First, the data comes only from takedowns on Facebook and Twitter. It tells us nothing about disinformation campaigns on other platforms, like Telegram, which is especially popular in some regions and has become a known haven for influence operations. Second, we don’t know what operations Facebook and Twitter failed to discover or chose not to publicize. Takedowns are likely more common in places that platforms and researchers disproportionately investigate, so the absence of takedowns in certain countries doesn’t mean those countries aren’t targeted. Third, some takedowns are not distinct operations but rather follow-ons from previous takedowns or parts of a single cross-platform campaign. That can lead to overcounting, as missed accounts resurface or networks respawn. It can also lead to double counting when the same perpetrators use both Twitter and Facebook as arms of one coordinated effort. At the same time, not all countries are equally likely to run disinformation operations. Russia, Iran, and Egypt were the most frequently attributed sources of foreign influence campaigns in 2020, just as they were in 2019. These countries have developed sophisticated capabilities and appear willing to use them repeatedly, often against the same set of targets.
Finally, the old standby of fake news outlets remains as popular as ever. In both 2019 and 2020, many of the operations removed by Facebook and Twitter included pages or accounts that pretended to be independent media organizations. This tactic is partly a byproduct of the democratization of the internet: new blogs and news sites appear every day, and a fake outlet can easily hide among them. One operation linked to the Islamic Movement of Nigeria, a group advocating for an Islamic form of government in Nigeria, included seven pages that claimed to represent NaijaFox.com, a pseudo-news site publishing anti-Western articles. The FBI has since seized the NaijaFox domain. Most fake media outlets are original creations, but some go further and impersonate real, existing outlets. A takedown in November of a network originating in Iran and Afghanistan included a Facebook page and Instagram account pretending to represent Afghanistan’s most popular TV channel. A common trick is “typosquatting,” changing just one letter in a URL so it looks legitimate at first glance. Researchers from CitizenLab documented how an Iran-aligned network impersonated Bloomberg as “bloomberq.com” and Politico as “policito.com.” As platforms get better at monitoring for information operations, bad actors see greater value in investing in domains, which are less vulnerable to suspension than social media accounts. They also drive traffic to third-party messaging apps that don’t actively monitor for disinformation, like Telegram. It has become routine for Facebook and Twitter to suspend fake news accounts that have linked Telegram channels, only for the Telegram channels to continue operating as if nothing happened. Interestingly, while these outlets are “fake,” their content is not always obviously false. Some of it is fabricated, like the absurd image of the Taliban apparently praying for President Donald Trump’s recovery from COVID-19. But much of the content is non-falsifiable: not true, but also not demonstrably false. Bad actors use their accounts to spread hyperpartisan material, like Saudi-linked Twitter accounts pretending to be ordinary Libyan citizens tweeting cartoons that mock Turkey and Qatar. The overall impact of these operations is still an open question. Some are remarkably ineffective, like a recent operation attributed to the Royal Thai Army in which the average tweet received only 0.26 engagements. But the broader strategic threat may be more significant. Even when individual operations fail, the knowledge that such manipulation exists erodes public trust in the entire information ecosystem. As governments and platforms work to respond, they must begin by understanding how these operations actually work, and then think carefully about the next round of tactical innovations that are surely already underway.