(Dis)information wars

Adrian Casillas, Maryam Farboodi, Layla Hashemi, Maryam Saeedi, and Steven Wilson

Source Link

Introduction

In countries with authoritarian regimes, traditional means of information dissemination such as newspapers, TV, and radio are heavily controlled by the central government. In the early 21st century, the introduction of social media and decentralized platforms proved to be a groundbreaking development in these countries. The rapid improvement of big data technologies enhanced convenient access at an unprecedented rate and made these platforms prominent vehicles for displaying dissidence during episodes of unrest.

Soon after, authoritarian regimes intervened by limiting internet access and censoring social media. In parallel, they started to spread disinformation on social media through propaganda accounts—accounts that are publicly pro-government and spread false news in an attempt to change the narrative in favor of the government. However, the widespread access of the public to multiple sources of information has reduced the effectiveness of this tactic. As such, governments have turned to a smarter approach to supply fake news. They engage in a “disinformation war,” i.e., they create imposter accounts who spread fake news on social media platforms while pretending to be unbiased, ordinary accounts (Hynes, 2021).

Disinformation wars have several advantages as they do not require exerting force and they are difficult to trace, yet they disrupt the flow of information. Therefore, they derange the opposition movement without apparent aggression.1 Furthermore, it is difficult to identify the pieces of fake news that have originated from imposter accounts, as these accounts imitate the behavior of ordinary accounts in many respects.

Although both strategies spread fake news, unlike classic propaganda, disinformation wars are not intended to control the narrative in favor of the central government. Rather, they are a means to disturb the narrative to derail and discredit the protest movement. To delineate the supply of fake news on social media platforms, it is crucial to understand both strategies concurrently.

As the prevalence of disinformation wars continues to grow, it becomes crucial to address the challenges they present. Previous studies have highlighted real-time content moderation and ex-post efforts to eliminate audience biases as essential strategies to contain the spread of fake news, but they have pressing limitations. Real-time fact-checking or content moderation is time-consuming, allowing disinformation to go viral before it can be addressed; moreover, ex-post debunking shows limited impact on debiasing the public.2 In this paper, we propose an alternative approach to restrict the supply of disinformation—ex-ante content moderation. We predict accounts likely to engage in spreading disinformation, even before they do so, and explore the effectiveness of a policy that uses this ex-ante information to limit the spread of fake news on social media.

We apply our method to the disinformation war launched on Farsi-X, formerly known as Farsi-Twitter, in the wake of recent protests in Iran. On September 16, 2022, a 22- year-old Iranian woman named Mahsa Amini died in a hospital in Tehran, Iran, after being arrested by the religious morality police of Iran’s government for not wearing the hijab per government standards. Eyewitnesses reported that she died as a result of police brutality which was denied by Iranian authorities. Amini’s death resulted in a series of widespread protests across Iran. These protests were primarily concentrated among the younger generation and were accompanied by a lot of activity on social media, particularly on Farsi-X, where the hashtag #MahsaAmini was created and widely used.3 Concurrently, a surge in disinformation supply across Farsi-X occurred during the protests. Multiple sock puppet accounts, which we call “imposter accounts,” were created by certain entities to disseminate fabricated, misleading, and false information.4 There is a variety of entities who engage in spreading disinformation including the government and certain opposition groups which further complicates identifying them.

We group all the accounts on Farsi-X into three groups: propaganda, ordinary and unsafe. Propaganda accounts openly engage in spreading pro-government propaganda. Ordinary accounts are those who generally do not engage in either propaganda or disinformation. Unsafe accounts constitute two groups. The first group is imposter accounts, those who start by pretending to be dissidents, posting pro-dissidence content and hashtags to build a network among the protesters. Subsequently, they intentionally start posting disinformation. This disinformation then spreads by other imposter accounts as well as some of the ordinary X accounts. The second group consists of normal X accounts who actively engaged in the spread of disinformation, albeit unintentionally.

We create a unique dataset comprising all posts, formerly called tweets, in Farsi from September 16, 2019, to March 14, 2023.5 We augment this data with a network of user re lationships, including follower-following connections and re-posts, formerly called retweets, interactions. We also gather a comprehensive set of user characteristics within this network, such as date created and activity rates. We then assemble a labeled dataset of unsafe, traditional propaganda, and ordinary accounts.

The paper has two methodological contributions. First, we propose social network-based characteristics that help identify unsafe accounts. We outline and implement an algorithm to construct these “network proximity measures” for our labeled dataset.

Second, to shed light on the supply and spread of disinformation, we devise a classifier to determine whether an account on Farsi-X is unsafe, propaganda, or ordinary. Our core classifier is a multinomial logit model that uses these network proximity measures along with non-network account characteristics. We train the multinomial logit model on a portion of our labeled dataset and use the remainder of the labeled data to test the model.

We then use the trained classifier to assign “disinformation scores” and “propaganda scores” to all Farsi-X accounts. The disinformation (propaganda) score of each X account reflects the probability of that account being an unsafe (propaganda) account. The disinformation score indicates the likelihood of an account acting as an unsafe account and actively engaging in the dissemination of disinformation, intentionally or not, even if it has not yet done so. The propaganda score indicates the likelihood that an account engages in explicit propaganda. We find that the unsafe accounts—those with a high disinformation score, have a disproportionate share in a wide-spread supply of fake news in the social network, compared to the traditional propaganda accounts.

The disinformation score is a useful instrument to guide the activities of ordinary users as well as to design policy. At the same time, it is potentially prone to manipulation by unsafe accounts. To address this concern, we propose an alternative classifier that only relies on network proximity measures and find that this limited classifier achieves 81.7% accuracy in detecting unsafe accounts. The significance of this finding is twofold: First, it highlights the importance of the structure of the social network in detecting adversarial behavior on social media and as such, closely ties the literature on social networks with media economics. Second, since the network proximity measures are difficult to manipulate, as they depend on the past and present structure of the social network, it shows that our disinformation scores are robust to manipulation by unsafe accounts.

We propose two policies using our disinformation score and assess their effectiveness in interrupting the flow of disinformation on social media. The first policy involves blocking posts by unsafe or propaganda accounts, or both, while the second policy involves disclosing either disinformation scores, propaganda scores, or both.

To evaluate the efficacy of our policies, we analyze verified instances of disinformation campaigns that occurred during the recent period of unrest in Iran. Our findings indicate that blocking unsafe accounts or disclosing disinformation scores significantly reduces the spread of disinformation among ordinary accounts, leading to a faster cessation of disinformation. However, blocking or disclosing information about propaganda accounts does not have a substantial impact. This finding highlights the importance of detecting unsafe accounts to contain the supply of information on social media.

A critical advantage of our methodology is that it proactively targets accounts with a high propensity to engage in disinformation campaigns, even before they do so. Early identification of these accounts provides a tangible opportunity to take preventative steps before the disinformation goes viral, making it an effective mechanism to combat the spread of disinformation. Furthermore, due to its high tractability, this methodology can be applied to a wide range of scenarios for which the spread of disinformation is a problem, thereby bolstering our ability to counteract the negative impact of disinformation proliferation.

Literature Review

Our paper contributes to the extensive body of literature on the economics of media. One strand of this literature considers media capture by governments and its consequences (Besley and Prat, 2006). A second strand studies the political economy of media censorship. Some papers focus on the government obstructing access to valuable information (Schedler, 2010; Shadmehr and Bernhardt, 2015), while others explore the effects of public demand for uncensored and non-ideological information (Gentzkow and Shapiro, 2006; Chen and Yang, 2019; Simonov and Rao, 2022), another strand studies the impact of change in technologies on news production (Cag´e et al., 2020; Angelucci et al., 2020; Levy, 2021).

We focus on a less explored intervention employed by authoritarian regimes to influence political outcomes: the deliberate spread of disinformation on social media platforms, aimed at distorting waves of unrest. Gottfried and Shearer (2016) emphasize the significance of social media, providing evidence that approximately three-fifths of adults in the United States access their news through these platforms. Cag´e et al. (2020) show that even mainstream media are impacted by social media news. In their research, Allcott and Gentzkow (2017) delve into the theoretical and empirical aspects of fake news dissemination on social media prior to the 2016 election. Moreover, Thomas et al. (2012) and Stukal et al. (2017) present evidence highlighting the extensive use of false information, particularly on Russian X.

Estimating the volume of misinformation circulating on social media between 2015 and 4 2018, Allcott et al. (2019) found that user interactions with false content increased steadily on Facebook and X until the end of 2016. However, they also discovered a sharp decline in interactions with false content on Facebook since then, while interactions on X continued to rise. Additionally, Bradshaw and Howard (2018) conducted an examination of organized social media manipulation campaigns in 28 countries worldwide, uncovering evidence of governments employing social media as a tactic for manipulation

Given the abundance of evidence regarding the use of social media in the manipulation of public opinion, several researchers have explored methods to combat this issue, see Bak-Coleman et al. (2022). Some researchers have proposed real-time fact-checking and moderation of information. However, Vosoughi et al. (2018) show that false information tends to spread faster than true information. They investigate the differential diffusion of true and false news stories using a comprehensive dataset of fact-checked rumor cascades on X spanning from its inception in 2006 to 2017. Due to the rapid spread of misinformation, real-time moderation appears futile, as disinformation often goes viral before being detected by content moderators. While ex-post rebuttals of disinformation have been extensively studied, their impact has been found to be limited (Kunda, 1990; Chan et al., 2017; Nyhan and Reifler, 2010; Ecker et al., 2022; Kahan et al., 2017).

There is also a strand of theoretical literature on information diffusion, information aggregation, and belief formation. The seminal work of Crawford and Sobel (1982) studies strategic dissemination of information using a cheap talk model. Acemoglu et al. (2010) consider the trade-off between information aggregation and propagation of misinformation on social media. Akbarpour et al. (2020) studies the optimal seeding strategy for information diffusion in networks. Related to our policy experiments, Budak et al. (2011) study theoretical approximation algorithms that can effectively limit the spread of misinformation on social media.

The rest of the paper is organized as follows. Section 2 provides details of the Farsi-X data that we use for estimation. Section 3 describes the algorithm we use to construct the network measures which we then use for estimation, as well as the estimation methodology for scoring. Section 4 presents the results. Section 5 reports the outcome of policy experiments guided by the disinformation scores estimated in the paper. Lastly, Section 6 concludes.