AI, Large Language Models and Targeted Disinformation
An interview with Matthew Gentzel,
Designation: Program Officer at Longview Philanthropy
Previously: OpenAI, National Security Commission on Artificial Intelligence
As part of our efforts to understand how Artificial Intelligence (AI) can change the future of conflict, we recently had the chance to interview Matthew Gentzel,1“Matthew Gentzel,” Longview Philanthropy, n.d., https://www.longview.org/about/matthew-gentzel Program Officer at Longview Philanthropy. During our discussion, the one topic that really interested us the potential for AI and large language models (LLMs) for targeted disinformation campaigns.2Mark Riedl, “A Very Gentle Introduction to Large Language Models without the Hype,” Medium, April 14, 2023, https://mark-riedl.medium.com/a-very-gentle-introduction-to-large-language-models-without-the-hype-5f67941fa59e In this piece, we share a summary of Matthew’s thoughts on the topic and what it might mean for the wider society.
Matthew began by sharing how he had initially been skeptical about LLMs being used for influence operations. However, his perspective shifted when he came across research highlighting how prompts from generative models were changing people’s perspectives on issues such as trade policy or policies in countries like Afghanistan and China. In some cases, their stance had shifted by up to 10 percentage points!
On what makes this different than other means of influence operations
Matthew agreed that while it is true that other techniques can also be used, what makes AI and LLMs different is the speed and level of automation in the process. These systems are also superior since they can perform what he called a “cultural context autocorrect.” For example, consider a Russian troll farm that is trying to influence voter behaviour in an upcoming election cycle. Such an operation would involve many propagandists trying to tweet as much as possible using false accounts on Twitter. However, their lack of fluency in English and their unfamiliarity with the cultural contexts of those that they’re attempting to influence would make it easy for such accounts to detected and taken down.
In contrast, when the same individuals use an LLM, suddenly their material is fluent, error-free and without any idiosyncrasies of language. Moreover, they fit perfectly with the local cultural context. Hence, LLMs enable such actors to use this “beautifully curated” propaganda material with strategic intent, which makes their operations much more effective than traditional troll farms. This was already the case with GPT3.0 and GPT3.5, but it has become much more powerful with GPT4.0.3Mohammed Haddad, “How Does GPT-4 Work and How Can You Start Using It in ChatGPT?,” March 15, 2023, https://www.aljazeera.com/news/2023/3/15/how-do-ai-models-like-gpt-4-work-and-how-can-you-start-using-it These techniques, when combined with the scale and reach of social media, can do great harm
On what makes GPT different than existing social media algorithms which can be manipulated by malicious actors
He once again spoke about his historical scepticism of LLMs, especially when compared to social media algorithms and how they had been shaped by different companies and advertisers. However, he added that it’s the speed and overall capabilities of LLMs which makes this a much more challenging problem than simple algorithm manipulation. Here, he gave the example of a conservative commentator who liked engaging in political arguments with his liberal friends.4 “American Political Attitudes and Participation,” American Government, n.d., https://www.ushistory.org/gov/4.asp He used one of these GPT models to reframe everything he wanted to say so it sounded much less “conservative”. Hence, he was able to use the language to fit the group that he was chatting with. This would then cause the “other side” to just argue amongst themselves and not kick him out of the group. Funny as though this might seem, it should be added that this technique allowed him to “blend in” as if he was an insider and use his presence to influence the group he was infiltrating.
Connecting all this back to social media, he acknowledged that these platforms were taking measures to identify and shut down false accounts. However, the problem here is that accounts using LLMs for targeted influence operations are not necessarily false accounts. Most of these are legitimate accounts who are propagating certain content with strategic content, all while using LLMs to sound much more articulate and persuasive.
Looking at the future and the power of GPT4
He stated that one argument that is given against the use of LLMs for such operations is that GPT4 has a subscription fee. To this, he argued that while a $20/month subscription fee might be expensive for some people, it’s definitely not prohibitively expensive. Even if one considers countries where the fee might be a lot in local currency terms, it’s still not beyond the reach of interest groups who would be looking to use LLMs for targeted operations in other jurisdictions.
In conclusion
He concluded by referring a new scepticism about the use of LLMs for targeted operations and arguing that some of the risks might be overblown. This is because he believes that if “one side” will these models to be more effective and persuasive, so will the “other side”. This might, perhaps, lead to a cancelling out effect. And lastly, there is a positive side to all this as well. LLMs like GPT are enabling many more people to communicate in a more effective manner, opening newer opportunities for them. Therefore, a lot comes down to the specific use cases, most of which he believes are going to be positive. This isn’t to say that negative ones won’t exist, it’s just that technologists, policy makers and other stakeholders will have to work together to counter the effects of such use cases.
Perhaps the biggest AI-enabled influence threats in general (not just LLMs) in the near-term are AI-enabled censorship and influence operations that aim to create strong stigmas and social pressure. People can sort through all sorts of nonsense with the right mindset and access to other information: when there is coercive social or legal pressure and the inability to find accurate information in the first place, then even the smartest people can end up making much worse decisions than they would otherwise. In short, coercive influence is the most concerning type of influence.
Note: This piece reflects the personal opinions of the interviewee and cannot be attributed to his current or former employers.
The opinions expressed in the articles on the Diplomacy, Law & Policy (DLP) Forum are those of the authors. They do not purport to reflect the opinions or views of the DLP Forum, its editorial team, or its affiliated organizations. Moreover, the articles are based upon information the authors consider reliable, but neither the DLP Forum nor its affiliates warrant its completeness or accuracy, and it should not be relied upon as such.
The DLP Forum hereby disclaims any and all liability to any party for any direct, indirect, implied, punitive, special, incidental or other consequential damages arising directly or indirectly from any use of its content, which is provided as is, and without warranties.
The articles may contain links to other websites or content belonging to or originating from third parties or links to websites and features in banners or other advertising. Such external links are not investigated, monitored, or checked for accuracy, adequacy, validity, reliability, availability or completeness by us and we do not warrant, endorse, guarantee, or assume responsibility for the accuracy or reliability of this information.