OpenAI dismantles Russian disinformation campaign run through ChatGPT

Full text

OpenAI says it has disrupted a social media influence campaign that is believed to be of Russian origin

What makes this campaign stand out is the underlying architecture, which features a "think tank" with highly questionable credibility

The objective of the campaign was to influence Western audiences into believing Russia was superior, and that Western countries were sacrificing their sovereignty

AI tools have once again been used for bad, rather than for good, and this time it has been as part of a campaign attributed to Russia that tried to discredit Western countries supportive of Ukraine.

The threat actors used ChatGPT accounts to promote the work of an “expert community” of academics known as the International Burke Institute (IBI). The site used stolen academic works attributed to the wrong authors in an attempt to appear legitimate while also hiding the true source of its academic ‘contributions’.

The IBI was registered to an address in Israel - with some evidence suggesting real individuals in Israel represented and promoted the page - but the main purpose of the misinformation campaign was to paint Russia in a favourable light compared to its Western adversaries.

International Burke Institute, or an institute run by international berks?

In its report on the campaign OpenAI points out that those behind the ChatGPT accounts used to promote the page took careful steps to hide their Russian origins. Many of the prompts included instructions to hide any linguistic clues that the operators used Russian language prompting.

One inclusion on the IBI website referred to Germany’s traffic light coalition as the “Svetofor coalition”. Svetofor is the word for ‘traffic light’ in numerous Slavic languages, including Russian, indicating that drafts were written in Slavic languages before being translated.

Some of the accounts were used to promote the IBI across X, LinkedIn, Facebook, Substack and Telegram with AI generated imagery and captions, with other accounts being used to automatically engage with comments by real users on Substack. The operators of the ChatGPT accounts regularly requested performance summaries of these pages in Russian, and used ChatGPT to generate matching profile pictures for some accounts.

Burke Sovereignty Index

One of the features of the IBI website is to advertise the ‘Burke Sovereignty Index’ - designed to measure how well a country performs compared to others across political, economic, technological, informational, cultural, cognitive and military factors.

The Index’s purpose is to make Western countries appear worse than Russia, OpenAI says. Having taken a look at the Index myself it's clear there is no consistency in measurement or comparison. For example, The Vatican City - which is less than half a square kilometer in size - sits above Spain in its average of scores. The Index also gave Russia the highest military score.

Questionable expert contributors

The website also includes a list of experts, whose affiliation to the IBI is not referenced. Some among them are pioneers in their fields of study, such as Francis Fukuyama and Noam Chomsky.

Others are former high-ranking members of the US government, such as Mike Pompeo and Joseph Nye. There are even listings for experts who passed away before the IBI was founded, such as Shlomo Avineri and Jiang Ping.

Much of the academic work cited on the IBI website is legitimate, but more often than not has been stolen from its actual author and misattributed to a different author to hide the work’s actual source, OpenAI said. This has been done to give the IBI website credibility and to make it appear authentic.

Impact of the IBI influence campaign

The overall impact of the IBI and its promotion using social media and ChatGPT is fairly limited, OpenAI noted. Some of the Telegram channels garnered followings between ten to twenty thousand showing a limited breakout to authentic audiences.

But the main thing to take away from this is the level of dedication placed behind the underlying infrastructure of the IBI. The site is designed to look authentic, uses authentic research (even if it is wrongly attributed), and presents itself as a collection of experts. To the layman, a cursory glance at the IBI website would give any of their social media presence a level of authenticity not seen in other social influence campaigns.