How ChatGPT Could Spread Disinformation Via Fake Reviews


In this article, Nico Dekens of ShadowDragon, covers how to identify AI-generated materials online that are intentionally spreading false information or even intended to incite violence.

Our latest research Opens a new window examines how AI, like ChatGPT, is being used maliciously to spread hate and misinformation via fake reviews and deepfakes. The rise of artificial intelligence (AI), particularly the development of ChatGPT, has led to both positive and negative consequences in the way we live, work and interact with the world online. While ChatGPT (and similar technologies) has the potential to provide valuable information and assistance, it is also being abused to spread disinformation, generate fake reviews and even create content that incites violence. This is especially true when tools like ChatGPT are combined with deepfake imagery and audio.

In this article, I will provide context on the definition of disinformation and practical tips on identifying AI-generated materials online, including potential hate speech or offensive content. 

Distinguishing Between Disinformation and Fake Reviews

Disinformation involves deliberately spreading false or misleading information to influence opinions or obscure the truth. Fake reviews, on the other hand, are reviews that are falsely created or manipulated to give an inaccurate impression of a product, service or business.  

ChatGPT utilizes AI and machine learning (ML) to generate responses based on its training from a large data corpus. This data includes decades of opinions and knowledge, including online reviews, that is publicly available on the internet. Based on this training, ChatGPT can create new reviews that appear to be written by real people, making it difficult to distinguish between what is genuine and what is fake. This misuse of AI has a negative impact on society, and it has become increasingly challenging to detect and combat disinformation due to the realistic nature of AI-generated content.

Methods for Identifying Disinformation Spread by AI

One method to combat this disinformation spread by AI is through open source intelligence (OSINT). By examining error messages generated by AI language models like ChatGPT, it is possible to identify potentially harmful content and patterns of disinformation. These error messages serve as keywords that can be monitored on various social media platforms, such as Twitter, Discord, Telegram, 4chan and Reddit. By monitoring these error messages, anyone can detect fake accounts, disinformation campaigns and even specific narratives. 

Examples of error messages generated by ChatGPT that may be seen within an online review or other location include: “as an AI language model,” “not a recognized word,” “cannot provide a phrase,” and “violates OpenAI’s content policy.” These error messages may seem innocuous, but they can serve as red flags for potentially harmful content. 

Moreover, AI-generated content can be identified by searching for specific error messages on social media platforms. For example, searching for “I’m sorry, I cannot generate” on Twitter can help identify accounts potentially using ChatGPT to spread disinformation. Similarly, using search operators like “inurl:post” on platforms like LinkedIn and Instagram, along with specific error messages, can help identify AI-generated content. By leveraging the same OSINT techniques as the experts investigating serious crimes and monitoring AI-generated error messages, it is possible to gain insights into the methods used to spread disinformation and false information online.

Fake reviews generated by AI language models like ChatGPT can also be identified by searching for specific error messages in combination with search operators on platforms like Amazon. For instance, searching for “as an AI language model” on Amazon can help identify reviews that have been generated by AI. These AI-generated reviews may appear “normal” and “human” at first glance, but they contain an error message that indicates their origin. By scrutinizing the content of reviews and looking for these error messages, it becomes possible to differentiate between genuine and AI-generated reviews.

How AI-generated Imagery and Audio Increase Disinformation

When AI language models are combined with generative AI that creates imagery and audio, the potential for spreading highly realistic disinformation increases.

Deepfakes, which are manipulated videos, images or audio recordings produced using AI, can be combined with AI language models, like ChatGPT, to create highly convincing disinformation. Deepfake technology can manipulate facial expressions, voice and even body movements to make it seem like someone is saying or doing something they never actually did. This combination of AI-generated text with manipulated multimedia can amplify the spread of disinformation by creating convincing and misleading content.

Detecting deepfakes and AI-generated content requires a combination of technical analysis and critical thinking. Here are some methods that can help to identify potential deepfakes:

  • Visual anomalies: Look for inconsistencies or unnatural movements in facial expressions, eye movements or body gestures. Deepfake videos may have artifacts, blurred edges or mismatched lighting and shadows.
  • Audio abnormalities: Pay attention to any unusual voice modulation or synthetic-sounding voices. Deepfake audio may lack natural intonation or exhibit glitches.
  • Source verification: Investigate the original source of the content. Check for metadata, image or video timestamps, and compare with other reliable sources to ensure authenticity.
  • Reverse image search: Perform a reverse image search to see if the same image or face has been used in other contexts or by different individuals.
  • Expert analysis: Consult experts in the field of deepfake detection who can employ advanced techniques and algorithms to identify manipulated content.
  • Movement inconsistencies: Pay attention to any unnatural or glitch-like movements in the video. Deepfake algorithms may struggle to accurately reproduce intricate details, resulting in slight abnormalities or inconsistencies in movement.
  • Contextual analysis: Consider the context in which the content is presented. If the information seems too sensational or unlikely, it could be a sign of disinformation. Cross-reference the information with reliable sources to verify its authenticity.
  • Public awareness and education: Promoting media literacy and educating the public about the existence and potential impact of deepfakes and AI-generated disinformation is key. By empowering individuals to critically analyze information, we can collectively combat the spread of disinformation.
  • Machine learning algorithms: For the more technical groups out there, developing and training machine learning models specifically designed to identify deepfakes will help the greater good combat this issue. These models can learn patterns and characteristics unique to deepfakes, helping to distinguish between real and manipulated content.

See More: What Is Deepfake? Meaning, Types of Frauds, Examples, and Prevention Best Practices for 2022

What Is Next for AI Like ChatGPT in the World of Disinformation?

As technology progresses, so does the sophistication of deepfakes and AI-generated disinformation. Adversarial AI, which involves training AI models to generate deepfakes that can specifically bypass detection algorithms, is an emerging concern. This ongoing arms race between the creators of deepfakes and those working to detect and mitigate them necessitates constant research and development.

In conclusion, while AI language models like ChatGPT have the potential to contribute to the spread of disinformation, there are various techniques and approaches available to identify and combat AI-generated disinformation. By employing a combination of technical analysis, critical thinking and expert assistance, we can mitigate the harmful effects of deepfakes and AI-generated disinformation. Continued research, public awareness, and the development of robust detection methods are essential to stay ahead of the evolving landscape of disinformation in the age of AI.

It’s important to note that detecting deepfakes and AI-generated content is an ongoing challenge as the technology continues to evolve. As AI models improve and become more sophisticated, identifying disinformation becomes increasingly complex. We must continue seeking solutions to keep the public both aware of the concerns of AI and armed to overcome those risks.

What challenges do you face with regard to AI-generated content? Share with us on  FacebookOpens a new window , TwitterOpens a new window , and LinkedInOpens a new window . We’d love to hear from you!

Image Source: Shutterstock