AI Hallucinations: Causes, Risks, and Mitigation Strategies

In this article, we will explain in detail about AI hallucination, its causes, the risks we face, and effective countermeasures. As the use of AI becomes more widespread, it is essential to deepen our understanding of hallucination and deal with it appropriately in order to use AI technology safely and effectively.

In recent years, generative AI, including ChatGPT, has made remarkable progress, bringing revolutionary changes to our lives and businesses. However, behind this evolution, a problem called “Hallucination” has come to light. Hallucination is a phenomenon in which AI generates information that is different from the facts or is erroneous as if it were the truth.

What is AI hallucination?

Definition and Overview of Hallucination

AI hallucination refers to the phenomenon in which an AI model generates information that is not based on fact. Specifically, it creates sentences, images, videos, etc. that appear to be true at first glance by fabricating information that does not exist in the training data or by incorrectly combining existing information.

This phenomenon is also called “AI lies” because it seems as if the AI is “lying.” However, AI does not lie intentionally like humans do. AI simply generates the most statistically likely output based on the patterns of training data. As a result, information that is different from the facts may be output.

The difference between hallucination and hallucinations

Hallucination is often confused with the word “hallucination,” but there is a clear difference between the two. Hallucination is a phenomenon in which the human brain produces sensations such as sight, hearing, and touch without external stimuli. On the other hand, AI hallucination is a phenomenon in which an AI model generates false information based on training data, and is not a phenomenon that occurs in the human brain.

Mechanism of Hallucination

Hallucination in AI is mainly caused by the following factors:

1. Biased or insufficient training data: AI models learn based on the information contained in the training data, so if the training data is biased or insufficient, it is susceptible to hallucination. For example, if there is a lack of information on a particular topic, the AI may generate inaccurate information in response to questions about that topic.
2. Model overfitting: Overfitting is a phenomenon in which an AI model becomes overly adapted to the training data and is unable to respond well to unknown data. When overfitting occurs, the AI may generate information that is not present in the training data.
3. Incorrect prompts: Prompts are instructions given to the AI. If the prompt is vague or contains inappropriate information, the AI may generate incorrect information.
4. The nature of probabilistic generation: Generative AI generates content based on probabilistic algorithms, so the correct output is not always guaranteed. In particular, large-scale language models may generate information that is out of context because they select words and phrases probabilistically.

These factors combine in a complex way to cause hallucination in AI. Hallucination is one of the important issues that must be resolved as AI technology evolves, and various countermeasures are being researched and developed.

Risks and Effects of Hallucination

AI hallucination may seem like a minor issue at first glance, but its impact on individuals, companies and organizations, and society as a whole could have serious consequences.

Risks to individuals

Hallucination can negatively impact an individual’s life in many different ways.

• Spread of misinformation and decline in credibility: If people believe the fake news and misinformation generated by generative AI, they may act based on their erroneous perceptions and suffer disadvantages. In addition, if information generated by AI spreads on the Internet, the credibility of the information may decrease, and the information environment in society as a whole may deteriorate.
• Promoting discrimination and prejudice: AI models may generate content that contains discriminatory or prejudiced content due to the biases contained in the training data. The spread of such content may promote discrimination and prejudice, deepening social inequality.
• Privacy violation: There is a risk that generative AI may generate personal information or information related to privacy. For example, if text or images containing the names, addresses, or phone numbers of real people are generated, this could lead to a privacy violation.
• Potential for fraud and misuse: Generative AI could also be used for malicious activities such as fraud and phishing. For example, there are concerns that it could be used to generate emails or messages that impersonate real people to defraud people of money or steal personal information.

Risks to companies and organizations

Hallucination also poses significant risks to businesses and organizations.

• Decisions based on incorrect information: Making important business decisions based on incorrect information generated by generative AI can cause significant damage to a company. For example, if the results of market analysis or competitive research are incorrect, a wrong marketing strategy can be created, leading to poor business performance.
• Damage to reputation: If generative AI generates inappropriate content and it becomes public, it could seriously damage a company’s reputation. In particular, content that contains discriminatory or prejudiced language could draw social criticism and seriously damage the brand image.
• Legal liability: If the content generated by generative AI raises legal issues, such as copyright infringement or defamation, companies may be held legally liable.

Risks to society

Hallucination can also have serious effects on society as a whole.

• Deterioration of the information environment: If generative AI generates and spreads large amounts of fake news and misinformation, the reliability of information will decrease, and the information environment throughout society could deteriorate.
• Increased social unrest: Fake news and deep fakes have the potential to increase social unrest. In particular, the spread of misinformation about politics and the economy could incite public anxiety and lead to social unrest.
• Threat to Democracy: If generative AI is misused to create fake news and deep fakes that can influence elections and public opinion, it could undermine the very foundations of democracy.

Hallucination Control: A Technical Approach

AI hallucination is both a technical and a social problem. Here we describe technical approaches to addressing hallucination.

Improving training data

AI models are highly dependent on the quality and quantity of training data. Improving training data is important to prevent hallucination.

• Data collection and selection:
  • It is important to collect data from reliable sources and select data that is free of bias and errors. For example, collecting data from reliable sources such as Wikipedia and government agency websites can improve the quality of the training data for AI models.
• Data cleaning and preprocessing:
  • The collected data may contain noise or errors. This data needs to be cleaned to make it suitable for analysis. For example, in the case of text data, typos and inappropriate expressions need to be corrected and duplicate data needs to be removed.
• Data Augmentation:
  • A technique called data augmentation is used to increase the amount of training data. Data augmentation is a technique that applies various transformations to existing data to create new data. For example, in the case of image data, the amount of data can be increased by applying transformations such as rotation, inversion, and enlargement/reduction.

Model Improvements

Hallucination can be suppressed by improving the structure and learning methods of the AI model.

• Countermeasures against overfitting:
  • Overfitting is a phenomenon in which an AI model becomes overly adapted to training data and is unable to respond well to unknown data. To prevent overfitting, techniques such as regularization and dropout are effective.
• Knowledge base enhancements:
  • By incorporating external knowledge into an AI model, it becomes possible to generate more accurate information. For example, by linking with a knowledge base such as Wikipedia, the AI can generate more reliable information.
• Model architecture improvements:
  • Hallucination can be suppressed by improving the structure of AI models. For example, introducing a self-attention mechanism into the Transformer model can improve its contextual understanding and generate more natural sentences.

Prompt Engineering

Prompt engineering is a technique for improving output results by devising instructions (prompts) given to an AI model.

• Clear and specific instructions:
  • It is important to be clear and specific about what you want the AI to do. Avoid vague or abstract language and include specific keywords and context to help the AI generate more accurate information.
• Explicit constraints:
  • By clearly specifying the output format, length, style, tone, and other requirements for the AI, you can more easily get the output you want.
• Few-shot learning, Chain-of-Thought prompting:
  • Few-shot learning is a technique that teaches an AI model to learn new tasks by showing a small number of examples. Chain-of-Thought prompting is a technique that improves the ability to solve complex problems by explicitly showing the thought process to the AI model. By utilizing these techniques, hallucination of AI can be suppressed.

Verifying the output results

Validating the output of generative AI is a critical step in preventing hallucination.

• Linkage with fact-checking tools: Information output by the generative AI can be automatically verified by linking it with fact-checking tools. This allows you to eliminate misinformation and unreliable information and use only reliable information.
  • Example: Google offers a tool called Fact Check Explorer that allows users to verify facts on the web.
• Human evaluation: It is also important for humans to check the output of the generative AI to see if it contains any factually incorrect information or inappropriate expressions. In particular, when making important decisions, it is desirable to have multiple experts evaluate the results.
  • Example: A company uses generative AI to generate articles that are then reviewed by human editors to fact-check and correct wording.
• Expressing uncertainty: Generative AI cannot always output the correct answer. If the output contains uncertainty, it is necessary to clearly state this to avoid misleading the user.
  • Example: If ChatGPT is unsure of an answer, it will display a warning such as “This answer may contain uncertain information.”

others

• Retrieval-Augmented Generation (RAG): RAG is a method in which a generative AI generates sentences by referencing an external knowledge base (e.g. Wikipedia). This allows the generative AI to output information that is not included in the training data, reducing the risk of hallucination.
  • Example: Facebook AI Research is using RAG to develop an open source chatbot called BlenderBot, which uses knowledge bases such as Wikipedia to make conversations more accurate and informative.
• Grounding: Grounding is a technique that increases the reliability of output results by linking text generated by generative AI with other modalities such as images and videos. For example, by generating text that explains the content of an image generated by an image generation AI, the risk of misinterpreting the image can be reduced.
  • Example: OpenAI’s CLIP (Contrastive Language-Image Pre-training) is a model that simultaneously trains on images and text, and can generate text that describes the content of an image.

Treating hallucination: a social approach

Countermeasures against hallucination require not only technical but also social approaches. Here, we explain social approaches from three perspectives: improving AI literacy, establishing laws and regulations, and international cooperation.

Improving AI literacy

AI literacy is the ability to understand the mechanisms, possibilities, and limitations of AI and to use it appropriately. Improving AI literacy is essential to recognize the risks of hallucination and to critically evaluate the output results of AI.

• AI education in educational institutions: It is important to teach basic knowledge and ethics regarding AI in school education. Not only children, but teachers and parents also need to acquire AI literacy.
  • Example: The Ministry of Education, Culture, Sports, Science and Technology has announced a policy to include content related to AI in the IT education curriculum from elementary school through high school.
• Information dissemination by the media: The media has a role to play in disseminating accurate information about AI and contributing to improving AI literacy. It is important to explain not only the benefits of AI but also its risks, such as hallucination, in an easy-to-understand manner.
  • Example: NHK broadcasts special programs on AI and provides information about AI on its website.
• AI ethics training for companies and organizations: Companies and organizations need to provide training on AI ethics to their employees and raise awareness on the ethical use of AI.
  • Example: A major IT company provides AI ethics training to all employees, providing opportunities for them to think about issues such as AI bias, discrimination, and privacy violations.

Legislation and Regulation

As the impact of the use of AI on society grows, it will become necessary to establish laws and regulations regarding AI.

• Mandatory labeling of AI-generated content: Content generated by AI must be labeled so that users can distinguish between AI-generated content and content created by humans.
  • Example: In the EU, a proposed AI regulation is under discussion that would require labelling of AI-generated content.
• Regulation of AI-related discrimination and privacy violations: In order to prevent AI-related discrimination and privacy violations, regulations are needed that establish clear rules and impose penalties for violations.
  • Example: In the United States, some states have passed laws banning discriminatory employment practices using AI.

International cooperation

Because AI is a technology that will be used across borders, international discussion and cooperation on AI ethics is essential.

• International discussion and collaboration on AI ethics: Governments and international organizations need to participate in the international discussion on AI ethics and develop common rules and principles.
  • Example: The OECD (Organization for Economic Cooperation and Development) has developed principles on AI and is encouraging member countries to adhere to them.
• Establishment of an AI governance framework: It is necessary to establish an international framework for the proper management of the development and use of AI, which will include not only ethical issues but also technical and security issues related to AI.
  • Example:
    • The United Nations is currently discussing restrictions on the military use of AI.
    • The World Economic Forum is facilitating a global dialogue on AI governance.

AI hallucination is not only a technical issue, but also a social issue. In parallel with technological development, it is essential for the healthy development of AI to deepen discussions on AI ethics throughout society and to establish appropriate rules and regulations.

Hallucination of generative AI: A case study

Hallucination in generative AI can occur in a variety of situations. Here, we introduce specific hallucination cases in ChatGPT and image generative AI. Through these cases, we can understand how hallucination manifests itself and what problems it causes.

Hallucination case of ChatGPT

While ChatGPT has advanced text generation capabilities, it also has a tendency to hallucinate. Below are some examples of hallucinations in ChatGPT.

• Citations of non-existent papers and books: ChatGPT may cite non-existent papers and books. This is likely due to the fact that the large amount of text data that ChatGPT has learned contains incorrect information or fictitious literature. For example, even if ChatGPT answers “According to the paper XX…”, it is necessary to check whether such a paper actually exists.
• Generation of Incorrect Historical Facts: ChatGPT may misinterpret historical facts or generate factually inaccurate information. This may be due to ChatGPT being influenced by misinformation or bias contained in the training data. It is important to confirm historical facts with reliable sources.
• Discriminatory and prejudiced expressions: ChatGPT may learn discriminatory and prejudiced expressions contained in the training data and generate sentences that reflect them. For example, making discriminatory remarks against a particular race or gender is ethically unacceptable. ChatGPT developers are taking measures to reduce such biases, but it is difficult to completely eliminate them.
• Generating information about non-existent people or organizations: ChatGPT may generate information about non-existent people or organizations, such as the names and achievements of non-existent scientists, or the names and products of non-existent companies. Please be aware that such information may be misleading.
• Numerical and calculation errors: ChatGPT can make mistakes even with simple calculation problems. This is because ChatGPT is a language model and is not specialized for numerical calculations. Be sure to check the numbers and calculation results output by ChatGPT with a calculator or similar.

Hallucination example of image generation AI

Image-generating AI can also cause hallucination. Below are some examples of hallucination in image-generating AI.

• Generating non-existent people and objects: Generative AI can generate faces of non-existent people or images of non-existent objects. These images can be so realistic that they can be indistinguishable from the real thing, posing a risk for misuse.
• Unnatural composite images: When image generation AI combines multiple images, it can sometimes produce unnatural aspects, such as distorted faces or unnatural blending of people and the background.
• Possibility of copyright infringement: If an image generated by image generation AI closely resembles an existing copyrighted work, it may be a copyright infringement. When using images generated by AI, it is necessary to be careful about copyright.
• Generation of ethically questionable images: Image generation AI may generate images that contain violent scenes or sexual content. Because such images are ethically questionable, developers and users of generative AI must take measures to avoid generating such images.

Summary: Understanding Hallucination and Using Generative AI Safely

AI hallucination is a new issue that has emerged with the advancement of AI technology. Hallucination can pose various risks, including misinformation, discrimination, and privacy violations.

However, hallucination shows the limitations of AI technology and is not a reason to reject AI. By understanding the causes and risks of hallucination and taking appropriate measures, we can use AI safely and effectively.

Effective technical measures include improving training data, improving models, prompt engineering, and verifying output results. Social measures include improving AI literacy, establishing laws and regulations, and international cooperation.

AI hallucination is one of the important issues that must be resolved as AI technology evolves. We need to face this issue seriously and build a better society that coexists with AI.