NSIM

Stories you may like

• The growing trend of Abacus education in this era,gives an opportunity to train children abacus based mental arithmetic as well as grows their cognitive skills,

• Inviting SkillTraining centres all over India to affiliate with BSS and Become B'Voc centres of Singhania University

• Inviting Panchayat Wise Skill Gap Testing and Training Centres all over India for Nsim's Life Long Learning Program

• Be a seller in E-commerce Platforms...

• How to do online marketing through e-commerce websites?

AI Hallucinations

AI hallucinations occur when a model generates incorrect, fabricated or misleading outputs that are not grounded in the input or training data. This typically happens when the model incorrectly identifies patterns, leading to unreliable results.

The impact can range from minor factual errors in text-based models to unrealistic outputs in generative systems. For example, a chatbot may provide incorrect answers despite appearing confident.

Real-World Example of an AI Hallucination

• Misinformation and Fabrication: AI news bots generating real-time reports may include unverified or fabricated details, leading to the spread of misinformation during critical events
• Misdiagnosis in Healthcare: Models analyzing medical data, such as skin lesions, may misclassify conditions (e.g., benign as malignant), resulting in unnecessary treatments or missed diagnoses
• Algorithmic Bias: Recruitment systems trained on historical data may favor certain demographics, leading to unfair filtering of qualified candidates
• Unexpected Outputs: Systems like Microsoft Tay chatbot can produce offensive or harmful content due to biased training data, while image models may misidentify patterns (e.g., clouds as birds)

Causes of AI Hallucinations

Some of the reasons (or causes) why Artificial Intelligence (AI) models do so are:

• Quality dataset: AI models rely on the training data. Incorrect labelled training data (adversarial examples), noise, bias or errors will result in model-generating hallucinations.
• Outdated Data: The world is constantly changing. AI models trained on outdated data might miss crucial information or trends, leading to hallucinations when encountering new situations.
• Missing context in training (or test) data: Wrong or contradictory input may result in hallucinations. This is in users control to provide the right context in the input.

More often, we rely on the results generated by an AI model, considering they might be accurate ones. But AI models can generate convincing information which can be false. This happens mostly with LLMs trained on data with the above-defined issues. But how can we detect Hallucinations?

Impact of AI Hallucinations

AI hallucinations, where AI systems generate incorrect information presented as fact, pose significant dangers across various sectors. Here's a breakdown of the potential problems in the areas you mentioned:

• Medical Misdiagnosis: Incorrect analysis of medical data (such as reports or scans) can lead to missed or false diagnoses, resulting in delayed treatment, unnecessary procedures or even harmful medical decisions
• Faulty Financial Predictions: Inaccurate predictions and biased data can cause financial losses, market instability and unfair outcomes like incorrect loan denials or higher interest rates
• Algorithmic Bias and Discrimination: Biased outputs can lead to unfair hiring decisions, exclusion of qualified candidates and discriminatory outcomes across different demographic groups
• Legal and Law Enforcement Risks: Errors in systems like facial recognition or predictive policing can result in misidentification, wrongful arrests and unjust legal actions

Detection

Users can verify model outputs by cross-checking with reliable sources, but this approach is often time-consuming and not always practical.

In computer vision tasks, this becomes even more challenging. For example, images of Chihuahuas and muffins can appear visually similar, making it difficult for AI models to distinguish between them accurately. While humans can usually identify the difference using common sense, models may struggle due to limitations such as insufficient training data or incorrectly labeled examples.

Prevention

AI hallucinations cannot be completely eliminated, but they can be reduced with the right practices:

• Constrain the output: Clearly define the expected response format (e.g., yes/no, list or short answer) to limit ambiguity.
• Provide specific instructions: Mention exactly what type of information is required.
• Set boundaries: Clearly state what information should be excluded from the response.
• Verify outputs: Cross-check generated results with reliable sources when accuracy is critical.