Tips for Managing AI Hallucinations in Customer Service

Don't be afraid of hallucinations! It's usually an early question in most talks I give on GenAI "But doesn't in hallucinate? How do you use a technology that makes things up?". It's a real issue, but it's a manageable one. 1. Decide what level of accuracy you really need in your GenAI application. For many applications it just needs to be better than a human, or good enough for a human first draft. It may not need to be perfect. 2. Control your inputs. If you do your "context engineering" well, you can point it to the data you want better. Well written prompts will also reduce the need for unwanted creativity! 3. Pick a "temperature". You can select a model setting that is more "creative" or one that sticks more narrowly to the facts. This adjusts the internal probabilities. The "higher temperature" results can often be more human-like and more interesting. 4. Cite your sources. RAG and other approaches allow you to be transparent about what the answers are based on, to give a degree of comfort to the user. 5. AI in the loop. You can build an AI "checker" to assess the quality of the output 6. Human in the loop. You aren't going to just rely on the AI checker of course! In the course of a few months we've seen concern around hallucinations go from a "show stopper" to a "technical parameter to be managed" for many business applications. It's by no means a fully solved problem, but we are highly encouraged by the pace of progress. #mckinseydigital #quantumblack #generativeai

❌  There is no denying it - AI makes mistakes and is sometimes way off the mark! Below are 5 ways leading technology players address this concern.    🔍 𝗧𝗛𝗘 𝗖𝗢𝗡𝗖𝗘𝗥𝗡    AI hallucinations and errors are not just myths; they're real and significant concerns, especially in finance and accounting.     The root of the issue is the nature of Generative AI and Large Language Models (LLMs)    As a form of automation built on top of statistical models, they are effective at dealing with ambiguity and unstructured data.     BUT this strength can also lead to less-than-accurate outcomes.     Contrast this with programmed automation that will always deliver the same result but cannot deal with variability outside of its programing.    💡 𝗧𝗛𝗘 𝗦𝗢𝗟𝗨𝗧𝗜𝗢𝗡    Leading AI software companies have crafted ingenious strategies that significantly mitigate these risks.     ➡️  𝗟𝗟𝗠 𝗙𝗶𝗻𝗲-𝗧𝘂𝗻𝗶𝗻𝗴 𝗮𝗻𝗱 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴  Beyond generic responses, models are tailored for specific finance and accounting needs. They evolve and learn delivering increasingly precise outcomes.    ➡️  𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗲𝗱 𝗣𝗿𝗼𝗺𝗽𝘁 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴  The (not so) secret sauce to getting high-quality outcomes lies in how we ask questions. AI technologies now simplify user input, transforming basic details into comprehensive prompts that yield exceptional results.    ➡️  𝗔𝗴𝗲𝗻𝘁𝗶𝘃𝗲 𝗔𝗜 𝗙𝗹𝗼𝘄  Imagine having a team of AI agents where one produces outcomes, and the others validate and refine (think a doer and reviewer process). This collaborative AI approach ensures the output's quality and relevance.    ➡️  𝗠𝘂𝗹𝘁𝗶-𝗠𝗼𝗱𝗲𝗹 𝗧𝗲𝗰𝗵𝗻𝗼𝗹𝗼𝗴𝘆  Diverse perspectives lead to accuracy. By leveraging multiple AI models for a single query, the consistency of responses significantly bolsters confidence in the results. Much like asking 2 experts the same question – a consistent response increases likelihood of accuracy.    ➡️  𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗮𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 (𝗥𝗔𝗚)  Combines the power of information retrieval with the generative capabilities of AI. AI retrieves relevant information from referenced databases and documents, providing context and creating more accurate and relevant outputs    🔑 𝗞𝗲𝘆 𝗧𝗮𝗸𝗲𝗮𝘄𝗮𝘆𝘀   Not all AI technologies are created equal.     Asking "How is your technology designed to improve the quality and relevance of outputs?" can be a game-changer in your tech evaluation process.    ❗ Remember, while AI can dramatically enhance decision-making accuracy, the buck stops with us. Ensuring effective management and control mechanisms are in place is non-negotiable. ❗    How do you ensure your AI technology delivers accurate outcomes? Share your strategies and insights in the comments below! 

Hallucination in LLMs refers to generating factually incorrect information. This is a critical issue because LLMs are increasingly used in areas where accurate information is vital, such as medical summaries, customer support, and legal advice. Errors in these applications can have significant consequences, underscoring the need to address hallucinations effectively. This paper (https://lnkd.in/ergsBcGP ) presents a comprehensive overview of the current research and methodologies addressing hallucination in LLMs. It categorizes over thirty-two different approaches, emphasizing the importance of Retrieval-Augmented Generation (RAG), Knowledge Retrieval, and other advanced techniques. These methods represent a structured approach to understanding and combating the issue of hallucination, which is critical in ensuring the reliability and accuracy of LLM outputs in various applications. Here are the three most effective and practical strategies that data scientists can implement currently: 1. Prompt Engineering: Adjusting prompts to provide specific context and expected outcomes, improving the accuracy of LLM responses. 2. Retrieval-Augmented Generation (RAG): Enhancing LLM responses by accessing external, authoritative knowledge bases, which helps in generating current, pertinent, and verifiable responses. 3. Supervised Fine-Tuning (SFT): Aligning LLMs with specific tasks using labeled data to increase the faithfulness of model outputs. This helps in better matching the model's output with input data or ground truth, reducing errors and hallucinations