Understanding the Limitations of LLMs

The "black box" nature of LLMs poses significant challenges for regulation and ensuring safety. Due to their opaque and complex internal workings, it is often not clear how these models arrive at specific answers or why they generate certain outputs. This lack of transparency complicates efforts to establish robust regulatory frameworks, as regulators find it difficult to assess compliance with ethical and legal standards, including privacy and fairness. Furthermore, without a clear understanding of how answers are generated, users may question the reliability and trustworthiness of the responses they receive. This uncertainty can deter wider adoption and reliance on LLMs. This study (https://lnkd.in/efjmvwiw) aims to address some of these issues by introducing CausalBench which is designed to address the limitations of existing causal evaluation methods by enhancing the complexity and diversity of the data, tasks, and prompt formats used in the assessments. The purpose of CausalBench is to test and understand the limits of LLMs in identifying and reasoning about causality particularly how well they can perform under conditions that mimic real-world examples. Using CausalBench, the authors then evaluated 19 leading LLMs on their capability to discern direct and indirect correlations, construct causal skeletons, and identify explicit causality from structured and unstructured data. Here are the key takeaways: • 𝗦𝗲𝗻𝘀𝗶𝘁𝗶𝘃𝗶𝘁𝘆 𝘁𝗼 𝗗𝗮𝘁𝗮𝘀𝗲𝘁 𝗦𝗰𝗮𝗹𝗲: LLMs are capable of recognizing direct correlations in smaller datasets, but their performance declines with larger, more complex datasets, particularly in detecting indirect correlations. This indicates a need for models trained on larger and more complex network structures. • 𝗣𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗘𝗱𝗴𝗲 𝗼𝗳 𝗖𝗹𝗼𝘀𝗲𝗱-𝘀𝗼𝘂𝗿𝗰𝗲 𝗟𝗟𝗠𝘀: Closed-source LLMs like GPT3.5-Turbo and GPT4 outperform open-source models in causality-related tasks, suggesting that the extensive training data and diverse datasets used for these models enhance their ability to handle complex causal queries. • 𝗜𝗺𝗽𝗮𝗰𝘁 𝗼𝗳 𝗣𝗿𝗼𝗺𝗽𝘁 𝗗𝗲𝘀𝗶𝗴𝗻: The effectiveness of LLMs varies with different prompt formats, with combinations of variable names with structured data or background knowledge proving particularly beneficial. The development of comprehensive benchmarks like CausalBench is pivotal in demystifying the "black box" nature of LLMs. This enhanced transparency aids in complex reasoning tasks, guiding the selection of appropriate models for specific applications based on empirical performance data. Additionally, a more granular understanding of LLM capabilities and behaviors facilitates more effective regulation and risk management, addressing both ethical and practical concerns in deploying these models in sensitive or high-stakes environments.

Most people assume large language models are like search engines or knowledge bases. They’re not. LLMs are stochastic text generators. That means: • They don’t store facts. • They don’t understand meaning. • They don’t retrieve answers from a database. Instead, they predict the most likely next word, one token at a time, based on the patterns they’ve seen in massive text datasets. This process is inherently probabilistic. The model doesn’t always give the same output. You can actually set a parameter called temperature to make it more or less “random.” Lower temperature = more deterministic. Higher temperature = more creative or chaotic. So when an LLM gives you: • A brilliant summary of a legal document • A wrong answer to a basic math question • A hallucinated source that doesn’t exist …it’s not being lazy. It’s doing exactly what it was trained to do: generate fluent, likely-sounding language. This doesn’t make LLMs useless. It just means we need to treat them as stochastic tools, not deterministic ones. And that’s why smart builders wrap LLMs with: • Prompting patterns (like chain-of-thought reasoning) • Retrieval (so the model can pull in factual context) • Post-processing (to catch or correct hallucinations) LLMs aren’t broken. They’re just uncertain by design Follow me for more clear, no-hype explanations of how this space is evolving. #LLMs #AIExplained #PromptEngineering #GenerativeAI #NLP #LanguageModels #AppliedAI

#LLM #GENAI #LAW: #Hallucinate, much? Much anecdotal evidence supports that thesis in #GenerativeAI's clumsy foray into law, but research at Stanford University Human-Centered #ArtificialIntellience delivers hard data. Here are just a couple of findings from Stanford's recent study: - "[I]n answering queries about a court’s core ruling (or holding), models hallucinate at least 75% of the time. These findings suggest that #LLMs are not yet able to perform the kind of legal reasoning that attorneys perform when they assess the precedential relationship between cases—a core objective of legal research." - "Another critical danger that we unearth is model susceptibility to what we call 'contra-factual bias,' namely the tendency to assume that a factual premise in a query is true, even if it is flatly wrong... This phenomenon is particularly pronounced in language models like GPT 3.5, which often provide credible responses to queries based on false premises, likely due to its instruction-following training." Read the full article here: https://lnkd.in/gEab43qK

When a person asks a question of an LLM, the LLM responds. But there’s a good chance of an some error in the answer. Depending on the model or the question, it could be a 10% chance or 20% or much higher. The inaccuracy could be a hallucination (a fabricated answer) or a wrong answer or a partially correct answer. So a person can enter in many different types of questions & receive many different types of answers, some of which are correct & some of which are not. In this chart, the arrow out of the LLM represents a correct answer. Askew arrows represent errors. Today, when we use LLMs, most of the time a human checks the output after every step. But startups are pushing the limits of these models by asking them to chain work. Imagine I ask an LLM-chain to make a presentation about the best cars to buy for a family of 5 people. First, I ask for a list of those cars, then I ask for a slide on the cost, another on fuel economy, yet another on color selection. The AI must plan what to do at each step. It starts with finding the car names. Then it searches the web, or its memory, for the data necessary, then it creates each slide. As AI chains these calls together the universe of potential outcomes explodes. If at the first step, the LLM errs : it finds 4 cars that exist, 1 car that is hallucinated, & a boat, then the remaining effort is wasted. The error compounds from the first step & the deck is useless. As we build more complex workloads, managing errors will become a critical part of building products. Design patterns for this are early. I imagine it this way : (third chart) At the end of every step, another model validates the output of the AI. Perhaps this is a classical ML classifier that checks the output of the LLM. It could also be an adversarial network (a GAN) that tries to find errors in the output. The effectiveness of the overall chained AI system will be dependent on minimizing the error rate at each step. Otherwise, AI systems will make a series of unfortunate decisions & its work won’t be very useful.