From the course: Machine Learning and AI Foundations: Producing Explainable AI (XAI) and Interpretable Machine Learning Solutions

Understanding the what and why your models predict - KNIME Tutorial

From the course: Machine Learning and AI Foundations: Producing Explainable AI (XAI) and Interpretable Machine Learning Solutions

Start my 1-month free trial Buy for my team

Understanding the what and why your models predict

“

- In an article in "Harvard Data Science Review", "Should We Trust Algorithms?", the author shares a slightly humorous yet powerful story of what issues can arise with computer predictions. When negotiating the narrow streets in the ancient university town of Coimbra, the GPS confidently told us to go left, and so we obeyed her, but we were somewhat taken aback, when the road abruptly turned into steps. We stopped in time, but after that, we were not so trusting. Since it didn't end in disaster, I think we can appreciate the everyday reality of AI models making mistakes, and can probably think of humorous times that our smart devices at home misunderstood what music we were requesting or what word we were pronouncing. I share this story because it raises three issues that we will be revisiting again and again. Is the model transparent? Is the reason for a specific individual prediction transparent? Is it high stakes? XAI or explainable AI is a set of techniques to explain a model's prediction even when the nature of the modeling algorithm would normally make this difficult. XAI is the primary subject of this course. Why would explanations be so difficult? Commonly, it's the very nature of the technique. For instance, deep neural networks are very complex models that are difficult to interpret. But even traditional neural networks, back when they had just one hidden layer, were already difficult to interpret. Other popular techniques like XGBoost or support vector machines would also fall into this category. In the case of the nearly averted car accident, the prediction itself was clear, "take a left," but why it made an error was not so clear. Whether it is high stakes, is somewhat subjective, but they narrowly avoided an accident. It wasn't just that they got caught up in traffic, so I think most would consider it high stakes. When we are trusting AI and machine learning to make decisions for us, these are all important issues. If the prediction being made is whether or not to buy a pair of shoes or adopt a particular clothing style, we want accuracy, but we're probably willing to sacrifice transparency. We are also probably fairly patient with search engine results. Maybe the first choice isn't that great, but as long as what we're searching for is on the first page, we likely move along without much of a thought. Even if we're curious, we don't urgently need to know why the top choice was not optimal. If, on the other hand, the problem is high stakes, like getting a favorable rate on a loan, or perhaps even being turned down for a loan, we very much want transparency. Frankly, if our business operates in a regulatory environment like insurance or banking, we are likely forced to have transparency. There will be government agencies that require justification for different prices or justification for decisions like denying an insurance claim. This course is about these issues, specifically when we do need transparency into why a model made a particular prediction, how do we go about providing that transparency? The techniques for revealing the inner workings of predictive models, is a complex and evolving world as AI and machine learning techniques get increasingly diverse and complex. While XAI will be our primary focus, we will explore two distinct approaches and world views. First, XAI, where so called black box models are built yet additional steps are taken to explain them, but also the other approach, interpretable machine learning, where you choose to build a transparent model from the start. We will be exploring both, and rehearsing how to perform each approach step by step.

Contents