Describe model management and deployment capabilities in Azure Machine Learning - Azure AI Services Tutorial

- [Tim] Again, I'm happy that we've at least seen this briefly in the previous lesson that Azure Machine Learning is an end-to-end lifecycle management solution where you can build your own ML model, you can refine it and publish it and monitor it all from within a single interface. There's a model registry where over time you've got a full audit trail of all of your experimentations and deployments. These are traditional REST APIs that use the traditional API tokens. Standard security rules apply for that as well as, ready? Another benefit of being in the Microsoft Cloud, bearer tokens and the ability to use Entra ID and other identity providers in a seamless way. Full Azure Monitor performance, you've got application insights. You need to have a conversational familiarity with these products by name. If you decide to go into the associate or expert level, then that's a different ball game. Our example here is Wingtip Toys, deploying a recommendation AI as an API for its e-commerce site. Exactly, so if we were at a whiteboard this solution schematically in Azure Machine Learning, it's hosting the web endpoint, and then here our e-commerce site may be in another cloud. It might even be on premises, but whenever it needs to do a recommendation lookup, it's got that API channel into the Azure Machine Learning Service. Without getting too geeky into the networking details, what's beautiful here is that we're talking about TCP 443 traffic, which is a firewall friendly port assignment, and that also ensures that we're using TLS encryption and making sure that our data in transit is fully protected. For our case study, Northwind Streamlines Ops with Azure Machine Learning. How do they do this? Well, they're using Azure Databricks notebooks and pipelines to automate data cleaning in compliance with their source data with responsible AI principles. They've created a pipeline that runs 24/7. They started with AutoML and then they moved over to their own pipeline for ongoing model training. They use Azure hosted Compute with Autoscale so that if they have a promotion and a big traffic spike, everybody still gets the same predictable service level. And they've got deployment choices. There are so many different ways to host these models. You can package and export it. Let me say that as a final point with the content here. You're not air gapped in a walled garden. You can take those models, package 'em, create a pickle file, whatever, and put 'em in another cloud. You are not trapped in Azure. For our AI 900 certification exam alert, I simply want to remind you of the class repository. My short link is go.techtrainertim.com/ai900, and in that repo you'll find lots and lots of whole bunches of supportive resources for the course and for the cert, including the demo files that I use in my demos. Let's do the demo now. I've cloned the course files down into VS Code on my local computer, and if I go under lessons, lesson four, you're going to see a bad data CSV and a good data CSV. And I've got a couple explainer files also that I encourage you to look at. And again, I just, I'm doing a survey here, we're not getting into the deep, deep weeds, but good data speaking as generically as I can is a data set that's complete. First of all, you'll notice here that I've got defined properties for every column, and if I scroll through, there's not a huge number of rows here, but it's a complete data set. I'm not seeing anything with my eyeball that stands out. This is something that it's very good to run by a generative AI. Do you see anything in this data that's problematic that we need to clean up? Okay, I also have an anti-pattern bad data file here that's got problems with different standards, with your column names, with CSV, you don't have any data type designation. When you switch this into a structured view, you're going to need to think about a data type to classify with each of these columns, you know, but look here, look at these gaps in the data. By definition, that's problematic because if the AI has one record that it's evaluating, that's a hundred percent complete and another one that's only 70% complete, that's not fair right out of the gate. Now, the good news is that Microsoft has lots and lots of tools, right? In Azure to automate all of that data cleaning, believe it or not, but for free, you also can download Power BI Desktop and load in, for example, that CSV file. Now, Power BI is another subject. It's a free download, but it is Windows only and it's old, so I can't even Zoom the interface. I apologize for the interface being so tiny. I just want to show you this that I've loaded in bad data and you'll want to look for ways to help. So here I can open on the homepage, I can say remove columns, I can do remove rows, and I can easily remove duplicates, remove blank rows. You can do all of this manually. And then what's so cool about Power BI is that you do your data modeling and your cleaning, and then you come back to the designer interface here, which you can see here, and you can start to build out your dashboards and your reports and so forth. All right, I'm not going to say anymore about Power BI. That was just a incidental side point. Yeah. I want to take us back into Azure Machine Learning Studio because I've gone further with our AutoML project. You'll recall that my bank marketing project, I ran AutoML, it found the voting ensemble to be the most effective algorithm to use, and we're predicting whether a new customer would say yes or no to a given promotion, all right? Now, what I did behind the scenes is that I went through the deploy wizard and I created a real time endpoint, didn't have to know much at all about Azure to do. It's pretty amazing. And then we can browse those endpoints. Not only I myself can browse these endpoints, but another point I want to call out is that Azure Machine Learning Studio is a multi-user collaborative interface. Particularly I'd encourage you to take a look at Jupyter Notebooks. It's a collaborative web format that's natively integrated into Machine Learning Studio. Just wanted to say that so your colleagues can come in here and test the endpoint just like you can. Now consume would mean that we're going to actually use Curl or a supported SDK down here to make the request. And this is just all helper code. You want to be really, really careful with your access keys. These are interchangeable and you want to definitely regenerate regularly. There's ways to automate that using Azure Key Vault. But if I go to test, what's neat is that it allows you to send a sample data set into the endpoint and give you a real response. It's a good way to smoke test. And so in this case, we can choose, I only have the one deployment that I've done, but remember I said that AML allows you to version everything. So if you publish a new version of an endpoint and it's bugged, you can easily roll back. That's definitely on the menu. Now, the input here, these are the properties, the parameters that are in the source data, but this is empty. I'm going to need to create a data array here. This is just as a placeholder. So let me do a Control + A and a Control + V, and I'm putting in one sample row of data. Why is it in JavaScript object notation? Well, because the lingua franca here, the way that all of the Azure AI services work, the way all the gen AIs that I know of work, it's restful Https traffic where the data payload is in JSON. So even if you're not quote unquote "a real program," or whatever that means, I would encourage you to lean into JSON, even if you're a business professional. By understanding this key value format, it's going to go a long way toward building your skills. So let's see here. Here in my data basket, we've got 35 admins. So these are answers to all those questions, right? Let's do a live test against the endpoint. In this case, the user is predicted to say no, and it comes back just as plain JSON. You might think, well, whoa, that's not very descriptive. You can use your Java shredding libraries or your JavaScript shredding libraries to format that output to your heart's content. Lastly, I never want to forget logging and monitoring and debugging. You'll find that across, not just Machine Learning Studio, but really across all of the Azure products. The software engineers work closely with customers to figure out what are the most important metrics and log searches that our customers need to see. Here, this is showing us requests for a minute. In other words, how busy is this endpoint right now in overtime? And then also, what is the latency or the delay that my users are seeing? And we can adjust the time grain, as you can see, and then logs allows you to do if you're instrumenting Machine Learning, allows you to go deeper and look at the raw data, specifically event by event, every single request that's happening. As you can see. Let me tie this all in a bow by going back to the homepage. AML is a great platform for everybody, whether you're a full-time AI ML engineer or not. It was built for multi-user collaboration and spend time on the home and learning pages to take advantage of all of the learning and education that Microsoft seamlessly it layers into their services.