From the course: AI Toolkit Essentials for Visual Studio Code

What is AI Toolkit for VS Code? - Visual Studio Code Tutorial

From the course: AI Toolkit Essentials for Visual Studio Code

Start my 1-month free trial Buy for my team

What is AI Toolkit for VS Code?

“

- [Instructor] AI Toolkit for Visual Studio Code is an extension to help create AI-enabled applications. It is targeted to both developers and AI engineers. So, this implies that a somewhat limited understanding of AI is needed to work with it. My experience of the tool needs to be to believe that it isn't targeting all developers, instead, ones that have crossed over into the AI world and understand, at very least, the basics of the technology. Like VS Code itself, AI Toolkit works on a couple of different platforms. Right now it is primarily Windows and Linux. However, Microsoft is working on the extension for macOS, though not everything is currently working in it. There is local work being done with the models and some sort of processor is needed. The AI Toolkit uses different solutions depending on the platform. For Windows, it can use the DirectML abstraction layer, the device's CPU, or for Copilot+ machines, the NPU. On Linux, we can use an NVIDIA processor or the CPU. Finally, when we run models locally, the AI Toolkit supports the Open Network Neural Exchange format. The ONNX format is an open cross-platform standard for representing machine learning models. There are several different capabilities of the AI Toolkit. One of the most interesting is the model catalog that allows us to browse and use models from many different sources, including ones hosted in GitHub, from various publishers, once we've downloaded, even ones we have imported into the system. With the model catalog, we should be able to find an LLM to support what we are trying to do. In order for us to import our own models into the model catalog, we have to have it hosted or deployed somewhere it is accessible from the internet. When we do find a model we want, we can use it in AI Toolkit's playground. The playground allows us to try the model with different prompts and parameter settings. We can also use multimodal models in the playground with attachment support. Multimodal models support a variety of different input formats, including text, voice, images, or even video. Attachment support allows us to attach files as part of the prompt for information or evaluation about their content. AI Toolkit also allows us to run many different prompts in a batch. This is useful for testing, as we don't have to enter them one prompt at a time and evaluate the results. Instead, AI Toolkit can consume multiple prompts as a dataset, and the dataset will then be updated with the results. We can even then export the dataset for evaluation of those results. The ability to use a dataset to run multiple prompts and then have it updated with the results can help us compare different models side by side or see how changes in our models parameters have impacted the results. Finally, AI Toolkit can help us fine-tune our models. When we are ready, we will need to deploy them as well. While deployment functionality isn't something really built into AI Toolkit, it is something crucial to getting our AI-enabled apps running.

Contents