From the course: Build with AI: SQL Agents with Large Language Models
Join today to access over 24,900 courses taught by industry experts.
Working with LLMs locally
From the course: Build with AI: SQL Agents with Large Language Models
Working with LLMs locally
- [Instructor] In the previous video, we looked at how to interact with LLM providers like OpenAI and Google Gemini using their APIs. In this chapter, we'll explore an alternative running models locally with a Docker Model runner. We'll be working inside this Jupyter notebook called 02_05, and you can find it under the chapter_2 folder. So what exactly is Docker Model Runner? Docker Model Runner, or DMR in short, is a feature of Docker Desktop that lets you run LLMs locally using your own resources. The following diagram illustrates how DMR works on your local machine. Think of it as a lightweight server running on your computer. It can serve models to both local virtual environment and containers. The nice thing about the DMR is that you do not need to have prior knowledge of docker and containers to use it. It can fit well to your workflow regardless of whether you use containers or not. Here how it works,…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.