From the course: Hands-On AI: Build a RAG Model from Scratch with Open Source
Join today to access over 24,900 courses taught by industry experts.
Overview of the RAG pipeline
From the course: Hands-On AI: Build a RAG Model from Scratch with Open Source
Overview of the RAG pipeline
- [Instructor] Welcome back. We've now set up nearly all of the necessary pre-processing and functions to generate our full RAG model. So let's go ahead and recap what we have left. Now, one of the first steps is gonna be for us to write a pipeline that collates the relevant content given a particular query, all in one place. And we've written most of the functionality to achieve this, but we have yet to address content overlaps. So let's briefly mention how we'll go about that. Overlaps occur when two of the matches, or more than two of the matches in our database, are close enough to each other that their context windows overlap. So consider the following database with these Index IDs one through nine here on the left. And let's consider that we have a match to our query for Index four and Index eight. Now, if we give a context window running two indices before and two indices after for each of the matches, then the…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.