From the course: Making Your AI Results More Predictable

Unlock this course with a free trial

Join today to access over 24,900 courses taught by industry experts.

Semantic similarity search

Semantic similarity search

From the course: Making Your AI Results More Predictable

Start my 1-month free trial Buy for my team

Semantic similarity search

“

- [Instructor] Giving a language model some context can really improve the accuracy and predictability of output. How would we go about finding that context to give to a model? For this, we can rely on semantic similarity or semantic search. Semantic similarity is similarity based on meaning. You can imagine language or words in a multidimensional space where you can find out what's similar to what. You can then leverage a vector representation of language in order to figure out what's the relevant context you're giving to a model. Technologies that leverage semantic search will usually have a language model for embeddings as well as a vector database to help perform this similarity search. Now, for us, it's easier to kind of understand a three-dimensional space and how things can be close and clumped together in this three-dimensional space. But models can have thousands of dimensions in a space that represents meanings of texts. OpenAI is not the only company that offers tools for…

Contents