TinkerThe map appears once a question reaches the index.
The diamond is “How much does a Kestrel K2 scanner cost per unit when ordering 50 or more?”.
Chapter 6 of 7
Meaning by neighbours
Each Larkspur document piece and each customer question now has an embedding. To answer a question, the assistant needs the pieces whose meaning is closest.
This chapter: How do you measure how close two meanings are, and find the nearest ones?
You will learn
- Cosine similarity: how close two embeddings point
- Nearest neighbours: the pieces closest to a question
- Why close in meaning is not the same as sharing words
Words you’ll meet (2)
- Cosine similarity
- A score for how alike two vectors point, from -1 to 1; 1 means the same direction.
- Nearest neighbours
- The vectors with the highest similarity to a given one.
The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.
Step 1 of 3
Every Larkspur piece and every question has an embedding. To compare two, take the cosine similarity: a score of how alike the two vectors point, from -1 to 1. A score of 1 means the same direction, the same meaning. The pieces with the highest scores are the question’s nearest neighbours, and they are what RAG’s search hands the model. A customer asks: “How much does a Kestrel K2 scanner cost per unit when ordering 50 or more?” Before you search: