Embeddings and Vectors

Introduction to Word Embeddings and Vectors

What's a Word Embedding?: Imagine trying to describe the vibe of your favorite song using just a few numbers. That's kind of what we're doing with word embeddings, but for words. We're turning words into lists of numbers (vectors) that capture their meaning.
Why Bother?: Computers don't get words like we do. They love numbers. So, if we want to teach a machine about words and their meanings, we need to translate those words into a language computers understand: numbers.

How LLMs Represent Words as High-Dimensional Vectors

High-Dimensional What Now?: Okay, so when we say "high-dimensional vectors", we mean lists of numbers that are super long. Like, instead of describing a word with 3 or 4 numbers, we might use hundreds or even thousands.
The LLM Magic: LLMs, like GPT, don't just assign random numbers to words. They analyze tons of text to figure out which words are used in similar contexts. Words that often appear in similar situations get similar number lists. So, "king" and "queen" might have number lists that look more alike than, say, "king" and "banana".

Importance of Embeddings in Semantic Search

Beyond Simple Search: Remember when you'd type something into a search bar, and it'd match your words exactly? That's old school. With embeddings, search engines get the vibe of what you're looking for, not just the exact words.
Capturing Nuance: Let's say you search for "big cat". Instead of just showing results with those exact words, a semantic search might understand you're interested in lions, tigers, and cheetahs, and show you info on those too.

Screenshot 2023-10-23 at 5.01.21 PM.png

Demonstrating the Power of Embeddings with Simple Examples

Word Relationships: One cool thing you can do with embeddings is figuring out word relationships. If you take the vector for "king", subtract the vector for "man", and then add the vector for "woman", you often get a vector super close to "queen". Mind blown, right?
Finding Similarities: Let's say you have a list of movies and their descriptions in vector form. You loved "Inception" and want something similar. By comparing the vector for "Inception" to other movie vectors, you can find movies with a similar vibe.