Embeddings (machine learning)

An embedding is a numerical representation of something—a word, image, user, or concept—as a point in high-dimensional space. Rather than storing raw data, embeddings compress meaningful patterns into dense vectors of numbers, making it possible for Machine learning models to process and compare abstract entities mathematically.

The magic lies in proximity: similar things end up near each other in embedding space. A model trained on text might place the word "king" close to "queen" and far from "toaster." This geometry enables Neural networks to reason about meaning, similarity, and relationships without explicit programming.

Embeddings emerged from Natural language processing work (see Word embeddings), but the concept generalizes far beyond language. They're now fundamental to Recommendation systems, Computer vision, and Knowledge graphs. The Transformer architecture's success partly stems from learning rich, contextual embeddings.

Creating embeddings typically requires training on large datasets—a Neuron-inspired process where models learn to position entities such that downstream tasks (prediction, classification, clustering) become easier. Pre-trained embeddings can then transfer knowledge across different problems, making them a cornerstone of modern Deep learning.

Related

Vector spaces, Representation learning, Dimensionality reduction, Similarity search, Transfer learning