Embedding

An Embedding is a dense vector representation of data (mostly words, sentences, or other discrete objects) in a continuous, low-dimensional space that captures semantic relationships and similarities. Unlike One-Hot-Encoding, which creates sparse, high-dimensional vectors, embeddings are compact, real-valued vectors trained through Machine Learning methods. Word Embeddings like Word2Vec, GloVe, or modern Transformer-based approaches arrange words in vector space so that similar words lie close together. Famous example: Vector('King') - Vector('Man') + Vector('Woman') ≈ Vector('Queen'). Embeddings enable neural networks to understand semantic meanings and are the foundation of modern NLP systems, from search engines to Large Language Models. They also work for other data types like images, documents, or user profiles.