Embeddings Using Modern Models

Learning Objectives

By the end of this session, you will be able to:

• Understand how embeddings are generated in practice

• Learn about modern embedding models

• Compare popular embedding providers

• Understand embedding dimensions

• Generate embeddings using AI platforms

• Evaluate embedding quality

• Choose the right embedding model for your RAG system

Introduction

In the previous session, we learned that embeddings are numerical representations of meaning.

We discovered that:

Similar Meaning
      ?
Similar Vectors

This principle enables:

• Semantic search

• RAG systems

• Recommendation engines

• Knowledge assistants

However, an important question remains:

How are embeddings actually created?

Fortunately, developers do not manually create embeddings.

Modern AI providers offer specialized embedding models that convert text into vectors automatically.

Just as Large Language Models generate text, embedding models generate vector representations.

Understanding how these models work and how to use them is essential for building production-grade RAG applications.

Why This Topic Matters

Imagine building an enterprise knowledge assistant.

You have:

50,000 Documents

Before retrieval becomes possible:

Every Document
       ?
Embedding Generation
       ?
Vector Database

If embedding quality is poor:

• Retrieval quality decreases

• Relevant documents may be missed

• Answer accuracy suffers

Selecting the right embedding model is one of the most important decisions in a RAG project.

Recap: What Is an Embedding?

Example text:

Remote Work Policy

Embedding:

[0.21, 0.45, -0.18, ...]

The vector captures semantic meaning.

Later:

Work From Home Policy

generates a similar vector.

This allows the retrieval system to identify related content.

Embedding Generation Workflow

A simplified workflow:

Text
 ?
Embedding Model
 ?
Vector
 ?
Storage

The embedding model performs the transformation automatically.

What Is an Embedding Model?

An embedding model is a machine learning model specifically trained to represent meaning as vectors.

Input:

Text

Output:

Vector

Example:

Artificial Intelligence

becomes:

[0.33, 0.82, -0.11, ...]

The model learns relationships between concepts during training.

Difference Between LLMs and Embedding Models

Many beginners confuse the two.

Think of them as complementary technologies.

Typical RAG architecture:

Embedding Model
        ?
Retrieval

LLM
        ?
Answer Generation

Both are required.

Popular Embedding Providers

Today, several providers offer high-quality embedding models.

OpenAI

Widely used in enterprise RAG systems.

Advantages:

• Strong semantic performance

• Easy integration

• Reliable API support

Google

Provides embedding capabilities within its AI ecosystem.

Advantages:

• Good multilingual support

• Strong ecosystem integration

Cohere

Well known for retrieval-focused embeddings.

Advantages:

• Search optimization

• Enterprise retrieval features

Open-Source Models

Popular examples:

• BGE

• E5

• Instructor

• GTE

• Sentence Transformers

Advantages:

• Full control

• Local deployment

• No API dependency

Embedding Generation Example

Suppose we have:

Employees receive 24 annual leave days.

Workflow:

Text
 ?
Embedding Model
 ?
Vector
 ?
Store in Vector Database

Generated output:

[0.42, -0.31, 0.88, ...]

The actual vector may contain hundreds or thousands of dimensions.

Understanding Embedding Dimensions

Embedding dimensions represent the size of the vector.

Examples:

128
256
512
1024
1536
3072

Example:

4-Dimensional Vector

[0.12, 0.45, 0.78, 0.91]

1536-Dimensional Vector

[0.12, 0.45, 0.78, ...]

containing 1536 values.

Higher dimensions generally capture more information.

Are Larger Embeddings Always Better?

Not necessarily.

Advantages:

• Richer representation

• Better semantic understanding

Disadvantages:

• More storage

• Higher computation cost

• Larger vector databases

Choosing dimensions involves balancing:

Quality
vs
Cost

OpenAI Embedding Example

Workflow:

Document
 ?
OpenAI Embedding Model
 ?
1536-Dimensional Vector
 ?
Vector Database

Later:

User Query
 ?
Embedding
 ?
Similarity Search

This workflow powers many modern RAG applications.

Embedding Similarity Example

Document A:

Vacation Policy

Vector:

[0.41, 0.72, ...]

Document B:

Annual Leave Policy

Vector:

[0.43, 0.70, ...]

The vectors are close.

Similarity search identifies the relationship.

Document C:

Football Match Schedule

Vector:

[-0.33, 0.04, ...]

The vector is very different.

The retrieval system ranks it lower.

Similarity Scores

Retrieval systems calculate similarity scores.

Example:

Higher scores indicate stronger relevance.

This process is the foundation of semantic retrieval.

Embedding Models in RAG

Recall the RAG pipeline:

Documents
 ?
Chunking
 ?
Embeddings
 ?
Vector Database

Question
 ?
Embedding
 ?
Similarity Search
 ?
Retrieved Chunks
 ?
LLM
 ?
Answer

Embeddings serve as the bridge between:

Knowledge
and
Retrieval

Without embeddings, semantic search would not work.

Real-World Example: University Assistant

Documents:

Scholarship Policy
Admission Rules
Hostel Regulations

Student asks:

What financial assistance options are available?

Keyword search:

May Miss Scholarship Policy

Embedding search:

Financial Assistance
˜
Scholarship

Correct information is retrieved.

Real-World Example: HR Assistant

Employee asks:

Can I work from home?

Knowledge base contains:

Remote Work Policy

Embeddings recognize the semantic relationship.

Relevant information is retrieved.

Batch Embedding Generation

Organizations rarely process one document at a time.

Example:

10,000 Documents

Workflow:

Document Batch
 ?
Embedding Generation
 ?
Vector Storage

Batch processing improves efficiency.

Most production systems ingest data in batches.

Embedding Storage Considerations

Suppose:

1 Million Chunks

Each chunk generates:

1536-Dimensional Vector

Storage requirements become significant.

Organizations must consider:

• Database size

• Search performance

• Infrastructure costs

This is one reason vector databases are important.

Choosing an Embedding Model

Selection depends on several factors.

Retrieval Quality

How accurately does the model capture meaning?

Cost

API-based models incur usage costs.

Language Support

Does the model support multiple languages?

Deployment Requirements

Cloud or local deployment?

Performance

Latency and throughput requirements.

There is no universally best model.

The right choice depends on the project.

Open-Source vs Hosted Embeddings

Both approaches are widely used.

Evaluating Embedding Quality

How do we know whether embeddings are good?

Common evaluation methods:

Retrieval Accuracy

Can relevant documents be found?

Ranking Quality

Are the best results ranked first?

User Satisfaction

Do users receive useful answers?

Benchmark Testing

Compare models on standard datasets.

Evaluation is a critical part of production RAG systems.

Common Embedding Mistakes

Using Poor-Quality Data

Bad data creates bad embeddings.

Ignoring Chunking

Even excellent embeddings cannot fix poor chunking.

Choosing Models Based Only on Cost

Cheaper is not always better.

Not Evaluating Retrieval

Retrieval quality should be tested regularly.

Mixing Different Embedding Models

Consistency is important.

Documents and queries should typically use the same embedding model.

Enterprise Architecture Example

Documents
      ?
Chunking
      ?
Embedding Model
      ?
Vector Database
      ?
Retriever
      ?
LLM
      ?
Answer

This architecture powers many enterprise knowledge assistants today.

Future of Embeddings

Embedding technology continues to evolve.

Modern trends include:

• Multilingual embeddings

• Multimodal embeddings

• Domain-specific embeddings

• Smaller and faster models

• Improved retrieval performance

These advances continue to improve RAG systems.

.NET Perspective

Popular .NET technologies include:

• Azure OpenAI Embeddings

• Semantic Kernel

• Azure AI Search

Typical workflow:

Documents
 ?
Embeddings
 ?
Azure AI Search
 ?
Retrieval

Many enterprise .NET applications follow this architecture.

Python Perspective

Popular Python tools include:

• OpenAI SDK

• Sentence Transformers

• LangChain

• LlamaIndex

• Hugging Face Transformers

Python provides extensive support for embedding generation and experimentation.

Assignment

Practical Exercise

Research three embedding models:

• OpenAI

• Cohere

• BGE

Compare:

• Features

• Dimensions

• Use cases

• Advantages

Design Activity

Create a model-selection report for a university knowledge assistant.

Include:

• Embedding model choice

• Reasons for selection

• Expected benefits

• Potential limitations

Key Takeaways

• Embedding models convert text into vector representations.

• Embeddings enable semantic search and retrieval.

• LLMs generate text, while embedding models generate vectors.

• Modern RAG systems rely heavily on embedding models.

• Model selection impacts retrieval quality and system performance.

• Higher dimensions can improve representation but increase costs.

• Understanding embedding generation is essential for building production RAG systems.

What's Next?

In Session 21, we will explore:

Vector Similarity Search

You will learn how vector databases compare embeddings, how similarity scoring works, the mathematics behind retrieval, and how modern RAG systems find the most relevant information from millions of documents.