How to Build Semantic Search in Go with Embeddings and Vector Retrieval

Traditional search systems usually look for matching words or phrases. This works well when users know exactly what to search for, but it can produce poor results when they describe a problem using different words from those stored in the documents.

For example, a developer might search for “how to recover an application after a server crash.” A document titled “Implementing fault recovery in distributed services” may contain the right answer, even though the two texts share few exact words.

Semantic search addresses this problem by comparing the meaning of text rather than relying only on keyword matches. It uses embeddings, which represent text as numerical vectors, and retrieves documents whose vectors are close to the vector of the user's query.

Go is a practical choice for building this type of service because it provides straightforward concurrency, efficient HTTP handling, and a simple way to integrate external embedding APIs and vector databases.

In this article, we will build a small semantic search engine in Go, implement vector similarity ranking, and examine how to extend the design for production applications.

What Is Semantic Search?

Semantic search retrieves information based on the meaning of a query. Instead of searching only for matching terms, it represents the query and documents as vectors generated by an embedding model.

An embedding model converts text into a list of floating-point numbers. Texts with related meanings may produce vectors that are close together in the embedding space.

For example, an application might store these documents:

How to restore a database after a failure.
Configuring authentication in a Go web service.
Recovering a distributed application after a server outage.

A query about recovering an application after a failure should rank the first and third documents above the authentication document, provided the embedding model represents their meanings appropriately.

Embeddings do not guarantee that every relevant document will be found. Results depend on the embedding model, document quality, chunking strategy, similarity metric, and retrieval configuration.

Understand the Semantic Search Pipeline

A semantic search system typically has two workflows: indexing documents and retrieving results.

During indexing, the application cleans and prepares documents, divides long content into manageable sections, generates embeddings, and stores the vectors alongside document metadata.

During retrieval, the application embeds the user's query, searches for nearby vectors, ranks the matching documents, and returns useful results.

The overall workflow looks like this:

Document Collection
        |
        v
Text Preparation and Chunking
        |
        v
Embedding Model
        |
        v
Vector Storage and Indexing
        |
        v
User Query
        |
        v
Query Embedding
        |
        v
Vector Similarity Search
        |
        v
Ranked Search Results

The same embedding model and compatible configuration should normally be used for documents and queries. If the model changes, existing document vectors may need to be regenerated before they can be compared meaningfully with new query vectors.

Choose an Embedding Model

An embedding model determines how text is represented in the vector space. Some models are designed for general text retrieval, while others target specific languages, domains, or multilingual workloads.

Before selecting a model, evaluate:

Whether it supports the languages in your documents and queries.
The number of dimensions in its output vectors.
The maximum input length it accepts.
The quality of retrieval on your actual dataset.
Its latency, operating cost, and deployment requirements.

The model's vector dimension matters because all vectors stored in the same vector index must follow the expected dimension. A model that produces 768-dimensional vectors cannot be mixed directly with a model that produces 1,024-dimensional vectors in the same fixed-dimension index.

The following Go interface keeps the application independent of a particular embedding provider.

package search

import "context"

type Embedder interface {
    Embed(
        ctx context.Context,
        text string,
    ) ([]float32, error)
}

A provider-specific implementation can call a hosted embedding API or a locally running embedding model. The interface also makes testing easier because a fake embedder can be supplied without making network requests.

Create a Document Model in Go

Each indexed document needs an identifier, its original text, an embedding vector, and any metadata required by the application.

package search

type Document struct {
    ID       string
    Title    string
    Content  string
    Category string
    Vector   []float32
}

The ID identifies the document or document chunk. The title and content provide the information that will be returned to the user, while the category supports filtering.

The Vector field stores the embedding generated from the content.

For a small demonstration, these records can be held in memory. In a production service, use durable storage so the application does not have to recreate every embedding whenever it restarts.

For long documents, store separate chunks rather than embedding an entire book or manual as one vector. A chunk should contain enough context to make sense independently while remaining within the embedding model's input limits.

Implement Vector Similarity

Once the application has embeddings for documents and a query, it needs a way to measure similarity.

Cosine similarity is a common choice. It compares the angle between two vectors and is calculated as:

[
\text{cosine}(A,B)=
\frac{A \cdot B}{|A||B|}
]

Here, (A \cdot B) is the dot product, while the denominator is the product of the vectors' magnitudes.

For nonzero vectors, the result ranges from -1 to 1. Higher values indicate greater directional similarity, although the meaning of a particular score depends on the embedding model and data.

The following implementation checks that the vectors have matching dimensions and rejects zero-length vectors.

package search

import (
    "errors"
    "math"
)

func CosineSimilarity(
    a, b []float32,
) (float64, error) {
    if len(a) == 0 || len(a) != len(b) {
        return 0, errors.New(
            "vectors must have matching nonzero dimensions",
        )
    }

    var dot float64
    var normA float64
    var normB float64

    for i := range a {
        x := float64(a[i])
        y := float64(b[i])

        dot += x * y
        normA += x * x
        normB += y * y
    }

    if normA == 0 || normB == 0 {
        return 0, errors.New(
            "cosine similarity is undefined for zero vectors",
        )
    }

    return dot / (math.Sqrt(normA) * math.Sqrt(normB)), nil
}

Using float64 for the intermediate calculations reduces rounding error in the similarity calculation, even though the stored vectors use float32.

This implementation compares two vectors directly. It is suitable for a small search collection, but it does not provide an efficient way to search millions of documents.

Build a Simple Retrieval Function

The next step is to embed the query, compare it with stored document vectors, and return the highest-ranked results.

For this demonstration, we will use a linear scan. The function calculates a similarity score for every document and sorts the results from highest to lowest.

package search

import (
    "context"
    "errors"
    "sort"
)

type Result struct {
    Document   Document
    Similarity float64
}

func Search(
    ctx context.Context,
    embedder Embedder,
    documents []Document,
    query string,
    limit int,
) ([]Result, error) {
    if limit <= 0 {
        return nil, errors.New(
            "limit must be greater than zero",
        )
    }

    queryVector, err := embedder.Embed(ctx, query)
    if err != nil {
        return nil, err
    }

    results := make([]Result, 0, len(documents))

    for _, doc := range documents {
        if err := ctx.Err(); err != nil {
            return nil, err
        }

        score, err := CosineSimilarity(
            queryVector,
            doc.Vector,
        )
        if err != nil {
            return nil, err
        }

        results = append(results, Result{
            Document:   doc,
            Similarity: score,
        })
    }

    sort.Slice(results, func(i, j int) bool {
        return results[i].Similarity >
            results[j].Similarity
    })

    if limit < len(results) {
        results = results[:limit]
    }

    return results, nil
}

The function accepts a context so cancellation can stop the scan between documents. It also returns errors from the embedding provider rather than silently producing incomplete results.

This example assumes the stored document vectors have already been generated and validated. The embedding model must produce vectors compatible with those stored values.

For an empty document collection, the function returns an empty result list. In a real application, you may also want to reject blank queries, filter documents by access permissions, and exclude results below a validated relevance threshold.

The search function performs a linear scan followed by sorting. Its approximate complexity is (O(ND + N\log N)), where (N) is the number of documents and (D) is the vector dimension. This is acceptable for small collections, but it becomes expensive as the collection grows.

Store Vectors in a Vector Database

For a larger dataset, use a vector database or a database extension that supports vector indexing.

Common choices include PostgreSQL with pgvector, Qdrant, Milvus, and other vector retrieval systems. These tools can store vectors alongside document identifiers and metadata, and many support approximate nearest-neighbor indexes.

Storage approach	Advantages	Limitations
In-memory Go slice	Simple to implement and test	Data disappears on restart; search requires scanning all vectors
PostgreSQL with pgvector	Combines relational data, metadata filtering, and vector search	Index configuration and query tuning require care
Dedicated vector database	Designed for vector retrieval and related operations	Adds another service to deploy and maintain

The right option depends on dataset size, filtering requirements, operational constraints, and query volume.

For example, a documentation platform may use PostgreSQL to store document metadata and pgvector to retrieve relevant chunks. A dedicated vector database may be more appropriate when vector retrieval is a major part of a larger search workload.

Improve Retrieval Quality

Vector similarity is only one part of search quality. A production implementation should evaluate how well the system retrieves the information users actually need.

Split Documents into Useful Chunks

Large documents may contain several unrelated topics. Embedding the entire document can produce a vector that does not represent the specific information needed for a query.

Split documents at meaningful boundaries, such as headings or paragraphs. Avoid chunks that are so small that they lose necessary context.

Preserve Metadata

Store the source document, title, section, category, and access-control information with each chunk. This makes results easier to display and supports filtering.

Apply authorization filters before returning search results. A semantically relevant document must not be exposed if the current user is not permitted to access it.

Evaluate Similarity Thresholds

A high-ranked result is not necessarily relevant. Measure retrieval quality against a labeled dataset and choose a threshold based on observed results rather than assuming a universal cosine score.

Consider Hybrid Search

Semantic search can struggle with exact identifiers, product codes, error messages, and uncommon technical terms. Hybrid search combines vector retrieval with keyword or full-text search.

For developer documentation, combining semantic retrieval with exact matching can help locate both conceptual explanations and specific compiler errors.

Best Practices for Production Applications

Reuse Embeddings

Generate document embeddings during indexing and reuse them until the document or embedding model changes. Recomputing every document vector for every query wastes processing time and money.

Validate Vector Dimensions

Reject vectors with unexpected dimensions or non-finite values. A single malformed vector should not cause unpredictable search results.

Batch Embedding Requests

When indexing many documents, use provider-supported batches where appropriate. Respect request-size limits and apply bounded concurrency to avoid overwhelming the embedding service.

Cache Carefully

Repeated queries may benefit from caching query embeddings or search results. Include the embedding model version, relevant filters, and authorization context in the cache design so stale or unauthorized results are not returned.

Monitor Retrieval Performance

Track embedding latency, search latency, indexing failures, result counts, and retrieval quality. If users consistently fail to find relevant content, review chunking and evaluation data before changing similarity thresholds blindly.

Common Mistakes and Troubleshooting

Problem	Likely cause	Recommended action
Search returns unrelated documents	Poor embeddings or unsuitable chunks	Review the model and chunk boundaries
Similarity calculation fails	Different vector dimensions	Verify the model configuration and stored vectors
Search becomes slow	Linear scanning of a large collection	Use a vector index
New documents do not appear	Indexing failed or updates were not persisted	Inspect indexing errors and storage state
Exact error codes are missed	Semantic retrieval alone is insufficient	Add keyword or full-text search
Results expose restricted content	Authorization filtering is missing	Enforce access controls in retrieval

Advantages and Disadvantages

Semantic search can find conceptually related documents even when users phrase queries differently from the original text. It is useful for documentation search, knowledge bases, support systems, and retrieval-augmented generation.

However, embedding models introduce processing costs and additional infrastructure. Semantic similarity is not a guarantee of factual relevance, and embedding quality can vary across domains and languages. Vector indexes also require maintenance as documents and models change.

Summary

Building semantic search in Go involves generating embeddings, storing them with document metadata, embedding user queries, and ranking documents by vector similarity.

A Go interface for embedding generation keeps the application independent of a particular provider, while cosine similarity and a linear scan provide a useful starting point for small collections. Larger applications should use vector storage with appropriate indexing, filtering, and monitoring.

The most important work is not simply calculating vector distances. Reliable semantic search depends on choosing a suitable embedding model, preparing useful document chunks, preserving access controls, and evaluating retrieval quality against real queries.