Improving Vector Search in AlloyDB With ScaNN Indexes

Vector search is becoming an important part of applications that use AI features such as semantic search, recommendation systems, document retrieval, and retrieval-augmented generation (RAG).

A traditional database query usually looks for exact matches. For example, an application might search for products where the category is Laptop or the product name contains a particular word. Semantic search works differently. It compares the meaning of the search query with stored data, even when the wording is different.

This requires converting text into numerical representations called embeddings. As the number of stored embeddings grows, searching through every vector becomes expensive.

Google AlloyDB for PostgreSQL supports ScaNN indexes to improve approximate nearest neighbor searches. ScaNN stands for Scalable Nearest Neighbors. It uses a tree-based indexing approach to find similar vectors without comparing the query against every stored vector.

In this article, we will build a product-search example using AlloyDB, PostgreSQL, vector embeddings, and ScaNN. We will also look at index configuration, query performance, recall, and common mistakes developers should avoid.

Why Vector Search Needs an Index

Consider an online shopping application with 500,000 products. Each product has a name, description, category, and embedding generated by an AI model.

A customer searches for:

A lightweight laptop for programming and travel

The application converts this sentence into an embedding and compares it with the embeddings stored in the database.

A basic vector query might look like this:

SELECT
    product_id,
    product_name
FROM products
ORDER BY embedding <=> '[0.12, 0.34, 0.56]'::vector
LIMIT 10;

The <=> operator calculates cosine distance between vectors. A smaller distance indicates greater similarity.

Without an appropriate index, the database may need to calculate distances against a large number of stored vectors. This becomes expensive as the dataset grows.

An approximate nearest neighbor (ANN) index reduces the amount of search work. Instead of examining every vector, it searches selected areas of the index to find likely matches.

The trade-off is that approximate search may not always return the exact nearest neighbors. Developers must balance search speed, recall, memory usage, and index maintenance.

What Is a ScaNN Index in AlloyDB?

ScaNN is a vector-search algorithm developed by Google. AlloyDB provides it through the alloydb_scann PostgreSQL extension.

The index organizes vectors into a tree-based structure using quantization. During a search, it identifies promising regions of the index and searches within them rather than scanning the entire dataset.

Compared with HNSW, ScaNN can offer:

Lower index-building time.
A smaller memory footprint.
Higher queries per second for some workloads.
Different options for balancing recall and search performance.

These benefits depend on the dataset, vector dimensions, query patterns, configuration, and available resources. ScaNN is not automatically faster for every workload, so benchmark it against alternatives using real application data.

Prepare the AlloyDB Database

Before creating a ScaNN index, you need a table containing vector embeddings.

The vector and alloydb_scann extensions provide the required functionality.

Run the following commands:

CREATE EXTENSION IF NOT EXISTS alloydb_scann CASCADE;

The alloydb_scann extension checks for the required vector extension and installs it if necessary.

Next, create a product table:

CREATE TABLE products
(
    product_id BIGSERIAL PRIMARY KEY,
    product_name VARCHAR(200) NOT NULL,
    description TEXT,
    category VARCHAR(100),
    embedding VECTOR(768)
);

The example uses 768 dimensions. Your vector dimension must match the output of the embedding model used by your application.

For example, if your embedding model returns 1,536 values, the column must use VECTOR(1536).

A vector column with the wrong dimension cannot store embeddings from a model that returns a different number of values.

Store Product Embeddings

The next step is to generate embeddings for product descriptions and store them in AlloyDB.

An embedding model converts text into a vector that represents its semantic meaning.

For example, these descriptions use different words but may have similar meanings:

Product A:
Lightweight laptop suitable for software development.

Product B:
Portable computer for coding and everyday programming.

A semantic search system can identify their relationship through embeddings.

You can generate embeddings outside the database using an application service, or use supported model integrations available in AlloyDB.

For this example, assume the application has already generated a vector with 768 dimensions.

A simplified insert statement looks like this:

INSERT INTO products
(
    product_name,
    description,
    category,
    embedding
)
VALUES
(
    'Developer Laptop',
    'Lightweight laptop for programming and travel',
    'Electronics',
    '[0.12, 0.34, 0.56]'::vector
);

The vector above is abbreviated to demonstrate the SQL structure. A real insert must contain all 768 values.

In production, keep the embedding model and its version consistent across the records being searched. Mixing embeddings from incompatible models can produce meaningless similarity results, even when the vector dimensions happen to match.

Create an Automatically Tuned ScaNN Index

AlloyDB supports automatically tuned and manually tuned ScaNN indexes.

For most applications starting with ScaNN, automatic tuning is a useful option because AlloyDB manages the index structure and maintenance.

Create an automatically tuned index using:

CREATE INDEX products_embedding_scann_idx
ON products
USING scann (embedding cosine)
WITH
(
    MODE = 'AUTO',
    OPTIMIZATION = 'SEARCH_OPTIMIZED',
    auto_maintenance = 'ON'
);

The important settings are:

MODE = 'AUTO' enables automatic index tuning.
OPTIMIZATION = 'SEARCH_OPTIMIZED' prioritizes vector-search recall and latency, potentially increasing index-building time.
auto_maintenance = 'ON' enables automatic index maintenance.

If your application needs a balance between index-building time and search performance, you can use:

CREATE INDEX products_embedding_scann_idx
ON products
USING scann (embedding cosine)
WITH
(
    MODE = 'AUTO',
    OPTIMIZATION = 'BALANCED',
    auto_maintenance = 'ON'
);

Choose one index definition for a given index name. Do not run both statements with the same index name unless you have removed or renamed the existing index.

Automatic tuning reduces the amount of manual configuration, but it does not remove the need to measure query performance.

Run Semantic Search Queries

After the index is created, the application can execute vector similarity queries.

Assume the search service converts the customer's query into a 768-dimensional embedding.

The application passes that vector to PostgreSQL:

SELECT
    product_id,
    product_name,
    description,
    embedding <=> $1::vector AS distance
FROM products
ORDER BY embedding <=> $1::vector
LIMIT 10;

Here, $1 represents a parameter supplied by the application.

Using a parameterized query is important. Avoid building SQL statements by concatenating user input or vector values into query strings.

The database uses the cosine distance operator to compare the query embedding with stored embeddings. The ScaNN index can accelerate this nearest-neighbor search.

The returned rows contain the closest matching products according to the selected distance function.

The distance value is useful for ranking, but it should not automatically be treated as a probability or a confidence percentage.

Choose the Correct Distance Function

ScaNN supports different distance functions. The index definition and query operator must use compatible distance calculations.

Distance function	Index definition	Query operator
Cosine distance	`cosine`	`<=>`
L2 distance	`l2`	`<->`
Dot product	`dot_product`	`<#>`

Cosine distance is commonly used for text embeddings because it compares vector direction.

L2 distance measures the straight-line distance between vectors.

Dot product measures the product of corresponding vector values. Its interpretation depends on how the embedding model produces and normalizes vectors.

Do not select a distance function only because it produces faster results in a small test. Follow the embedding model's recommended similarity measure and validate the results against relevant search examples.

Tune ScaNN for Larger Datasets

Automatic tuning is a good starting point, but some applications need more control.

ScaNN also supports manually tuned indexes. These allow developers to configure the number of leaves, quantization method, and tree structure.

A two-level tree index can be created with:

CREATE INDEX products_scann_manual_idx
ON products
USING scann (embedding cosine)
WITH
(
    mode = 'MANUAL',
    num_leaves = 1000,
    quantizer = 'SQ8',
    auto_maintenance = 'ON'
);

The values are starting examples, not universal recommendations. The appropriate number of leaves depends on the dataset size and search requirements.

The num_leaves setting controls how the index divides vectors into partitions. Increasing the number of leaves changes the search structure and can affect recall, query latency, and index-building cost.

ScaNN also provides quantization choices:

SQ8 provides a balance between search performance and recall.
FLAT prioritizes recall at the cost of search performance.
AH uses asymmetric hashing and is available as a preview option in the documented configuration.

For a production application, begin with the automatically tuned index. Move to manual tuning only when measurements show that the default configuration does not meet your requirements.

Measure Recall and Query Performance

A fast query is not necessarily a good query. A search system must return relevant results as well as respond quickly.

Two useful measurements are latency and recall.

Latency measures how long a query takes. It is useful to examine median latency and higher-percentile values such as p95 and p99 rather than relying only on average response time.

Recall measures how many of the expected nearest neighbors are returned by approximate search.

For example, if an exact search returns ten expected results and the approximate search returns eight of those results, recall at ten is 80%.

A practical benchmarking process is:

Prepare a representative dataset.
Run exact vector searches to establish a baseline.
Run approximate searches using ScaNN.
Compare the returned results.
Measure latency under realistic concurrency.
Repeat the tests with different index configurations.
Check how inserts and updates affect performance.

Use the same query set for each test. Otherwise, the comparison may reflect differences in search input rather than index performance.

You can also inspect index creation progress with:

SELECT *
FROM pg_stat_progress_create_index;

This view helps identify the current phase of an index-building operation.

After index creation, run EXPLAIN on representative search queries to check whether the query planner is using the expected index.

Common Mistakes to Avoid

Creating an index before inserting enough data

ScaNN needs sufficient data to build a useful index. Creating an index on an empty or very small table can fail or produce an unsuitable structure.

For small tables, use a normal exact search until there is enough data. AlloyDB also documents deferred index creation for supported configurations.

Mixing embedding dimensions

A vector column must match the dimensions returned by the embedding model. Changing models may require a new column or a controlled re-embedding process.

Ignoring recall

Optimizing only for latency can reduce search quality. Test whether the returned results are relevant to actual user queries.

Choosing manual settings without benchmarks

A large num_leaves value is not automatically better. Test index configurations against the size and distribution of your own data.

Ignoring index maintenance

Vector data changes over time. Inserts, updates, and deletes can affect index quality and performance. Choose an appropriate maintenance strategy and monitor the index as the dataset grows.

Advantages and Limitations of ScaNN

ScaNN is useful when an application needs approximate nearest neighbor search over a growing collection of embeddings.

Its main advantages include efficient search, lower memory requirements than HNSW in many workloads, and automatic tuning options.

However, it also has limitations. Approximate search does not guarantee exact nearest neighbors. Index creation requires sufficient data, and manual tuning requires testing. The best configuration depends on the workload, database resources, vector dimensions, and required recall.

For smaller datasets, a sequential exact search may be simpler and sufficiently fast. For other workloads, HNSW may provide a better result. Compare the options instead of assuming one index type is always the right choice.

Summary

ScaNN indexes help AlloyDB applications perform approximate nearest neighbor searches over stored vector embeddings. They are useful for semantic search, product recommendations, document retrieval, and other AI-powered database features.

Developers can start with an automatically tuned index and use PostgreSQL vector operators to search for similar records. Larger or more demanding workloads can use manual tuning to control index structure and quantization.

The important part is to measure both search speed and result quality. Test with real embeddings, realistic query patterns, and production-like data volumes before selecting an index configuration.