Elasticsearch is designed to search large collections of documents, but a search request can become slow even when the cluster appears healthy. A query that usually returns results quickly may start taking several seconds as the index grows, filters become more complex, or multiple searches compete for cluster resources.
The cause is not always the same. A query may examine too many documents, run expensive aggregations, execute scripts for many records, or spend time collecting results from several shards. In other cases, the query itself is reasonable, but the cluster is under pressure from garbage collection, disk latency, or an uneven distribution of shards.
Elasticsearch provides a Profile API that helps developers understand how query and aggregation components execute. It exposes detailed timing information about parts of the search process, making it easier to identify expensive operations.
However, profiling is not a complete performance diagnosis. It adds overhead, and the reported timings do not represent the entire time a user waits for a response. You must interpret profiling data alongside the request, execution plan, shard distribution, and cluster metrics.
This article explains how to profile Elasticsearch queries, identify expensive query components, investigate aggregations and shard-level delays, and make performance improvements based on evidence.
Understand What Makes an Elasticsearch Query Slow
An Elasticsearch search request passes through several stages. Depending on the request, the work may include parsing the query, executing it on shards, matching documents, collecting scores, calculating aggregations, sorting results, and returning the response.
A query can become slow at any of these stages.
For example, consider an application that searches support tickets by status, customer, and free-text content. If the search matches a large portion of the index and sorts every matching document by a computed score, Elasticsearch may need to perform substantially more work than a query that filters on exact values and returns a small number of results.
Common causes of slow searches include:
Broad queries that match too many documents.
Expensive wildcard or regular-expression queries.
Scripts executed against large sets of documents.
Aggregations over high-cardinality fields.
Sorting on fields that are not mapped appropriately.
Large result windows or unnecessarily high page sizes.
Shards that are too large, too numerous, or unevenly distributed.
CPU, memory, disk, or network contention.
The first task is to identify which part of the request consumes the most time. The Profile API can help with the query execution portion of that investigation.
Enable the Elasticsearch Profile API
The Profile API is enabled by adding "profile": true to a search request.
Suppose an index named support-tickets contains documents with fields such as status, customerId, subject, and description. You can profile a query that filters tickets by status and searches for a phrase in the subject.
{
"profile": true,
"query": {
"bool": {
"filter": [
{
"term": {
"status": "open"
}
}
],
"must": [
{
"match_phrase": {
"subject": "database connection"
}
}
]
}
}
}
The request returns the search results along with profiling details for the participating shards.
The profile information includes the query tree, timing information for query components, and collector details. Depending on the query and Elasticsearch version, it can also expose details about how particular query implementations were executed.
Why profiling should be temporary
Profiling adds work to the search request. Elasticsearch must collect detailed timing information while executing the query, which can affect latency and resource consumption.
For that reason, avoid enabling profiling on every production request or turning it on globally for an extended period.
A better approach is to reproduce the slow query with representative data, profile a limited number of requests, inspect the results, and then disable profiling.
If the issue occurs only under heavy traffic, capture the query during a controlled test or investigate it alongside cluster metrics. A profile collected under light load may not reveal contention that appears only during peak traffic.
Read the Profile Response
A Profile API response can contain several sections. The most relevant are query timings, collector information, and aggregation profiling when aggregations are present.
Query timing information
The query section describes the operations Elasticsearch used to evaluate the query. A complex query may contain multiple nested query components.
For each profiled component, the response can include information such as:
The query type and description.
The number of times the component was executed.
The number of matching documents.
The reported execution time.
Child query components, where applicable.
The timing values help you identify which query operations deserve closer investigation.
For example, if a broad text query dominates the reported time while a term filter takes relatively little time, examine the text query's selectivity, analyzer configuration, and target fields.
Do not simply add every timing value together and assume the result equals the total request duration. Nested query timings can overlap conceptually because parent operations include work performed by child operations.
Collector information
Collectors gather matching documents and help process search results. Collector timings can reveal that the query-matching stage is not the only expensive part of a search.
For example, a query may identify matching documents quickly but spend additional time scoring, sorting, or collecting a large number of results.
If collector activity is significant, investigate the requested result size, sort configuration, scoring requirements, and whether the application actually needs every returned field.
Find Expensive Query Components
Start with the query tree and identify the operations with the highest reported execution time. Then check whether those operations are necessary for the application's search requirements.
Consider the following examples.
Investigate broad text searches
A query that searches several large text fields can perform more work than expected, especially when the search terms match a large portion of the index.
Inspect the fields targeted by the query and verify that they use the intended mappings and analyzers. A field intended for full-text search should generally use a text mapping, while exact values and sorting often require a keyword or another suitable field type.
For example:
{
"query": {
"match": {
"description": {
"query": "database connection timeout",
"operator": "and"
}
}
}
}
The AND operator requires all analyzed query terms to match, which can reduce the number of matches compared with the default OR behavior. However, it can also change the meaning of the search, so use it only when it fits the application's requirements.
Do not narrow a query merely to improve its benchmark result if users expect broader matching behavior. Validate relevance and recall alongside latency.
Examine wildcard and regular-expression queries
Wildcard and regular-expression queries can be expensive, especially when the pattern requires Elasticsearch to examine many possible terms.
For example, a leading wildcard such as "*timeout" can be more costly than a prefix pattern because the search cannot use the term dictionary in the same way as a straightforward prefix lookup.
If the Profile API identifies a wildcard or regular-expression operation as expensive, consider whether the application can use a different field mapping or query design.
For frequently searched identifiers, normalize the value at indexing time and use exact matching when appropriate. For prefix searches, consider a mapping or query approach designed for that requirement.
Do not replace a wildcard query with an exact term query unless the expected search behavior is equivalent.
Review scripts and computed fields
Script queries and script-based sorting may execute additional logic for many candidate documents.
For example, calculating a score from several document fields for every matching record can be expensive when the query matches a large number of documents.
If a script is a bottleneck, determine whether the calculation can be performed at indexing time or represented by a field that Elasticsearch can access more efficiently.
For frequently used calculations, storing a derived field may reduce repeated computation. However, this introduces data consistency requirements because the derived value must be updated whenever the underlying values change.
Profile Aggregations Separately
Aggregations can consume significant resources even when the search query itself is fast.
A dashboard might filter support tickets by date and calculate counts by customer, status, and product. If an aggregation operates over a large number of documents or a high-cardinality field, it can become a major source of latency and memory use.
The Profile API can include aggregation profiling for requests that contain aggregations.
For example:
{
"profile": true,
"size": 0,
"query": {
"term": {
"status": "open"
}
},
"aggs": {
"tickets_by_customer": {
"terms": {
"field": "customerId",
"size": 20
}
}
}
}
The size value at the top level is set to zero because this request needs aggregation results rather than individual search hits.
The terms aggregation returns a limited number of buckets. However, the size setting does not guarantee that Elasticsearch performs work only for those 20 customers. It may still need to inspect many matching documents and collect candidate buckets across shards.
Understand high-cardinality aggregations
A high-cardinality field contains many distinct values. Aggregating over such a field can require substantial memory and processing.
If an aggregation is slow, check whether the application really needs a bucket for every distinct value. Consider filtering the input documents, limiting returned buckets, or using a different aggregation strategy that better fits the reporting requirement.
For very large analytical workloads, composite aggregations can support bucket pagination. This is useful when an application needs to process many buckets incrementally instead of returning them all in one response.
Be careful when changing aggregation size or filters. These changes can alter the completeness or meaning of the report.
Distinguish Query Time from Total Request Latency
One of the most important limitations of the Profile API is that it does not measure every part of a search request.
Profile timings focus on the profiled search execution. They do not fully account for all network time, request coordination, queueing, response serialization, or other overhead that contributes to end-to-end latency.
Consider a request that takes 1.5 seconds from the application's perspective, while the profiled query operations account for a much smaller portion of that duration.
The difference may involve shard coordination, queue delays, network communication, response handling, or contention outside the profiled operations.
To investigate this situation, compare:
Application-side request duration.
Elasticsearch's overall response time.
Profile API timings.
Search thread-pool queue and rejection metrics.
CPU and memory pressure.
Garbage collection activity.
Disk latency and storage throughput.
Shard count and shard distribution.
If the query components are relatively fast but the request remains slow, focus on the cluster and request path rather than rewriting the query without evidence.
Check Whether Shards Are Causing Delays
Elasticsearch distributes data across shards. A search that targets multiple shards may require work to be performed across several locations before the results can be combined.
A high shard count can increase coordination overhead, while uneven data distribution may cause one shard to do substantially more work than others.
Use the Profile API's per-shard information to compare execution behavior. If one shard consistently takes much longer than others, investigate its size, document distribution, node resources, and local query behavior.
However, the slowest profiled shard is not automatically the root cause. A shard may have more documents or may be running on a node with different resource pressure.
Check the cluster's shard allocation and node-level metrics before making changes. Reindexing, changing shard counts, or relocating data can be expensive and should be planned rather than used as an immediate response to one slow query.
Use Explain for a Specific Document
The Explain API answers a different question from the Profile API.
While profiling helps you understand query execution cost, Explain helps you understand why a particular document matches a query and how its relevance score is calculated.
For example, you can use Explain when a search returns a document with an unexpectedly high or low score.
{
"query": {
"match": {
"description": "database connection timeout"
}
}
}
Send the query to the Explain endpoint for a specific document in the index. The response describes how the document's score was calculated.
Explain is useful for relevance debugging, but it is not a substitute for profiling a slow search. Its output can be detailed, and using it repeatedly across large numbers of documents can be expensive.
Common Mistakes When Profiling Elasticsearch
Treating profile time as total request time
Profile timings describe query execution details, not the complete application-to-Elasticsearch round trip. Use end-to-end latency and cluster metrics alongside the profile.
Profiling every production request
Detailed instrumentation can add overhead and generate large responses. Use targeted profiling sessions and disable profiling when the investigation is complete.
Optimizing only for speed
A faster query may return fewer relevant documents or change the meaning of a report. Validate search quality and correctness after every query change.
Assuming a slow query always needs a new index
An inefficient query, expensive aggregation, queueing delay, or overloaded node may be responsible. Inspect the execution plan and cluster state before changing mappings or adding indexes.
Ignoring query concurrency
A query that runs quickly in isolation may become slow when many users execute it simultaneously. Test realistic concurrency and inspect thread-pool queues, resource utilization, and shard-level contention.
Best Practices for Production Elasticsearch Searches
Maintain a set of representative slow queries and benchmark them against realistic data. Record the complete request, index mappings, execution plan, query duration, and relevant cluster metrics so that results can be compared over time.
Use the Profile API to identify expensive query components, but combine the findings with application latency and node-level measurements. Separate query-matching cost from aggregation work, result collection, and cluster coordination.
Keep search requests focused. Return only the fields the application needs, avoid unnecessarily large result windows, and use filters for exact constraints when scoring is not required for those conditions.
Review mappings before changing query syntax. Use text fields for full-text search and appropriate exact-value fields for filtering, sorting, and aggregations. Evaluate expensive wildcard patterns and scripts carefully.
Finally, test query changes under realistic concurrency and validate the results against expected search behavior. The goal is to reduce unnecessary work while preserving the relevance and completeness users depend on.
Advantages and Disadvantages of Query Profiling
Advantages
Reveals expensive query and aggregation components.
Helps distinguish query execution problems from broader cluster latency.
Provides per-shard information for investigating uneven execution.
Supports evidence-based changes to query structure and field mappings.
Helps developers understand how Elasticsearch executes complex searches.
Disadvantages
Adds overhead and should not be enabled indiscriminately.
Produces detailed output that can be difficult to interpret.
Does not measure every source of end-to-end request latency.
Does not automatically identify the root cause of node resource contention.
Requires realistic test data and workload measurements to validate improvements.
Summary
Elasticsearch's Profile API provides detailed information about query execution and can help identify expensive text queries, wildcard operations, scripts, aggregations, and result collection.
Start by profiling the exact request that is slow, then inspect the query tree and identify the operations responsible for the largest share of the reported work. Review mappings, filters, aggregations, and result sizes before changing the query.
If profiling does not explain the overall response time, investigate shard coordination, thread-pool queues, CPU, memory, garbage collection, and storage latency. Use Explain when you need to understand relevance scoring rather than execution performance.
The most effective optimization process combines profiling with realistic benchmarking, cluster monitoring, and relevance testing. This makes it possible to improve search performance without sacrificing the results that users expect.
Join the conversation! Your thoughts help the community grow.