A Java application can experience slow requests, inconsistent response times, and reduced throughput even when CPU utilization appears normal. In multithreaded applications, one possible reason is lock contention: multiple threads compete for the same shared resource, forcing some of them to wait while another thread holds a lock.
Lock contention can be difficult to diagnose in production. A thread dump may show that several threads are waiting, but it does not always provide enough information to explain how frequently the problem occurs, how long threads wait, or which code paths are responsible.
Java Flight Recorder (JFR) provides a way to investigate these problems by collecting runtime events with relatively low overhead in typical production configurations. Developers can analyze those events to understand thread behavior, synchronization delays, CPU usage, garbage collection, and other aspects of application performance.
This article explains how JFR works, how to capture a recording from a running Java application, how to identify lock contention, and how to use the collected evidence to improve application performance.
What Is Java Flight Recorder?
Java Flight Recorder is a diagnostic and profiling framework included in modern Java Development Kit distributions. It collects information about JVM activity, including thread execution, garbage collection, class loading, synchronization, and other runtime events.
A JFR recording contains timestamped events that can be analyzed after collection or, depending on the workflow, examined while the application is running.
JFR is particularly useful when an application exhibits intermittent performance problems. Rather than relying exclusively on a snapshot of the application's state, developers can inspect a time-based record of what happened during the affected period.
For example, consider an API service that normally processes requests quickly but occasionally experiences a sudden increase in response time. If many requests need to acquire the same synchronized resource, threads may spend substantial time waiting for access.
A JFR recording can help identify the affected threads, the relevant synchronization events, and the methods associated with the contention.
What Is Lock Contention?
Lock contention occurs when a thread attempts to acquire a lock that another thread currently holds.
Consider a shared object protected by a synchronized method:
public class InventoryService {
private int availableItems = 100;
public synchronized boolean reserveItem() {
if (availableItems <= 0) {
return false;
}
availableItems--;
return true;
}
}
The synchronized keyword ensures that only one thread at a time can execute a synchronized instance method on the same object.
This protects the shared inventory count from concurrent modifications through this method. However, if many threads call reserveItem() simultaneously, they must compete for the object's monitor.
If the method performs only a small amount of work, the contention may be negligible. If the critical section becomes expensive or the method is called at a very high rate, waiting threads can reduce throughput and increase latency.
The goal of performance analysis is not to eliminate every lock. Locks are often necessary for correctness. The goal is to determine whether a particular lock creates an unnecessary or excessive bottleneck.
How Java Flight Recorder Captures Lock Contention
JFR records selected events emitted by the JVM. The available events and their settings depend on the Java version and recording configuration.
For synchronization analysis, useful events can include monitor contention and thread parking.
These events represent different mechanisms.
Monitor contention commonly involves threads waiting to acquire intrinsic locks, such as those used by synchronized.
Thread parking can occur when threads wait through mechanisms such as LockSupport.park(), which is used by several concurrency utilities.
A thread waiting on a ReentrantLock, for example, may spend time parked rather than appear exclusively as a monitor-contention event.
This distinction matters because examining only one category of event may miss important waiting behavior.
Why JFR Is Useful in Production
JFR offers several advantages for production diagnostics:
It can collect events over a defined time window.
It can record thread and stack-trace information for relevant events.
It helps correlate synchronization delays with other JVM activity.
It can be started on a running JVM without requiring an application restart in supported configurations.
It can help distinguish lock-related delays from CPU-intensive work or garbage collection.
JFR is not completely free of overhead. The impact depends on the enabled events, recording duration, stack-trace settings, event volume, and application workload.
For production use, choose a suitable recording configuration and validate its overhead in an environment representative of the deployment.
Step 1: Verify Java Flight Recorder Availability
First, verify the Java version used by the application.
java -version
JFR is included in modern JDK distributions. Its availability and supported commands depend on the specific runtime, so use a JDK version that supports the diagnostic workflow you intend to follow.
If the application runs inside a container, execute the diagnostic commands in the same container or an appropriately configured diagnostic environment.
Next, identify the running Java process:
jcmd -l
Example output:
12345 com.example.OrderApplication
In this example, 12345 is the JVM process ID.
The process must be accessible to the account running the diagnostic command. Container boundaries, operating-system permissions, and JVM attachment restrictions can prevent jcmd from communicating with the target process.
For containerized applications, verify that the required diagnostic utilities are available. A minimal runtime image may not include jcmd, even if the JVM supports JFR.
Step 2: Start a JFR Recording
The jcmd utility can control diagnostic operations on a running JVM.
To start a recording with the default JFR profile, use:
jcmd 12345 JFR.start \
name=LockContention \
settings=profile \
duration=120s \
filename=/tmp/lock-contention.jfr
Replace 12345 with the actual process ID.
This command starts a recording named LockContention, uses the JFR profile settings, and specifies a duration of 120 seconds.
The recording is written to the requested file when the recording completes.
The profile settings are a useful starting point for many investigations, but they do not necessarily enable every event that might be relevant to a particular lock-contention problem.
For a controlled investigation, confirm that the recording configuration includes the synchronization events you need.
Capture an Explicit Recording Window
If you want to capture an incident that occurs during a known period, you can start a recording without specifying a duration:
jcmd 12345 JFR.start \
name=LockContention \
settings=profile
After reproducing the problem or observing the performance degradation, dump the recording:
jcmd 12345 JFR.dump \
name=LockContention \
filename=/tmp/lock-contention.jfr
Then stop the recording if it is no longer needed:
jcmd 12345 JFR.stop name=LockContention
This workflow gives you more control over the capture window. Be aware that a recording without a duration can continue until it is stopped or the JVM exits.
For production systems, avoid leaving unnecessary diagnostic recordings active indefinitely.
Step 3: Configure Synchronization Events
A useful investigation must collect events that match the locking mechanism used by the application.
JFR event configuration can be customized through a JFR settings file. The following example illustrates how to enable monitor-contention events and capture stack traces.
<?xml version="1.0" encoding="UTF-8"?>
<configuration version="2.0"
label="Lock Contention Analysis"
description="Records monitor contention events"
provider="Example">
<event name="jdk.JavaMonitorEnter">
<setting name="enabled">true</setting>
<setting name="stackTrace">true</setting>
<setting name="threshold">10 ms</setting>
</event>
</configuration>
Save the file as lock-contention.jfc.
This configuration enables the jdk.JavaMonitorEnter event and requests stack traces for events that meet the configured threshold.
The threshold is an example, not a universal recommendation. A 10-millisecond threshold can be useful for identifying longer waits, but it may miss frequent shorter waits that collectively affect throughput.
Also, this configuration focuses on monitor-entry contention. It does not comprehensively capture every possible lock or parking mechanism. For applications that use ReentrantLock, semaphores, futures, or other concurrency primitives, examine the relevant thread-parking and synchronization events as well.
Use the settings files and event names supported by the target JDK. JFR configuration syntax and event availability should be verified against the runtime version in use.
Start a Recording With Custom Settings
Use the custom configuration when starting the recording:
jcmd 12345 JFR.start \
name=LockContention \
settings=/opt/jfr/lock-contention.jfc \
duration=120s \
filename=/tmp/lock-contention.jfr
The path to the settings file must be accessible in the environment where the JVM processes the command.
For containers, ensure the file exists inside the relevant filesystem namespace or use a diagnostic workflow that makes it accessible to the JVM.
Before deploying a custom configuration broadly, test it against a representative workload. Excessively detailed event collection can increase the recording size and diagnostic overhead.
Step 4: Analyze the Recording
After capturing the recording, open the .jfr file in Java Mission Control (JMC), a graphical tool for examining JFR recordings.
The exact layout depends on the JMC version and available event data, but the investigation should focus on several areas.
Identify the Most Frequent Contention Events
Look for monitor-contention events and inspect:
The event duration.
The thread that attempted to acquire the monitor.
The stack trace associated with the waiting thread.
The monitor or class involved, when available.
The time at which the event occurred.
A large number of events can indicate a frequently contended lock, but event count alone does not establish the severity of the problem.
For example, thousands of very short waits may be less harmful than a smaller number of long waits that block latency-sensitive requests.
Find Repeated Stack Traces
Repeated stack traces can help identify the application method associated with contention.
Suppose several requests are waiting in the same inventory-related service method. That pattern may indicate that a shared synchronized operation is limiting concurrency.
However, the waiting stack trace does not necessarily identify every operation performed by the thread that currently owns the lock.
Use the event data to identify the contended monitor, then inspect the corresponding code and surrounding call paths to understand why the lock is held for so long.
Correlate Contention With Other JVM Activity
Lock contention may occur alongside other performance problems.
For example:
A thread may hold a lock while performing expensive computation.
A synchronized method may include blocking database access.
A large allocation rate may increase garbage-collection activity.
A limited worker pool may cause additional requests to queue.
Review the recording's timeline and related events to determine whether these problems overlap.
The purpose of JFR analysis is to establish a plausible cause from runtime evidence, rather than assuming that every slow request is caused by synchronization.
Step 5: Build a Reproducible Contention Example
The following example creates a shared lock that is held for a noticeable period. It demonstrates how contention can occur when multiple threads compete for the same monitor.
public class LockContentionDemo {
private static final Object LOCK = new Object();
public static void main(String[] args) {
Runnable task = () -> {
for (int i = 0; i < 20; i++) {
synchronized (LOCK) {
try {
Thread.sleep(50);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return;
}
}
}
};
for (int i = 0; i < 8; i++) {
Thread thread = new Thread(
task,
"worker-" + i
);
thread.start();
}
}
}
All eight worker threads attempt to enter the synchronized block using the same lock object.
The Thread.sleep(50) call intentionally keeps the lock held while the thread sleeps. Other threads attempting to enter the block must wait.
This is a diagnostic demonstration, not a recommended application design. Sleeping while holding a shared lock is usually undesirable because it prevents other threads from entering the critical section even though the current thread is not doing useful work.
Compile and run the program:
javac LockContentionDemo.java
java LockContentionDemo
In another terminal, identify the JVM process and start a JFR recording:
jcmd -l
Then use the process ID to start the recording with the appropriate synchronization events enabled.
Open the resulting recording in JMC and inspect monitor-contention events.
Depending on the JDK version and event settings, the recording may show threads waiting to enter the synchronized block.
This controlled example helps establish what a known contention pattern looks like before investigating a more complex production application.
Step 6: Fix the Underlying Lock Bottleneck
After identifying the relevant code path, choose a solution based on why the lock is needed and what work occurs while it is held.
Reduce the Critical Section
A common improvement is to keep the locked region as small as possible.
For example, if a method acquires a lock and then performs logging, formatting, or network communication, determine whether those operations can safely occur outside the critical section.
Do not move operations outside a lock if doing so would break the consistency of shared state.
The goal is to protect the minimum amount of state required for correctness.
Avoid Blocking Operations While Holding Locks
Database calls, HTTP requests, file operations, and other blocking tasks can keep a lock occupied for a long time.
If multiple requests need the same lock, these operations can serialize otherwise independent work.
Where possible, separate state coordination from external I/O. If the operation requires a consistent snapshot, capture the necessary state under the lock and perform independent work afterward.
The correct approach depends on the concurrency guarantees the application must preserve.
Reduce Shared Mutable State
Sometimes the real problem is excessive sharing.
If many threads need to update the same mutable object, consider whether the design can use independent state, immutable data, partitioned resources, or message-based coordination.
For example, partitioning a counter by customer or account may reduce contention if the business rules allow independent updates. Aggregating the partitions later can be more scalable than forcing every request through a single global lock.
However, partitioning introduces complexity when operations need a consistent global view. Validate the resulting consistency model before making the change.
Choose Concurrency Utilities Carefully
Java provides several synchronization mechanisms, including synchronized, ReentrantLock, atomic classes, concurrent collections, and semaphores.
Changing synchronized to ReentrantLock does not automatically eliminate contention. Both mechanisms can serialize access to shared state.
Choose a concurrency primitive based on the required semantics, such as interruptible acquisition, timed acquisition, fairness requirements, or atomic updates.
The most effective optimization is often reducing the need for shared coordination rather than replacing one lock implementation with another.
Common Mistakes When Diagnosing Lock Contention
Looking Only at CPU Usage
A process can have low CPU utilization while many threads wait for locks.
CPU metrics are useful, but they do not show the full cost of blocked threads or the effect on request latency.
Use JFR synchronization events alongside application metrics and request traces.
Treating Every Lock as a Problem
Locks are necessary in many correct concurrent designs.
The presence of a synchronized method does not prove that the application has a performance issue.
Look for evidence that waiting time, contention frequency, or serialized execution affects an important workload.
Capturing Too Much Data
Enabling every possible event with detailed stack traces can create large recordings and increase overhead.
Start with a focused recording configuration, reproduce the problem, and expand the event set only when additional evidence is needed.
Fixing the Wrong Code Path
A waiting thread's stack trace shows where it attempted to acquire a lock. It may not reveal why the lock holder is taking so long.
Inspect the shared resource, the lock's owning code path, and the operations performed inside the critical section.
Ignoring Thread Parking
Applications that use explicit locks and concurrency utilities may spend substantial time parked rather than waiting exclusively on intrinsic monitors.
If monitor-contention events do not explain the problem, investigate thread-parking events and other relevant synchronization behavior.
Best Practices for Production Diagnostics
Capture JFR recordings around the period when the problem occurs.
Start with a standard profile configuration and add targeted synchronization events as needed.
Include stack traces for the events most relevant to the investigation.
Correlate JFR findings with request latency, throughput, and error rates.
Compare the behavior of affected and unaffected periods.
Reproduce suspected contention in a controlled environment before changing production synchronization.
Validate that optimizations preserve thread safety and application correctness.
Keep recording durations and event settings appropriate for the environment.
Protect recording files because stack traces and runtime metadata can expose internal implementation details.
Repeat the same performance measurements after implementing a fix.
Advantages and Limitations
Advantages | Limitations |
|---|
Captures timestamped JVM runtime events | Event availability depends on the JDK version and configuration |
Helps investigate intermittent contention | A recording may not include every relevant event |
Supports stack-trace analysis for selected events | Stack traces do not always identify the lock holder's complete behavior |
Can collect data from a running JVM | Diagnostic overhead depends on the selected events and workload |
Helps correlate synchronization with other runtime activity | Requires careful interpretation and knowledge of Java concurrency |
Conclusion
Java Flight Recorder is a useful tool for investigating lock contention in running Java applications. It provides runtime evidence that can help developers identify waiting threads, inspect synchronization events, and correlate contention with other performance problems.
A reliable investigation starts by capturing a focused recording during the affected period. Analyze monitor contention and thread parking where appropriate, inspect repeated stack traces, and determine which shared resource is responsible for the waiting behavior.
Once the bottleneck is understood, reduce unnecessary work inside critical sections, avoid blocking operations while holding locks, and reconsider whether the application needs so much shared mutable state.
The goal is not to remove synchronization indiscriminately. It is to preserve correctness while reducing unnecessary waiting and improving throughput and response-time consistency.
Join the conversation! Your thoughts help the community grow.