When a Python application becomes slow in production, knowing that the process is using 100% CPU is only the beginning.
The harder question is:
What is actually using that CPU?
A Python profiler can tell you which Python functions are busy. But modern Python applications also spend time in C extensions, native libraries, system calls, database drivers, compression libraries, and other code outside the Python layer.
That is where native profiling becomes useful.
Python 3.15 improves support for profiling Python applications with system-level tools by enabling frame pointers in official builds on supported platforms. This makes it easier for tools such as Linux perf and other sampling profilers to reconstruct Python call stacks.
The result is a much more useful view of application performance:
Python application
↓
Python function
↓
C extension
↓
Native library
↓
Operating system
Instead of seeing only a native function or a large interpreter frame, the profiler can retain more of the Python call context.
This article explains what frame pointers are, why they matter for Python profiling, how Python 3.15 improves the situation, and how to use native profiling as part of a production performance investigation.
What Is a Frame Pointer?
A frame pointer is a register or piece of stack information used to help describe the current call stack.
When one function calls another:
main()
↓
process_request()
↓
calculate()
↓
native_function()
the system needs a way to understand that chain.
A profiler can then reconstruct something similar to:
main
└── process_request
└── calculate
└── native_function
That call stack is extremely useful when diagnosing CPU problems.
Without reliable stack information, a sampling profiler may have difficulty reconstructing the complete chain of calls.
Why Does This Matter for Python?
Python code does not always spend its time executing Python bytecode.
Consider:
import numpy as np
result = np.dot(matrix_a, matrix_b)
The Python code is tiny.
The actual CPU work may happen inside native code used by NumPy and its underlying numerical libraries.
A Python-only profile might tell you:
calculate_matrix()
78%
But that still leaves an important question:
Where inside that operation is the CPU time actually going?
Native profiling can provide another layer of visibility.
You may discover a call path such as:
calculate_matrix
↓
numpy
↓
native numerical routine
↓
CPU-intensive operation
That can change the optimization strategy completely.
Python 3.15 and Frame Pointers
Python 3.15 makes native profiling easier on supported configurations by building the interpreter with frame pointers enabled.
This is particularly useful for system profilers that rely on stack unwinding.
Python's profiling documentation describes frame pointers as an important part of getting useful Python stack information from tools such as Linux perf.
The goal is not to make Python applications automatically faster.
The goal is to make them easier to profile.
That distinction is important.
Enabling frame pointers is primarily an observability and profiling improvement.
What Is Stack Unwinding?
Suppose the CPU samples a running process at a random moment.
The profiler sees the instruction currently being executed.
For example:
native_library_function()
That is useful, but incomplete.
The profiler also wants to know how execution reached that function:
handle_request()
↓
process_data()
↓
calculate_statistics()
↓
native_library_function()
This process of reconstructing the call stack is called stack unwinding.
Frame pointers provide one mechanism for making that reconstruction reliable.
A simplified representation looks like:
Current stack frame
↓
Previous frame
↓
Previous frame
↓
Caller
The profiler follows those relationships to build the stack.
Why Native Profiling Is Different From Python Profiling
There are several layers at which you can profile an application.
Application-Level Profiling
You might measure:
HTTP request duration
Database latency
Queue processing time
This tells you what users are experiencing.
Python-Level Profiling
A Python profiler can show:
process_request()
calculate_report()
parse_data()
This helps you understand Python execution.
Native/System-Level Profiling
A tool such as perf can show:
Python function
↓
CPython interpreter
↓
C extension
↓
native library
This can expose CPU activity below the Python level.
All three views can be useful.
A Typical Production Problem
Imagine a Python API server suddenly starts consuming a lot of CPU.
Monitoring shows:
CPU usage: 95%
Memory: normal
Request rate: normal
The application team runs a Python profiler and finds:
process_request() 70%
That still does not explain enough.
The function might be calling:
def process_request(data):
return library.transform(data)
The expensive work may actually happen inside library.transform().
Native profiling can help answer what happens underneath that Python call.
The investigation becomes:
High CPU
↓
Python profiler
↓
Identify hot Python path
↓
Native profiler
↓
Inspect C/native execution
↓
Find actual hotspot
This layered approach is much more useful than trying to solve every performance problem with one profiler.
Using Linux perf
On Linux systems, perf is one of the most useful tools for CPU profiling.
A basic sampling command can look like:
perf record -F 99 -g -- python app.py
The important option here is:
-g
which tells perf to collect call graphs.
After the workload finishes:
perf report
can be used to inspect the collected profile.
For a long-running application, you can also profile a running process rather than starting a new one.
For example:
perf record -F 99 -g -p 12345
where 12345 is the process ID.
This can be useful when the problem occurs only in an environment that is difficult to reproduce locally.
Why the Call Graph Matters
Without a call graph, you might see something like:
native_function 43%
That tells you the function is consuming CPU.
But you may not know why it is being called.
With stack information, you can get something closer to:
api_handler
└── process_request
└── parse_payload
└── native_function
Now you know which application path leads to the hotspot.
That context is often more valuable than the hotspot name itself.
A Python Example
Consider a CPU-heavy application:
def calculate(values):
result = 0
for value in values:
result += value * value
return result
def process():
values = range(10_000_000)
for _ in range(10):
calculate(values)
if __name__ == "__main__":
process()
You could start with Python's statistical profiler.
But if the workload involves native extensions, the native profile can provide additional information.
For example, an application using image processing might look like:
from PIL import Image
def resize_image(path):
image = Image.open(path)
return image.resize((1920, 1080))
The Python function itself is small.
The resizing work may involve native code.
A system-level profile can help identify whether CPU time is being spent in the Python interpreter, the extension, or a lower-level library.
Sampling Frequency
A profiler usually samples the application at a particular frequency.
For example:
perf record -F 99 -g -- python app.py
samples approximately 99 times per second.
The exact frequency you choose depends on the workload.
Higher sampling rates can provide more observations but also increase profiling overhead.
For many investigations, a moderate sampling frequency is enough to identify major CPU hotspots.
You should not automatically use the highest possible frequency.
The goal is to collect enough useful data while keeping the profiling environment reasonably close to the workload you are trying to measure.
What Frame Pointers Do Not Do
Frame pointers do not:
Automatically optimize Python code.
Reduce application CPU usage by themselves.
Replace Python profilers.
Identify database bottlenecks automatically.
Explain network latency.
Fix memory leaks.
Guarantee that every stack will always be available.
They improve the information available to profiling tools.
Think of frame pointers as an infrastructure improvement for observability rather than a performance optimization.
Frame Pointers vs DWARF Unwinding
Native profiling tools can use different techniques to reconstruct call stacks.
One common alternative is DWARF-based unwinding.
DWARF information can provide detailed information about how to unwind a stack, but it can also involve additional processing.
Frame-pointer-based unwinding follows a simpler chain of stack frames.
A simplified comparison is:
Approach | Main idea | Typical benefit |
|---|
Frame pointers | Follow frame-pointer chain | Fast stack walking |
DWARF | Use debug/unwind metadata | More detailed information |
No unwind information | Limited stack reconstruction | Less useful call graphs |
The exact behavior depends on the operating system, compiler, binary, and profiler.
Python 3.15's focus on frame pointers makes common sampling workflows easier on supported builds.
Why This Helps Production Observability
Production systems are often the hardest systems to profile.
A developer may be unable to reproduce:
High CPU at 2:15 PM
on a local machine.
The production workload may involve:
Different traffic patterns
Different data
More concurrent requests
Different native libraries
Different hardware
Different operating-system behavior
Different background jobs
A lightweight sampling profile can provide evidence from the actual environment.
That makes it possible to investigate:
What was the process doing when CPU usage was high?
rather than trying to recreate the exact situation from memory.
Keep Profiling Data Separate From Application Logs
A profiling output file can contain detailed information about application execution.
Treat it as operational data.
Do not automatically upload profiles to public storage.
Depending on the profiling tool and workload, the output may expose:
Use the same security controls you would use for logs and diagnostic dumps.
Profiling a Running Service
Suppose a service is running as:
python server.py
and monitoring shows a sustained CPU spike.
First identify the process:
ps aux | grep server.py
Suppose the process ID is:
12345
You can use:
perf record -F 99 -g -p 12345
After collecting enough samples, stop the recording and inspect:
perf report
Look for repeated call stacks.
For example:
server_request
process_payload
decode_data
native_decode
If the same path dominates the profile, you now have a much stronger starting point for investigation.
Combine Python and Native Profiles
The strongest workflow is often to use both levels.
Start with Python-level profiling:
python -m profiling.sampling run app.py
Find the important Python call path.
Then use a system profiler:
perf record -F 99 -g -- python app.py
Compare the results.
For example:
Python profile:
generate_report()
64%
Native profile:
generate_report()
↓
compression_library
↓
compression_worker()
51%
The second profile explains what the first one could not.
The optimization target is now much clearer.
CPU Profiling Is Not the Same as Performance Monitoring
Profiling and monitoring answer different questions.
Monitoring might tell you:
CPU = 92%
Requests = 1,200/sec
Latency = 420 ms
Profiling tells you:
Which code paths are responsible for the CPU usage?
You need both in a production environment.
A monitoring system can detect the problem.
A profiler can help explain it.
A useful operational flow is:
Metrics
↓
Detect abnormal CPU
↓
Logs / traces
↓
Identify affected workload
↓
Sampling profiler
↓
Find CPU hotspot
↓
Optimize
Frame Pointers and Containers
Containers add another layer to the problem.
The Python process may run inside a container, while profiling tools run from the host or another privileged environment.
You need to consider:
For example, perf may require additional permissions depending on the environment.
If profiling works locally but fails inside a container, the problem may not be Python at all.
Check the container's security and kernel configuration before changing application code.
Production Profiling Should Be Short and Targeted
A common mistake is to leave a profiler running indefinitely.
You usually do not need hours of profiling data to identify a major CPU hotspot.
A better approach is:
Detect problem
↓
Start short profiling session
↓
Collect representative workload
↓
Stop profiler
↓
Analyze
For example, if CPU usage is consistently high for five minutes, collecting a profile during that period can be more useful than profiling an entire day.
The profile should answer a specific question.
Common Mistakes
Looking Only at Python Functions
A Python function may simply be the entry point to expensive native code.
Assuming High CPU Means Python Code Is Bad
The CPU may be consumed by a native dependency or external component.
Ignoring the Call Stack
A hotspot without caller information may not tell you which application path caused it.
Profiling Without a Baseline
Capture normal behavior when possible.
Then compare it with the problematic workload.
Collecting Too Much Data
Large profiles can become difficult to analyze.
Start with a focused profiling period.
Ignoring Production Permissions
System profilers often require additional privileges.
Do not weaken security controls blindly just to get a profile.
When Frame Pointers Are Most Useful
Frame-pointer support becomes particularly valuable when:
Python code calls native extensions.
CPU usage is unexpectedly high.
A Python-level profile is not detailed enough.
You need Linux perf call graphs.
You want better visibility into mixed Python/native workloads.
You are investigating a production-only performance problem.
You need to connect application-level code with native execution.
For a simple Python script that spends all its time in ordinary Python code, you may never notice the difference.
For a large production service, the additional visibility can be significant.
A Practical Investigation Checklist
When investigating high CPU usage in a Python 3.15 service, follow a simple process.
1. Confirm the Problem
Check:
CPU utilization
Request rate
Latency
Error rate
2. Determine Whether CPU Is Actually the Bottleneck
Do not profile CPU simply because an application feels slow.
The problem could be I/O.
3. Use Python-Level Sampling
Find the major Python call paths.
4. Use Native Profiling
If native code is involved, collect a perf profile or another system-level sample.
5. Examine Call Stacks
Look beyond the first function name.
6. Identify the Real Hotspot
Determine whether the cost comes from:
Python code
C extension
Native library
Interpreter overhead
System call
7. Make One Change
Do not rewrite several components at once.
8. Measure Again
Compare the new workload with the original baseline.
This final step tells you whether the change actually helped.
Final Thoughts
Frame pointers may sound like a low-level compiler detail, but they have a practical impact on application debugging.
Modern Python applications are rarely pure Python. They often depend on native libraries for databases, numerical processing, cryptography, compression, image processing, networking, and many other tasks.
When something goes wrong, you need visibility across those boundaries.
Python 3.15's improved frame-pointer support makes that kind of investigation easier on supported platforms and builds. Tools such as perf can use the resulting stack information to produce more useful call graphs.
The important part is not the frame pointer itself.
The important part is the visibility it provides.
When a production process is consuming too much CPU, you want to move from:
"The server is slow."
to:
"This request path reaches this native operation,
which is responsible for most of the sampled CPU time."
That level of evidence gives you a much better basis for deciding what to optimize.
Summary
Python 3.15 improves native profiling support by making frame-pointer-based stack collection more practical on supported builds.
Frame pointers help profilers reconstruct the call stack while a program is running. This is especially useful when Python code calls C extensions or other native libraries, because a Python-level profile may not show what is happening underneath the Python function.
Tools such as Linux perf can use call graphs to connect Python application code with native execution. This makes frame pointers particularly useful when investigating high CPU usage in production.
They do not make applications faster by themselves. Their value is in better observability. Measure the problem first, profile the actual workload, follow the complete call stack, and then make changes based on evidence.
Join the conversation! Your thoughts help the community grow.