Compression is often introduced into an application for one simple reason: reduce the amount of data that needs to be stored or transferred.
The engineering trade-off is more complicated.
Compression consumes CPU and can introduce additional memory and latency costs. A highly compressed payload may save network bandwidth while requiring substantially more work to produce. A faster compression configuration may reduce CPU usage while producing a larger output.
.NET 11 adds native Zstandard support to System.IO.Compression, alongside existing compression technologies such as GZip, Brotli, Deflate, and ZLib. The new APIs include ZstandardStream, ZstandardEncoder, ZstandardDecoder, compression options, and dictionary support.
That makes Zstandard an interesting option for .NET applications handling:
But adding another compression algorithm does not automatically make an application faster.
The useful question is:
How does Zstandard behave for the specific data, CPU budget, and throughput requirements of an application?
What Is Zstandard?
Zstandard, commonly called zstd, is a general-purpose lossless compression format developed by Facebook/Meta.
Its design focuses on providing configurable compression while maintaining fast decompression.
The practical trade-off looks like this:
Less Compression
|
v
More Speed
|
v
Larger Output
versus:
More Compression
|
v
More CPU Work
|
v
Smaller Output
The correct point on this curve depends on the workload.
For a network-heavy application, smaller payloads may be valuable.
For a CPU-constrained service, compression speed may be more important.
Zstandard Support in .NET 11
.NET 11 exposes Zstandard through System.IO.Compression.
A stream-based example is:
using System.IO.Compression;
await using var output =
File.Create("data.zst");
await using var zstd =
new ZstandardStream(
output,
CompressionLevel.Fastest);
await input.CopyToAsync(zstd);
The API follows the same general stream model used by other .NET compression implementations. Microsoft documents constructors for compression and decompression as well as configurable Zstandard options.
This is important for existing .NET applications because developers can use the familiar Stream abstraction rather than introducing a completely separate compression programming model.
Why Benchmark Compression?
Compression performance has multiple dimensions.
A useful benchmark should measure at least:
Compression speed
Decompression speed
Compressed size
CPU consumption
Memory usage
Allocation behavior
For example:
Input
|
v
Compression
|
+---- CPU
+---- Memory
+---- Time
|
v
Compressed Output
Optimizing only one measurement can produce the wrong architecture.
Compression Ratio vs Compression Speed
One of the most important metrics is the compression ratio.
A simple definition is:
Compression Ratio =
Uncompressed Size / Compressed Size
For example, if:
Uncompressed = 10 MB
Compressed = 2 MB
then:
Compression Ratio = 5
The higher the ratio, the smaller the resulting compressed representation.
However, ratio alone does not tell you whether the configuration is suitable.
Consider:
Configuration A
Small output
High CPU cost
Configuration B
Larger output
Low CPU cost
If the application is CPU-bound, B may be preferable.
If the application is bandwidth-constrained, A may be preferable.
Compare Zstandard With Existing Algorithms
A benchmark should not evaluate Zstandard in isolation.
.NET applications already have several compression choices.
| Algorithm | .NET API | Typical Consideration |
|---|
| GZip | GZipStream | Broad compatibility |
| Deflate | DeflateStream | General-purpose compression |
| Brotli | BrotliStream | Strong web-oriented use cases |
| ZLib | ZLibStream | zlib-compatible workloads |
| Zstandard | ZstandardStream | New high-performance general-purpose option |
The correct choice depends on the application and interoperability requirements.
A benchmark should therefore compare algorithms using the same input data and equivalent test conditions.
Create a Representative Dataset
Compression results are heavily influenced by the input.
Do not benchmark only:
Hello world
or:
AAAAAAAAAAAAAAAAAAAA
Highly repetitive data can produce results that are irrelevant to real applications.
Use representative datasets such as:
JSON
CSV
Logs
XML
Binary records
Application traces
Database exports
For example:
{
"id": 1001,
"customer": "Alice",
"status": "active",
"orders": [
{
"id": 5001,
"amount": 125.50
}
]
}
A real application should use production-shaped data where possible.
Create Multiple Dataset Sizes
One dataset size is not enough.
Use several workload sizes:
| Dataset | Purpose |
|---|
| Small | Per-operation overhead |
| Medium | Typical payload |
| Large | Throughput |
| Very large | Sustained processing |
For example:
100 KB
1 MB
10 MB
100 MB
The actual sizes should reflect the application being tested.
Do not assume that an algorithm's behavior at 100 KB represents its behavior on a 1 GB export.
Benchmark With BenchmarkDotNet
Create a dedicated benchmark project:
dotnet new console -n CompressionBenchmarks
cd CompressionBenchmarks
dotnet add package BenchmarkDotNet
Then create the benchmark class:
using BenchmarkDotNet.Attributes;
[MemoryDiagnoser]
public class CompressionBenchmark
{
private byte[] _data = [];
[Params(100_000, 1_000_000, 10_000_000)]
public int Size { get; set; }
[GlobalSetup]
public void Setup()
{
_data = GenerateData(Size);
}
private static byte[] GenerateData(int size)
{
var data = new byte[size];
Random.Shared.NextBytes(data);
return data;
}
}
This creates a baseline for testing different payload sizes.
However, random bytes are generally difficult to compress.
For a realistic compression experiment, generate datasets with characteristics similar to the application's actual data.
Benchmark Zstandard Compression
A basic benchmark can use ZstandardStream:
[Benchmark]
public byte[] Zstandard()
{
using var output =
new MemoryStream();
using (var zstd =
new ZstandardStream(
output,
CompressionLevel.Fastest))
{
zstd.Write(_data);
}
return output.ToArray();
}
The benchmark measures the complete operation:
Input
|
v
Zstandard
|
v
MemoryStream
|
v
byte[]
This is useful for an initial experiment.
However, returning the resulting byte[] also introduces a copy and allocation.
For a precise production study, measure the exact API pattern used by the application.
Benchmark GZip
Create a comparable benchmark:
[Benchmark]
public byte[] Gzip()
{
using var output =
new MemoryStream();
using (var gzip =
new GZipStream(
output,
CompressionLevel.Fastest))
{
gzip.Write(_data);
}
return output.ToArray();
}
The important point is consistency.
Use:
Do not change several variables between tests.
Benchmark Brotli
You can also include Brotli:
[Benchmark]
public byte[] Brotli()
{
using var output =
new MemoryStream();
using (var brotli =
new BrotliStream(
output,
CompressionLevel.Fastest))
{
brotli.Write(_data);
}
return output.ToArray();
}
This provides a three-way comparison:
Zstandard
vs
GZip
vs
Brotli
The result should be interpreted as workload-specific rather than as a universal ranking.
Measure Compressed Size
Execution time alone is insufficient.
Capture the output size:
private static long CompressWithZstd(
byte[] data)
{
using var output =
new MemoryStream();
using (var zstd =
new ZstandardStream(
output,
CompressionLevel.Fastest))
{
zstd.Write(data);
}
return output.Length;
}
Then calculate:
double ratio =
(double)data.Length /
compressedLength;
This gives the compression ratio.
Measure Compression Throughput
A useful metric is:
Throughput =
Input Bytes / Compression Time
For example:
Throughput = MB processed per second
BenchmarkDotNet can provide execution time, after which throughput can be derived from the known input size.
Do not calculate throughput from one manual stopwatch measurement.
Use multiple benchmark iterations.
Benchmark Decompression Separately
Compression and decompression have different characteristics.
First create a compressed payload:
private byte[] Compress()
{
using var output =
new MemoryStream();
using (var zstd =
new ZstandardStream(
output,
CompressionLevel.Fastest))
{
zstd.Write(_data);
}
return output.ToArray();
}
Then benchmark decompression:
[Benchmark]
public byte[] ZstandardDecompress()
{
using var input =
new MemoryStream(_compressed);
using var zstd =
new ZstandardStream(
input,
CompressionMode.Decompress);
using var output =
new MemoryStream();
zstd.CopyTo(output);
return output.ToArray();
}
This separates:
Compression Cost
from:
Decompression Cost
That distinction matters for client/server systems.
Why Decompression Matters
Suppose a server compresses every response but thousands of clients decompress them.
The cost distribution becomes:
Server
Compression
|
v
Network
|
v
Many Clients
Decompression
A compression strategy should therefore consider both ends of the pipeline.
For distributed systems, measure:
Server CPU
+
Client CPU
+
Network Bytes
+
End-to-End Latency
Benchmark Streaming
Compression is often used with streams rather than complete byte arrays.
For example:
await using var output =
File.Create("archive.zst");
await using var zstd =
new ZstandardStream(
output,
CompressionLevel.Fastest);
await input.CopyToAsync(zstd);
This is fundamentally different from:
byte[] data =
await File.ReadAllBytesAsync(
"large-file.dat");
followed by compression.
The streaming design can avoid requiring the entire input in memory.
Benchmark both approaches when the application has a choice.
Measure Memory Usage
Add:
[MemoryDiagnoser]
to the benchmark.
Then examine:
Allocated bytes
Gen 0 collections
Gen 1 collections
Gen 2 collections
For high-throughput services, allocation behavior can matter as much as raw compression time.
A compression implementation that is slightly faster but creates significantly more temporary allocations may behave differently under sustained concurrency.
Avoid Benchmarking Only MemoryStream
MemoryStream is useful because it removes storage latency.
But production applications may use:
FileStream
NetworkStream
Pipe
HTTP response stream
Cloud storage stream
After measuring compression itself, benchmark the actual stream used by the application.
The test progression can be:
Phase 1
MemoryStream
|
v
Algorithm cost
Phase 2
FileStream
|
v
Algorithm + disk
Phase 3
Network
|
v
Algorithm + network
This helps separate bottlenecks.
Test Compression Levels
Compression APIs expose a CompressionLevel abstraction, and Zstandard also provides more detailed configuration through ZstandardCompressionOptions. Microsoft documents constructors for Zstandard streams using both CompressionLevel and explicit Zstandard options.
Start with the common levels:
CompressionLevel.Fastest
CompressionLevel.Optimal
Then test more detailed Zstandard settings if the application needs them.
Do not assume Optimal is the right production setting.
The correct configuration depends on:
CPU budget
Payload size
Bandwidth
Latency target
Compression ratio
Request frequency
Build a Compression Matrix
A useful benchmark table is:
| Algorithm | Level | Time | Output Size | Ratio | Allocations |
|---|
| GZip | Fastest | Measure | Measure | Calculate | Measure |
| Brotli | Fastest | Measure | Measure | Calculate | Measure |
| Zstandard | Fastest | Measure | Measure | Calculate | Measure |
| GZip | Optimal | Measure | Measure | Calculate | Measure |
| Brotli | Optimal | Measure | Measure | Calculate | Measure |
| Zstandard | Higher setting | Measure | Measure | Calculate | Measure |
The values should come from the actual benchmark environment.
Zstandard Dictionaries
Zstandard also supports dictionaries.
The concept is useful when many small payloads share common structure.
For example:
Payload 1
{
"customerId": ...
"transactionId": ...
}
Payload 2
{
"customerId": ...
"transactionId": ...
}
A dictionary can contain patterns that frequently occur in the data.
.NET 11 exposes ZstandardDictionary and related encoder/decoder APIs.
Conceptually:
Training Data
|
v
Dictionary
|
+--------+
| |
v v
Encoder Decoder
| |
v v
Payload Payload
Dictionary-based compression should be benchmarked separately from ordinary Zstandard compression.
When Dictionaries May Help
Dictionaries are particularly interesting for:
Small messages
Repeated schemas
Protocol records
RPC payloads
Structured logs
Similar JSON objects
But dictionary effectiveness depends strongly on whether the dictionary represents the actual production data.
A dictionary generated from unrelated data may provide little benefit.
Benchmark Dictionary-Based Compression
A simplified example using the explicit encoder API could look like:
using System.IO.Compression;
using var dictionary =
new ZstandardDictionary(
dictionaryBytes);
using var encoder =
new ZstandardEncoder(
dictionary);
The exact encoder API should be used according to the .NET 11 SDK version being tested because the Zstandard API is part of the evolving .NET 11 surface. Microsoft's current API documentation lists dictionary-aware encoder and stream constructors.
For a production benchmark, record:
Dictionary size
Training dataset
Payload size
Compression ratio
Compression time
Decompression time
Benchmark Highly Compressible Data
Create a dataset with repeated structures.
For example:
var text = string.Join(
Environment.NewLine,
Enumerable.Repeat(
"INFO User authenticated successfully",
100_000));
Convert it to bytes:
byte[] data =
Encoding.UTF8.GetBytes(text);
This represents a highly repetitive workload.
It is useful for testing compression effectiveness, but it should not be the only dataset.
Benchmark Low-Compressibility Data
Use data that already has high entropy.
For example:
var data =
RandomNumberGenerator.GetBytes(
10 * 1024 * 1024);
This helps answer a practical question:
How much CPU does compression consume when the input does not compress well?
This can be important for applications processing encrypted or already-compressed data.
Do Not Compress Everything
Compression is not always beneficial.
Avoid blindly compressing:
JPEG
PNG
MP4
ZIP
GZIP
Zstandard
Encrypted payloads
Many of these formats are already compressed or have high entropy.
Compression can add CPU cost without significantly reducing size.
A better strategy is to understand the data first.
Compression and Encryption
Encryption should generally happen after compression when both operations are required:
Original Data
|
v
Compression
|
v
Encryption
|
v
Network / Storage
Compressing encrypted data is generally ineffective because encrypted output should not retain the predictable structure compression algorithms depend on.
For sensitive data, also consider whether compression introduces side-channel risks in the specific protocol design.
Do not add compression to security-sensitive protocols without reviewing the threat model.
HTTP Compression
For HTTP APIs, compression is often negotiated between the client and server.
Conceptually:
Accept-Encoding: zstd, br, gzip
The server can select a supported encoding.
The exact HTTP framework and infrastructure configuration determines whether Zstandard can be negotiated and emitted automatically.
Do not assume that adding ZstandardStream automatically enables HTTP content encoding.
A custom implementation may require explicit middleware or server configuration.
Storage Workloads
Compression can reduce storage consumption.
For example:
Raw Data
|
v
Zstandard
|
v
Object Storage
This can be useful for:
But storage cost is only one variable.
Measure:
Compression CPU
+
Storage reduction
+
Upload time
+
Download time
+
Decompression time
A smaller file is not automatically a better architecture if compression consumes excessive CPU during every request.
Common Benchmarking Mistakes
Using Random Data Only
Random data often compresses poorly.
You need representative structured datasets.
Using Highly Repetitive Data Only
Highly repetitive data can make compression appear unrealistically effective.
Use multiple datasets.
Measuring Only Compression Time
Measure output size and decompression as well.
Including Disk I/O Without Realizing It
Use MemoryStream to isolate algorithm cost.
Then separately benchmark real storage.
Using Different Input Data Per Algorithm
Every algorithm must receive identical input.
Ignoring Allocations
High-throughput services can be affected by allocation and GC pressure.
Testing One Payload Size
Small messages and large files can behave very differently.
Publishing Unverified Benchmarks
Do not report specific throughput, CPU, or compression-ratio numbers unless they were actually measured.
Troubleshooting
Zstandard Output Is Larger Than Expected
Check the input.
Already-compressed or high-entropy data may not benefit much from additional compression.
Also verify the selected compression settings.
Compression Uses Too Much CPU
Test a faster compression configuration.
Then measure whether the larger output causes an acceptable network or storage trade-off.
Decompression Is the Bottleneck
Benchmark the consumer separately.
The fastest compressor is not automatically the best choice if clients have constrained CPU resources.
Memory Usage Is High
Check whether the application buffers the complete input and output.
Prefer streaming APIs when the workload supports them.
Dictionary Compression Does Not Improve Results
Verify that the dictionary was trained on representative data.
Dictionary effectiveness depends on similarity between training and production payloads.
Production Decision Framework
A compression decision can be expressed as:
Is bandwidth expensive?
|
+-----+-----+
Yes No
| |
v v
Evaluate ratio Is CPU limited?
|
+----+----+
Yes No
| |
v v
Prefer speed Benchmark
This is only a starting point.
Actual decisions should use measurements.
For example:
Scenario A
CPU expensive
Network cheap
|
v
Favor faster compression
Scenario B
CPU available
Network expensive
|
v
Favor stronger compression
Scenario C
Small repeated messages
|
v
Evaluate dictionaries
Scenario D
Already compressed data
|
v
Consider skipping compression
Best Practices
Benchmark Zstandard against the algorithms already used by the application.
Use production-shaped datasets.
Test multiple payload sizes.
Measure compression and decompression separately.
Measure compressed size and compression ratio.
Measure allocations and memory usage.
Use MemoryStream to isolate algorithm performance.
Then test the actual production stream.
Evaluate compression levels independently.
Use dictionaries only when the workload justifies them.
Do not compress already-compressed data blindly.
Consider both CPU and network/storage costs.
Document hardware, runtime, input data, and configuration.
Do not publish fabricated benchmark numbers.
Frequently Asked Questions
Does .NET 11 support Zstandard natively?
Yes. .NET 11 adds Zstandard APIs to System.IO.Compression, including ZstandardStream and ZstandardEncoder.
Is Zstandard faster than GZip?
There is no universal result that applies to every payload and configuration.
Compression level, input data, hardware, implementation, and workload all affect the outcome.
Benchmark your actual data.
Is Zstandard better than Brotli?
Neither is universally better.
The appropriate choice depends on compression ratio, compression speed, decompression speed, compatibility, and application requirements.
Should I use Zstandard for every API response?
No.
Evaluate payload size, client support, CPU cost, network savings, and infrastructure compatibility before enabling it broadly.
Are Zstandard dictionaries useful?
They can be useful for small, structurally similar messages.
Their effectiveness depends on the quality of the dictionary and similarity between training and production data.
Should I benchmark compression with real files?
Yes.
If your production workload processes files, benchmark representative files.
Also use MemoryStream when you need to isolate compression cost from storage performance.
Conclusion
Zstandard support in .NET 11 gives developers another native compression option without requiring a separate compression abstraction. ZstandardStream, ZstandardEncoder, dictionaries, and related configuration APIs are now part of System.IO.Compression.
But the existence of a new compression algorithm does not answer the architectural question.
The correct choice depends on the complete workload:
Input Data
|
v
Compression
|
+-- CPU
+-- Memory
+-- Time
|
v
Compressed Data
|
+-- Storage
+-- Network
|
v
Decompression
|
v
Consumer
A useful benchmark therefore measures more than compression ratio.
It should compare:
Compression Time
+
Decompression Time
+
Output Size
+
CPU
+
Memory
+
Allocations
+
End-to-End Cost
The most important principle is:
Do not choose a compression algorithm because it produces the smallest file or the fastest benchmark in isolation. Choose it because its compression, decompression, CPU, memory, and storage/network characteristics fit the actual workload.
.NET 11's native Zstandard APIs make that evaluation easier to perform directly within the .NET ecosystem. The next step for any production team should be a controlled benchmark using its own representative data rather than relying on generic algorithm rankings.