Exploring Frozen Collections in .NET 8 With Benchmarking

Introduction

In the realm of multi-threaded programming, ensuring data consistency and thread safety is paramount. To address this challenge, .NET 8 introduces Frozen Collections, immutable data structures designed to provide thread-safe and efficient handling of collections in concurrent environments. These frozen collections offer developers a powerful toolset for building robust and reliable applications. Let's delve into the world of Frozen Collections in .NET 8 and explore their key features, benefits, and usage scenarios.

Frozen Collections is a new .NET 8 feature that can be used to create Dictionaries and Sets for faster read operations when you don’t need to make changes after the creation. In this article, I present how to work with these collections and demonstrate the performance difference when compared with other collections.

The new System.Collections.Frozen namespace includes the collection types FrozenDictionary<TKey,TValue> and FrozenSet<T>. These types are optimized for fast lookup operations. They take a bit more time during the creation, but the read operations are faster when compared with a Dictionary or a Set.

Syntax

To create a FrozenDictionary, you can use the ToFrozenDictionary method:

FrozenDictionary<int, int> frozenDictionary = 
    Enumerable.Range(0, 10).ToFrozenDictionary(key => key);

To create a FrozenSet, you can use the ToFrozenSet method:

FrozenSet<int> frozenSet = Enumerable.Range(0, 10).ToFrozenSet();

Benchmark

For demonstration purposes, I created a series of methods that do three different operations for different collection types: Create the collection, execute the TryGetValue method, and execute a Lookup operation and to run the benchmark, I used the BenchmarkDotNet package.

To execute the benchmarks, you need to set the Visual Studio to Release and execute the project or use the following commands via the command line (When running via the command line, the results will be stored in a folder named BenchmarkDotNet.Artifacts):

// Build:
dotnet build -c Release .\FrozenCollections

// Run:
dotnet FrozenCollections\bin\Release\net8.0\FrozenCollections.dll

In the benchmark result, you can see the following information:

Method: the name of the benchmarked method.
Mean: Arithmetic mean of all measurements.
Error: Half of 99.9% confidence interval.
StdDev: Standard deviation of all measurements. A lower standard deviation indicates more consistent results.
Rank: the rank performance position (from fastest to slowest).

For the benchmark results, smaller values are better.

Create benchmark

In the class below, you can find the methods that create a series of collection types:

[Orderer(SummaryOrderPolicy.FastestToSlowest)]
[MemoryDiagnoser(false)]
[MarkdownExporter]
public class CreateBenchmark
{
    private const int itemsCount = 10_000;

    [Benchmark]
    public void CreateDictionary()
    {
        Dictionary<int, int> dictionary = Enumerable.Range(0, itemsCount).ToDictionary(key => key);
    }

    [Benchmark]
    public void CreateImmutableDictionary()
    {
        ImmutableDictionary<int, int> dictionary = Enumerable.Range(0, itemsCount).ToImmutableDictionary(key => key);
    }

    [Benchmark]
    public void CreateFrozenDictionary()
    {
        FrozenDictionary<int, int> frozenDictionary = Enumerable.Range(0, itemsCount).ToFrozenDictionary(key => key);
    }

    [Benchmark]
    public void CreateList()
    {
        List<int> list = Enumerable.Range(0, itemsCount).ToList();
    }

    [Benchmark]
    public void CreateImmutableList()
    {
        ImmutableList<int> list = Enumerable.Range(0, itemsCount).ToImmutableList();
    }

    [Benchmark]
    public void CreateHashSet()
    {
        HashSet<int> hashSet = Enumerable.Range(0, itemsCount).ToHashSet();
    }

    [Benchmark]
    public void CreateImmutableHashSet()
    {
        ImmutableHashSet<int> hashSet = Enumerable.Range(0, itemsCount).ToImmutableHashSet();
    }

    [Benchmark]
    public void CreateFrozenSet()
    {
        FrozenSet<int> frozenSet = Enumerable.Range(0, itemsCount).ToFrozenSet();
    }
}

This is the benchmark result:

| Method                    | Mean        | Error      | StdDev     | Median      |
|-------------------------- |------------:|-----------:|-----------:|------------:|
| CreateList                |    33.13 ns |   0.689 ns |   0.943 ns |    32.87 ns |
| CreateHashSet             |   597.05 ns |  11.883 ns |  13.684 ns |   590.95 ns |
| CreateDictionary          |   683.53 ns |  13.587 ns |  15.647 ns |   681.10 ns |
| CreateImmutableList       | 1,022.82 ns |  20.180 ns |  39.833 ns | 1,005.03 ns |
| CreateFrozenSet           | 1,604.11 ns |  29.915 ns |  56.187 ns | 1,588.61 ns |
| CreateFrozenDictionary    | 1,805.45 ns |  34.253 ns |  35.175 ns | 1,797.17 ns |
| CreateImmutableHashSet    | 5,647.38 ns | 112.673 ns | 129.754 ns | 5,599.78 ns |
| CreateImmutableDictionary | 7,645.07 ns | 151.149 ns | 235.320 ns | 7,644.09 ns |

The creation of a FrozenDictionary and a FrozenSet are slower when compared with the creation of other collections, but as we are going to see in the next methods, the reading operation will be faster.

TryGetValue benchmark:

In the class below, you can find a series of collection types being initialized as private properties with 100.000 items. Each class method executes the TryGetValue method, searching for the key 500:

[Orderer(SummaryOrderPolicy.FastestToSlowest)]
[MarkdownExporter]
public class TryGetValueBenchmark
{
    private const int itemsCount = 100_000;
    private const int keyToFind = 500;

    private Dictionary<int, int> _dictionary = Enumerable.Range(0, itemsCount).ToDictionary(key => key);
    private ImmutableDictionary<int, int> _immutableDictionary = Enumerable.Range(0, itemsCount).ToImmutableDictionary(key => key);
    
    private HashSet<int> _hashSet = Enumerable.Range(0, itemsCount).ToHashSet();
    private ImmutableHashSet<int> _immutableHashSet = Enumerable.Range(0, itemsCount).ToImmutableHashSet();
    
    private FrozenDictionary<int, int> _frozenDictionary = Enumerable.Range(0, itemsCount).ToFrozenDictionary(key => key);
    private FrozenSet<int> _frozenSet = Enumerable.Range(0, itemsCount).ToFrozenSet();

    [Benchmark]
    public void TryGetValueDictionary()
    {
        _dictionary.TryGetValue(keyToFind, out _);
    }

    [Benchmark]
    public void TryGetValueImmutableDictionary()
    {
        _immutableDictionary.TryGetValue(keyToFind, out _);
    }

    [Benchmark]
    public void TryGetValueFrozenDictionary()
    {
        _frozenDictionary.TryGetValue(keyToFind, out _);
    }

    [Benchmark]
    public void TryGetValueHashSet()
    {
        _hashSet.TryGetValue(keyToFind, out _);
    }

    [Benchmark]
    public void TryGetValueImmutableHashSet()
    {
        _immutableHashSet.TryGetValue(keyToFind, out _);
    }

    [Benchmark]
    public void TryGetValueFrozenSet()
    {
        _frozenSet.TryGetValue(keyToFind, out _);
    }
}

This is the benchmark result:

| Method                         | Mean      | Error     | StdDev    |
|------------------------------- |----------:|----------:|----------:|
| TryGetValueFrozenDictionary    |  1.871 ns | 0.1156 ns | 0.3371 ns |
| TryGetValueFrozenSet           |  3.372 ns | 0.1231 ns | 0.3493 ns |
| TryGetValueDictionary          |  3.781 ns | 0.1381 ns | 0.4006 ns |
| TryGetValueHashSet             |  4.466 ns | 0.1464 ns | 0.4270 ns |
| TryGetValueImmutableHashSet    | 18.406 ns | 0.6214 ns | 1.7930 ns |
| TryGetValueImmutableDictionary | 27.926 ns | 1.0242 ns | 2.9713 ns |

Executing the TryGetValue in a FrozenDictionary and in a FrozenSet were the fastest operations when compared with these other collections.

Lookup Benchmark

In the class below, you can find a series of collection types being initialized as private properties with 100.000 items. On each class method, there is a loop (using a for), and for each iteration, the method ContainsKey is executed:

[Orderer(SummaryOrderPolicy.FastestToSlowest)]
[MarkdownExporter]
public class LookupBenchmark
{
    private const int itemsCount = 100_000;
    private const int iterations = 1_000;

    private Dictionary<int, int> _dictionary = Enumerable.Range(0, itemsCount).ToDictionary(key => key);
    private ImmutableDictionary<int, int> _immutableDictionary = Enumerable.Range(0, itemsCount).ToImmutableDictionary(key => key);

    private List<int> _list = Enumerable.Range(0, itemsCount).ToList();
    private ImmutableList<int> _immutableList = Enumerable.Range(0, itemsCount).ToImmutableList();

    private HashSet<int> _hashSet = Enumerable.Range(0, itemsCount).ToHashSet();
    private ImmutableHashSet<int> _immutableHashSet = Enumerable.Range(0, itemsCount).ToImmutableHashSet();

    private FrozenDictionary<int, int> _frozenDictionary = Enumerable.Range(0, itemsCount).ToFrozenDictionary(key => key);
    private FrozenSet<int> _frozenSet = Enumerable.Range(0, itemsCount).ToFrozenSet();

    [Benchmark]
    public void LookupDictionary()
    {
        for (int i = 0; i < iterations; i++)
            _ = _dictionary.ContainsKey(i);
    }

    [Benchmark]
    public void LookupImmutableDictionary()
    {
        for (int i = 0; i < iterations; i++)
            _ = _immutableDictionary.ContainsKey(i);
    }

    [Benchmark]
    public void LookupFrozenDictionary()
    {
        for (var i = 0; i < iterations; i++)
            _ = _frozenDictionary.ContainsKey(i);
    }

    [Benchmark]
    public void LookupList()
    {
        for (int i = 0; i < iterations; i++)
            _ = _list.Contains(i);
    }

    [Benchmark]
    public void LookupImmutableList()
    {
        for (int i = 0; i < iterations; i++)
            _ = _immutableList.Contains(i);
    }

    [Benchmark]
    public void LookupHashSet()
    {
        for (var i = 0; i < iterations; i++)
            _ = _hashSet.Contains(i);
    }

    [Benchmark]
    public void LookupImmutableHashSet()
    {
        for (var i = 0; i < iterations; i++)
            _ = _immutableHashSet.Contains(i);
    }

    [Benchmark]
    public void LookupFrozenSet()
    {
        for (var i = 0; i < iterations; i++)
            _ = _frozenSet.Contains(i);
    }
}

This is the benchmark result:

| Method                    | Mean         | Error        | StdDev      |
|-------------------------- |-------------:|-------------:|------------:|
| LookupFrozenSet           |     2.308 us |     5.256 us |   0.2881 us |
| LookupFrozenDictionary    |     2.353 us |     3.256 us |   0.1785 us |
| LookupHashSet             |     3.604 us |     2.191 us |   0.1201 us |
| LookupDictionary          |     3.945 us |     6.932 us |   0.3800 us |
| LookupImmutableHashSet    |    32.816 us |     9.960 us |   0.5459 us |
| LookupList                |    38.407 us |    12.993 us |   0.7122 us |
| LookupImmutableDictionary |    44.991 us |    11.125 us |   0.6098 us |
| LookupImmutableList       | 1,523.117 us | 1,922.508 us | 105.3791 us |

As expected, the lookup in a FrozendDictionary and in a FrozenSet were the fastest when compared with the other collections.

Benefits of Frozen Collections:

Thread Safety: Frozen Collections are inherently thread-safe, making them ideal for use in multi-threaded applications. Since their contents cannot be modified after creation, there is no risk of data corruption or race conditions due to concurrent access.
Predictable Behavior: Immutable collections exhibit predictable behavior, as their contents remain constant throughout their lifetime. This predictability simplifies reasoning about program behavior and reduces the likelihood of bugs related to mutable state.
Efficiency: Despite their immutability, Frozen Collections are designed to be efficient in terms of memory usage and performance. They leverage efficient internal data structures and algorithms to provide fast read access and minimal memory overhead.
Functional Programming: Frozen Collections align with the principles of functional programming, emphasizing immutability and side-effect-free operations. This makes them well-suited for functional programming paradigms and immutable data modeling.

Usage Scenarios

Configuration Data: Frozen Collections are ideal for storing configuration data that remains constant throughout the application's execution. Since configuration data is typically read-only, using immutable collections ensures its integrity and thread safety.
Caching: Immutable collections are well-suited for caching scenarios where the cached data does not change frequently. By using Frozen Collections for caching, developers can eliminate the need for synchronization mechanisms and ensure consistent access to cached data across multiple threads.
Functional Programming: Functional programming techniques, such as pure functions and immutable data structures, are gaining popularity in .NET development. Frozen Collections provide a natural fit for functional programming paradigms, enabling developers to write concise and predictable code.

Conclusion

Frozen Collections are collections optimized for situations where you have collections that will be frequently accessed, and you do not need to change the keys and values after creating. These collections are a bit slower during the creation, but reading operations are faster.