.net, C# and programming (EN)

Collections search

Sergey Rogatnev — Thu, 09 Apr 2026 00:00:00 GMT

During interviews, we often hear — or say ourselves — that searching in an array is slower than in a hashtable. Some might even recall that array search has linear complexity, or O(n), while a hash table has constant complexity, O(1). But does this hold true in practice? What if there are situations where searching in an array turns out to be faster? Let's not rush to conclusions.

So, let's compare searching in an array and a HashSet on small collections, using different types: a primitive int and a reference type string.

Full benchmark code. Click to expand.

using System.Collections.Generic;
using System.Linq;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;

namespace Benchmarks
{
[ShortRunJob(RuntimeMoniker.Net60)]
[ShortRunJob(RuntimeMoniker.Net80)]
[ShortRunJob(RuntimeMoniker.Net10_0)]
[HideColumns("Job", "Error", "StdDev", "Gen0", "Median")]
[DisassemblyDiagnoser(printSource: true, maxDepth: 2)]
public class HashSetContainsBenchmark
{
[Params(1, 10, 100, 1000)]
public int Size { get; set; }

        private int[] _arrayInt;
        private HashSet<int> _hashSetInt;
        private int _missingInt;

        private string[] _arrayString;
        private HashSet<string> _hashSetString;
        private string _missingString;

        [GlobalSetup]
        public void Setup()
        {
            // --- Int Setup ---
            var intData = Enumerable.Range(0, Size).ToArray();
            _arrayInt = intData;
            _hashSetInt = new HashSet<int>(intData);
            _missingInt = Size + 1;

            // --- String Setup ---
            var stringData = Enumerable.Range(0, Size).Select(i => $"value-{i}").ToArray();
            _arrayString = stringData;
            _hashSetString = new HashSet<string>(stringData);
            _missingString = "value-a";
        }

        // --- Int Benchmarks ---
        [Benchmark(Description = "Array.Contains (Int)")]
        public bool Int_Array() => _arrayInt.Contains(_missingInt);

        [Benchmark(Description = "HashSet.Contains (Int)")]
        public bool Int_HashSet() => _hashSetInt.Contains(_missingInt);

        // --- String Benchmarks ---
        [Benchmark(Description = "Array.Contains (String)")]
        public bool String_Array() => _arrayString.Contains(_missingString);

        [Benchmark(Description = "HashSet.Contains (String)")]
        public bool String_HashSet() => _hashSetString.Contains(_missingString);
    }
}

| Method                      | Runtime   | Size |     Mean | Code Size |
|-----------------------------|-----------|------|---------:|----------:|
|  Array.Contains (Int)       | .NET 6.0  | 1    | 3.443 ns |      50 B |
|  HashSet.Contains (Int)     | .NET 6.0  | 1    | 2.052 ns |     447 B |
|  Array.Contains (String)    | .NET 6.0  | 1    | 8.621 ns |      57 B |
|  HashSet.Contains (String)  | .NET 6.0  | 1    | 4.741 ns |     737 B |
|  Array.Contains (Int)       | .NET 8.0  | 1    | 2.676 ns |      40 B |
|  HashSet.Contains (Int)     | .NET 8.0  | 1    | 1.954 ns |     385 B |
|  Array.Contains (String)    | .NET 8.0  | 1    | 9.639 ns |      51 B |
|  HashSet.Contains (String)  | .NET 8.0  | 1    | 4.599 ns |     351 B |
|  Array.Contains (Int)       | .NET 10.0 | 1    | 1.193 ns |     669 B |
|  HashSet.Contains (Int)     | .NET 10.0 | 1    | 1.666 ns |     377 B |
|  Array.Contains (String)    | .NET 10.0 | 1    | 2.285 ns |     548 B |
|  HashSet.Contains (String)  | .NET 10.0 | 1    | 3.698 ns |     233 B |

Intuitively, it seems that iterating over one element in an array should be faster than the more complex mechanism of hash-based searching. As we can see, even when searching in a collection of a single int, the array loses to HashSet up until .NET 10:

And even then, the advantage is quite small.

We observe the same picture for strings:

Notice how in .NET 10, array search for strings (2.285 ns) even beats HashSet (3.698 ns) — we'll explain why later.

Alright, collections of one element are weird. Let's take the larger size — 10 elements. Surely, the advantage of HashSet should become more obvious there:

| Method                      | Runtime   | Size |      Mean | Code Size |
|-----------------------------|-----------|------|----------:|----------:|
|  Array.Contains (Int)       | .NET 6.0  | 10   |  4.059 ns |      50 B |
|  HashSet.Contains (Int)     | .NET 6.0  | 10   |  2.901 ns |     447 B |
|  Array.Contains (String)    | .NET 6.0  | 10   | 33.465 ns |      57 B |
|  HashSet.Contains (String)  | .NET 6.0  | 10   |  5.807 ns |     737 B |
|  Array.Contains (Int)       | .NET 8.0  | 10   |  3.137 ns |      40 B |
|  HashSet.Contains (Int)     | .NET 8.0  | 10   |  2.497 ns |     397 B |
|  Array.Contains (String)    | .NET 8.0  | 10   | 31.164 ns |      51 B |
|  HashSet.Contains (String)  | .NET 8.0  | 10   |  5.937 ns |     377 B |
|  Array.Contains (Int)       | .NET 10.0 | 10   |  1.639 ns |     729 B |
|  HashSet.Contains (Int)     | .NET 10.0 | 10   |  2.230 ns |     377 B |
|  Array.Contains (String)    | .NET 10.0 | 10   | 18.063 ns |     548 B |
|  HashSet.Contains (String)  | .NET 10.0 | 10   |  4.583 ns |     233 B |

Do you see the oddities already? Here they are:

The search time in HashSet remains roughly the same regardless of the framework version, but array search keeps getting better and better. Moreover, in .NET 10, the array overtakes HashSet! How come? Does array search also work in constant time now? It seems we are on the verge of a grand discovery. Let's test our hypothesis on collections of 100 elements:

| Method                      | Runtime   | Size |       Mean | Code Size |
|-----------------------------|-----------|------|-----------:|----------:|
|  Array.Contains (Int)       | .NET 6.0  | 100  |  13.073 ns |      50 B |
|  HashSet.Contains (Int)     | .NET 6.0  | 100  |   2.116 ns |     447 B |
|  Array.Contains (String)    | .NET 6.0  | 100  | 168.924 ns |      57 B |
|  HashSet.Contains (String)  | .NET 6.0  | 100  |   8.306 ns |     737 B |
|  Array.Contains (Int)       | .NET 8.0  | 100  |   4.381 ns |      40 B |
|  HashSet.Contains (Int)     | .NET 8.0  | 100  |   1.857 ns |     385 B |
|  Array.Contains (String)    | .NET 8.0  | 100  |  93.214 ns |      51 B |
|  HashSet.Contains (String)  | .NET 8.0  | 100  |   7.938 ns |     377 B |
|  Array.Contains (Int)       | .NET 10.0 | 100  |   3.401 ns |     736 B |
|  HashSet.Contains (Int)     | .NET 10.0 | 100  |   1.774 ns |     377 B |
|  Array.Contains (String)    | .NET 10.0 | 100  |  71.121 ns |     562 B |
|  HashSet.Contains (String)  | .NET 10.0 | 100  |   6.806 ns |     233 B |

Unfortunately, a miracle didn't happen. Now the array consistently loses to HashSet in searching. But the difference for the int type is still not that significant:

Let's look at the growth of search time in an array relative to its size. For clarity, let's add an array of 1000 elements:

As we can see, only in .NET 6 do we begin to observe linear growth after 100 elements. And in versions 8 and 10, the growth, if not logarithmic, is heavily smoothed out.

It's time to dive deeper, specifically into exactly how array search is performed. Calling the Contains method on an array ultimately leads to calling the static Array.IndexOf method. But this is where the differences begin.

.NET 6

Using some loop-unrolling optimizations and fast access to array elements, .NET checks each element for equality:

...
while (length >= 8)
{
    length -= 8;

    if (value.Equals(Unsafe.Add(ref searchSpace, index)))
        goto Found;
    if (value.Equals(Unsafe.Add(ref searchSpace, index + 1)))
        goto Found1;
    if (value.Equals(Unsafe.Add(ref searchSpace, index + 2)))
        goto Found2;
    if (value.Equals(Unsafe.Add(ref searchSpace, index + 3)))
        goto Found3;
    if (value.Equals(Unsafe.Add(ref searchSpace, index + 4)))
        goto Found4;
    if (value.Equals(Unsafe.Add(ref searchSpace, index + 5)))
        goto Found5;
    if (value.Equals(Unsafe.Add(ref searchSpace, index + 6)))
        goto Found6;
    if (value.Equals(Unsafe.Add(ref searchSpace, index + 7)))
        goto Found7;

    index += 8;
}
// the remainder of the array (less than 8 elements) is checked next

As you can see, we sequentially check each element and get our honest O(N) complexity. Nothing unusual so far.

.NET 8

Starting with this version, the situation changes drastically. The SpanHelpers.NonPackedIndexOfValueType method comes into play, which, in addition to loop unrolling, contains a powerful hardware optimization—vectorization:

if (Vector512.IsHardwareAccelerated && length >= Vector512<TValue>.Count)
{
    Vector512<TValue> left = Vector512.Create<TValue>(value);
    //...
}
else if (Vector256.IsHardwareAccelerated && length >= Vector256<TValue>.Count)
{
    Vector256<TValue> left = Vector256.Create<TValue>(value);
    //...
}
else
{
    Vector128<TValue> left = Vector128.Create<TValue>(value);
    //...
}

We can see the use of new types: Vector128, Vector256, Vector512 — collections for working with vector data. And pay attention to the IsHardwareAccelerated property being checked in every condition — this is where the magic lies. It determines whether your processor supports working with vectors of 128, 256, and 512 bits in length, respectively.

Using the Vector512 class allows operating on 16 int array elements at once. 512 bits = 64 bytes, meaning the vector holds 16 values of type int at 4 bytes each.

Unfortunately, we now have to dive into assembler to better understand how this works:

cmp r8d,4
jl near ptr M01_L03          ; If elements < 4, process one by one
cmp r8d,10h                  ; 0x10 = 16 decimal
jl near ptr M01_L18          ; If elements < 16, switch to processing using a 256-bit register (ymm)
vpbroadcastd zmm0,edx        ; Load the target value edx into the 512-bit register zmm0. Now zmm0 looks like [edx, edx, edx, ..., edx]
...
vmovups zmm1,[rcx]           ; Load the first 16 elements from memory into the 512-bit register zmm1
vpcmpeqd k1,zmm1,zmm0        ; Compare zmm1 and zmm0 for equality
kortestw k1,k1               ; Check if there is at least one match
...

vpcmpeqd is an instruction that compares two vectors element-by-element and saves the result into a special register k1. If the fifth element matches, the fifth bit in k1 will be set to 1.
kortestw checks this bitmask. If it equals 0, it means the target number is not present in the first 16 elements.

At first glance, it looks complicated, but essentially, this is a continuation of the loop unrolling concept, just at the hardware level. And the new framework versions hide all the complexity of such optimizations from us. Ultimately, there is no magic, and array search is still O(N). Even if it is O(N/16).

This magic only works on modern processors that support AVX-512 instructions (or AVX2 for 256-bit). If you run this code on older hardware, .NET will gracefully fall back to Vector128 or a regular loop.

.NET 10

Let me remind you that in this version, the results for an array of length 10 got even better and even surpassed HashSet. At the same time, the search algorithm itself hasn't changed; the code is exactly the same. Registers larger than 512 bits haven't been invented yet. So, what's the deal? I assume that PGO (Profile-Guided Optimization) kicked in, and the compiler decided that searches will most often occur in medium-sized arrays (8–15 elements), rearranged the comparison blocks to avoid unnecessary jumps, and saved the processor a couple more clock cycles.

And remember how earlier we made a mental note of why string array search became so good in .NET 10? It's time to figure that out as well.

Let's look at the benchmark once again:

| Method                  | Runtime   | Size |     Mean | Code Size |
|-------------------------|-----------|------|---------:|----------:|
| Array.Contains (String) | .NET 8.0  | 1    | 9.639 ns |      51 B |
| Array.Contains (String) | .NET 10.0 | 1    | 2.285 ns |     548 B |

Besides the nearly fourfold difference in speed, we also see a multiple-fold difference in Code Size. For .NET 8, the compiler didn't think twice and redirected the Contains method call to the generic Array.IndexOf method, which calls the virtual bool Equals(object obj) method for each array element. It's reliable, but not fast.

However, in .NET 10, the compiler applied a new technique called "devirtualization" — more specifically, Guarded Devirtualization. It is understood that our collection is an array of strings and only strings. This means we can skip all those generic calls and bypass the virtual method table dispatch, and simply inline the entire string comparison loop. That is why we get 548 bytes of code versus 51. But! We get specific code for searching a string in an array, which works much faster. To see this, let's look at the assembly code again:

...
mov r8,offset MT_System.String
cmp [rcx],r8                ; Verify that we definitely have a string. Hence Guarded Devirtualization.
jne short M00_L08           ; Otherwise, compare via classic Equals
...
mov r8d,[rcx+8]             ; Read the length of the current string
cmp r8d,[rsi+8]             ; Compare it with the length of the target string
je short M00_L07            ; If lengths match - compare the strings themselves byte by byte
...
M00_L07:
lea rax,[rcx+0C]
...
lea rdx,[rsi+0C]            ; Get a pointer to the byte array where strings are stored
mov rcx,rax
call qword ptr [7FFD2A46C330] ; System.SpanHelpers.SequenceEqual - call the method that compares 2 byte arrays
...

At the same time, SequenceEqual also supports vector operations and hardware acceleration. Wow! That's powerful! I didn't even suspect what a tremendous amount of work goes on behind a simple string search in an array.

Conclusions

Once again, the obvious conclusion suggests itself: simply updating .NET will yield a performance boost (and therefore resource savings). Also, there is no magic; O(N) is still O(N), which is clearly noticeable at large values. However, for small values (5–10 elements), things might not be so straightforward.

But can we do better? To avoid O(N), even with hardware acceleration, but also without constantly paying the hashing overhead? We'll see...

Analyzer

I've added a rule to my analyzer that will suggest replacing an array (or list) with a HashSet if Contains is called on it. This is useful for .NET 8 and below, and remains relevant for .NET 10 with collections larger than ~15 elements. For finer tuning, you can adjust the trigger threshold in .editorconfig — for example, only starting from five elements:

dotnet_diagnostic.CI0008.min_items_count = 5

Collection initializer: List

Sergey Rogatnev — Thu, 27 Nov 2025 00:00:00 GMT

We're all used to write new List<int> { 1, 2, 3, 4 } or new int[] { 1, 2, 3, 4} to initialize collections with values. Syntactically, they look similar, but the behavior is different, and you should be careful if you are worried about performance.

Array

As we know, arrays contain a sequence of fixed-size elements. Once created, their size cannot be changed for the entire lifetime of the array.

var array = new int[] { 1, 2, 3, 4};

List<T>

When we don't know the final size of a collection, or we need to add/remove elements during its life cycle, using List<T> type is appropriate.

var list = new List<int>();
list.Add(1);
list.Add(2);

At first glance, it might seem like we can always use a list instead of an array — it has all the capabilities of an array, but it can also be dynamically resized. But to decide whether to use a list, we need to understand more about its internal structure.

Part of the source code:

public class List<T> : IList<T>, IList, IReadOnlyList<T>
{
    private const int DefaultCapacity = 4;

    internal T[] _items;
    internal int _size;

    private static readonly T[] s_emptyArray = new T[0];

    // Constructs a List. The list is initially empty and has a capacity
    // of zero. Upon adding the first element to the list the capacity is
    // increased to DefaultCapacity, and then increased in multiples of two
    // as required.
    public List()
    {
        _items = s_emptyArray;
    }
    ...

    public int Capacity
    {
        get => _items.Length;
        set {...}
    }

    public int Count => _size;
}

_items - internal array for storing elements;
_size - the number of elements in the array and the total size of the list;
Capacity - the size of the _items array and the maximum number of elements that can fit into it without resizing.

Resizing a list

In simple terms, we can say that a list is an array that can be resized as needed.

Every time we try to add another item, the list checks if there is enough free space in _items, otherwise it sets the new capacity:

internal void Grow(int capacity)
{
    ...
    int newCapacity = _items.Length == 0 ? DefaultCapacity : 2 * _items.Length;
    ...
    Capacity = newCapacity;
}

Let's take a closer look at the property Capacity:

// Gets and sets the capacity of this list.  The capacity is the size of
// the internal array used to hold items.  When set, the internal
// array of the list is reallocated to the given capacity.
public int Capacity
{
    get => _items.Length;
    set
    {
        if (value < _size)
        {
            ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.value, ExceptionResource.ArgumentOutOfRange_SmallCapacity);
        }

        if (value != _items.Length)
        {
            if (value > 0)
            {
                T[] newItems = new T[value];
                if (_size > 0)
                {
                    Array.Copy(_items, newItems, _size);
                }
                _items = newItems;
            }
            else
            {
                _items = s_emptyArray;
            }
        }
    }
}

So, what do we see? Initially, each list is created with an empty internal array. After the first element is added, the list creates a new array with 4 elements (DefaultCapacity is 4). And when the current array is exhausted, a new one is created with double the size, and all elements are copied.

Performance

What happens when we create a new list and initialize it with values?

var list = new List<int> { 1, 2, 3, 4, 5 };

This looks like array initialization, but it works completely differently. According to the documentation, any type that implements IEnumerable and has an Add method can be used with a collection initializer.

So, the previous example is just a shorthand form of successive calls to the Add method:

var list = new List<int>();
list.Add(1);
list.Add(2);
list.Add(3);
list.Add(4);
list.Add(5);

The compiler simply transforms the short collection initializer and automatically adds the necessary calls. This can result in a performance penalty.

Let's take a closer look at the process of adding 5 elements:

Before the first call to Add the list is empty, the internal array is empty.
The first element added creates a new internal array with 4 elements.
Elements 2, 3, 4 do not change anything when added.
When we add the fifth element, the internal array is full and needs to be resized. A new array of size 8 is created, all elements are copied from the previous array, and the fifth element is added.

So, we end up with a list with 5 elements and an internal array with 8 elements. We've created two arrays, and the final one is wasting 37.5% of its space. As you might have guessed, creating new arrays and copying elements results in memory allocation and takes additional time.

This could be an unpleasant surprise in critical areas. Do we have a solution? Yes!

Capacity

If we know or guess the finite size of the list, we can create it with an initial capacity (Capacity).

var list = new List<int>(5);
list.Add(1);
list.Add(2);
list.Add(3);
list.Add(4);
list.Add(5);

var list = new List<int>(5) { 1, 2, 3, 4, 5 };

Now we immediately create one internal array of 5 elements, and there are no more unnecessary memory allocations.

Benchmark

Let's compare creating lists with and without initial capacity.

Full benchmark code. Click to expand.

using System.Collections.Generic;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;

namespace ListBenchmark
{
    [ShortRunJob(RuntimeMoniker.Net48)]
    [ShortRunJob(RuntimeMoniker.NetCoreApp31)]
    [ShortRunJob(RuntimeMoniker.Net80)]
    [ShortRunJob(RuntimeMoniker.Net10_0)]
    [MemoryDiagnoser]
    [HideColumns("Job", "Error", "StdDev", "Gen0")]
    public class InitListBenchmark
    {
        [BenchmarkCategory("One")]
        [Benchmark]
        public List<int> InitList1()
        {
            return new List<int> {1};
        }

        [BenchmarkCategory("One")]
        [Benchmark]
        public List<int> InitListWithSize1()
        {
            return new List<int>(1) {1};
        }

        [BenchmarkCategory("Five")]
        [Benchmark]
        public List<int> InitList5()
        {
            return new List<int> {1, 2, 3, 4, 5};
        }
        
        [BenchmarkCategory("Five")]
        [Benchmark]
        public List<int> InitListWithSize5()
        {
            return new List<int>(5) {1, 2, 3, 4, 5};
        }
        
        [BenchmarkCategory("Ten")]
        [Benchmark]
        public List<int> InitList10()
        {
            return new List<int> {1, 2, 3, 4, 5, 6, 7, 8, 9, 0};
        }
        
        [BenchmarkCategory("Ten")]
        [Benchmark]
        public List<int> InitListWithSize10()
        {
            return new List<int>(10) {1, 2, 3, 4, 5, 6, 7, 8, 9, 0};
        }
    }
}

BenchmarkDotNet v0.15.6, Windows 10 (10.0.19045.6456/22H2/2022Update)
AMD Ryzen 7 7840H with Radeon 780M Graphics 3.80GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK 10.0.100
[Host]                      : .NET 10.0.0 (10.0.0, 10.0.25.52411), X64 RyuJIT x86-64-v4
ShortRun-.NET 10.0          : .NET 10.0.0 (10.0.0, 10.0.25.52411), X64 RyuJIT x86-64-v4
ShortRun-.NET 8.0           : .NET 8.0.22 (8.0.22, 8.0.2225.52707), X64 RyuJIT x86-64-v4
ShortRun-.NET Core 3.1      : .NET Core 3.1.32 (3.1.32, 4.700.22.55902), X64 RyuJIT VectorSize=256
ShortRun-.NET Framework 4.8 : .NET Framework 4.8.1 (4.8.9310.0), X64 RyuJIT VectorSize=256

IterationCount=3 LaunchCount=1 WarmupCount=3

| Method             | Runtime            |      Mean | Allocated |
|--------------------|--------------------|----------:|----------:|
| InitList1          | .NET Framework 4.8 | 13.904 ns |      80 B |
| InitListWithSize1  | .NET Framework 4.8 |  6.658 ns |      72 B |
| InitList1          | .NET Core 3.1      | 11.091 ns |      72 B |
| InitListWithSize1  | .NET Core 3.1      |  7.407 ns |      64 B |
| InitList1          | .NET 8.0           | 10.084 ns |      72 B |
| InitListWithSize1  | .NET 8.0           |  6.838 ns |      64 B |
| InitList1          | .NET 10.0          |  8.170 ns |      72 B |
| InitListWithSize1  | .NET 10.0          |  6.638 ns |      64 B |
| InitList5          | .NET Framework 4.8 | 31.298 ns |     136 B |
| InitListWithSize5  | .NET Framework 4.8 | 12.013 ns |      88 B |
| InitList5          | .NET Core 3.1      | 26.466 ns |     128 B |
| InitListWithSize5  | .NET Core 3.1      |  9.446 ns |      80 B |
| InitList5          | .NET 8.0           | 23.714 ns |     128 B |
| InitListWithSize5  | .NET 8.0           | 15.587 ns |      80 B |
| InitList5          | .NET 10.0          | 20.002 ns |     128 B |
| InitListWithSize5  | .NET 10.0          |  8.712 ns |      80 B |
| InitList10         | .NET Framework 4.8 | 53.488 ns |     225 B |
| InitListWithSize10 | .NET Framework 4.8 | 18.185 ns |     104 B |
| InitList10         | .NET Core 3.1      | 44.371 ns |     216 B |
| InitListWithSize10 | .NET Core 3.1      | 12.496 ns |      96 B |
| InitList10         | .NET 8.0           | 38.707 ns |     216 B |
| InitListWithSize10 | .NET 8.0           | 12.024 ns |      96 B |
| InitList10         | .NET 10.0          | 33.854 ns |     216 B |
| InitListWithSize10 | .NET 10.0          | 15.822 ns |      96 B |

Setting the initial capacity prevents unnecessary memory allocations and results in better performance. And there's no excess memory traffic. The more elements we add to the list, the greater the difference we see in the benchmarks.

Analyzer

If performance is critical for your project, you should pay attention to these situations. Automatic diagnostics can help. I maintain a set of diagnostic tools based on Roslyn – Collections.Analyzer, and now it can detect lists with a collection initializer and without an initial capacity.

It's important to understand that the analyzer only sets the initial collection size to avoid unnecessary allocations during the initial collection phase. He cannot predict how the size of the collection will change in the future.

Recommendations

If you need a static collection that you won't modify (add or remove elements), use arrays.
If you are creating a list and exactly know its future size, set the initial capacity to that size.
If you are creating a list and don't know its future size, set the expected size.

Dapper: How caching can be harmful

Sergey Rogatnev — Sat, 06 Sep 2025 00:00:00 GMT

Dapper is a popular library that allows mapping objects from a database to C# types. Unlike Entity Framework, it is not a full-fledged ORM, but it is very popular due to its minimalism. In this article, I will explain how the default behavior can lead to a significant increase in memory consumption.

Profiling

This story began with one application at work. Metrics showed that over time it consumed more and more memory, although there were no obvious reasons for it. I collected an application dump and opened it in a profiler.

The problem was visible almost immediately — the application was holding 2 gigabytes of some strings in memory, related to Dapper. The strings themselves contained the body of the SQL query. Let's study what objects in memory they are associated with.

Dapper internals

The profiler points to the SqlMapper.Identity type and the associated SqlMapper.CacheInfo. From the name, it's obvious that it is about caching. Let's take a closer look at the creation of the SqlMapper.Identity type during query execution:

  private static async Task<int> ExecuteImplAsync(
    IDbConnection cnn,
    CommandDefinition command,
    object? param)
  {
    SqlMapper.CacheInfo cacheInfo = SqlMapper.GetCacheInfo(new SqlMapper.Identity(command.CommandText, new CommandType?(command.CommandTypeDirect), cnn, (Type) null, param?.GetType()), param, command.AddToCache);
    // ...
  }

As we can see, at the beginning of the method, we try to get something from the cache using SqlMapper.Identity as a key, which contains the full text of the query in command.CommandText and other parameters.

The type stored in the cache is SqlMapper.CacheInfo:

  private class CacheInfo
  {
    private int hitCount;
    public SqlMapper.DeserializerState Deserializer { get; set; }
    public Func<DbDataReader, object>[]? OtherDeserializers { get; set; }
    public Action<IDbCommand, object?>? ParamReader { get; set; }
    public int GetHitCount() => Interlocked.CompareExchange(ref this.hitCount, 0, 0);
    public void RecordHit() => Interlocked.Increment(ref this.hitCount);
  }

This is a set of different deserializers associated with a specific SQL query, which allows Dapper to reuse objects for similar queries, speeding up data materialization.

A fairly reasonable optimization. But why is our application holding such a large cache?

SQl + json = 🚩

The problem is how we construct the SQL query. For example, the method for getting users by a list of identifiers looks like this:

public async Task<ICollection<string>> SearchAsync(
    ThemeSearchRequest searchRequest, 
    int skip = 0,
    int take = int.MaxValue, 
    TimeSpan? timeout = null)
{
    var selectScript = @$"SELECT theme_base 
                          FROM [dbo].Themes 
                          {searchRequest.ToWhereExpression()}
                          GROUP BY theme_base
                          ORDER BY MAX(count_base) DESC, theme_base
                          OFFSET {skip} ROWS 
                          FETCH NEXT {take} ROWS ONLY";
    
        await using var connection = new SqlConnection(this.connectionString);
        
        var result = (await connection.QueryAsync<string>(
            selectScript,
            searchRequest.ToParametersWithValues(),
            commandTimeout: (int)(timeout?.TotalSeconds ?? DefaultTimeout.TotalSeconds))).AsList();
        
        return result;
}

The SearchAsync method forms a dynamic query to the database — some search conditions, pagination. And that's the problem. Almost every query will be unique, and according to Dapper's basic logic, it will be cached for fast materialization in the future.

The same is written in the documentation:

Dapper caches information about every query it runs, this allows it to materialize objects quickly and process parameters quickly. The current implementation caches this information in a ConcurrentDictionary object. Statements that are only used once are routinely flushed from this cache. Still, if you are generating SQL strings on the fly without using parameters it is possible you may hit memory issues.

Dapper caching

Let's take a closer look at the caching mechanism in Dapper. We have already found out that each SQL query is used as a key and stores additional information for fast materialization of objects. Will this cache grow indefinitely? Let's look at the SqlMapper code:

private static bool TryGetQueryCache(Identity key, [NotNullWhen(true)] out CacheInfo? value)
{
    if (_queryCache.TryGetValue(key, out value!))
    {
        value.RecordHit();
        return true;
    }
    value = null;
    return false;
}

The TryGetQueryCache method searches for data in the cache and, if successful, increments the hit counter — the number of times the data has been returned for this key.

private const int COLLECT_PER_ITEMS = 1000;
private static void SetQueryCache(Identity key, CacheInfo value)
{
    if (Interlocked.Increment(ref collect) == COLLECT_PER_ITEMS)
    {
        CollectCacheGarbage();
    }
    _queryCache[key] = value;
}

The SetQueryCache method tries to add data to the cache until it reaches a size of 1000 elements. When this limit is reached, the CollectCacheGarbage() method is called:

private const int COLLECT_HIT_COUNT_MIN = 0;
private static void CollectCacheGarbage()
{
    try
    {
        foreach (var pair in _queryCache)
        {
            if (pair.Value.GetHitCount() <= COLLECT_HIT_COUNT_MIN)
            {
                _queryCache.TryRemove(pair.Key, out var _);
            }
        }
    }
    finally
    {
        Interlocked.Exchange(ref collect, 0);
    }
}

It tries to remove all cached items that have never been accessed. In this process, the GetHitCount() method resets the internal hit counter:

public int GetHitCount() { return Interlocked.CompareExchange(ref hitCount, 0, 0); }

Thus, after CollectCacheGarbage() is run, the value of all hitCounts is reset, which allows finding unused elements during the next cleanup. But in the case of active use of dynamic SQL queries, it is highly likely that all queries will be used once and will be constantly evicted from the cache.

Using one-time queries not only hits your application's memory but also creates an additional load on the SQL server. It will have to parse, compile, and build an execution plan for each new query.

Solution

The correct solution here would be to use parameterized queries so that a single query will be reused many times, including on the SQL server:

public async Task<ICollection<TaskRules>> GetTaskRulesAsync(long taskId)
{
    var sql = $@"SELECT * FROM TaskRules WHERE TaskID = @taskId";
    await using var connection = new SqlConnection(this.connectionString);
    await connection.OpenAsync();
    var result = await connection
        .QueryAsync<TaskRules>(sql, new { taskId }, commandTimeout: 180);

    return result.AsList();
}

The GetTaskRulesAsync method uses a parameterized SQL query, where data is selected by @taskId. This allows using caching at all levels — in the application and on the SQL server.

A quick solution might be to disable caching for a specific query — this can be done by managing CommandFlags when creating the command:

new CommandDefinition(sql, CommandFlags.NoCache)

This way, "one-time" queries will not get into the cache and clutter up the memory.

Conclusion

Many solutions try to optimize frequently used scenarios through strict rules or some of their own heuristics. Such important rules are usually described somewhere prominently. In Dapper's case — right in the Readme.md.

Recommendations

Perform periodic profiling, see how applications use memory. For example, using dotMemory.
Use parameterized queries for working with SQL.
If any solution uses caching, make sure it is useful for you.

String of Digits

Sergey Rogatnev — Sat, 05 Jul 2025 00:00:00 GMT

A popular task is to check if a string contains only digits. For example, you need to check if the user entered the correct phone number, index or a tax code of an organization. There are several ways to solve this task, which differ in their efficiency. Let's take a look at the most popular ones.

Regex

Probably the most popular way to solve this task is to use a regular expression. It's straightforward and easy to use an expression ^[0-9]*$ (or ^\d*$).

Below is a naive implementation of a regular expression check:

Regex regex = new Regex("^[0-9]*$");
var value = "123456789000";
var isValid = regex.IsMatch(value);

I'm sure someone sees the problem. This implementation is only suitable for a one-time launch. In production code, when you check thousands of strings, this solution will not be efficient. .NET provides the ability to compile a regular expression at runtime, by using the RegexOptions.Compiled option:

Regex regex = new Regex("^[0-9]*$", RegexOptions.Compiled);
var value = "123456789000";
var isValid = regex.IsMatch(value);

Calling a constructor with this option will generate IL code that will be called through DynamicMethod inside Regex.IsMatch, which will be faster than the usual regular expression processing. The downside is that it takes longer to create a Regex object due to the time spent on compilation at runtime, but this quickly pays off with repeated use.

Let's compare both solutions.

Benchmark code. Click to expand.

[MemoryDiagnoser]
[SimpleJob(RuntimeMoniker.Net90)]
[SimpleJob(RuntimeMoniker.Net60)]
[SimpleJob(RuntimeMoniker.NetCoreApp31)]
[SimpleJob(RuntimeMoniker.Net48)]
[HideColumns("Job", "Error", "StdDev", "Gen0")]
public partial class DigitsBenchmarks
{
    private static string value = "123456789000";

    private static readonly Regex regex = new Regex("^[0-9]*$");
    private static readonly Regex compiledRegex = new Regex("^[0-9]*$", RegexOptions.Compiled);

    [Benchmark]
    public bool Regex()
    {
        return regex.IsMatch(value);
    }
    
    [Benchmark]
    public bool CompiledRegex()
    {
        return compiledRegex.IsMatch(value);
    }
}

Results:

Method	Runtime	Mean	Median	Allocated
Regex	.NET Framework 4.8	165.4417 ns	166.2537 ns	-
CompiledRegex	.NET Framework 4.8	115.9377 ns	115.9720 ns	-
Regex	.NET Core 3.1	118.1540 ns	118.1887 ns	-
CompiledRegex	.NET Core 3.1	89.7392 ns	89.6514 ns	-
Regex	.NET 6.0	57.8247 ns	57.8031 ns	-
CompiledRegex	.NET 6.0	21.2952 ns	21.2616 ns	-
Regex	.NET 9.0	47.2579 ns	47.3506 ns	-
CompiledRegex	.NET 9.0	24.2419 ns	24.2547 ns	-

The advantage of using compiled expressions is quite obvious. Also, with each new version of .NET, the contribution of developers to performance is more visible. This is another argument in favor of upgrading to modern versions of the .NET Framework.

Regex source generators

Again, compiling regular expressions has one drawback - creating a Regex object at runtime will take some time. Is it possible to get rid of this?

Starting with .NET 7, this capability is available through source generators. Strictly speaking, source generators appeared in .NET 5, but the regular expression solution was implemented only in the seventh version.

Source generators allow you to create C# code at compile time, which means you can view and debug it as if it was your own code. And regular expressions can be converted to C# code at compile time! .NET has a special attribute for this - GeneratedRegex:

namespace DigitBenchmark
{
    public partial class DigitsBenchmarks
    {
        private static readonly Regex generatedRegex = GenerateRegex();
        
        [GeneratedRegex("^[0-9]*$")]
        private static partial Regex GenerateRegex();
    }
}

Let's figure out what's going on here.

First, we need to mark our DigitsBenchmarks class as partial, since some of the generated code for this class will be in another file. Next we need to create a partial method that will return an object of type Regex and mark it with the GeneratedRegex attribute specifying the regular expression pattern. You don't need to specify the RegexOptions.Compiled option, it will be ignored.

The implementation of the GenerateRegex method will be in another file. You can find it in the project and view the source code:

namespace DigitBenchmark
{
    partial class DigitsBenchmarks
    {
        /// <remarks>
        /// Pattern:<br/>
        /// <code>^[0-9]*$</code><br/>
        /// Explanation:<br/>
        /// <code>
        /// ○ Match if at the beginning of the string.<br/>
        /// ○ Match a character in the set [0-9] atomically any number of times.<br/>
        /// ○ Match if at the end of the string or if before an ending newline.<br/>
        /// </code>
        /// </remarks>
        [global::System.CodeDom.Compiler.GeneratedCodeAttribute("System.Text.RegularExpressions.Generator", "8.0.12.21506")]
        private static partial global::System.Text.RegularExpressions.Regex GenerateRegex() => global::System.Text.RegularExpressions.Generated.GenerateRegex_0.Instance;
    }
}

As you can see, the file with the same class was automatically created and contains the implementation of our method for generating a regular expression.

And then we can use this Regex object as usual. Since this is real C# code, there is no need to generate anything at runtime, so there is no point in specifying the RegexOptions.Compiled option.

What benefit do we get from this? My benchmarks do not include .NET 7 and 8, let's compare the performance of the latest one at the moment:

Method	Runtime	Mean	Median	Allocated
Regex	.NET 9.0	47.2579 ns	47.3506 ns	-
CompiledRegex	.NET 9.0	24.2419 ns	24.2547 ns	-
GeneratedRegex	.NET 9.0	17.2548 ns	17.2603 ns	-

We see that the time has been reduced by almost 30%! The compiler has much more opportunity to optimize the source code at the compilation stage than at runtime.

char.IsDigit

Another popular way is to use the static char.IsDigit method in combination with the LINQ All method:

var value = "123456789000";
var isValid = value.All(char.IsDigit);

Let's check how good this method is in terms of performance.

Benchmark code. Click to expand.

[MemoryDiagnoser]
[SimpleJob(RuntimeMoniker.Net90)]
[SimpleJob(RuntimeMoniker.Net60)]
[SimpleJob(RuntimeMoniker.NetCoreApp31)]
[SimpleJob(RuntimeMoniker.Net48)]
[HideColumns("Job", "Error", "StdDev", "Gen0")]
public partial class DigitsBenchmarks
{
    [Benchmark]
    public bool LinqCharIsDigit()
    {
        return value.All(char.IsDigit);
    }
}

Results:

Method	Runtime	Mean	Median	Allocated
LinqCharIsDigit	.NET Framework 4.8	92.1679 ns	92.2549 ns	96 B
LinqCharIsDigit	.NET Core 3.1	72.0987 ns	72.6419 ns	96 B
LinqCharIsDigit	.NET 6.0	74.2609 ns	74.4256 ns	96 B
LinqCharIsDigit	.NET 9.0	31.0294 ns	31.0501 ns	32 B

And let's compare this method with previous solutions.

If in older versions of the framework, LINQ validation and the IsDigit method has an advantage over regular expressions, then later we see that such an implementation starts to lose.

Also, please note that each call results in the allocation of additional memory. This value consists of two parts:

Creating a lambda expression in a method parameter All(c => char.IsDigit(c)).
Creating an iterator inside the All method.

It is noteworthy that .NET 9 allocates three times less memory. Before .NET 9, the All method consisted of a foreach loop with a condition:

public static bool All<TSource>(this IEnumerable<TSource> source, Func<TSource, bool> predicate)
{
  //...
  foreach (TSource source1 in source)
  {
    if (!predicate(source1))
      return false;
  }
  return true;
}

But in .NET 9, an important optimization was added:

public static bool All<TSource>(this IEnumerable<TSource> source, Func<TSource, bool> predicate)
{
  //...
  ReadOnlySpan<TSource> span;
  if (source.TryGetSpan<TSource>(out span))
  {
    ReadOnlySpan<TSource> readOnlySpan = span;
    for (int index = 0; index < readOnlySpan.Length; ++index)
    {
      TSource source1 = readOnlySpan[index];
      if (!predicate(source1))
        return false;
    }
  }
  else
  {
    foreach (TSource source2 in source)
    {
      if (!predicate(source2))
        return false;
    }
  }
  return true;
}

Instead of an unconditional foreach loop, the All method tries to get a ReadOnlySpan - a read-safe contiguous block of memory - from the source. And then a simple for loop is used, which does not lead to the creation of an iterator, reducing the amount of additional memory. You can get rid of this completely by rewriting the All method to a regular loop:

public bool ForIsDigit()
{
    for (var i = 0; i < value.Length; i++)
    {
        if (!char.IsDigit(value[i]))
            return false;
    }

    return true;
}

In addition to the absence of unnecessary memory traffic, this solution is rapid.

What is a digit?

It seems that we have found the optimal solution for checking a string for the presence of only digits. But try to guess what the following code will output:

Console.WriteLine(char.IsDigit('0'));
Console.WriteLine(char.IsDigit('a'));
Console.WriteLine(char.IsDigit('٨'));
Console.WriteLine(char.IsDigit('৯'));

Click to find out the answer.

Console.WriteLine(char.IsDigit('0')); //True
Console.WriteLine(char.IsDigit('a')); //False
Console.WriteLine(char.IsDigit('٨')); //True
Console.WriteLine(char.IsDigit('৯')); //True

I think the result surprises you. But there is nothing unusual about it, the IsDigit method considers as digits not only the usual symbols from the set 0-9, but also all other symbols that are related to digits in the Unicode encoding. And there are actually a lot of them. This can be a problem if you rely on such a check in your business code.

I think this is the reason for the new char.IsAsciiDigit method to appear starting with .NET 7. Now it really only checks characters from the 0-9 set.

Its implementation is very similar to manually checking each character in the loop, let's compare both solutions:

[Benchmark]
public bool ForCompare()
{
    for (var i = 0; i < value.Length; i++)
    {
        if (value[i] < '0' || value[i] > '9')
            return false;
    }

    return true;
}

[Benchmark]
public bool ForIsAsciiDigit()
{
    for (var i = 0; i < value.Length; i++)
    {
        if (!char.IsAsciiDigit(value[i]))
            return false;
    }

    return true;
}

Method	Runtime	Mean	Median	Allocated
ForCompare	.NET 9.0	4.8587 ns	4.8656 ns	-
ForIsAsciiDigit	.NET 9.0	4.7515 ns	4.4976 ns	-

Both methods show equivalent results.

Conclusion

We've looked at several different ways to check whether a string consists only of digits or not. And it's important to understand the specifics of how some of them work, because in addition to trivial performance issues, you can get errors in your business logic if you don't check your input data well enough.

Recommendations:

If you write your applications for .NET 7 or higher, then use the generated regular expressions. Otherwise, specify the RegexOptions.Compiled option.
If you write your applications for .NET 7, use the char.IsAsciiDigit method to check characters. Otherwise, it is better to write the check yourself.

Links

Full benchmark code. Click to expand.

using System.Linq;
using System.Text.RegularExpressions;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;

namespace DigitBenchmark
{
    [MemoryDiagnoser]
    [SimpleJob(RuntimeMoniker.Net90)]
    [SimpleJob(RuntimeMoniker.Net60)]
    [SimpleJob(RuntimeMoniker.NetCoreApp31)]
    [SimpleJob(RuntimeMoniker.Net48)]
    [HideColumns("Job", "Error", "StdDev", "Gen0")]
    public partial class DigitsBenchmarks
    {
        private static string value = "123456789000";

        private static readonly Regex regex = new Regex("^[0-9]*$");
        private static readonly Regex compiledRegex = new Regex("^[0-9]*$", RegexOptions.Compiled);
        private static readonly Regex generatedRegex = GenerateRegex();

        [GeneratedRegex("^[0-9]*$")]
        private static partial Regex GenerateRegex();

        [Benchmark]
        public bool Regex()
        {
            return regex.IsMatch(value);
        }
        
        [Benchmark]
        public bool CompiledRegex()
        {
            return compiledRegex.IsMatch(value);
        }

        [Benchmark]
        public bool GeneratedRegex()
        {
            return generatedRegex.IsMatch(value);
        }

        [Benchmark]
        public bool LinqCharIsDigit()
        {
            return value.All(char.IsDigit);
        }

        [Benchmark]
        public bool ForCompare()
        {
            for (var i = 0; i < value.Length; i++)
            {
                if (value[i] < '0' || value[i] > '9')
                    return false;
            }

            return true;
        }

        [Benchmark]
        public bool ForIsDigit()
        {
            for (var i = 0; i < value.Length; i++)
            {
                if (!char.IsDigit(value[i]))
                    return false;
            }

            return true;
        }

        [Benchmark]
        public bool ForIsAsciiDigit()
        {
            for (var i = 0; i < value.Length; i++)
            {
                if (!char.IsAsciiDigit(value[i]))
                    return false;
            }

            return true;
        }

        [Benchmark]
        public bool LinqCharIsAsciiDigit()
        {
            return value.All(char.IsAsciiDigit);
        }
    }
}

Full results. Click to expand.

BenchmarkDotNet v0.15.0, Windows 10 (10.0.19045.5917/22H2/2022Update)
AMD Ryzen 7 7840H with Radeon 780M Graphics 3.80GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK 9.0.204
  [Host]             : .NET 9.0.5 (9.0.525.21509), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  .NET 6.0           : .NET 6.0.36 (6.0.3624.51421), X64 RyuJIT AVX2
  .NET 9.0           : .NET 9.0.5 (9.0.525.21509), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  .NET Core 3.1      : .NET Core 3.1.32 (CoreCLR 4.700.22.55902, CoreFX 4.700.22.56512), X64 RyuJIT AVX2
  .NET Framework 4.8 : .NET Framework 4.8.1 (4.8.9310.0), X64 RyuJIT VectorSize=256

Method	Runtime	Mean	Median	Allocated
Regex	.NET Framework 4.8	165.4417 ns	166.2537 ns	-
CompiledRegex	.NET Framework 4.8	115.9377 ns	115.9720 ns	-
GeneratedRegex	.NET Framework 4.8	N/A	N/A	-
LinqCharIsDigit	.NET Framework 4.8	92.1679 ns	92.2549 ns	96 B
ForCompare	.NET Framework 4.8	4.8703 ns	4.8602 ns	-
ForIsDigit	.NET Framework 4.8	8.3920 ns	8.3952 ns	-
ForIsAsciiDigit	.NET Framework 4.8	N/A	N/A	-
LinqCharIsAsciiDigit	.NET Framework 4.8	N/A	N/A	-
Regex	.NET Core 3.1	118.1540 ns	118.1887 ns	-
CompiledRegex	.NET Core 3.1	89.7392 ns	89.6514 ns	-
GeneratedRegex	.NET Core 3.1	N/A	N/A	-
LinqCharIsDigit	.NET Core 3.1	72.0987 ns	72.6419 ns	96 B
ForCompare	.NET Core 3.1	5.3070 ns	5.3077 ns	-
ForIsDigit	.NET Core 3.1	9.0998 ns	9.1106 ns	-
ForIsAsciiDigit	.NET Core 3.1	N/A	N/A	-
LinqCharIsAsciiDigit	.NET Core 3.1	N/A	N/A	-
Regex	.NET 6.0	57.8247 ns	57.8031 ns	-
CompiledRegex	.NET 6.0	21.2952 ns	21.2616 ns	-
GeneratedRegex	.NET 6.0	N/A	N/A	-
LinqCharIsDigit	.NET 6.0	74.2609 ns	74.4256 ns	96 B
ForCompare	.NET 6.0	5.6198 ns	5.5922 ns	-
ForIsDigit	.NET 6.0	10.3160 ns	10.1789 ns	-
ForIsAsciiDigit	.NET 6.0	N/A	N/A	-
LinqCharIsAsciiDigit	.NET 6.0	N/A	N/A	-
Regex	.NET 9.0	47.2579 ns	47.3506 ns	-
CompiledRegex	.NET 9.0	24.2419 ns	24.2547 ns	-
GeneratedRegex	.NET 9.0	17.2548 ns	17.2603 ns	-
LinqCharIsDigit	.NET 9.0	31.0294 ns	31.0501 ns	32 B
ForCompare	.NET 9.0	4.8587 ns	4.8656 ns	-
ForIsDigit	.NET 9.0	7.1207 ns	7.7792 ns	-
ForIsAsciiDigit	.NET 9.0	4.7515 ns	4.4976 ns	-
LinqCharIsAsciiDigit	.NET 9.0	32.6861 ns	32.5483 ns	32 B

The N/A mark is given to methods that do not exist in the current version of the framework.

Any() vs Count: part 2

Sergey Rogatnev — Sun, 11 May 2025 00:00:00 GMT

In Part 1, we compared Any() and Count methods for different collections and proposed optimization approaches.

Array

Let's start with the simplest collection type - arrays. We'll check if there's any difference between array.Any() and array.Length != 0.

The following configuration is used throughout:

BenchmarkDotNet v0.14.0, Windows 10 (10.0.19045.5796/22H2/2022Update)
AMD Ryzen 7 7840H with Radeon 780M Graphics, 1 CPU, 16 logical and 8 physical cores
.NET SDK 9.0.203
  [Host]             : .NET 9.0.4 (9.0.425.16305), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  .NET 5.0           : .NET 5.0.17 (5.0.1722.21314), X64 RyuJIT AVX2
  .NET 6.0           : .NET 6.0.36 (6.0.3624.51421), X64 RyuJIT AVX2
  .NET 8.0           : .NET 8.0.15 (8.0.1525.16413), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  .NET 9.0           : .NET 9.0.4 (9.0.425.16305), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  .NET Core 3.1      : .NET Core 3.1.32 (CoreCLR 4.700.22.55902, CoreFX 4.700.22.56512), X64 RyuJIT AVX2
  .NET Framework 4.8 : .NET Framework 4.8.1 (4.8.9310.0), X64 RyuJIT VectorSize=256

Benchmark code. Click to expand.

using System.Linq;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;

namespace AnyVsCount
{
    [MemoryDiagnoser]
    [SimpleJob(RuntimeMoniker.Net90)]
    [SimpleJob(RuntimeMoniker.Net80)]
    [SimpleJob(RuntimeMoniker.Net60)]
    [SimpleJob(RuntimeMoniker.Net50)]
    [SimpleJob(RuntimeMoniker.NetCoreApp31)]
    [SimpleJob(RuntimeMoniker.Net48)]
    [HideColumns("Job", "Error", "StdDev", "Gen0")]
    public class ArrayBenchmark
    {
        [Params(10, 10000)]
        public int N;

        private int[] array;

        [GlobalSetup]
        public void SetUp()
        {
            array = new int[N];
        }

        [Benchmark]
        public bool ArrayAny()
        {
            return array.Any();
        }

        [Benchmark]
        public bool ArrayCount()
        {
            return array.Length != 0;
        }
    }
}

Results

Method	Runtime	N	Mean	Median	Allocated
ArrayAny	.NET Framework 4.8	10	8.7962 ns	8.8213 ns	32 B
ArrayCount	.NET Framework 4.8	10	0.0073 ns	0.0028 ns	-
ArrayAny	.NET Core 3.1	10	8.0450 ns	8.0462 ns	32 B
ArrayCount	.NET Core 3.1	10	0.0433 ns	0.0399 ns	-
ArrayAny	.NET 5.0	10	6.2080 ns	6.1852 ns	-
ArrayCount	.NET 5.0	10	0.0030 ns	0.0000 ns	-
ArrayAny	.NET 6.0	10	6.0673 ns	6.0671 ns	-
ArrayCount	.NET 6.0	10	0.0135 ns	0.0135 ns	-
ArrayAny	.NET 8.0	10	5.0980 ns	5.2591 ns	-
ArrayCount	.NET 8.0	10	0.0000 ns	0.0000 ns	-
ArrayAny	.NET 9.0	10	2.1643 ns	2.1216 ns	-
ArrayCount	.NET 9.0	10	0.0030 ns	0.0034 ns	-
ArrayAny	.NET Framework 4.8	10000	9.0093 ns	9.0043 ns	32 B
ArrayCount	.NET Framework 4.8	10000	0.0123 ns	0.0137 ns	-
ArrayAny	.NET Core 3.1	10000	7.9624 ns	7.7794 ns	32 B
ArrayCount	.NET Core 3.1	10000	0.0311 ns	0.0295 ns	-
ArrayAny	.NET 5.0	10000	4.8357 ns	4.8352 ns	-
ArrayCount	.NET 5.0	10000	0.0008 ns	0.0007 ns	-
ArrayAny	.NET 6.0	10000	6.0913 ns	6.0747 ns	-
ArrayCount	.NET 6.0	10000	0.0147 ns	0.0153 ns	-
ArrayAny	.NET 8.0	10000	4.7691 ns	4.7521 ns	-
ArrayCount	.NET 8.0	10000	0.0110 ns	0.0078 ns	-
ArrayAny	.NET 9.0	10000	2.2933 ns	2.2906 ns	-
ArrayCount	.NET 9.0	10000	0.0121 ns	0.0109 ns	-

Values close to zero (e.g., 0.0000 ns) should be considered negligible — they are within measurement error.

Legend for all tables:

  N         : Value of the 'N' parameter
  Mean      : Arithmetic mean of all measurements
  Median    : Value separating the higher half of all measurements (50th percentile)
  Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  1 ns      : 1 Nanosecond (0.000000001 sec)

For arrays, results are nearly independent of size, so we'll focus on N = 10000. Directly accessing the Length property is at least 2 orders of scale faster than Any(). However, in generic code working with IEnumerable<T>, we often use:

public bool IsEmpty<T>(IEnumerable<T> collection)
{
    return !collection.Any();
}

Let's analyze why Any() performance varies across .NET versions. From .NET 4.8 to 9.0, Any() execution time decreased 4x:

In .NET 4.8 and Core 3.1, Any() creates an iterator and checks MoveNext(), allocating 32 bytes:

public static bool Any<TSource>(this IEnumerable<TSource> source)
{
    if (source == null)
    {
        ThrowHelper.ThrowArgumentNullException(ExceptionArgument.source);
    }

    using (IEnumerator<TSource> e = source.GetEnumerator())
    {
        return e.MoveNext();
    }
}

Starting with .NET 5, optimizations were introduced:

public static bool Any<TSource>(this IEnumerable<TSource> source)
{
    if (source == null)
    {
        ThrowHelper.ThrowArgumentNullException(ExceptionArgument.source);
    }

    if (source is ICollection<TSource> collectionoft)
    {
        return collectionoft.Count != 0;
    }
    else if (source is IIListProvider<TSource> listProv)
    {
        int count = listProv.GetCount(onlyIfCheap: true);
        if (count >= 0)
        {
            return count != 0;
        }
    }
    else if (source is ICollection collection)
    {
        return collection.Count != 0;
    }

    using (IEnumerator<TSource> e = source.GetEnumerator())
    {
        return e.MoveNext();
    }
}

This solution is similar to the one we used in Part 1. Firstly it checks interface ICollection<T>, that has property Count. And then we see usage of a new interface IIListProvider<TSource> (internal interface of .NET to optimize LINQ methods):

internal interface IIListProvider<TElement> : IEnumerable<TElement>
{
    TElement[] ToArray();

    List<TElement> ToList();

    int GetCount(bool onlyIfCheap);
}

The interface IIListProvider<TSource> is part of a major rework of LINQ that has resulted in significant optimizations noticeable even in the simple Any() method. Interestingly, in .NET 6 the performance has degraded somewhat, although the implementation of the Any() method has not changed.

In the eighth version of .NET, the code that calculates the size of an enumeration (without performing the enumeration) was moved to a separate method TryGetNonEnumeratedCount(). This method runs in constant time but may not always return a value.

In .NET 9, the concept of using IIListProvider<TSource> was further developed and the LINQ methods were refactored to use the new Iterator<TSource> class, further improving performance. As we can see, the obvious solution with checking the enumeration type and the Count property has been implemented in new versions of .NET.

Will this be effective for all collections?

ConcurrentDictionary

Let's look at the most common thread-safe collection - ConcurrentDictionary<TKey, TValue>. It seems that there may be problems with the new implementation.

Benchmark code. Click to expand.

using System.Collections.Concurrent;
using System.Linq;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;

namespace AnyVsCount
{
    [MemoryDiagnoser]
    [SimpleJob(RuntimeMoniker.Net90)]
    [SimpleJob(RuntimeMoniker.Net80)]
    [SimpleJob(RuntimeMoniker.Net60)]
    [SimpleJob(RuntimeMoniker.Net50)]
    [SimpleJob(RuntimeMoniker.NetCoreApp31)]
    [SimpleJob(RuntimeMoniker.Net48)]
    [HideColumns("Job", "Error", "StdDev", "Gen0")]
    public class ConcurrentDictionaryBenchmark
    {
        [Params(10, 10000)]
        public int N;

        private ConcurrentDictionary<int, string> dictionary;

        [GlobalSetup]
        public void SetUp()
        {
            dictionary = new ConcurrentDictionary<int, string>();

            for (int i = 0; i < N; i++)
            {
                dictionary[i] = i.ToString();
            }
        }

        [Benchmark]
        public bool DictionaryAny()
        {
            return dictionary.Any();
        }

        [Benchmark]
        public bool DictionaryCount()
        {
            return dictionary.Count != 0;
        }
    }
}

Results

Method	Runtime	N	Mean	Median	Allocated
DictionaryAny	.NET Framework 4.8	10	18.48 ns	18.47 ns	64 B
DictionaryCount	.NET Framework 4.8	10	110.77 ns	110.75 ns	-
DictionaryAny	.NET Core 3.1	10	18.21 ns	18.17 ns	64 B
DictionaryCount	.NET Core 3.1	10	108.27 ns	110.12 ns	-
DictionaryAny	.NET 5.0	10	92.98 ns	92.81 ns	-
DictionaryCount	.NET 5.0	10	97.02 ns	96.42 ns	-
DictionaryAny	.NET 6.0	10	87.96 ns	87.79 ns	-
DictionaryCount	.NET 6.0	10	88.35 ns	88.40 ns	-
DictionaryAny	.NET 8.0	10	76.10 ns	76.08 ns	-
DictionaryCount	.NET 8.0	10	72.26 ns	72.27 ns	-
DictionaryAny	.NET 9.0	10	83.02 ns	83.52 ns	-
DictionaryCount	.NET 9.0	10	80.43 ns	79.34 ns	-
DictionaryAny	.NET Framework 4.8	10000	18.87 ns	18.71 ns	64 B
DictionaryCount	.NET Framework 4.8	10000	6,870.38 ns	6,859.65 ns	-
DictionaryAny	.NET Core 3.1	10000	18.04 ns	18.06 ns	64 B
DictionaryCount	.NET Core 3.1	10000	7,000.55 ns	7,000.95 ns	-
DictionaryAny	.NET 5.0	10000	5,877.30 ns	5,877.50 ns	-
DictionaryCount	.NET 5.0	10000	5,962.47 ns	5,958.17 ns	-
DictionaryAny	.NET 6.0	10000	5,993.15 ns	5,994.17 ns	-
DictionaryCount	.NET 6.0	10000	6,030.51 ns	6,029.08 ns	-
DictionaryAny	.NET 8.0	10000	5,082.70 ns	5,082.69 ns	-
DictionaryCount	.NET 8.0	10000	5,370.34 ns	5,368.76 ns	-
DictionaryAny	.NET 9.0	10000	5,917.83 ns	5,917.32 ns	-
DictionaryCount	.NET 9.0	10000	5,847.50 ns	5,848.04 ns	-

Some features are already visible:

Execution time depends on the size of the collection.
In older versions of the framework, the Any() method was much faster.
In newer versions of the framework, the Any() method is on par with Count in performance:

Let's look at the implementation of the Count property:

public int Count
{
    get
    {
        int locksAcquired = 0;
        try
        {
            AcquireAllLocks(ref locksAcquired);

            return GetCountNoLocks();
        }
        finally
        {
            ReleaseLocks(locksAcquired);
        }
    }
}

private void AcquireAllLocks(ref int locksAcquired)
{
    //...

    // First, acquire lock 0, then acquire the rest. _tables won't change after acquiring lock 0.
    AcquireFirstLock(ref locksAcquired);
    AcquirePostFirstLock(_tables, ref locksAcquired);
    Debug.Assert(locksAcquired == _tables._locks.Length);
}

Counting all elements requires acquiring locks on a special internal array _tables._locks. Each element of this array locks a part of the dictionary. Thus, starting with .NET 5, the Any() method of ConcurrentDictionary uses the Count property through ICollection<T>, which results in the same locks as calling Count directly.

Any() и IsEmpty

Let's reconstruct the implementation of Any() with an iterator and check the performance of this method:

[Benchmark]
public bool DictionaryEnumerator()
{
    using (var enumerator = dictionary.GetEnumerator())
    {
        return enumerator.MoveNext();
    }
}

Method	Runtime	N	Mean	Median	Allocated
DictionaryEnumerator	.NET Framework 4.8	10	15.51 ns	15.52 ns	64 B
DictionaryEnumerator	.NET Core 3.1	10	14.60 ns	14.60 ns	64 B
DictionaryEnumerator	.NET 5.0	10	15.16 ns	15.14 ns	64 B
DictionaryEnumerator	.NET 6.0	10	18.60 ns	18.67 ns	64 B
DictionaryEnumerator	.NET 8.0	10	12.97 ns	12.96 ns	64 B
DictionaryEnumerator	.NET 9.0	10	12.80 ns	12.81 ns	64 B
DictionaryEnumerator	.NET Framework 4.8	10000	15.41 ns	15.38 ns	64 B
DictionaryEnumerator	.NET Core 3.1	10000	15.03 ns	14.96 ns	64 B
DictionaryEnumerator	.NET 5.0	10000	15.90 ns	15.92 ns	64 B
DictionaryEnumerator	.NET 6.0	10000	18.50 ns	18.50 ns	64 B
DictionaryEnumerator	.NET 8.0	10000	13.38 ns	12.92 ns	64 B
DictionaryEnumerator	.NET 9.0	10000	12.97 ns	12.53 ns	64 B

We see that the results do not depend on the number of elements and have remained virtually unchanged in newer versions of .NET. But we spend a little memory creating the iterator each time.

But you can use a specialized collection property - IsEmpty:

[Benchmark]
public bool DictionaryIsEmpty()
{
    return !dictionary.IsEmpty;
}

Method	Runtime	N	Mean	Median	Allocated
DictionaryIsEmpty	.NET Framework 4.8	10	99.023 ns	98.709 ns	-
DictionaryIsEmpty	.NET Core 3.1	10	2.313 ns	2.313 ns	-
DictionaryIsEmpty	.NET 5.0	10	2.545 ns	2.545 ns	-
DictionaryIsEmpty	.NET 6.0	10	2.247 ns	2.283 ns	-
DictionaryIsEmpty	.NET 8.0	10	2.567 ns	2.568 ns	-
DictionaryIsEmpty	.NET 9.0	10	10.917 ns	10.902 ns	-
DictionaryIsEmpty	.NET Framework 4.8	10000	6,027.991 ns	6,026.648 ns	-
DictionaryIsEmpty	.NET Core 3.1	10000	2.320 ns	2.315 ns	-
DictionaryIsEmpty	.NET 5.0	10000	2.513 ns	2.512 ns	-
DictionaryIsEmpty	.NET 6.0	10000	2.756 ns	2.759 ns	-
DictionaryIsEmpty	.NET 8.0	10000	2.673 ns	2.673 ns	-
DictionaryIsEmpty	.NET 9.0	10000	3.801 ns	3.804 ns	-

We see that starting with .NET Core 3.1, the implementation of this property does not depend on the size of the collection, and in the case of non-empty collections, calling this property is not blocking. For .NET 4.8, we got results similar to calling the Count property - a lock on the entire collection is also used. For .NET 9, the results were slightly worse, but I don't have a quick answer as to why this happened.

Conclusion

The implementation of the Any() method is now optimized enough to be used in generic code, but there is still some room for its use in thread-safe collections.

Recommendations

Use Count properties for simple collections if you care about performance.
Use Any() method for generic code and IEnumerable<T>.
Use IsEmpty property if collections support it, starting with .NET Core 3.1.

nvarchar(max) vs nvarchar(N)

Sergey Rogatnev — Sat, 19 Apr 2025 00:00:00 GMT

MS SQL Server offers several data types for storing strings. The most popular are nvarchar(N) (1 ≤ N ≤ 4000) and nvarchar(max), which allow storing Unicode-encoded string data. Application developers often transfer their experience with the string type from programming languages like C# or Java to databases and automatically choose nvarchar(max), which can store strings up to 2GB. However, databases store and process data in fundamentally different ways. In this article, I will explain and demonstrate the consequences of unjustified use of the nvarchar(max) type.

Preparation

Let’s create three tables to store text data:

CREATE TABLE [dbo].[nv100]
(
	[Id] [int] NOT NULL,
	[CreateDate] [datetime] NOT NULL,
	[Comment] [nvarchar](100) NOT NULL,
	CONSTRAINT [PK_nv100_Id] PRIMARY KEY CLUSTERED ([Id] ASC)
)

CREATE TABLE [dbo].[nv1000]
(
	[Id] [int] NOT NULL,
	[CreateDate] [datetime] NOT NULL,
	[Comment] [nvarchar](1000) NOT NULL,
	CONSTRAINT [PK_nv1000_Id] PRIMARY KEY CLUSTERED ([Id] ASC)
)

CREATE TABLE [dbo].[nvmax]
(
	[Id] [int] NOT NULL,
	[CreateDate] [datetime] NOT NULL,
	[Comment] [nvarchar](max) NOT NULL,
	CONSTRAINT [PK_nvmax_Id] PRIMARY KEY CLUSTERED ([Id] ASC)
)

The tables differ in the Comment column type. Let’s populate them with 200,000 rows like this:

Id          CreateDate              Comment
----------- ----------------------- ----------
1           2025-01-01 00:00:01.000 1
2           2025-01-01 00:00:02.000 2
3           2025-01-01 00:00:03.000 3
4           2025-01-01 00:00:04.000 4
5           2025-01-01 00:00:05.000 5
6           2025-01-01 00:00:06.000 6
7           2025-01-01 00:00:07.000 7
8           2025-01-01 00:00:08.000 8
9           2025-01-01 00:00:09.000 9
10          2025-01-01 00:00:10.000 10

Testing Data Select

Let’s select all data from the tables and examine the execution plans:

select *
from nv100

select *
from nv1000

select *
from nvmax

At first glance, all three plans look identical:

Let’s look at the single arrow in the first query’s plan:

SQL Server correctly estimated the number of rows to return: 200,000. This is predictable since we’re selecting all data from the table. The more interesting metric is Estimated Row Size — the estimated size of one row in bytes. How is this calculated? It includes row metadata and the actual data. For our case: 4 bytes for the int identifier, 8 bytes for the datetime field, and according to documentation, nvarchar(n) requires 2*n bytes. For nvarchar(100), this is 200 bytes. Total: 4 + 8 + 200 = 212 bytes. Why does SQL Server estimate 123 bytes?

When generating the query plan, SQL Server estimates nvarchar(n) columns as half their maximum byte size. Thus, 100 bytes for the first table. The estimated row size becomes 4 + 8 + 100 = 112 bytes, plus metadata. Let’s check the second table’s plan:

Here, Estimated Row Size increases to 1023 bytes, with 4 + 8 + 1000 = 1012 bytes for data. For nvarchar(max), which can store up to 2GB, does SQL Server estimate 1GB per row? No, it uses 4000 bytes as the default estimate:

If we exclude the Comment column, SQL Server provides accurate estimates:

select Id, CreateDate
from nvmax

Sorting

When compiling a query plan, SQL Server considers factors like table size, indexes, data types, and Estimated Row Size to determine memory allocation and whether to use TempDB (e.g., for sorting large datasets).

Let’s sort the data by Comment:

select *
from nv100
order by Comment desc

select *
from nv1000
order by Comment desc

select *
from nvmax
order by Comment desc

Execution plans:

The plans appear similar, but the second query shows a warning (Excessive Grant) on the SELECT operator:

SQL Server requested 295,616 KB (~300 MB) for execution but used only 14,584 KB (~15 MB) — 20 times less. The third query is worse:

It requested 1,122,496 KB (~1.1 GB) but used only 19,960 KB (~20 MB) — 56 times less. This unnecessary memory allocation can deprive other queries of resources.

Filtering

Let’s filter by the Comment column:

select *
from nv100
where Comment = N'12345'

select *
from nv1000
where Comment = N'12345'

select *
from nvmax
where Comment = N'12345'

Execution plans:

All queries scan the clustered index, but the nvmax table has an extra Filter operation. Why? nvarchar(max) data can be stored in two ways:

IN_ROW_DATA: Stored directly in the row if ≤ 8000 bytes.
LOB_DATA: Stored in separate pages if > 8000 bytes, with a pointer in the row.

Check storage types:

SELECT
	t.Name as TableName,
	a.type_desc as StorageType
FROM sys.tables t
JOIN sys.columns c ON t.object_id = c.object_id
JOIN sys.partitions p ON t.object_id = p.object_id
JOIN sys.allocation_units a ON p.partition_id = a.container_id
WHERE c.system_type_id = 231 -- nvarchar(max)
  AND a.type_desc IN ('IN_ROW_DATA', 'LOB_DATA')

Result:

TableName  StorageType
---------- -------------
nv100      IN_ROW_DATA
nv1000     IN_ROW_DATA
nvmax      IN_ROW_DATA
nvmax      LOB_DATA

For nvmax, data is stored in both formats, forcing SQL Server to retrieve data before applying filters, hence the extra Filter step.

Indexes

You cannot create an index on nvarchar(max). Attempting to index nvarchar(1000) generates a warning:

CREATE NONCLUSTERED INDEX [IX_nv1000_Comment] ON [dbo].[nv1000]
(
	[Comment] ASC
)

Warning! The maximum key length for a nonclustered index is 1700 bytes. The index 'IX_nv1000_Comment' has maximum length of 2000 bytes. For some combination of large values, the insert/update operation will fail.

Inserting a 1000-character string fails:

insert into nv1000
values(-1, GETUTCDATE(), REPLICATE(N'A',1000))

Msg 1946, Level 16, State 3, Line 1
Operation failed. The index entry of length 2000 bytes for the index 'IX_nv1000_Comment' exceeds the maximum length of 1700 bytes for nonclustered indexes.

Comparison

	nvarchar(100)	nvarchar(1000)	nvarchar(max)
Data Storage	IN_ROW_DATA	IN_ROW_DATA	IN_ROW_DATA/LOB_DATA
Estimated Row Size	100 bytes	1000 bytes	4000 bytes
Index Support	Yes	Yes (with restrictions)	No

Conclusions

The n in nvarchar(n) is not just a maximum length — it helps SQL Server optimize query plans.
Use nvarchar(max) only for genuinely large strings. Apply business logic constraints (e.g., user input limits) to the database.
The optimal nvarchar(n) size is twice the average stored string length.

References

nchar и nvarchar (Transact-SQL)

List initializer

Sergey Rogatnev — Sat, 06 Jan 2024 00:00:00 GMT

We all used to type new List<int> { 1, 2, 3, 4 } or new int[] { 1, 2, 3, 4} to initialize collections with some values. It looks similar in syntax but differs in behavior, and you should be careful if you are concerned about performance.

Array

As we know, arrays contains a sequence of elements with a fixed size. Once it was created it can't be changed during the lifetime of the instance.

var array = new int[] { 1, 2, 3, 4};

List<T>

When we don't know final size of a collection, or we need to add/delete elements during the lifetime, it's suitable to use the List<T> type.

var list = new List<int>();
list.Add(1);
list.Add(2);

At first glance it may seem that we can always use the list instead of the array — it has all array's features, but also can be dynamically changed. But to decide whether to use the list, we need to know more about its internal structure.

Part of the source code:

public class List<T> : IList<T>, IList, IReadOnlyList<T>
{
    private const int DefaultCapacity = 4;

    internal T[] _items;
    internal int _size;

    private static readonly T[] s_emptyArray = new T[0];

    // Constructs a List. The list is initially empty and has a capacity
    // of zero. Upon adding the first element to the list the capacity is
    // increased to DefaultCapacity, and then increased in multiples of two
    // as required.
    public List()
    {
        _items = s_emptyArray;
    }
    ...

    public int Capacity
    {
        get => _items.Length;
        set {...}
    }

    public int Count => _size;
}

_items - array with stored elements;
_size - count of elements stored in the list and size of the whole list;
Capacity - size of _items and maximum number of elements can be stored without a resize.

List resize

To put it simply, we can say that the list is just the array that can resize when needed. Every time we try to add one more item the list ensures _items has enough free space, otherwise it sets new capacity:

internal void Grow(int capacity)
{
    ...
    int newCapacity = _items.Length == 0 ? DefaultCapacity : 2 * _items.Length;
    ...
    Capacity = newCapacity;
}

Let's look closer to the Capacity property:

// Gets and sets the capacity of this list.  The capacity is the size of
// the internal array used to hold items.  When set, the internal
// array of the list is reallocated to the given capacity.
public int Capacity
{
    get => _items.Length;
    set
    {
        if (value < _size)
        {
            ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.value, ExceptionResource.ArgumentOutOfRange_SmallCapacity);
        }

        if (value != _items.Length)
        {
            if (value > 0)
            {
                T[] newItems = new T[value];
                if (_size > 0)
                {
                    Array.Copy(_items, newItems, _size);
                }
                _items = newItems;
            }
            else
            {
                _items = s_emptyArray;
            }
        }
    }
}

So, what do we see? Initially, every list is created with an empty internal array. After adding the first element the list creates the new array for 4 elements (DefaultCapacity is 4). And when the current array is exhausted, the new one is created with doubled size, and all elements are copied.

Performance

What happens when we create a new list and initialize it with some values?

var list = new List<int> { 1, 2, 3, 4, 5 };

It looks like array initialization but works very different way. According to the documentation, any type that implements IEnumerable and has Add method can be used with collection initializer.

So the previous example is just short form of serial calls of Add method:

var list = new List<int>();
list.Add(1);
list.Add(2);
list.Add(3);
list.Add(4);
list.Add(5);

The compiler just transforms short collection initializer and adds the necessary calls automatically. And this may cause performance degradation.

Let's take a closer look at the process of adding 5 elements:

Before the first Add call the list is empty, the internal array is empty.
The first added element creates the new internal array for 4 elements.
Elements 2, 3, 4 change nothing.
When we add the fifth element, the internal array is full and needs to be resized. The new array of size 8 is created, all elements are copied from the previous array, and the fifth element is added.

At the end we have the list with 5 elements and an internal array for 8 elements. At the same time, we created 2 arrays, and the final one wastes 37.5% of its space for nothing.

As you might guess, creating new arrays and copying elements causes memory allocations and takes additional time.

This could be an unpleasant surprise for critical places. Do we have a solution? Yes!

Capacity

If we know or suggest the final size of the list, we can create it with initial capacity.

var list = new List<int>(5);
list.Add(1);
list.Add(2);
list.Add(3);
list.Add(4);
list.Add(5);

var list = new List<int>(5) { 1, 2, 3, 4, 5 };

Now we immediately create one array for 5 elements and no more unnecessary allocations.

Benchmarks

Let's compare creating lists with initial capacity and without it.

Benchmark code. Click to expand.

using System.Collections.Generic;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;

namespace Benchmarks
{
    [ShortRunJob(RuntimeMoniker.Net48)]
    [ShortRunJob(RuntimeMoniker.Net60)]
    [ShortRunJob(RuntimeMoniker.Net80)]
    [MemoryDiagnoser]
    public class InitListBenchmark
    {
        [BenchmarkCategory("One")]
        [Benchmark]
        public List<int> InitList1()
        {
            return new List<int> {1};
        }

        [BenchmarkCategory("One")]
        [Benchmark]
        public List<int> InitListWithSize1()
        {
            return new List<int>(1) {1};
        }

        [BenchmarkCategory("Five")]
        [Benchmark]
        public List<int> InitList5()
        {
            return new List<int> {1, 2, 3, 4, 5};
        }
        
        [BenchmarkCategory("Five")]
        [Benchmark]
        public List<int> InitListWithSize5()
        {
            return new List<int>(5) {1, 2, 3, 4, 5};
        }
        
        [BenchmarkCategory("Ten")]
        [Benchmark]
        public List<int> InitList10()
        {
            return new List<int> {1, 2, 3, 4, 5, 6, 7, 8, 9, 0};
        }
        
        [BenchmarkCategory("Ten")]
        [Benchmark]
        public List<int> InitListWithSize10()
        {
            return new List<int>(10) {1, 2, 3, 4, 5, 6, 7, 8, 9, 0};
        }
    }
}


BenchmarkDotNet v0.13.8, Windows 10 (10.0.19045.3930/22H2/2022Update)
AMD Ryzen 7 7840H with Radeon 780M Graphics, 1 CPU, 16 logical and 8 physical cores
.NET SDK 8.0.101
  [Host]                      : .NET 6.0.26 (6.0.2623.60508), X64 RyuJIT AVX2
  ShortRun-.NET 6.0           : .NET 6.0.26 (6.0.2623.60508), X64 RyuJIT AVX2
  ShortRun-.NET 8.0           : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2
  ShortRun-.NET Framework 4.8 : .NET Framework 4.8.1 (4.8.9195.0), X64 RyuJIT VectorSize=256

IterationCount=3  LaunchCount=1  WarmupCount=3

Method	Runtime	Mean	Gen0	Allocated
InitList1	.NET Framework 4.8	14.523 ns	0.0127	80 B
InitListWithSize1	.NET Framework 4.8	6.678 ns	0.0115	72 B
InitList5	.NET Framework 4.8	33.884 ns	0.0216	136 B
InitListWithSize5	.NET Framework 4.8	12.232 ns	0.0140	88 B
InitList10	.NET Framework 4.8	54.156 ns	0.0357	225 B
InitListWithSize10	.NET Framework 4.8	18.609 ns	0.0166	104 B
InitList1	.NET 6.0	10.599 ns	0.0086	72 B
InitListWithSize1	.NET 6.0	5.727 ns	0.0076	64 B
InitList5	.NET 6.0	22.426 ns	0.0153	128 B
InitListWithSize5	.NET 6.0	8.611 ns	0.0096	80 B
InitList10	.NET 6.0	37.512 ns	0.0258	216 B
InitListWithSize10	.NET 6.0	12.740 ns	0.0115	96 B
InitList1	.NET 8.0	9.515 ns	0.0086	72 B
InitListWithSize1	.NET 8.0	6.638 ns	0.0076	64 B
InitList5	.NET 8.0	23.910 ns	0.0153	128 B
InitListWithSize5	.NET 8.0	9.153 ns	0.0096	80 B
InitList10	.NET 8.0	38.765 ns	0.0258	216 B
InitListWithSize10	.NET 8.0	12.288 ns	0.0115	96 B

When we set initial capacity, it doesn't lead to unnecessary allocations, and we see better performance. And there is no excess memory traffic. The more elements we add to the list, the more difference we see in benchmarks.

Analyzer

If performance is critical to your project, you should pay attention to these situations. Automatic diagnostics can help you. I support a set of roslyn-based diagnostics - Collections.Analyzer and now it can detect lists with collection initializer and without initial capacity.

Opinion

If you need a static collection, you won't change (add or remove elements) — use arrays.
If you create a list, and you know exactly its future size — set initial capacity with this size.
If you create a list, and you don't know its future size — set the expected size.

Harmful collection transformations. Part 3: collections

Sergey Rogatnev — Sat, 08 Jan 2022 00:00:00 GMT

Starting with string in the first post we continue to study examples with collection transformations and how they affect our applications.

Collections

.NET contains hundreds of different collections for any purpose, but it looks like developers used to use List<T> or Array so much that they convert anything into an array just in case. In most cases, this is unnecessary. Sometimes it's harmful as we found out earlier. Let's look at some examples:

Count elements

Sometimes we don't need all elements but only count them. And we can see the code like this:

return certificates.Where(x => certificateStatesByFormId.SafeGet(x.Id) == Form.CertificateState.Revoked || x.ValidToTicks <= nowTicks - 1)
   .ToArray().Length;

Or another example:

public int GetTotalAmount(Guid managerId)
{
   return filterSettingsHandler
       .Select<ClientFilterDbo>(managerId)
       .Where(IsVisible)
       .ToArray().Length;
}

Creating an array to count elements is unnecessary. LINQ provides a Count() method that does what we need without harmful overhead. And we can easily rewrite examples:

return certificates.Count(x => certificateStatesByFormId.SafeGet(x.Id) == Form.CertificateState.Revoked || x.ValidToTicks <= nowTicks - 1);

public int GetTotalAmount(Guid managerId)
{
   return filterSettingsHandler
       .Select<ClientFilterDbo>(managerId)
       .Count(IsVisible);
}

We get the cleaner and more readable code. Besides that, we got some performance improvements.

Benchmark code. Click to expand.

namespace ToCharArrayBenchmark
{
    using System.Linq;
    using BenchmarkDotNet.Attributes;
    using BenchmarkDotNet.Jobs;

    [HtmlExporter]
    [RPlotExporter]
    [SimpleJob(RuntimeMoniker.Net48, baseline: true)]
    [SimpleJob(RuntimeMoniker.NetCoreApp31)]
    [SimpleJob(RuntimeMoniker.Net50)]
    [MemoryDiagnoser]
    public class CollectionsCountBenchmark
    {
        [Params(10, 20, 50, 100, 500)]
        public int N { get; set; }
        
        private string[] array;
        
        [GlobalSetup]
        public void SetUp()
        {
            array = new string[N];
            for (int i = 0; i < N; i++)
            {
                array[i] = i.ToString();
            }
        }

        [Benchmark(Baseline = true)]
        public int Count()
        {
            return array.Where(o => o != "1").Count();
        }
        
        [Benchmark]
        public int ArrayLength()
        {
            return array.Where(o => o != "1").ToArray().Length;
        }
    }
}

Full results in table format. Click to expand.


BenchmarkDotNet=v0.13.0, OS=Windows 10.0.19043.1348 (21H1/May2021Update)
AMD Ryzen 7 4700U with Radeon Graphics, 1 CPU, 8 logical and 8 physical cores
  [Host]             : .NET Framework 4.8 (4.8.4420.0), X64 RyuJIT
  .NET 5.0           : .NET 5.0.7 (5.0.721.25508), X64 RyuJIT
  .NET Core 3.1      : .NET Core 3.1.20 (CoreCLR 4.700.21.47003, CoreFX 4.700.21.47101), X64 RyuJIT
  .NET Framework 4.8 : .NET Framework 4.8 (4.8.4420.0), X64 RyuJIT

Method	Runtime	N	Mean	Error	StdDev	Ratio	RatioSD	Gen 0	Allocated
Count	.NET 5.0	10	106.4 ns	1.78 ns	1.67 ns	0.70	0.02	0.0229	48 B
ArrayLength	.NET 5.0	10	263.5 ns	3.93 ns	3.28 ns	1.72	0.04	0.1798	376 B
Count	.NET Core 3.1	10	108.2 ns	1.66 ns	1.39 ns	0.71	0.02	0.0229	48 B
ArrayLength	.NET Core 3.1	10	286.0 ns	3.52 ns	3.12 ns	1.88	0.03	0.1798	376 B
Count	.NET Framework 4.8	10	151.7 ns	2.52 ns	2.90 ns	1.00	0.00	0.0267	56 B
ArrayLength	.NET Framework 4.8	10	361.4 ns	6.58 ns	9.65 ns	2.39	0.10	0.2141	449 B

Count	.NET 5.0	20	139.5 ns	0.79 ns	0.74 ns	0.61	0.01	0.0229	48 B
ArrayLength	.NET 5.0	20	378.4 ns	7.24 ns	6.77 ns	1.67	0.04	0.2713	568 B
Count	.NET Core 3.1	20	136.5 ns	0.63 ns	0.59 ns	0.60	0.01	0.0229	48 B
ArrayLength	.NET Core 3.1	20	419.6 ns	4.89 ns	4.57 ns	1.85	0.03	0.2713	568 B
Count	.NET Framework 4.8	20	227.0 ns	3.01 ns	2.82 ns	1.00	0.00	0.0267	56 B
ArrayLength	.NET Framework 4.8	20	595.2 ns	6.18 ns	5.48 ns	2.63	0.03	0.3862	810 B

Count	.NET 5.0	50	253.4 ns	0.99 ns	0.83 ns	0.59	0.00	0.0229	48 B
ArrayLength	.NET 5.0	50	708.7 ns	8.07 ns	7.55 ns	1.66	0.02	0.5121	1,072 B
Count	.NET Core 3.1	50	248.0 ns	2.19 ns	2.05 ns	0.58	0.00	0.0229	48 B
ArrayLength	.NET Core 3.1	50	763.3 ns	12.10 ns	10.72 ns	1.79	0.03	0.5121	1,072 B
Count	.NET Framework 4.8	50	426.5 ns	2.19 ns	1.94 ns	1.00	0.00	0.0267	56 B
ArrayLength	.NET Framework 4.8	50	1,334.4 ns	26.38 ns	25.91 ns	3.12	0.06	0.7572	1,589 B

Count	.NET 5.0	100	458.0 ns	2.64 ns	2.34 ns	0.60	0.00	0.0229	48 B
ArrayLength	.NET 5.0	100	1,185.5 ns	11.97 ns	11.20 ns	1.55	0.02	0.9060	1,896 B
Count	.NET Core 3.1	100	414.5 ns	1.15 ns	0.96 ns	0.54	0.00	0.0229	48 B
ArrayLength	.NET Core 3.1	100	1,236.0 ns	23.07 ns	22.65 ns	1.62	0.04	0.9060	1,896 B
Count	.NET Framework 4.8	100	765.6 ns	4.88 ns	4.32 ns	1.00	0.00	0.0267	56 B
ArrayLength	.NET Framework 4.8	100	2,521.6 ns	48.92 ns	61.87 ns	3.30	0.09	1.4496	3,041 B

Count	.NET 5.0	500	2,010.3 ns	7.45 ns	6.97 ns	0.61	0.02	0.0229	48 B
ArrayLength	.NET 5.0	500	4,572.5 ns	86.28 ns	80.71 ns	1.38	0.05	4.0283	8,432 B
Count	.NET Core 3.1	500	1,893.9 ns	9.48 ns	8.40 ns	0.57	0.02	0.0229	48 B
ArrayLength	.NET Core 3.1	500	4,395.2 ns	86.59 ns	88.92 ns	1.33	0.05	4.0283	8,432 B
Count	.NET Framework 4.8	500	3,298.3 ns	64.73 ns	79.49 ns	1.00	0.00	0.0267	56 B
ArrayLength	.NET Framework 4.8	500	11,853.8 ns	235.81 ns	280.71 ns	3.59	0.12	5.9204	12,470 B

// * Legends *
  N         : Value of the 'N' parameter
  Mean      : Arithmetic mean of all measurements
  Error     : Half of 99.9% confidence interval
  StdDev    : Standard deviation of all measurements
  Ratio     : Mean of the ratio distribution ([Current]/[Baseline])
  RatioSD   : Standard deviation of the ratio distribution ([Current]/[Baseline])
  Gen 0     : GC Generation 0 collects per 1000 operations
  Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  1 us      : 1 Microsecond (0.000001 sec)

The following diagram shows execution time of both methods (in microseconds) for .NET 5 platform:

As we see, converting to array is about 2.5 times slower even for the most optimized .NET 5. Other frameworks are up to 4 times slower.

And if this is not enough, let's look at memory allocations (in bytes):

The LINQ method creates few objects and does not lead to additional GC regardless of the size of the original collection.

We can confidently say that creating an array to count elements is a harmful transformation.

`ToArray()` with LINQ

The widespread pattern - to call ToArray() (just in case) before LINQ-methods.

var correctPaymentsByBillId = correctPayments
   .GroupBy(x => x.BillId)
   .ToDictionary(x => x.Key, x => x.ToArray().Last());

Here GroupBy returns IGrouping<out TKey, out TElement> interface and it, in turns, implements IEnumerable<TElement>. Based on this we can simply remove this ToArray():

var correctPaymentsByBillId = correctPayments
   .GroupBy(x => x.BillId)
   .ToDictionary(x => x.Key, x => x.Last());

Array is often used as a return type of methods like this:

private OrderPackage[] ReadRootPackages(params Guid[] packageIds)
{
    var rootPackages = new Dictionary<Guid, OrderPackage>();
    //...
    return rootPackages.Values.ToArray();
}

And the result doesn't have to be an array or any specific collection:

public OrderPackageDto[] SelectTariffForestByPackageIds(Guid[] packageIds)
{
	var rootOrderPackages = ReadRootPackages(packageIds);
	var masterPackageIds = rootOrderPackages.Select(package => package.Id).ToArray();
	...
}

As we see, ReadRootPackages can return ICollection<OrderPackage> or IEnumerable<OrderPackage> and ToArray() is redundant:

private IEnumerable<OrderPackage> ReadRootPackages(params Guid[] packageIds)
{
    var rootPackages = new Dictionary<Guid, OrderPackage>();
    //...
    return rootPackages.Values;
}

The most obvious example is declaring method return type as IEnumerable but returning an array:

public IEnumerable<Division> GetDivisions(HashSet<SupportUserAccessRight> accessRights)
{
   return divisions.Where(d => d.AccessRights.Intersect(accessRights).Any()).OrderBy(d => d.Priority).ToArray();
}

Like the previous example GetDivisions is used in LINQ-methods chain, and we can remove ToArray() call:

public IEnumerable<Division> GetDivisions(HashSet<SupportUserAccessRight> accessRights)
{
   return divisions.Where(d => d.AccessRights.Intersect(accessRights).Any()).OrderBy(d => d.Priority);
}

Usually, you don't need a collection between LINQ-methods, because of the lazy evaluation nature of LINQ. How does ToArray() affect performance depending on different collections? Let's compare two versions - with ToArray() and without it.

Benchmark code. Click to expand.

namespace ToCharArrayBenchmark
{
    using System.Collections.Generic;
    using System.Linq;
    using BenchmarkDotNet.Attributes;
    using BenchmarkDotNet.Engines;
    using BenchmarkDotNet.Jobs;

    [HtmlExporter]
    [RPlotExporter]
    [SimpleJob(RuntimeMoniker.Net48, baseline: true)]
    [SimpleJob(RuntimeMoniker.NetCoreApp31)]
    [SimpleJob(RuntimeMoniker.Net50)]
    [MemoryDiagnoser]
    public class CollectionToArrayBenchmark
    {
        [Params(10, 20, 50, 100, 500)]
        public int N { get; set; }

        private List<string> list;
        private string[] array;
        private HashSet<string> hashSet;
        private Dictionary<long, string> dictionary;
        private readonly Consumer consumer = new Consumer();

        [GlobalSetup]
        public void SetUp()
        {
            array = new string[N];
            list = new List<string>(N);
            dictionary = new Dictionary<long, string>(N);
            hashSet = new HashSet<string>(N);

            for (int i = 0; i < N; i++)
            {
                list.Add(i.ToString());
                array[i] = i.ToString();
                hashSet.Add(i.ToString());
                dictionary[i] = i.ToString();
            }
        }

        [Benchmark(Baseline = true)]
        public void FilterArray()
        {
            GetFilteredEnumerable(array).Consume(consumer);
        }

        [Benchmark]
        public void FilterList()
        {
            GetFilteredEnumerable(list).Consume(consumer);
        }

        [Benchmark]
        public void FilterHashSet()
        {
            GetFilteredEnumerable(hashSet).Consume(consumer);
        }

        [Benchmark]
        public void FilterDictionary()
        {
            GetFilteredEnumerable(dictionary.Values).Consume(consumer);
        }

        [Benchmark]
        public void FilterArray_ToArray()
        {
            GetFilteredArray(array).Consume(consumer);
        }

        [Benchmark]
        public void FilterList_ToArray()
        {
            GetFilteredArray(list).Consume(consumer);
        }

        [Benchmark]
        public void FilterHashSet_ToArray()
        {
            GetFilteredArray(hashSet).Consume(consumer);
        }

        [Benchmark]
        public void FilterDictionary_ToArray()
        {
            GetFilteredArray(dictionary.Values).Consume(consumer);
        }

        private static IEnumerable<string> GetFilteredEnumerable(IEnumerable<string> strings)
        {
            return strings.Where(o => o.Length > 1).Select(o => o);
        }

        private static string[] GetFilteredArray(IEnumerable<string> strings)
        {
            return strings.Where(o => o.Length > 1).ToArray();
        }
    }
}

Full results in table format. Click to expand.

BenchmarkDotNet=v0.13.0, OS=Windows 10.0.19043.1288 (21H1/May2021Update)
AMD Ryzen 7 4700U with Radeon Graphics, 1 CPU, 8 logical and 8 physical cores
  [Host]             : .NET Framework 4.8 (4.8.4420.0), X64 RyuJIT
  .NET 5.0           : .NET 5.0.7 (5.0.721.25508), X64 RyuJIT
  .NET Core 3.1      : .NET Core 3.1.20 (CoreCLR 4.700.21.47003, CoreFX 4.700.21.47101), X64 RyuJIT
  .NET Framework 4.8 : .NET Framework 4.8 (4.8.4420.0), X64 RyuJIT

Method	Runtime	N	Mean	Error	StdDev	Median	Ratio	RatioSD	Gen 0	Allocated
FilterArray	.NET 5.0	10	88.86 ns	0.937 ns	0.876 ns	88.73 ns	0.80	0.01	0.0497	104 B
FilterList	.NET 5.0	10	156.24 ns	1.318 ns	1.169 ns	155.86 ns	1.40	0.02	0.0725	152 B
FilterHashSet	.NET 5.0	10	212.50 ns	3.097 ns	2.746 ns	211.43 ns	1.91	0.03	0.0763	160 B
FilterDictionary	.NET 5.0	10	209.59 ns	2.441 ns	2.283 ns	209.03 ns	1.89	0.03	0.0763	160 B
FilterArray_ToArray	.NET 5.0	10	88.34 ns	1.332 ns	1.180 ns	88.73 ns	0.79	0.01	0.0229	48 B
FilterList_ToArray	.NET 5.0	10	100.15 ns	1.151 ns	0.961 ns	100.02 ns	0.90	0.01	0.0343	72 B
FilterHashSet_ToArray	.NET 5.0	10	197.00 ns	1.438 ns	1.345 ns	196.58 ns	1.77	0.03	0.0458	96 B
FilterDictionary_ToArray	.NET 5.0	10	206.34 ns	3.901 ns	3.649 ns	204.96 ns	1.86	0.04	0.0458	96 B
FilterArray	.NET Core 3.1	10	100.98 ns	1.096 ns	0.972 ns	100.91 ns	0.91	0.01	0.0497	104 B
FilterList	.NET Core 3.1	10	166.74 ns	1.152 ns	1.021 ns	166.57 ns	1.50	0.02	0.0725	152 B
FilterHashSet	.NET Core 3.1	10	240.48 ns	4.734 ns	7.644 ns	236.99 ns	2.16	0.09	0.0763	160 B
FilterDictionary	.NET Core 3.1	10	210.68 ns	1.404 ns	1.173 ns	210.80 ns	1.89	0.03	0.0763	160 B
FilterArray_ToArray	.NET Core 3.1	10	94.07 ns	1.784 ns	2.615 ns	93.07 ns	0.84	0.03	0.0229	48 B
FilterList_ToArray	.NET Core 3.1	10	97.77 ns	1.811 ns	1.606 ns	97.25 ns	0.88	0.02	0.0343	72 B
FilterHashSet_ToArray	.NET Core 3.1	10	210.94 ns	3.940 ns	5.650 ns	210.06 ns	1.92	0.06	0.0458	96 B
FilterDictionary_ToArray	.NET Core 3.1	10	204.44 ns	3.835 ns	3.587 ns	203.66 ns	1.83	0.04	0.0458	96 B
FilterArray	.NET Framework 4.8	10	111.34 ns	1.837 ns	1.534 ns	111.21 ns	1.00	0.00	0.0573	120 B
FilterList	.NET Framework 4.8	10	161.72 ns	2.305 ns	2.156 ns	161.68 ns	1.45	0.03	0.0725	152 B
FilterHashSet	.NET Framework 4.8	10	213.74 ns	3.929 ns	3.675 ns	213.37 ns	1.92	0.05	0.0763	160 B
FilterDictionary	.NET Framework 4.8	10	212.36 ns	3.331 ns	3.116 ns	211.33 ns	1.91	0.04	0.0763	160 B
FilterArray_ToArray	.NET Framework 4.8	10	125.26 ns	2.469 ns	2.310 ns	124.86 ns	1.12	0.03	0.0381	80 B
FilterList_ToArray	.NET Framework 4.8	10	182.54 ns	2.822 ns	2.640 ns	182.31 ns	1.64	0.04	0.0458	96 B
FilterHashSet_ToArray	.NET Framework 4.8	10	223.26 ns	4.104 ns	3.638 ns	223.15 ns	2.01	0.05	0.0572	120 B
FilterDictionary_ToArray	.NET Framework 4.8	10	214.85 ns	1.389 ns	1.084 ns	214.84 ns	1.93	0.03	0.0572	120 B

FilterArray	.NET 5.0	20	204.77 ns	2.235 ns	1.981 ns	204.25 ns	0.81	0.02	0.0496	104 B
FilterList	.NET 5.0	20	335.74 ns	3.112 ns	2.911 ns	335.81 ns	1.34	0.03	0.0725	152 B
FilterHashSet	.NET 5.0	20	416.95 ns	2.229 ns	2.085 ns	415.96 ns	1.66	0.04	0.0763	160 B
FilterDictionary	.NET 5.0	20	437.06 ns	6.477 ns	6.058 ns	438.21 ns	1.74	0.04	0.0763	160 B
FilterArray_ToArray	.NET 5.0	20	338.20 ns	5.587 ns	4.953 ns	339.08 ns	1.34	0.04	0.1988	416 B
FilterList_ToArray	.NET 5.0	20	358.90 ns	4.098 ns	3.422 ns	358.15 ns	1.42	0.04	0.2103	440 B
FilterHashSet_ToArray	.NET 5.0	20	557.83 ns	4.063 ns	3.800 ns	558.38 ns	2.22	0.06	0.2213	464 B
FilterDictionary_ToArray	.NET 5.0	20	574.07 ns	7.046 ns	6.246 ns	574.29 ns	2.28	0.05	0.2213	464 B
FilterArray	.NET Core 3.1	20	215.45 ns	1.700 ns	1.419 ns	215.22 ns	0.86	0.02	0.0496	104 B
FilterList	.NET Core 3.1	20	320.02 ns	2.990 ns	2.651 ns	319.63 ns	1.27	0.03	0.0725	152 B
FilterHashSet	.NET Core 3.1	20	437.02 ns	4.356 ns	3.861 ns	436.05 ns	1.74	0.05	0.0763	160 B
FilterDictionary	.NET Core 3.1	20	452.16 ns	8.495 ns	7.947 ns	453.44 ns	1.80	0.06	0.0763	160 B
FilterArray_ToArray	.NET Core 3.1	20	340.41 ns	4.727 ns	4.190 ns	339.43 ns	1.35	0.04	0.1988	416 B
FilterList_ToArray	.NET Core 3.1	20	353.83 ns	6.133 ns	7.532 ns	354.83 ns	1.42	0.04	0.2103	440 B
FilterHashSet_ToArray	.NET Core 3.1	20	576.33 ns	8.148 ns	7.223 ns	575.38 ns	2.29	0.06	0.2213	464 B
FilterDictionary_ToArray	.NET Core 3.1	20	584.74 ns	11.442 ns	10.143 ns	580.87 ns	2.32	0.08	0.2213	464 B
FilterArray	.NET Framework 4.8	20	248.80 ns	4.782 ns	6.546 ns	247.30 ns	1.00	0.00	0.0572	120 B
FilterList	.NET Framework 4.8	20	329.58 ns	3.299 ns	2.925 ns	328.61 ns	1.31	0.03	0.0725	152 B
FilterHashSet	.NET Framework 4.8	20	443.56 ns	8.577 ns	9.177 ns	441.12 ns	1.77	0.07	0.0763	160 B
FilterDictionary	.NET Framework 4.8	20	461.62 ns	7.660 ns	6.791 ns	459.31 ns	1.83	0.06	0.0763	160 B
FilterArray_ToArray	.NET Framework 4.8	20	469.41 ns	7.360 ns	6.525 ns	470.37 ns	1.86	0.03	0.2332	489 B
FilterList_ToArray	.NET Framework 4.8	20	575.07 ns	6.210 ns	5.809 ns	575.53 ns	2.29	0.06	0.2403	505 B
FilterHashSet_ToArray	.NET Framework 4.8	20	686.90 ns	13.321 ns	14.253 ns	687.12 ns	2.75	0.10	0.2518	530 B
FilterDictionary_ToArray	.NET Framework 4.8	20	677.82 ns	10.143 ns	9.488 ns	675.06 ns	2.70	0.08	0.2518	530 B

FilterArray	.NET 5.0	50	519.68 ns	8.640 ns	7.215 ns	517.65 ns	0.89	0.01	0.0496	104 B
FilterList	.NET 5.0	50	819.43 ns	11.886 ns	11.118 ns	814.16 ns	1.41	0.02	0.0725	152 B
FilterHashSet	.NET 5.0	50	972.46 ns	5.558 ns	5.199 ns	973.23 ns	1.67	0.01	0.0763	160 B
FilterDictionary	.NET 5.0	50	972.48 ns	2.824 ns	2.359 ns	972.40 ns	1.67	0.01	0.0763	160 B
FilterArray_ToArray	.NET 5.0	50	810.60 ns	12.451 ns	11.647 ns	808.18 ns	1.39	0.02	0.4930	1,032 B
FilterList_ToArray	.NET 5.0	50	851.79 ns	11.612 ns	10.294 ns	848.90 ns	1.46	0.02	0.5045	1,056 B
FilterHashSet_ToArray	.NET 5.0	50	1,479.79 ns	28.211 ns	26.389 ns	1,471.44 ns	2.54	0.06	0.5684	1,192 B
FilterDictionary_ToArray	.NET 5.0	50	1,401.39 ns	26.216 ns	24.522 ns	1,396.34 ns	2.41	0.05	0.5684	1,192 B
FilterArray	.NET Core 3.1	50	534.26 ns	5.393 ns	4.781 ns	533.28 ns	0.92	0.01	0.0496	104 B
FilterList	.NET Core 3.1	50	775.65 ns	6.106 ns	5.413 ns	775.77 ns	1.33	0.01	0.0725	152 B
FilterHashSet	.NET Core 3.1	50	1,016.81 ns	15.748 ns	14.731 ns	1,020.79 ns	1.74	0.02	0.0763	160 B
FilterDictionary	.NET Core 3.1	50	1,056.04 ns	20.453 ns	26.595 ns	1,056.13 ns	1.83	0.05	0.0763	160 B
FilterArray_ToArray	.NET Core 3.1	50	940.75 ns	14.501 ns	13.564 ns	942.47 ns	1.61	0.03	0.4921	1,032 B
FilterList_ToArray	.NET Core 3.1	50	977.71 ns	19.087 ns	34.417 ns	983.53 ns	1.67	0.08	0.5035	1,056 B
FilterHashSet_ToArray	.NET Core 3.1	50	1,453.36 ns	27.352 ns	25.585 ns	1,443.28 ns	2.50	0.05	0.5684	1,192 B
FilterDictionary_ToArray	.NET Core 3.1	50	1,368.01 ns	11.451 ns	10.151 ns	1,365.62 ns	2.35	0.02	0.5684	1,192 B
FilterArray	.NET Framework 4.8	50	582.74 ns	3.555 ns	2.969 ns	582.90 ns	1.00	0.00	0.0572	120 B
FilterList	.NET Framework 4.8	50	783.91 ns	1.892 ns	1.580 ns	784.02 ns	1.35	0.01	0.0725	152 B
FilterHashSet	.NET Framework 4.8	50	1,071.60 ns	21.374 ns	28.533 ns	1,072.29 ns	1.83	0.05	0.0763	160 B
FilterDictionary	.NET Framework 4.8	50	1,139.82 ns	21.384 ns	20.003 ns	1,141.93 ns	1.96	0.04	0.0763	160 B
FilterArray_ToArray	.NET Framework 4.8	50	1,367.50 ns	27.057 ns	65.861 ns	1,331.99 ns	2.52	0.08	0.7381	1,549 B
FilterList_ToArray	.NET Framework 4.8	50	1,597.52 ns	23.829 ns	19.899 ns	1,603.19 ns	2.74	0.04	0.7458	1,565 B
FilterHashSet_ToArray	.NET Framework 4.8	50	1,808.94 ns	23.400 ns	21.888 ns	1,811.74 ns	3.11	0.04	0.7572	1,589 B
FilterDictionary_ToArray	.NET Framework 4.8	50	1,832.78 ns	34.906 ns	34.282 ns	1,840.36 ns	3.14	0.07	0.7572	1,589 B

FilterArray	.NET 5.0	100	1,049.10 ns	10.403 ns	8.687 ns	1,047.02 ns	0.87	0.02	0.0496	104 B
FilterList	.NET 5.0	100	1,516.00 ns	29.028 ns	27.153 ns	1,510.02 ns	1.26	0.02	0.0725	152 B
FilterHashSet	.NET 5.0	100	1,938.09 ns	29.526 ns	27.619 ns	1,924.95 ns	1.61	0.03	0.0763	160 B
FilterDictionary	.NET 5.0	100	2,045.10 ns	15.618 ns	14.609 ns	2,035.70 ns	1.70	0.03	0.0763	160 B
FilterArray_ToArray	.NET 5.0	100	1,499.65 ns	21.245 ns	19.872 ns	1,498.60 ns	1.25	0.03	0.8869	1,856 B
FilterList_ToArray	.NET 5.0	100	1,555.10 ns	31.036 ns	31.871 ns	1,560.42 ns	1.30	0.03	0.8984	1,880 B
FilterHashSet_ToArray	.NET 5.0	100	2,457.19 ns	34.939 ns	30.973 ns	2,451.15 ns	2.05	0.04	1.0147	2,128 B
FilterDictionary_ToArray	.NET 5.0	100	2,599.85 ns	39.720 ns	37.155 ns	2,585.06 ns	2.17	0.04	1.0147	2,128 B
FilterArray	.NET Core 3.1	100	1,102.10 ns	4.401 ns	4.117 ns	1,101.34 ns	0.92	0.01	0.0496	104 B
FilterList	.NET Core 3.1	100	1,423.55 ns	3.328 ns	2.599 ns	1,423.44 ns	1.19	0.02	0.0725	152 B
FilterHashSet	.NET Core 3.1	100	2,160.32 ns	42.036 ns	51.624 ns	2,159.38 ns	1.81	0.06	0.0763	160 B
FilterDictionary	.NET Core 3.1	100	2,190.92 ns	42.564 ns	53.830 ns	2,194.61 ns	1.81	0.05	0.0763	160 B
FilterArray_ToArray	.NET Core 3.1	100	1,530.44 ns	20.335 ns	19.021 ns	1,526.51 ns	1.28	0.03	0.8869	1,856 B
FilterList_ToArray	.NET Core 3.1	100	1,583.65 ns	31.314 ns	36.061 ns	1,568.48 ns	1.32	0.05	0.8984	1,880 B
FilterHashSet_ToArray	.NET Core 3.1	100	2,711.94 ns	54.067 ns	127.443 ns	2,673.45 ns	2.33	0.16	1.0147	2,128 B
FilterDictionary_ToArray	.NET Core 3.1	100	2,509.05 ns	43.450 ns	40.643 ns	2,508.83 ns	2.09	0.04	1.0147	2,128 B
FilterArray	.NET Framework 4.8	100	1,199.84 ns	21.589 ns	19.139 ns	1,195.87 ns	1.00	0.00	0.0572	120 B
FilterList	.NET Framework 4.8	100	1,614.03 ns	31.386 ns	27.823 ns	1,602.08 ns	1.35	0.04	0.0725	152 B
FilterHashSet	.NET Framework 4.8	100	2,035.95 ns	39.087 ns	32.640 ns	2,023.10 ns	1.70	0.04	0.0763	160 B
FilterDictionary	.NET Framework 4.8	100	2,295.34 ns	9.565 ns	8.947 ns	2,296.43 ns	1.91	0.03	0.0763	160 B
FilterArray_ToArray	.NET Framework 4.8	100	2,767.37 ns	17.560 ns	16.426 ns	2,768.58 ns	2.31	0.04	1.4305	3,001 B
FilterList_ToArray	.NET Framework 4.8	100	3,164.04 ns	11.241 ns	9.386 ns	3,164.16 ns	2.64	0.04	1.4381	3,017 B
FilterHashSet_ToArray	.NET Framework 4.8	100	3,660.94 ns	70.542 ns	65.985 ns	3,654.40 ns	3.06	0.08	1.4458	3,041 B
FilterDictionary_ToArray	.NET Framework 4.8	100	3,699.41 ns	21.608 ns	18.044 ns	3,693.30 ns	3.08	0.04	1.4458	3,041 B

FilterArray	.NET 5.0	500	5,357.99 ns	103.933 ns	149.057 ns	5,286.35 ns	0.93	0.03	0.0458	104 B
FilterList	.NET 5.0	500	7,633.34 ns	81.790 ns	72.504 ns	7,610.14 ns	1.33	0.02	0.0610	152 B
FilterHashSet	.NET 5.0	500	9,670.59 ns	67.964 ns	53.062 ns	9,665.57 ns	1.68	0.01	0.0763	160 B
FilterDictionary	.NET 5.0	500	9,571.07 ns	112.495 ns	93.938 ns	9,531.54 ns	1.67	0.03	0.0763	160 B
FilterArray_ToArray	.NET 5.0	500	6,904.08 ns	66.606 ns	62.303 ns	6,874.80 ns	1.20	0.01	4.0054	8,392 B
FilterList_ToArray	.NET 5.0	500	6,968.44 ns	83.854 ns	70.022 ns	6,975.29 ns	1.21	0.02	4.0131	8,416 B
FilterHashSet_ToArray	.NET 5.0	500	10,971.07 ns	109.378 ns	96.961 ns	10,951.74 ns	1.91	0.03	4.0741	8,536 B
FilterDictionary_ToArray	.NET 5.0	500	11,998.38 ns	211.175 ns	197.533 ns	11,998.62 ns	2.09	0.03	4.0741	8,536 B
FilterArray	.NET Core 3.1	500	5,347.93 ns	33.686 ns	28.129 ns	5,338.97 ns	0.93	0.01	0.0458	104 B
FilterList	.NET Core 3.1	500	7,483.89 ns	45.716 ns	35.692 ns	7,498.14 ns	1.30	0.01	0.0610	152 B
FilterHashSet	.NET Core 3.1	500	9,377.21 ns	116.563 ns	97.335 ns	9,337.82 ns	1.63	0.02	0.0763	160 B
FilterDictionary	.NET Core 3.1	500	9,933.11 ns	61.978 ns	51.755 ns	9,907.55 ns	1.73	0.01	0.0763	160 B
FilterArray_ToArray	.NET Core 3.1	500	7,688.61 ns	149.282 ns	223.439 ns	7,655.68 ns	1.35	0.04	4.0054	8,392 B
FilterList_ToArray	.NET Core 3.1	500	8,520.42 ns	170.171 ns	315.423 ns	8,541.70 ns	1.48	0.05	4.0131	8,416 B
FilterHashSet_ToArray	.NET Core 3.1	500	11,357.74 ns	167.598 ns	139.952 ns	11,288.67 ns	1.98	0.03	4.0741	8,536 B
FilterDictionary_ToArray	.NET Core 3.1	500	11,174.15 ns	183.748 ns	180.465 ns	11,096.25 ns	1.95	0.04	4.0741	8,536 B
FilterArray	.NET Framework 4.8	500	5,735.48 ns	35.151 ns	39.070 ns	5,726.53 ns	1.00	0.00	0.0534	120 B
FilterList	.NET Framework 4.8	500	8,031.11 ns	136.072 ns	186.257 ns	7,934.82 ns	1.40	0.04	0.0610	153 B
FilterHashSet	.NET Framework 4.8	500	10,124.26 ns	199.856 ns	266.803 ns	10,041.80 ns	1.78	0.04	0.0763	161 B
FilterDictionary	.NET Framework 4.8	500	11,615.37 ns	147.565 ns	123.223 ns	11,591.23 ns	2.02	0.02	0.0763	161 B
FilterArray_ToArray	.NET Framework 4.8	500	13,573.07 ns	128.080 ns	113.540 ns	13,595.41 ns	2.36	0.03	5.9052	12,434 B
FilterList_ToArray	.NET Framework 4.8	500	17,131.55 ns	340.744 ns	510.009 ns	17,036.32 ns	2.98	0.09	5.9204	12,445 B
FilterHashSet_ToArray	.NET Framework 4.8	500	18,801.99 ns	353.812 ns	295.449 ns	18,881.88 ns	3.28	0.06	5.9204	12,470 B
FilterDictionary_ToArray	.NET Framework 4.8	500	18,045.81 ns	133.857 ns	118.661 ns	18,032.49 ns	3.14	0.04	5.9204	12,470 B

// * Legends *
  N         : Value of the 'N' parameter
  Mean      : Arithmetic mean of all measurements
  Error     : Half of 99.9% confidence interval
  StdDev    : Standard deviation of all measurements
  Ratio     : Mean of the ratio distribution ([Current]/[Baseline])
  RatioSD   : Standard deviation of the ratio distribution ([Current]/[Baseline])
  Gen 0     : GC Generation 0 collects per 1000 operations
  Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  1 us      : 1 Microsecond (0.000001 sec)

The following diagram shows execution time of both methods (in microseconds) for .NET 5 platform:

The results are quite comparable, but .NET Framework is not so good:

Additional ToArray() call is about 2.5 times slower.

.NET made huge improvements in Core 3.1 and 5 versions. The following diagram shows execution time (in microseconds) of different .NET versions depending on the number of elements:

Array arguments

We used to use arrays as arguments in methods, but this can be a place where we can make another unnecessary transformation. Let's look at the example:

private async Task<OrderPackage[]> ReadRootPackagesAsync(params Guid[] packageIds)
{
	var queryPackageIds = new HashSet<Guid>(packageIds);
	...
	var packageIds = queryPackageIds.ToArray();
	var orderPackages = await orderPackageService.SelectByIdsAsync(packageIds).ConfigureAwait(false);
	...
}

public Task<OrderPackage[]> SelectByIdsAsync(params Guid[] packageIds)
{
	...
	return GetTable().Where(o => packageIds.Contains(o.Id));
}

GetTable() is a method that returns IQueryable<OrderPackage>, so the Where expression will be converted to a SQL code, and it's not necessary to use any specific collection. Here we can use IEnumerable<Guid> again, but I prefer an interface IReadOnlyCollection<T> that was introduced in .NET 4.5 Framework. The IReadOnlyCollection<T> interface indicates that a collection is materialized unlike IEnumerable<T> but we are not interested in a collection's structure (HashSet, List, Array, ... - does not matter). And on the other hand, using the IReadOnlyCollection<T> prevents the possibility of multiple enumeration of IEnumerable<T>. So, our modified example:

private async Task<OrderPackage[]> ReadRootPackagesAsync(params Guid[] packageIds)
{
	var queryPackageIds = new HashSet<Guid>(packageIds);
	...
	var orderPackages = await orderPackageService.SelectByIdsAsync(queryPackageIds).ConfigureAwait(false);
	...
}

public Task<OrderPackage[]> SelectByIdsAsync(IReadOnlyCollection<Guid> packageIds)
{
	...
	return GetTable().Where(o => packageIds.Contains(o.Id));
}

We pass the HashSet<Guid> directly to the SelectByIdsAsync method without using intermediate variables.

But we should be careful using common interfaces like IEnumerable<T> or IReadOnlyCollection<T> to avoid antipatterns like this:

public UserSearchModel BuildUserSearchModel(IEnumerable<IUserProps> userPropses)
{
	var users = new List<UserModel>();
	var allUserPropses = userPropses.ToArray();
	foreach (var userProps in allUserPropses)
	{
		...
	}
	
	return new UserSearchModel
	{
		Users = users.ToArray(),
		QueryModel = queryModel,
		TotalCount = total,
		AvailableCount = users.Count,
		HasMore = allUserPropses.Length >= queryModel.PageSize,
		QueryForm = queryFormBuilder.Build(queryModel)
	};
}

The argument userPropses converted to a local array allUserPropses to avoid multiple enumeration as we learned before. userPropses has type IEnumerable<IUserProps>, so it deserves our attention. But what we will see, if we look at the BuildUserSearchModel call?

public async Task<SearchModel> SearchAsync(SearchQuery searchQuery)
{
	var userPropses = new List<IUserProps>();
	...
	return searchModelBuilder.BuildUserSearchModel(userPropses);
}

It becomes obvious that IEnumerable<IUserProps> is not the best choice. It confuses developers because it says that userPropses is not a collection, but it is in fact. The better variant - replace it with IReadOnlyCollection<IUserProps>:

public UserSearchModel BuildUserSearchModel(IReadOnlyCollection<IUserProps> userPropses)
{
	var users = new List<UserModel>();
	foreach (var userProps in userPropses)
	{
		...
	}
	
	return new UserSearchModel
	{
		Users = users.ToArray(),
		QueryModel = queryModel,
		TotalCount = total,
		AvailableCount = users.Count,
		HasMore = userPropses.Count >= queryModel.PageSize,
		QueryForm = queryFormBuilder.Build(queryModel)
	};
}

Code became cleaner, no conversions, no overhead.

Fluent assertions in unit tests

I would like to make a small note about the Fluent Assertions library. We write many tests where we need to compare two collections:

[Test]
public void TestSingleWriteInSameTimeline()
{
	var testData = new List<TestTimelineData>();
	...
	
	var actual = timeline.Select<TestTimelineData>(timelineId, ticks, TestWriteCount);

	actual.Select(x => x.Data).ToArray().Should().BeEquivalentTo(testData.ToArray());
}

But the method BeEquivalentTo works fine with enumerations. We can verify this by looking at its definition:

public AndConstraint<TAssertions> BeEquivalentTo<TExpectation>(IEnumerable<TExpectation> expectation,
	string because = "", params object[] becauseArgs)
{
	return BeEquivalentTo(expectation, config => config, because, becauseArgs);
}

Fluent Assertions will create arrays anyway, so you will get few performance benefits but still can keep your code cleaner:

[Test]
public void TestSingleWriteInSameTimeline()
{
	var testData = new List<TestTimelineData>();
	...
	
	var actual = timeline.Select<TestTimelineData>(timelineId, ticks, TestWriteCount);

	actual.Select(x => x.Data).Should().BeEquivalentTo(testData));
}

Analyzer

If you are worried about performance and memory management or just purity of code, you should be very attentive with collection manipulations. Sometimes, it's challenging to notice a problem, especially after refactorings: you change a method signature from int[] to IEnumerable<int> but don't check all callings - it can lead to some issues like the ones we reviewed earlier.

Unfortunately, there is not a built-in tool to detect problems like this because many of them need complex analysis. I started Collections.Analyzer project that can warn about potential problems:

And more obvious cases:

Harmful collection transformations. Part 2: automatic diagnostics

Sergey Rogatnev — Wed, 13 Oct 2021 00:00:00 GMT

The previous post contains code examples that can be simplified by removing ToArray() or ToCharArray() calls. And an interesting example with List<char> constructor. But we need to be very attentive to find such places in our code. It would be much safer to do it automatically. This post shows us about automatic compiler diagnostics that can detect issues.

Analyzers

Since the release of the new .NET compiler platform, Roslyn developers have got powerful tools for code analysis. Roslyn compilers offer APIs for C# and Visual Basic that we can use for code-related applications. One of the possible applications is a code analyzer that produces warnings and can offer an automatic code fix in IDE.

One of the most popular analyzers built on Roslyn is ConfigureAwaitChecker.Analyzer. The analyzer helps to find awaited methods without ConfigureAwait(false) call and offers automatic code fix.

Collections.Analyzer

Collections.Analyzer is a set of roslyn-based diagnostics for C#-projects that detect potential issues with different collections and produce warnings.

Installation

Every analyzer can be installed as a usual nuget-package. Add a package reference to a project:

<PackageReference Include="Collections.Analyzer" Version="0.1.9" />

The analyzer will work only in the project it was added to. If you want to analyze all projects in your solution, you can add file Directory.build.props to the solution directory with content:

<Project>
  <ItemGroup>
    <PackageReference Include="Collections.Analyzer" Version="0.1.9" />
  </ItemGroup>
</Project>

MSBuild will read these properties.

Build

If your code has problems, you'll see warnings as usual.

PS C:\example> dotnet build
C:\example\App\Program.cs(12,31): warning CI0001: Redundant string conversion [C:\example\App\App.csproj]
C:\example\App\Program.cs(17,21): warning CI0001: Redundant string conversion [C:\example\App\App.csproj]
  App -> C:\example\App\bin\Debug\net472\App.exe

    Warnings: 2
    Errors: 0

CI0001: Redundant string conversion is a new diagnostic described in the Collections.Analyzer package.

IDE

Besides compiler warnings, all diagnostics are integrated in IDE (Visual Studio or Rider) with automatic code fixes. So, we can correct our code on the fly. Let's see how it works for strings.

LINQ

foreach statement

`List<char>` constructor

The case with List<char> constructor is very seldom and specific, but analyzer always helps us:

LINQ on method

Analyzer works on methods that return strings:

These are the most popular string-to-array issues that I have encountered. But if you have any ideas, feel free to write to me or add them to GitHub issues.

In the next article we will continue to look at other collections, possible issues and add automatic diagnostics if it's possible.

Harmful collection transformations. Part 1: string to array of chars

Sergey Rogatnev — Thu, 23 Sep 2021 00:00:00 GMT

An array — the simplest collection. It's just a memory block with elements of one type. Arrays are often used when we need to iterate a collection, to search for elements or transform them. C# and .net evolved, many new collections were introduced, interfaces allowed to create generalized algorithms, LINQ simplified the work of programmers, but we continue to use arrays in inappropriate places and perform excess and sometimes harmful transformations. This post begins a series about collections, their features, performance and design. All posts will contain benchmarks, code examples, my opinion and recommendations.

I want to start with a string, some specific collection of .net.

Part 1: string to an array of chars
Part 2: automatic diagnostics
Part 3: collections

String

Any large enough project operates with strings. Usually strings store a user input and are used as data for other types. For example, we can read text input and convert it to a number type:

static void Main()
{
    var input = Console.ReadLine();
    var value = int.Parse(input);
}

All we know, user input must be validated. Here we need to check that all chars are digits. It's a widespread check. And often the following code is used:

private bool IsDigitString(string str)
{
   return str.ToCharArray().All(char.IsDigit);
}

string bik = ...;
if (bik.ToArray().Any(x => !char.IsDigit(x)))
{
   message = "BIK must contain only digits";
   return false;
}

Looks simple: get a collection of chars and call LINQ-method to check every digit. The only unnecessary and harmful thing is transformation of string to an array of chars with methods ToCharArray() or ToArray(). We used to use LINQ-methods with collections, but forget that the string is an immutable collection that implements IEnumerable<char> interface:

[Serializable]
public sealed partial class String : IComparable, IEnumerable, IConvertible, IEnumerable<char>, IComparable<string>, IEquatable<string>, ICloneable
{
...
}

So, we can modify our examples and remove excessive calls:

private bool IsDigitString(string str)
{
   return str.All(char.IsDigit);
}

and

string bik = ...;
if (bik.Any(x => !char.IsDigit(x)))
{
   message = "BIK must contain only digits";
   return false;
}

Ok, our code became simpler and more readable. But what do we get besides code quality benefits? Benchmarks can help us. Let's compare three methods: with ToArray(), with ToCharArray() and without additional calls for different string lengths and frameworks:

Benchmark code. Click to expand.

namespace ToCharArrayBenchmark
{
    using System.Linq;
    using BenchmarkDotNet.Attributes;
    using BenchmarkDotNet.Jobs;

    [SimpleJob(RuntimeMoniker.Net48, baseline: true)]
    [SimpleJob(RuntimeMoniker.NetCoreApp31)]
    [SimpleJob(RuntimeMoniker.Net50)]
    [DisassemblyDiagnoser]
    [MemoryDiagnoser]
    public class IsDigitBenchmark
    {
        [Params(10, 100, 1000)]
        public int N { get; set; }

        private string value;

        [GlobalSetup]
        public void SetUp()
        {
            var chars = new char[N];
            for (int i = 0; i < N; i++)
            {
                chars[i] = (char)(i % 10 + '0');
            }

            value = new string(chars);
        }

        [Benchmark]
        public bool IsDigitString_ToCharArray()
        {
            return value.ToCharArray().All(char.IsDigit);
        }

        [Benchmark]
        public bool IsDigitString_ToArray()
        {
            return value.ToArray().All(char.IsDigit);
        }

        [Benchmark(Baseline = true)]
        public bool IsDigitString()
        {
            return value.All(char.IsDigit);
        }
    }
}

Full results in table format. Click to expand.

BenchmarkDotNet=v0.13.0, OS=Windows 10.0.19043.1237 (21H1/May2021Update)
AMD Ryzen 7 4700U with Radeon Graphics, 1 CPU, 8 logical and 8 physical cores
  [Host]             : .NET Framework 4.8 (4.8.4400.0), X64 RyuJIT
  .NET 5.0           : .NET 5.0.7 (5.0.721.25508), X64 RyuJIT
  .NET Core 3.1      : .NET Core 3.1.11 (CoreCLR 4.700.20.56602, CoreFX 4.700.20.56604), X64 RyuJIT
  .NET Framework 4.8 : .NET Framework 4.8 (4.8.4400.0), X64 RyuJIT

Method	Runtime	N	Mean	Error	StdDev	Ratio	RatioSD	Gen 0	Allocated
IsDigitString	.NET 5.0	10	94.54 ns	1.924 ns	3.267 ns	1.01	0.04	0.0459	96 B
IsDigitString_ToCharArray	.NET 5.0	10	97.01 ns	0.602 ns	0.503 ns	1.02	0.02	0.0688	144 B
IsDigitString_ToArray	.NET 5.0	10	250.71 ns	3.346 ns	2.794 ns	2.64	0.04	0.1376	288 B

IsDigitString	.NET Core 3.1	10	98.74 ns	1.595 ns	1.492 ns	1.04	0.02	0.0459	96 B
IsDigitString_ToCharArray	.NET Core 3.1	10	105.02 ns	1.217 ns	1.079 ns	1.10	0.02	0.0688	144 B
IsDigitString_ToArray	.NET Core 3.1	10	294.50 ns	0.838 ns	0.700 ns	3.11	0.05	0.1373	288 B

IsDigitString	.NET Framework 4.8	10	94.96 ns	1.863 ns	1.913 ns	1.00	0.00	0.0459	96 B
IsDigitString_ToCharArray	.NET Framework 4.8	10	90.48 ns	1.345 ns	1.258 ns	0.95	0.02	0.0688	144 B
IsDigitString_ToArray	.NET Framework 4.8	10	249.17 ns	0.617 ns	0.482 ns	2.63	0.04	0.1452	305 B

IsDigitString	.NET 5.0	100	617.01 ns	2.901 ns	2.714 ns	0.91	0.02	0.0458	96 B
IsDigitString_ToCharArray	.NET 5.0	100	730.71 ns	6.392 ns	5.666 ns	1.08	0.02	0.1526	320 B
IsDigitString_ToArray	.NET 5.0	100	1,471.00 ns	25.149 ns	62.630 ns	2.21	0.16	0.3891	816 B

IsDigitString	.NET Core 3.1	100	695.67 ns	13.740 ns	22.576 ns	1.03	0.03	0.0458	96 B
IsDigitString_ToCharArray	.NET Core 3.1	100	707.81 ns	14.127 ns	40.760 ns	1.12	0.05	0.1526	320 B
IsDigitString_ToArray	.NET Core 3.1	100	1,638.57 ns	18.990 ns	16.834 ns	2.42	0.05	0.3891	816 B

IsDigitString	.NET Framework 4.8	100	677.60 ns	13.072 ns	12.839 ns	1.00	0.00	0.0458	96 B
IsDigitString_ToCharArray	.NET Framework 4.8	100	692.45 ns	10.962 ns	9.717 ns	1.02	0.02	0.1526	321 B
IsDigitString_ToArray	.NET Framework 4.8	100	1,696.70 ns	26.955 ns	22.509 ns	2.51	0.05	0.4768	1,003 B

IsDigitString	.NET 5.0	1000	8,340.66 ns	159.513 ns	141.404 ns	1.26	0.02	0.0458	96 B
IsDigitString_ToCharArray	.NET 5.0	1000	7,902.88 ns	153.280 ns	209.812 ns	1.20	0.04	1.0071	2,120 B
IsDigitString_ToArray	.NET 5.0	1000	12,526.66 ns	215.331 ns	201.421 ns	1.90	0.03	2.1820	4,568 B

IsDigitString	.NET Core 3.1	1000	6,771.54 ns	42.026 ns	35.093 ns	1.03	0.01	0.0458	96 B
IsDigitString_ToCharArray	.NET Core 3.1	1000	6,820.17 ns	77.269 ns	60.326 ns	1.03	0.01	1.0071	2,120 B
IsDigitString_ToArray	.NET Core 3.1	1000	14,993.39 ns	249.113 ns	255.821 ns	2.27	0.04	2.1820	4,568 B

IsDigitString	.NET Framework 4.8	1000	6,609.99 ns	51.380 ns	48.061 ns	1.00	0.00	0.0458	96 B
IsDigitString_ToCharArray	.NET Framework 4.8	1000	6,922.85 ns	102.382 ns	90.759 ns	1.05	0.01	1.0071	2,127 B
IsDigitString_ToArray	.NET Framework 4.8	1000	15,401.52 ns	117.030 ns	103.744 ns	2.33	0.02	3.0823	6,475 B

// * Legends *
  N         : Value of the 'N' parameter
  Mean      : Arithmetic mean of all measurements
  Error     : Half of 99.9% confidence interval
  StdDev    : Standard deviation of all measurements
  Ratio     : Mean of the ratio distribution ([Current]/[Baseline])
  RatioSD   : Standard deviation of the ratio distribution ([Current]/[Baseline])
  Gen 0     : GC Generation 0 collects per 1000 operations
  Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  1 us      : 1 Microsecond (0.000001 sec)

Let's examine a diagram with execution times (in microseconds) for strings with 100 chars:

As we see, method IsDigitString is about 5% faster than calling ToCharArray(). ToArray() is up to 3 times slower. Both ToCharArray() and ToArray() do the same, but ToCharArray() is a string class method and very optimized with unsafe code. ToArray() is an extension method for IEnumerable<T> and is not optimized so much.

So, we know, there is no need to transform string to array to use LINQ-methods, this transformation slightly hurts performance and there is one more thing - GC. Every time when we call ToArray() or ToCharArray() we create at least one new object — a new array. We no longer need the source string, and GC will collect at some time.

The next diagram shows us memory allocations (in bytes) for strings with 100 chars:

Cons

Should we always remove ToArray() and ToCharArray() calls? It depends on context. ToArray() generates a lot of overhead, so I can't find a situation where to use it. But as we saw before, ToCharArray() is very fast, and almost does not affect performance. Can you imagine that calling ToCharArray() will save time? But it does! Look at this example:

var convertParameterSymbols = attributes["ConvertParameterSymbols"];
if (convertParameterSymbols != null)
	ConvertParameterSymbols = new List<char>(convertParameterSymbols.ToCharArray());

It looks like we need to remove ToCharrArray(). But before doing this, we need to go back to the string class definition. As we remember, it implements IEnumerable<char>, but not ICollection<char>. ICollection<T> interface describes the contract for mutable collections (Add, Clear, Remove) that is not suitable for strings. And look at List<T> constructor:

public List(IEnumerable<T> collection)
{
  if (collection == null)
	ThrowHelper.ThrowArgumentNullException(ExceptionArgument.collection);
  if (collection is ICollection<T> objs)
  {
	int count = objs.Count;
	if (count == 0)
	{
	  this._items = List<T>._emptyArray;
	}
	else
	{
	  this._items = new T[count];
	  objs.CopyTo(this._items, 0);
	  this._size = count;
	}
  }
  else
  {
	this._size = 0;
	this._items = List<T>._emptyArray;
	foreach (T obj in collection)
	  this.Add(obj);
  }
}

It's optimized for ICollection<T> objects. Elements are just copied as a block with CopyTo instead of adding one by one that causes resizes of the list. What types do implement ICollection<T> interface? You will be surprised, but any array (T[]) can be converted to ICollection<T>. And using new List<char>(convertParameterSymbols.ToCharArray()) is faster than new List<char>(convertParameterSymbols). Benchmarks can prove this.

Benchmark code. Click to expand.

namespace ToCharArrayBenchmark
{
    using System.Collections.Generic;
    using System.Linq;
    using BenchmarkDotNet.Attributes;
    using BenchmarkDotNet.Jobs;

    [SimpleJob(RuntimeMoniker.Net48, baseline: true)]
    [SimpleJob(RuntimeMoniker.NetCoreApp31)]
    [SimpleJob(RuntimeMoniker.Net50)]
    [DisassemblyDiagnoser]
    [MemoryDiagnoser]
    public class ConstructorBenchmark
    {
        [Params(10, 100, 1000)]
        public int N { get; set; }

        private string value;

        [GlobalSetup]
        public void SetUp()
        {
            var chars = new char[N];
            for (int i = 0; i < N; i++)
            {
                chars[i] = (char)(i % 10 + '0');
            }
            
            value = new string(chars);
        }
        
        [Benchmark(Baseline = true)]
        public List<char> Constructor()
        {
            return new List<char>(value);
        }

        [Benchmark]
        public List<char> Constructor_ToCharArray()
        {
            return new List<char>(value.ToCharArray());
        }

        [Benchmark]
        public List<char> Constructor_ToArray()
        {
            return new List<char>(value.ToArray());
        }
    }
}

Full results in table format. Click to expand.

BenchmarkDotNet=v0.13.0, OS=Windows 10.0.19043.1288 (21H1/May2021Update)
AMD Ryzen 7 4700U with Radeon Graphics, 1 CPU, 8 logical and 8 physical cores
  [Host]             : .NET Framework 4.8 (4.8.4420.0), X64 RyuJIT
  .NET 5.0           : .NET 5.0.7 (5.0.721.25508), X64 RyuJIT
  .NET Core 3.1      : .NET Core 3.1.20 (CoreCLR 4.700.21.47003, CoreFX 4.700.21.47101), X64 RyuJIT
  .NET Framework 4.8 : .NET Framework 4.8 (4.8.4420.0), X64 RyuJIT

Method	Runtime	N	Mean	Error	StdDev	Ratio	RatioSD	Gen 0	Allocated
Constructor	.NET 5.0	10	115.76 ns	1.517 ns	1.345 ns	0.74	0.01	0.0917	192 B
Constructor_ToCharArray	.NET 5.0	10	35.95 ns	0.698 ns	0.653 ns	0.23	0.00	0.0612	128 B
Constructor_ToArray	.NET 5.0	10	194.70 ns	2.219 ns	1.967 ns	1.25	0.01	0.1299	272 B
Constructor	.NET Core 3.1	10	138.96 ns	2.804 ns	2.880 ns	0.89	0.02	0.0918	192 B
Constructor_ToCharArray	.NET Core 3.1	10	52.31 ns	0.897 ns	0.839 ns	0.34	0.01	0.0612	128 B
Constructor_ToArray	.NET Core 3.1	10	228.65 ns	0.703 ns	0.657 ns	1.47	0.00	0.1299	272 B
Constructor	.NET Framework 4.8	10	155.57 ns	0.166 ns	0.147 ns	1.00	0.00	0.0956	201 B
Constructor_ToCharArray	.NET Framework 4.8	10	71.37 ns	1.098 ns	1.027 ns	0.46	0.01	0.0650	136 B
Constructor_ToArray	.NET Framework 4.8	10	232.85 ns	1.850 ns	1.730 ns	1.50	0.01	0.1414	297 B

Constructor	.NET 5.0	100	648.65 ns	7.217 ns	6.751 ns	0.80	0.02	0.3405	712 B
Constructor_ToCharArray	.NET 5.0	100	51.29 ns	0.921 ns	0.817 ns	0.06	0.00	0.2295	480 B
Constructor_ToArray	.NET 5.0	100	791.75 ns	15.823 ns	20.012 ns	0.99	0.03	0.4663	976 B
Constructor	.NET Core 3.1	100	684.45 ns	4.868 ns	4.065 ns	0.85	0.02	0.3405	712 B
Constructor_ToCharArray	.NET Core 3.1	100	70.10 ns	0.527 ns	0.493 ns	0.09	0.00	0.2294	480 B
Constructor_ToArray	.NET Core 3.1	100	962.36 ns	4.630 ns	3.867 ns	1.19	0.03	0.4654	976 B
Constructor	.NET Framework 4.8	100	800.13 ns	14.134 ns	17.875 ns	1.00	0.00	0.3443	722 B
Constructor_ToCharArray	.NET Framework 4.8	100	95.32 ns	1.880 ns	1.931 ns	0.12	0.00	0.2333	489 B
Constructor_ToArray	.NET Framework 4.8	100	960.90 ns	6.956 ns	5.431 ns	1.19	0.03	0.5569	1,171 B

Constructor	.NET 5.0	1000	5,289.46 ns	104.687 ns	153.449 ns	0.83	0.03	2.0828	4,368 B
Constructor_ToCharArray	.NET 5.0	1000	235.86 ns	4.589 ns	4.507 ns	0.04	0.00	1.9505	4,080 B
Constructor_ToArray	.NET 5.0	1000	5,943.57 ns	111.670 ns	204.195 ns	0.95	0.04	3.1128	6,528 B
Constructor	.NET Core 3.1	1000	5,196.81 ns	70.120 ns	65.590 ns	0.82	0.02	2.0828	4,368 B
Constructor_ToCharArray	.NET Core 3.1	1000	247.25 ns	1.114 ns	0.987 ns	0.04	0.00	1.9503	4,080 B
Constructor_ToArray	.NET Core 3.1	1000	7,100.14 ns	85.990 ns	80.435 ns	1.13	0.01	3.1128	6,528 B
Constructor	.NET Framework 4.8	1000	6,306.12 ns	69.763 ns	65.257 ns	1.00	0.00	2.0905	4,389 B
Constructor_ToCharArray	.NET Framework 4.8	1000	275.66 ns	2.318 ns	2.055 ns	0.04	0.00	1.9541	4,101 B
Constructor_ToArray	.NET Framework 4.8	1000	7,507.12 ns	20.504 ns	17.121 ns	1.19	0.01	4.0131	8,453 B

// * Legends *
  N         : Value of the 'N' parameter
  Mean      : Arithmetic mean of all measurements
  Error     : Half of 99.9% confidence interval
  StdDev    : Standard deviation of all measurements
  Ratio     : Mean of the ratio distribution ([Current]/[Baseline])
  RatioSD   : Standard deviation of the ratio distribution ([Current]/[Baseline])
  Gen 0     : GC Generation 0 collects per 1000 operations
  Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  1 us      : 1 Microsecond (0.000001 sec)

Time diagram (in microseconds) for strings with 100 chars:

Incredible! A list created from an array is about 10 times faster than created from a source string. And memory usage is more optimal (in bytes):

But we should remember, it's a very specific case.

Conclusion

In common, we should be more attentive and avoid unnecessary transformations for reasons:

Performance. Such small optimizations can positively influence the processing of big data.
Additional memory traffic — GC has to collect more data, GC pauses can be longer.
Code quality. We can reduce code but not lose functionality.

Some cases require special attention and a deeper understanding of the .net platform.

If you really need to convert a string to an array use ToCharArray() instead of ToArray().

More examples

Here are more examples I found.

Lowercase string

There is a very popular way to lowercase the first letter of a string. We remember — strings are immutable and name[0] = char.ToLowerInvariant(name[0]) won't compile. So, we need to create a new string from a char array.

public override string Format(string name)
{
	if (string.IsNullOrEmpty(name) || char.IsLower(name[0])) return name;
	var chars = name.ToCharArray();
	chars[0] = char.ToLowerInvariant(chars[0]);
	return new string(chars);
}

Reverse string

There is no built-in method to reverse a string, so we have to create our own using Array.Reverse method, so we transform a source string to an array.

public static string ReverseString(this string input)
{
	var charArray = input.ToCharArray();
	Array.Reverse(charArray);
	return new string(charArray);
}

Iterate string

public static byte[] ToBytes(string value)
{
	char[] chars = value.ToCharArray();
	byte[] res = new byte[chars.Length / 2];
	for (int i = 0, j = 0; i < chars.Length; i += 2, j++)
		res[j] = (byte)(FromHex((byte)chars[i]) << 4 ^ FromHex((byte)chars[i + 1]));
	return res;
}

We iterate through a string, and no need to create a new array. The code can be simplified:

public static byte[] ToBytes(string value)
{
	byte[] res = new byte[value.Length / 2];
	for (int i = 0, j = 0; i < value.Length; i += 2, j++)
		res[j] = (byte)(FromHex((byte)value[i]) << 4 ^ FromHex((byte)value[i + 1]));
	return res;
}

.net, C# and programming (EN)

Collections search

.NET 6

.NET 8

.NET 10

Conclusions

Analyzer

Collection initializer: List

Array

List<T>

Resizing a list

Performance

Capacity

Benchmark

Analyzer

Recommendations

Links

Dapper: How caching can be harmful

Profiling

Dapper internals

SQl + json = 🚩

Dapper caching

Solution

Conclusion

Recommendations

String of Digits

Regex

Regex source generators

char.IsDigit

What is a digit?

Conclusion

Links

Any() vs Count: part 2

Array

Results

ConcurrentDictionary

Results

Any() и IsEmpty

Conclusion

Recommendations

nvarchar(max) vs nvarchar(N)

Preparation

Testing Data Select

Sorting

Filtering

Indexes

Comparison

Conclusions

References

List initializer

Array

List<T>

List resize

Performance

Capacity

Benchmarks

Analyzer

Opinion

Links

Harmful collection transformations. Part 3: collections

Collections

Count elements

ToArray() with LINQ

Array arguments

Fluent assertions in unit tests

Analyzer

Links

Harmful collection transformations. Part 2: automatic diagnostics

Analyzers

Collections.Analyzer

Installation

Build

IDE

LINQ

foreach statement

List<char> constructor

LINQ on method

Links

Harmful collection transformations. Part 1: string to array of chars

String

`ToArray()` with LINQ

`List<char>` constructor