Memory and Span pt.1

Starting from .NET Core 2.0 and .NET Framework 4.5 we can use new data types: Span and Memory. To use them, you just need to install the System.Memory nuget package:

PM> Install-Package System.Memory

These data types are notable because the CLR team has done a great job to implement their special support inside the code of .NET Core 2.1+ JIT compiler by embedding these data types right into the core. What kind of data types are these and why are they worth a whole chapter?

If we talk about problems that made these types appear, I should name three of them. The first one is unmanaged code.

Both the language and the platform have existed for many years along with means to work with unmanaged code. So, why release another API to work with unmanaged code if the former basically existed for many years? To answer this question, we should understand what we lacked before.

This chapter was translated from Russian jointly by author and by professional translators. You can help us with translation from Russian or English into any other language, primarily into Chinese or German.

Also, if you want thank us, the best way you can do that is to give us a star on github or to fork repository github/sidristij/dotnetbook.

The platform developers already tried to facilitate the use of unmanaged resources for us. They implemented auto wrappers for imported methods and marshaling that works automatically in most cases. Here also belongs stackalloc, mentioned in the chapter about a thread stack. However, as I see it, the first C# developers came from C++ world (my case), but now they shift from more high-level languages (I know a developer who wrote in JavaScript before). This means people are getting more suspicious to unmanaged code and C/C+ constructs, so much the more to Assembler.

As a result, projects contain less and less unsafe code and the confidence in the platform API grows more and more. This is easy to check if we search for stackalloc use cases in public repositories — they are scarce. However, let’s take any code that uses it:

Interop.ReadDir class
/src/mscorlib/shared/Interop/Unix/System.Native/Interop.ReadDir.cs

unsafe
{
    // s_readBufferSize is zero when the native implementation does not support reading into a buffer.
    byte* buffer = stackalloc byte[s_readBufferSize];
    InternalDirectoryEntry temp;
    int ret = ReadDirR(dir.DangerousGetHandle(), buffer, s_readBufferSize, out temp);
    // We copy data into DirectoryEntry to ensure there are no dangling references.
    outputEntry = ret == 0 ?
                new DirectoryEntry() { InodeName = GetDirectoryEntryName(temp), InodeType = temp.InodeType } :
                default(DirectoryEntry);

    return ret;
}

We can see why it is not popular. Just skim this code and question yourself whether you trust it. I guess the answer is ‘No’. Then, ask yourself why. It is obvious: not only do we see the word Dangerous, which kind of suggests that something may go wrong, but there is the unsafe keyword and byte* buffer = stackalloc byte[s_readBufferSize]; line (specifically — byte*) that change our attitude. This is a trigger for you to think: “Wasn’t there another way to do it”? So, let’s get deeper into psychoanalysis: why might you think that way? On the one hand, we use language constructs and the syntax offered here is far from, for example, C++/CLI, which allows anything (even inserting pure Assembler code). On the other hand, this syntax looks unusual.

The second issue developers thought of implicitly or explicitly is incompatibility of string and char[] types. Although, logically a string is an array of characters, but you can’t cast a string to char[]: you can only create a new object and copy the content of a string to an array. This incompatibility is introduced to optimize strings in terms of storage (there are no readonly arrays). However, problems appear when you start working with files. How to read them? As a string or as an array? If you choose an array you cannot use some methods designed to work with strings. What about reading as a string? It may be too long. If you need to parse it then, what parser should you choose for primitive data types: you don’t always want to parse them manually (integers, floats, given in different formats). We have a lot of proven algorithms that do it quicker and more efficiently, don’t we? However, such algorithms often work with strings that contain nothing but a primitive type itself. So, there is a dilemma.

The third problem is that the data required by an algorithm rarely make a continuous, solid data slice within a section of an array read from some source. For example, in case of files or data read from a socket, we have some part of those already processed by an algorithm, followed by a part of data that must be processed by our method, and then by not yet processed data. Ideally, our method wants only the data for which this method was designed. For example, a method that parses integers won’t be happy with a string that contains some words with an expected number somewhere among them. This method wants a number and nothing else. Or, if we pass an entire array, there is a requirement to indicate, for example, the offset for a number from the beginning of the array.

int ParseInt(char[] input, int index)
{
    while(char.IsDigit(input[index]))
    {
        // ...
        index++;
    }
}

However, this approach is poor, as this method gets unnecessary data. In other words the method is called for contexts it was not designed for, and has to solve some external tasks. This is a bad design. How to avoid these problems? As an option we can use the ArraySegment<T> type that can give access to a section of an array:

int ParseInt(IList<char>[] input)
{
    while(char.IsDigit(input.Array[index]))
    {
        // ...
        index++;
    }
}

var arraySegment = new ArraySegment(array, from, length);
var res = ParseInt((IList<char>)arraySegment);

However, I think this is too much both in terms of logic and a decrease in performance. ArraySegment is poorly designed and slows the access to elements 7 times more in comparison with the same operations done with an array.

So how do we solve these problems? How do we get developers back to using unmanaged code and give them a unified and quick tool to work with heterogeneous data sources: arrays, strings and unmanaged memory. It was necessary to give them a sense of confidence that they can’t do a mistake unknowingly. It was necessary to give them an instrument that doesn’t diminish native data types in terms of performance but solves the listed problems. Span<T> and Memory<T> types are exactly these instruments.

Span<T>, ReadOnlySpan<T>

Span type is an instrument to work with data within a section of a data array or with a subrange of its values. As in case of an array it allows both reading and writing to the elements of this subrange, but with one important constraint: you get or create a Span<T> only for a temporary work with an array, Just to call a group of methods. However, to get a general understanding let’s compare the types of data which Span is designed for and look at its possible use scenarios.

The first type of data is a usual array. Arrays work with Span in the following way:

    var array = new [] {1,2,3,4,5,6};
    var span = new Span<int>(array, 1, 3);
    var position = span.BinarySearch(3);
    Console.WriteLine(span[position]);  // -> 3

At first, we create an array of data, as shown by this example. Next, we create Span (or a subset) which references to the array, and makes a previously initialized value range accessible to code that uses the array.

Here we see the first feature of this type of data i.e. the ability to create a certain context. Let’s expand our idea of contexts:

void Main()
{
    var array = new [] {'1','2','3','4','5','6'};
    var span = new Span<char>(array, 1, 3);
    if(TryParseInt32(span, out var res))
    {
        Console.WriteLine(res);
    }
    else
    {
        Console.WriteLine("Failed to parse");
    }
}

public bool TryParseInt32(Span<char> input, out int result)
{
    result = 0;
    for (int i = 0; i < input.Length; i++)
    {
        if(input[i] < '0' || input[i] > '9')
            return false;
    result = result * 10 + ((int)input[i] - '0');
    }
    return true;
}

-----
234

As we see Span<T> provides abstract access to a memory range both for reading and writing. What does it give us? If we remember what else we can use Span for, we will think about unmanaged resources and strings:

// Managed array
var array = new[] { '1', '2', '3', '4', '5', '6' };
var arrSpan = new Span<char>(array, 1, 3);
if (TryParseInt32(arrSpan, out var res1))
{
    Console.WriteLine(res1);
}

// String
var srcString = "123456";
var strSpan = srcString.AsSpan();
if (TryParseInt32(strSpan, out var res2))
{
    Console.WriteLine(res2);
}

// void *
Span<char> buf = stackalloc char[6];
buf[0] = '1'; buf[1] = '2'; buf[2] = '3';
buf[3] = '4'; buf[4] = '5'; buf[5] = '6';

if (TryParseInt32(buf, out var res3))
{
    Console.WriteLine(res3);
}

-----
234
234
234

That means Span<T> is a tool to unify ways of working with memory, both managed and unmanaged. It ensures safety while working with such data during Garbage Collection. That is if memory ranges with unmanaged resources start to move, it will be safe.

However, should we be so excited? Could we achieve this earlier? For example, in case of managed arrays there is no doubt about it: you just need to wrap an array in one more class (e.g. long-existing [ArraySegment] (https://referencesource.microsoft.com/#mscorlib/system/arraysegment.cs,31)) thus giving a similar interface and that is it. Moreover, you can do the same with strings — they have necessary methods. Again, you just need to wrap a string in the same type and provide methods to work with it. However, to store a string, a buffer and an array in one type you will have much to do with keeping references to each possible variant in a single instance (with only one active variant, obviously).

public readonly ref struct OurSpan<T>
{
    private T[] _array;
    private string _str;
    private T * _buffer;

    // ...
}

Or, based on architecture you can create three types that implement a uniform interface. Thus, it is not possible to create a uniform interface between these data types that is different from Span<T> and keep the maximum performance.

Next, there is a question of what is ref struct in respect to Span? These are exactly those “structures existing only on stack” that we hear about during job interviews so often. It means this data type can be allocated on the stack only and cannot go to the heap. This is why Span, which is a ref structure, is a context data type that enables work of methods but not that of objects in memory. That is what we need to base on when trying to understand it.

Now we can define the Span type and the related ReadOnlySpan type:

Span is a data type that implements a uniform interface to work with heterogeneous types of data arrays and enables passing a subset of an array to a method so that the speed of access to the original array would be constant and highest regardless of the depth of the context.

Indeed, if we have a code like

public void Method1(Span<byte> buffer)
{
    buffer[0] = 0;
    Method2(buffer.Slice(1,2));
}
Method2(Span<byte> buffer)
{
    buffer[0] = 0;
    Method3(buffer.Slice(1,1));
}
Method3(Span<byte> buffer)
{
    buffer[0] = 0;
}

the speed of access to the original buffer will be the highest as you work with a managed pointer and not a managed object. That means you work with an unsafe type in a managed wrapper, but not with a .NET managed type.

This chapter was translated from Russian jointly by author and by professional translators. You can help us with translation from Russian or English into any other language, primarily into Chinese or German.

Also, if you want thank us, the best way you can do that is to give us a star on github or to fork repository github/sidristij/dotnetbook.