Memory<T> and ReadOnlyMemory<T>

There are two visual differences between Memory<T> and Span<T>. The first one is that Memory<T> type doesn’t contain ref modifier in the header of the type. In other words, the Memory<T> type can be allocated both on the stack while being either a local variable, or a method parameter, or its returned value and on the heap, referencing some data in memory from there. However, this small difference creates a huge distinction in the behavior and capabilities of Memory<T> compared to Span<T>. Unlike Span<T> that is an instrument for some methods to use some data buffer, the Memory<T> type is designed to store information about the buffer, but not to handle it. Thus, there is the difference in API.

Memory<T> doesn’t have methods to access the data that it is responsible for. Instead, it has the Span property and the Slice method that return an instance of the Span type.
Additionally, Memory<T> contains the Pin() method used for scenarios when a stored buffer data should be passed to unsafe code. If this method is called when memory is allocated in .NET, the buffer will be pinned and will not move when GC is active. This method will return an instance of the MemoryHandle structure, which encapsulates GCHandle to indicate a segment of a lifetime and to pin array buffer in memory.

This chapter was translated from Russian jointly by author and by professional translators. You can help us with translation from Russian or English into any other language, primarily into Chinese or German.

Also, if you want thank us, the best way you can do that is to give us a star on github or to fork repository github/sidristij/dotnetbook.

However, I suggest we get familiar with the whole set of classes. First, let’s look at the Memory<T> structure itself (here I show only those type members that I found most important):

    public readonly struct Memory<T>
    {
        private readonly object _object;
        private readonly int _index, _length;

        public Memory(T[] array) { ... }
        public Memory(T[] array, int start, int length) { ... }
        internal Memory(MemoryManager<T> manager, int length) { ... }
        internal Memory(MemoryManager<T> manager, int start, int length) { ... }

        public int Length => _length & RemoveFlagsBitMask;
        public bool IsEmpty => (_length & RemoveFlagsBitMask) == 0;

        public Memory<T> Slice(int start, int length);
        public void CopyTo(Memory<T> destination) => Span.CopyTo(destination.Span);
        public bool TryCopyTo(Memory<T> destination) => Span.TryCopyTo(destination.Span);

        public Span<T> Span { get; }
        public unsafe MemoryHandle Pin();
    }

As we see the structure contains the constructor based on arrays, but stores data in the object. This is to additionally reference strings that don’t have a constructor designed for them, but can be used with the AsMemory() string method, it returns ReadOnlyMemory. However, as both types should be binary similar, Object is the type of the _object field.

Next, we see two constructors based on MemoryManager. We will talk about them later. The properties of obtaining Length (size) and IsEmpty check for an empty set. Also, there is the Slice method for getting a subset as well as CopyTo and TryCopyTo methods of copying.

Talking about Memory I want to describe two methods of this type in detail: the Span property and the Pin method.

Memory<T>.Span

public Span<T> Span
{
    get
    {
        if (_index < 0)
        {
            return ((MemoryManager<T>)_object).GetSpan().Slice(_index & RemoveFlagsBitMask, _length);
        }
        else if (typeof(T) == typeof(char) && _object is string s)
        {
            // This is dangerous, returning a writable span for a string that should be immutable.
            // However, we need to handle the case where a ReadOnlyMemory<char> was created from a string
            // and then cast to a Memory<T>. Such a cast can only be done with unsafe or marshaling code,
            // in which case that's the dangerous operation performed by the dev, and we're just following
            // suit here to make it work as best as possible.
            return new Span<T>(ref Unsafe.As<char, T>(ref s.GetRawStringData()), s.Length).Slice(_index, _length);
        }
        else if (_object != null)
        {
            return new Span<T>((T[])_object, _index, _length & RemoveFlagsBitMask);
        }
        else
        {
            return default;
        }
    }
}

Namely, the lines that handle strings management. They say that if we convert ReadOnlyMemory<T> to Memory<T> (these things are the same in binary representation and there is even a comment these types must coincide in a binary way as one is produced from another by calling Unsafe.As) we will get an ~access to a secret chamber~ with an opportunity to change strings. This is an extremely dangerous mechanism:

unsafe void Main()
{
    var str = "Hello!";
    ReadOnlyMemory<char> ronly = str.AsMemory();
    Memory<char> mem = (Memory<char>)Unsafe.As<ReadOnlyMemory<char>, Memory<char>>(ref ronly);
    mem.Span[5] = '?';

    Console.WriteLine(str);
}
---
Hello?

This mechanism combined with string interning can produce dire consequences.

Memory<T>.Pin

The second method that draws strong attention is Pin:

public unsafe MemoryHandle Pin()
{
    if (_index < 0)
    {
        return ((MemoryManager<T>)_object).Pin((_index & RemoveFlagsBitMask));
    }
    else if (typeof(T) == typeof(char) && _object is string s)
    {
        // This case can only happen if a ReadOnlyMemory<char> was created around a string
        // and then that was cast to a Memory<char> using unsafe / marshaling code.  This needs
        // to work, however, so that code that uses a single Memory<char> field to store either
        // a readable ReadOnlyMemory<char> or a writable Memory<char> can still be pinned and
        // used for interop purposes.
        GCHandle handle = GCHandle.Alloc(s, GCHandleType.Pinned);
        void* pointer = Unsafe.Add<T>(Unsafe.AsPointer(ref s.GetRawStringData()), _index);
        return new MemoryHandle(pointer, handle);
    }
    else if (_object is T[] array)
    {
        // Array is already pre-pinned
        if (_length < 0)
        {
            void* pointer = Unsafe.Add<T>(Unsafe.AsPointer(ref array.GetRawSzArrayData()), _index);
            return new MemoryHandle(pointer);
        }
        else
        {
            GCHandle handle = GCHandle.Alloc(array, GCHandleType.Pinned);
            void* pointer = Unsafe.Add<T>(Unsafe.AsPointer(ref array.GetRawSzArrayData()), _index);
            return new MemoryHandle(pointer, handle);
        }
    }
    return default;
}

It is also an important instrument for unification because if we want to pass a buffer to unmanaged code, we just need to call the Pin() method and pass a pointer to this code no matter what type of data Memory<T> refers to. This pointer will be stored in the property of a resulting structure.

void PinSample(Memory<byte> memory)
{
    using(var handle = memory.Pin())
    {
        WinApi.SomeApiMethod(handle.Pointer);
    }
}

It doesn’t matter what Pin() was called for in this code: it can be Memory that represents either T[], or a string or a buffer of unmanaged memory. Merely arrays and string will get a real GCHandle.Alloc(array, GCHandleType.Pinned) and in case of unmanaged memory nothing will happen.

MemoryManager, IMemoryOwner, MemoryPool

Besides indicating structure fields, I want to note that there are two other internal type constructors based on an other entity – MemoryManager. This is not a classic memory manager that you might have thought of and we are going to talk about it later. classic memory manager that you might have thought of and we are going to talk about it later. Like Span, Memory has a reference to a navigated object, an offset, and a size of an internal buffer. Note that you can use the new operator to create Memory from an array only. Or, you can use extension methods to create Memory from a string, an array or ArraySegment. I mean it is not designed to be created from unmanaged memory manually. However, we can see that there is an internal method to create this structure using MemoryManager.

File MemoryManager.cs

public abstract class MemoryManager<T> : IMemoryOwner<T>, IPinnable
{
    public abstract MemoryHandle Pin(int elementIndex = 0);
    public abstract void Unpin();

    public virtual Memory<T> Memory => new Memory<T>(this, GetSpan().Length);
    public abstract Span<T> GetSpan();
    protected Memory<T> CreateMemory(int length) => new Memory<T>(this, length);
    protected Memory<T> CreateMemory(int start, int length) => new Memory<T>(this, start, length);

    void IDisposable.Dispose()
    protected abstract void Dispose(bool disposing);
}

This structure indicates the owner of a memory range. In other words, Span is an instrument to work with memory, Memory is a tool to store the information about a particular memory range and MemoryManager is a tool to control the lifetime of this range, i.e. its owner. For example, we can look at NativeMemoryManager<T> type. Although it is used for tests, this type clearly represents the concept of “ownership”.

File NativeMemoryManager.cs

internal sealed class NativeMemoryManager : MemoryManager<byte>
{
    private readonly int _length;
    private IntPtr _ptr;
    private int _retainedCount;
    private bool _disposed;

    public NativeMemoryManager(int length)
    {
        _length = length;
        _ptr = Marshal.AllocHGlobal(length);
    }

    public override void Pin() { ... }

    public override void Unpin()
    {
        lock (this)
        {
            if (_retainedCount > 0)
            {
                _retainedCount--;
                if (_retainedCount== 0)
                {
                    if (_disposed)
                    {
                        Marshal.FreeHGlobal(_ptr);
                        _ptr = IntPtr.Zero;
                    }
                }
            }
        }
    }

    // Other methods
}

That means the class allows for nested calls of the Pin() method, thus counting generated references from the unsafe world.

Another entity closely tied with Memory is MemoryPool that pools MemoryManager instances (IMemoryOwner in fact):

File MemoryPool.cs

public abstract class MemoryPool<T> : IDisposable
{
    public static MemoryPool<T> Shared => s_shared;

    public abstract IMemoryOwner<T> Rent(int minBufferSize = -1);

    public void Dispose() { ... }
}

It is used to lease buffers of a necessary size for temporary use. The leased instances with implemented IMemoryOwner<T> interface have the Dispose() method to return the leased array back to the pool of arrays. By default, you can use the shareable pool of buffers built on ArrayMemoryPool:

File ArrayMemoryPool.cs

internal sealed partial class ArrayMemoryPool<T> : MemoryPool<T>
{
    private const int MaximumBufferSize = int.MaxValue;
    public sealed override int MaxBufferSize => MaximumBufferSize;
    public sealed override IMemoryOwner<T> Rent(int minimumBufferSize = -1)
    {
        if (minimumBufferSize == -1)
            minimumBufferSize = 1 + (4095 / Unsafe.SizeOf<T>());
        else if (((uint)minimumBufferSize) > MaximumBufferSize)
            ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.minimumBufferSize);

        return new ArrayMemoryPoolBuffer(minimumBufferSize);
    }
    protected sealed override void Dispose(bool disposing) { }
}

Based on this architecture, we have the following picture:

Span data type should be used as a method parameter if you want to read data (ReadOnlySpan) or read and write data (Span). However, it is not supposed to be stored in a field of a class for future use.
If you need to store a reference from a field of a class to a data buffer, you need to use Memory<T> or ReadOnlyMemory<T> depending on your goals.
MemoryManager<T> is the owner of a data buffer (optional ). It may be necessary if you need to count Pin() calls for example. Or, if you need to know how to release memory.
If Memory is built around an unmanaged memory range, Pin() can do nothing. However, this uniforms working with different types of buffers: for both managed and unmanaged code the interaction interface will be the same.
Every type has public constructors. That means you can use Span directly or get its instance from Memory. For Memory as such, you can create it individually or you can create a memory range owned by IMemoryOwner and referenced by Memory. Any type based on MemoryManger can be regarded as a specific case which it owns some local memory range (e.g. accompanied by counting the references from the unsafe world). In addition, if you need to pool such buffers (the frequent traffic of almost equally sized buffers is expected) you can use the MemoryPool type.
If you intend to work with unsafe code by passing a data buffer there, you should use the Memory type which has the Pin() method that automatically pins a buffer on the .NET heap if it was created there.
If you have some traffic of buffers (for example you parse a text of a program or DSL), it is better to use the MemoryPool type. You can properly implement it to output the buffers of a necessary size from a pool (for example a slightly bigger buffer if there is no suitable one, but using originalMemory.Slice(requiredSize) to avoid pool fragmentation).

Performance

To measure the performance of new data types I decided to use a library that has already become standard BenchmarkDotNet:

[Config(typeof(MultipleRuntimesConfig))]
public class SpanIndexer
{
    private const int Count = 100;
    private char[] arrayField;
    private ArraySegment<char> segment;
    private string str;

    [GlobalSetup]
    public void Setup()
    {
        str = new string(Enumerable.Repeat('a', Count).ToArray());
        arrayField = str.ToArray();
        segment = new ArraySegment<char>(arrayField);
    }

    [Benchmark(Baseline = true, OperationsPerInvoke = Count)]
    public int ArrayIndexer_Get()
    {
        var tmp = 0;
        for (int index = 0, len = arrayField.Length; index < len; index++)
        {
            tmp = arrayField[index];
        }
        return tmp;
    }

    [Benchmark(OperationsPerInvoke = Count)]
    public void ArrayIndexer_Set()
    {
        for (int index = 0, len = arrayField.Length; index < len; index++)
        {
            arrayField[index] = '0';
        }
    }

    [Benchmark(OperationsPerInvoke = Count)]
    public int ArraySegmentIndexer_Get()
    {
        var tmp = 0;
        var accessor = (IList<char>)segment;
        for (int index = 0, len = accessor.Count; index < len; index++)
        {
            tmp = accessor[index];
        }
        return tmp;
    }

    [Benchmark(OperationsPerInvoke = Count)]
    public void ArraySegmentIndexer_Set()
    {
        var accessor = (IList<char>)segment;
        for (int index = 0, len = accessor.Count; index < len; index++)
        {
            accessor[index] = '0';
        }
    }

    [Benchmark(OperationsPerInvoke = Count)]
    public int StringIndexer_Get()
    {
        var tmp = 0;
        for (int index = 0, len = str.Length; index < len; index++)
        {
            tmp = str[index];
        }

        return tmp;
    }

    [Benchmark(OperationsPerInvoke = Count)]
    public int SpanArrayIndexer_Get()
    {
        var span = arrayField.AsSpan();
        var tmp = 0;
        for (int index = 0, len = span.Length; index < len; index++)
        {
            tmp = span[index];
        }
        return tmp;
    }

    [Benchmark(OperationsPerInvoke = Count)]
    public int SpanArraySegmentIndexer_Get()
    {
        var span = segment.AsSpan();
        var tmp = 0;
        for (int index = 0, len = span.Length; index < len; index++)
        {
            tmp = span[index];
        }
        return tmp;
    }

    [Benchmark(OperationsPerInvoke = Count)]
    public int SpanStringIndexer_Get()
    {
        var span = str.AsSpan();
        var tmp = 0;
        for (int index = 0, len = span.Length; index < len; index++)
        {
            tmp = span[index];
        }
        return tmp;
    }

    [Benchmark(OperationsPerInvoke = Count)]
    public void SpanArrayIndexer_Set()
    {
        var span = arrayField.AsSpan();
        for (int index = 0, len = span.Length; index < len; index++)
        {
            span[index] = '0';
        }
    }

    [Benchmark(OperationsPerInvoke = Count)]
    public void SpanArraySegmentIndexer_Set()
    {
        var span = segment.AsSpan();
        for (int index = 0, len = span.Length; index < len; index++)
        {
            span[index] = '0';
        }
    }
}

public class MultipleRuntimesConfig : ManualConfig
{
    public MultipleRuntimesConfig()
    {
        Add(Job.Default
            .With(CsProjClassicNetToolchain.Net471) // Span not supported by CLR
            .WithId(".NET 4.7.1"));

        Add(Job.Default
            .With(CsProjCoreToolchain.NetCoreApp20) // Span supported by CLR
            .WithId(".NET Core 2.0"));

        Add(Job.Default
            .With(CsProjCoreToolchain.NetCoreApp21) // Span supported by CLR
            .WithId(".NET Core 2.1"));

        Add(Job.Default
            .With(CsProjCoreToolchain.NetCoreApp22) // Span supported by CLR
            .WithId(".NET Core 2.2"));
    }
}

Now, let’s see the results.

Performance chart

Looking at them we can get the following information:

ArraySegment is awful. But if you wrap it in Span you can make it less awful. In this case, performance will increase 7 times.
If we consider .NET Framework 4.7.1 (the same thing is for 4.5), the use of Span will significantly lower the performance when working with data buffers. It will decrease by 30–35 %.
However, if we look at .NET Core 2.1+ the performance remains similar or even increases given that Span can use a part of a data buffer, creating the context. The same functionality can be found in ArraySegment, but it works very slowly.

Thus, we can draw simple conclusions regarding the use of these data types:

for .NET Framework 4.5+ и .NET Core they have the only advantage: they are faster than ArraySegment when dealing with a subset of an original array;
in .NET Core 2.1+ their use gives an undeniable advantage over both ArraySegment and any manual implementation of Slice;
all three ways are as productive as possible and that cannot be achieved with any tool to unify arrays.

This chapter was translated from Russian jointly by author and by professional translators. You can help us with translation from Russian or English into any other language, primarily into Chinese or German.

Memory and Span pt.3