ValueTask - why, why and how?

Preface to the translation

Unlike scientific articles, articles of this type are difficult to translate "close to the text", and a rather strong adaptation has to be done. For this reason, I apologize for some liberties, for my part, in dealing with the text of the original article. I am guided by only one goal - to make the translation understandable, even if in places it strongly deviates from the original article. I would be grateful for constructive criticism and corrections / additions to the translation.

Introduction

The System.Threading.Tasks



namespace and the Task



class were first introduced in the .NET Framework 4. Since then, this type, and its derived class Task<TResult>



, have firmly entered the practice of programming in .NET and have become key aspects of the asynchronous model. implemented in C # 5, with its async/await



. In this article, I will talk about the new ValueTask/ValueTask<TResult>



types that were introduced in order to improve the performance of asynchronous code in cases where overhead when working with memory plays a key role.

Task

Task



serves several purposes, but the main one is “promise” - an object that represents the ability to wait for an operation to complete. You initiate the operation and get Task



. This Task



will be completed when the operation itself is completed. In this case, there are three options:

1. The operation completes synchronously in the initiator thread. For example, when accessing some data that is already in the buffer .
2. The operation is performed asynchronously, but manages to complete by the time the initiator receives the Task
   
   
   
   . For example, when performing quick access to data that has not yet been buffered
3. The operation is performed asynchronously, and ends after the initiator receives the Task
   
   
   
   An example would be receiving data over a network .

To obtain the result of an asynchronous call, the client can either block the calling thread while waiting for completion, which often contradicts the idea of ​​asynchrony, or provide a callback method that will be executed upon completion of the asynchronous operation. The callback model in .NET 4 was presented explicitly, using the ContinueWith



method of an object of the Task



class, which received a delegate that was called upon completion of the asynchronous operation.

 SomeOperationAsync().ContinueWith(task => { try { TResult result = task.Result; UseResult(result); } catch (Exception e) { HandleException(e); } });
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

With .NET Frmaework 4.5 and C # 5, getting the result of an asynchronous operation has been simplified by introducing the async/await



keywords and the mechanism behind them. This mechanism, the generated code, is able to optimize all the cases mentioned above, correctly handling completion despite the path in which it was reached.

 TResult result = await SomeOperationAsync(); UseResult(result);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

The Task



class is quite flexible and has several advantages. For example, you can "expect" an object of this class several times, you can expect the result competitively, by any number of consumers. Instances of a class can be stored in a dictionary for any number of subsequent calls, with the goal of "waiting" in the future. The described scenarios allow you to consider Task



objects as a kind of cache of results obtained asynchronously. In addition, Task



provides the ability to block the waiting thread until the operation is completed if the script requires it. There is also the so-called. combinators for various strategies for waiting for the completion of task sets, for example, "Task.WhenAny" - asynchronously waiting for the completion of the first, from many, tasks.

But, nevertheless, the most common use case is simply starting an asynchronous operation and then waiting for the result of its execution. Such a simple case, quite common, does not require the above flexibility:

 TResult result = await SomeOperationAsync(); UseResult(result);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

This is very similar to how we write synchronous code (e.g. TResult result = SomeOperation();



). This option is naturally translated into async/await



.

In addition, for all its merits, the Task



type has a potential flaw. Task



is a class, which means that every operation that creates an instance of a task allocates an object on the heap. The more objects we create, the more work is required from the GC, and the more resources are spent on the work of the garbage collector, resources that could be used for other purposes. This becomes a clear problem for the code, in which, on the one hand, Task



instances are created often, and on the other, which has increased demands on throughput and performance.

The runtime and major libraries, in many situations, manage to mitigate this effect. For example, if you write a method like the one below:

 public async Task WriteAsync(byte value) { if (_bufferedCount == _buffer.Length) { await FlushAsync(); } _buffer[_bufferedCount++] = value; }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

and, most often, there will be enough space in the buffer, the operation will end synchronously. If so, then there is nothing special about the returned task, there is no return value, and the operation is already completed. In other words, we are dealing with Task



, the equivalent of a synchronous void



operation. In such situations, the runtime simply caches the Task



object, and uses it every time as a result for any async Task



- a method that finishes synchronously ( Task.ComletedTask



). Another example, let's say you write:

 public async Task<bool> MoveNextAsync() { if (_bufferedCount == 0) { await FillBuffer(); } return _bufferedCount > 0; }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Suppose, in the same way, that in most cases, there is some data in the buffer. The method checks _bufferedCount



, sees that the variable is greater than zero, and returns true



. Only if at the time of verification the data was not buffered, an asynchronous operation is required. Anyway, there are only two possible logical results ( true



and false



), and only two possible return states through Task<bool>



. Based on synchronous completion, or asynchronous, but before exiting the method, the runtime caches two instances of Task<bool>



(one for true



and one for false



), and returns the desired one, avoiding additional allocations. The only option when you have to create a new Task<bool>



object is the case of asynchronous execution, which ends after the "return". In this case, the method has to create a new Task<bool>



object, because at the time of exiting the method, the result of the operation is not yet known. The returned object must be unique, because it will ultimately store the result of the asynchronous operation.

There are other examples of similar caching from the runtime. But such a strategy is not applicable everywhere. For example, the method:

 public async Task<int> ReadNextByteAsync() { if (_bufferedCount == 0) { await FillBuffer(); } if (_bufferedCount == 0) { return -1; } _bufferedCount--; return _buffer[_position++]; }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

also often ends synchronously. But, unlike the previous example, this method returns an integer result that has approximately four billion possible values. To cache Task<int>



, in this situation, hundreds of gigabytes of memory would be required. The environment here also supports a small cache for Task<int>



, for several small values. So, for example, if the operation completes synchronously (data is present in the buffer), with a result of 4, the cache will be used. But if the result, albeit synchronous, the completion is 42, a new Task<int>



object will be created, similar to calling Task.FromResult(42)



.

Many library implementations try to mitigate such situations by supporting their own caches. One example is the overload of MemoryStream.ReadAsync



. This operation, introduced in the .NET Framework 4.5, always ends synchronously, because it is just a reading from memory. ReadAsync



returns a Task<int>



, where the integer result represents the number of bytes read. Quite often, in the code, a situation occurs when ReadAsync



used in a loop. Moreover, if there are the following symptoms:

• The number of bytes requested does not change for most iterations of the loop;
• In most iterations, ReadAsync
  
  
  
  can read the requested number of bytes.

That is, for repeated calls, ReadAsync



runs synchronously and returns a Task<int>



object, with the same result from iteration to iteration. It is logical that MemoryStream



caches the last successfully completed task, and for all subsequent calls, if the new result matches the previous one, it returns an instance from the cache. If the result does not match, then Task.FromResult



used to create a new instance, which, in turn, is also cached before returning.

But, nevertheless, there are many cases when an operation is forced to create new Task<TResult>



objects, even when synchronously completed.

ValueTask <TResult> and synchronous completion

All this, ultimately, served as the motivation for introducing a new type of ValueTask<TResult>



into .NET Core 2.0. Also, through the nuget package System.Threading.Tasks.Extensions



, this type was made available in other .NET releases.

ValueTask<TResult>



was introduced in .NET Core 2.0 as a structure capable of wrapping TResult



or Task<TResult>



. This means that objects of this type can be returned from the async



method. The first plus from the introduction of this type is immediately visible: if the method completed successfully and synchronously, there is no need to create anything on the heap, it is simple enough to create an instance of ValueTask<TResult>



with the result value. Only if the method exits asynchronously, we need to create a Task<TResult>



. In this case, ValueTask<TResult>



used as a wrapper over Task<TResult>



. The decision to make ValueTask<TResult>



able to aggregate Task<TResult>



was made with the aim of optimization: in case of success and in case of failure, the asynchronous method creates Task<TResult>



, from the point of view of memory optimization, it is better to aggregate the Task<TResult>



object itself Task<TResult>



than keeping additional fields in a ValueTask<TResult>



for various completion cases (for example, to store an exception).

Given the above, there is no longer a need for caching in methods such as the above MemoryStream.ReadAsync



, but instead can be implemented as follows:

 public override ValueTask<int> ReadAsync(byte[] buffer, int offset, int count) { try { int bytesRead = Read(buffer, offset, count); return new ValueTask<int>(bytesRead); } catch (Exception e) { return new ValueTask<int>(Task.FromException<int>(e)); } }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

ValueTask <TResult> and Asynchronous Termination

Having the ability to write asynchronous methods that do not require additional memory allocations for the result, with synchronous completion, is really a big plus. As stated above, this was the main goal for introducing the new ValueTask<TResult>



in .NET Core 2.0. All new methods that are expected to be used on the "hot paths" now use ValueTask<TResult>



instead of Task<TResult>



as the return type. For example, a new overload of the ReadAsync



method for Stream



, in .NET Core 2.1 (which takes Memory<byte>



instead of byte[]



as the parameter), returns an instance of ValueTask<int>



. This allowed to significantly reduce the number of allocations when working with streams (very often the ReadAsync



method finishes synchronously, as in the example with MemoryStream



).

However, when developing services with high bandwidth, in which asynchronous termination is not uncommon, we need to do our best to avoid additional allocations.

As mentioned earlier, in the async/await



model, any operation that completes asynchronously must return a unique object in order to wait for completion. Unique because it will serve as a channel for performing callbacks. Note, however, that this construction does not say anything about whether the returned wait object can be reused after the completion of the asynchronous operation. If an object can be reused, then the API can maintain a pool for these kinds of objects. But, in this case, this pool cannot support concurrent access - an object from the pool will go from the "completed" state to the "not completed" state and vice versa.

To support the possibility of working with such pools, the IValueTaskSource<TResult>



interface was added to .NET Core 2.1, and the ValueTask<TResult>



structure was expanded: now objects of this type can wrap not only objects of the TResult



or Task<TResult>



, but also instances of IValueTaskSource<TResult>



. The new interface provides basic functionality that allows ValueTask<TResult>



objects to work with IValueTaskSource<TResult>



in the same manner as with Task<TResult>



:

 public interface IValueTaskSource<out TResult> { ValueTaskSourceStatus GetStatus(short token); void OnCompleted( Action<object> continuation, object state, short token, ValueTaskSourceOnCompletedFlags flags); TResult GetResult(short token); }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

GetStatus



intended for use in the ValueTask<TResult>.IsCompleted/IsCompletedSuccessfully



- allows you to find out whether the operation completed or not (successfully or not). OnCompleted



used in ValueTask<TResult>



to trigger a callback. GetResult



used to get the result, or to propagate an exception that has occurred.

Most developers are unlikely to ever need to deal with the IValueTaskSource<TResult>



interface, because asynchronous methods, when returned, hide it behind the ValueTask<TResult>



instance. The interface itself is primarily intended for those who develop high-performance APIs and seeks to avoid unnecessary work with a bunch.

In .NET Core 2.1, there are several examples of this kind of API. The most famous of these is the new overloads of the Socket.ReceiveAsync



and Socket.SendAsync



methods. For instance:

 public ValueTask<int> ReceiveAsync( Memory<byte> buffer, SocketFlags socketFlags, CancellationToken cancellationToken = default);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Objects of type ValueTask<int>



are used as the return value.

If the method exits synchronously, then it returns a ValueTask<int>



with the corresponding value:

 int result = …; return new ValueTask<int>(result);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

If the operation completes asynchronously, then a cached object is used that implements the IValueTaskSource<TResult>



interface:

 IValueTaskSource<int> vts = …; return new ValueTask<int>(vts);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

The Socket



implementation supports one cached object for receiving, and one for sending data, as long as each of them is used without competition (no, for example, competitive data sending). This strategy reduces the amount of additional memory allocated, even in the case of asynchronous execution.

The described optimization of Socket



in .NET Core 2.1 positively influenced the performance of NetworkStream



. Its overload is the ReadAsync



method of the Stream



class:

 public virtual ValueTask<int> ReadAsync( Memory<byte> buffer, CancellationToken cancellationToken);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

just delegates the work to the Socket.ReceiveAsync



method. Increasing the efficiency of the socket method, in terms of working with memory, increases the efficiency of the NetworkStream



method.

Non-generic ValueTask

I have noted several times that the original goal of ValueTask<T>



, in .NET Core 2.0, was to optimize cases of synchronous completion of methods with a "non-empty" result. This means that there was no need for a non-typed ValueTask



: in cases of synchronous completion, methods use a singleton via the Task.CompletedTask



property, and the runtime for async Task



methods is also implicitly received.

But, with the advent of the ability to avoid unnecessary allocations and with asynchronous execution, the need for a non-typed ValueTask



again became relevant. For this reason, in .NET Core 2.1, we introduced non- ValueTask



and IValueTaskSource



. They are analogues of the corresponding generic types, and are used in the same way, but for methods with an empty ( void



) return.

Implementation of IValueTaskSource / IValueTaskSource <T>

Most developers will not need to implement these interfaces. And their implementation is not an easy task. If you decide that you need to implement them yourself, then, inside .NET Core 2.1, there are several implementations that can serve as examples:

• AwaitableSocketAsyncEventArgs
• AsyncOperation <TResult>
• DefaultPipeReader

To simplify these tasks (implementations of IValueTaskSource / IValueTaskSource<T>



), we plan to introduce the type ManualResetValueTaskSourceCore<TResult>



in .NET Core 3.0. This structure will encapsulate all the necessary logic. The ManualResetValueTaskSourceCore<TResult>



instance can be used in another object that implements IValueTaskSource<TResult>



and / or IValueTaskSource



, and delegate most of the work to it. You can learn more about this at ttps: //github.com/dotnet/corefx/issues/32664.

The correct model for using ValueTasks

Even a cursory examination shows that ValueTask



and ValueTask<TResult>



more limited than Task



and Task<TResult>



. And this is normal, even desirable, because their main goal is to wait for the completion of asynchronous execution.

In particular, significant limitations arise because ValueTask



and ValueTask<TResult>



can aggregate reusable objects. In general, the following operations * NEVER should be performed when using ValueTask



/ ValueTask<TResult>



* ( let me reformulate through "Never" *):

• Never use the same ValueTask
  
  
  
  / ValueTask<TResult>
  
  
  
  object repeatedly

Motivation: The Task



and Task<TResult>



instances never go from the "completed" state to the "incomplete" state; we can use them to wait for the result as many times as we want - after completion we will always get the same result. On the contrary, since ValueTask



/ ValueTask<TResult>



can act as wrappers over reusable objects, which means that their state can change, because the state of reused objects changes by definition - to move from "completed" to "incomplete" and vice versa.

• Never ValueTask
  
  
  
  / ValueTask<TResult>
  
  
  
  in competitive mode.

Motivation: A wrapped object expects to work with only one callback, from a single consumer at a time, and attempting to compete competitively can easily lead to race conditions and subtle programming errors. Competitive expectations, this is one of the options described above multiple expectations . Note that Task



/ Task<TResult>



allow any number of competitive expectations.

• Never use .GetAwaiter().GetResult()
  
  
  
  until the operation completes .

Motivation: Implementations of IValueTaskSource



/ IValueTaskSource<TResult>



should not support locking until the operation completes. Blocking, in fact, leads to a race condition, it is unlikely that this will be the expected behavior on the part of the consumer. While Task



/ Task<TResult>



allows you to do this, thereby blocking the calling thread until the operation completes.

But what if, nevertheless, you need to do one of the operations described above, and the called method returns instances of ValueTask



/ ValueTask<TResult>



? For such cases, ValueTask



/ ValueTask<TResult>



provides the .AsTask()



method. By calling this method, you will get an instance of Task



/ Task<TResult>



, and you can already perform the necessary operation with it. Reusing the original object after calling .AsTask()



is not allowed .

A short rule says : When working with an instance of ValueTask



/, ValueTask<TResult>



you must either expect ( await



) it directly (or, if necessary, c .ConfigureAwait(false)



), or call it .AsTask()



, and never use the original object ValueTask



/ again ValueTask<TResult>



.

 // Given this ValueTask<int>-returning method… public ValueTask<int> SomeValueTaskReturningMethodAsync(); … // GOOD int result = await SomeValueTaskReturningMethodAsync(); // GOOD int result = await SomeValueTaskReturningMethodAsync().ConfigureAwait(false); // GOOD Task<int> t = SomeValueTaskReturningMethodAsync().AsTask(); // WARNING ValueTask<int> vt = SomeValueTaskReturningMethodAsync(); ... // storing the instance into a local makes it much more likely it'll be misused, // but it could still be ok // BAD: awaits multiple times ValueTask<int> vt = SomeValueTaskReturningMethodAsync(); int result = await vt; int result2 = await vt; // BAD: awaits concurrently (and, by definition then, multiple times) ValueTask<int> vt = SomeValueTaskReturningMethodAsync(); Task.Run(async () => await vt); Task.Run(async () => await vt); // BAD: uses GetAwaiter().GetResult() when it's not known to be done ValueTask<int> vt = SomeValueTaskReturningMethodAsync(); int result = vt.GetAwaiter().GetResult();
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

There is another additional “advanced” usage pattern that some programmers may decide to apply (I hope only after careful measurements, with justification of the benefits of its use).

ValueTask



/ ValueTask<TResult>



, . , IsCompleted



true



, ( , ), — false



, IsCompletedSuccessfully



true



. " " , , , , , . await



/ .AsTask()



.Result



. , SocketsHttpHandler



.NET Core 2.1, .ReadAsync



, ValueTask<int>



. , , , . , .. . Because , , , , :

 int bytesRead; { ValueTask<int> readTask = _connection.ReadAsync(buffer); if (readTask.IsCompletedSuccessfully) { bytesRead = readTask.Result; } else { using (_connection.RegisterCancellation()) { bytesRead = await readTask; } } }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

, .. ValueTask<int>



, .Result



, await



, .

API ValueTask / ValueTask<TResult>?

, . Task



/ ValueTask<TResult>



.

, Task



/ Task<TResult>



. , "" / , Task



/ Task<TResult>



. , , ValueTask<TResult>



Task<TResult>



: , , await



Task<TResult>



ValueTask<TResult>



. , (, API Task



Task<bool>



), , , Task



( Task<bool>



). , ValueTask



/ ValueTask<TResult>



. , async-, ValueTask



/ ValueTask<TResult>



, .

, ValueTask



/ ValueTask<TResult>



, :

1. , API ,
2. API ,
3. , , , .

, abstract



/ virtual



, , / ?

What's next?

.NET, API, Task



/ Task<TResult>



. , , API c ValueTask



/ ValueTask<TResult>



, . IAsyncEnumerator<T>



, .NET Core 3.0. IEnumerator<T>



MoveNext



, . — IAsyncEnumerator<T>



MoveNextAsync



. , Task<bool>



, , . , , , ( ), , , await foreach



-, , MoveNextAsync



, ValueTask<bool>



. , , , " " , . , C# , .