Counting download speed in your application

Background

I have a small and cozy pet-project, which allows you to download files from the Internet. The files are grouped together and the user is not shown each file, but some grouping. And the entire download process (and the display of this process) was highly dependent on the data. The data was obtained on the fly, i.e. the user starts to download and there is no information how much you have to download in reality.

The naive implementation of at least some kind of informing is made simple - the download progress is displayed as the ratio of the number of downloaded to the total number. There is not much information for the user - just a creeping strip, but this is better than nothing, and it is noticeably better than the currently popular loading mechanism without indicating progress.

And then a user appears with a logical problem - in a large group it is not clear why progress is barely creeping - do I need to download a lot of files or a low speed? As I mentioned above - the number of files is not known in advance. Therefore, I decided to add a speed counter.

Analysis

It’s good practice to see those who have already solved a similar problem so as not to reinvent the wheel. Different software closes these different tasks, but the display looks pretty much the same:

uTorrent	Downloadmaster

The key point that I have identified for myself is that the first display of speed is needed at the current time. Not what speed was average, not what speed as a whole was average from the moment it started, namely what this figure is at the current moment. In fact, this is important when I get to the code - I will explain it separately.

So, we need a simple digit like 10 MB/s



or something like that. How do we calculate it?

Theory and practice

The existing download implementation used HttpWebRequest



and I decided not to redo the download itself - do not touch the working mechanism.

So, the initial implementation without any calculation:

  var request = WebRequest.Create(uri); var response = await request.GetResponseAsync(); using (var ms = new MemoryStream()) { await response.GetResponseStream().CopyToAsync(ms); return ms.ToArray(); }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

At the level of such an API, you can only respond to a full file download, for small groups (or even for one file), the speed can not actually be calculated. We follow the CopyToAsync source code , copy-paste the simple logic from there:

  byte[] buffer = new byte[bufferSize]; int bytesRead; while ((bytesRead = await ReadAsync(buffer, 0, buffer.Length, cancellationToken).ConfigureAwait(false)) != 0) { await destination.WriteAsync(buffer, 0, bytesRead, cancellationToken).ConfigureAwait(false); }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Now we can respond to every buffer given to us over the network.

So, firstly, what we do instead of the boxed CopyToAsync:

  public static async Task<byte[]> GetBytesAsync(this Stream from) { using (var memory = new MemoryStream()) { byte[] buffer = new byte[81920]; int bytesRead; while ((bytesRead = await from.ReadAsync(buffer, 0, buffer.Length).ConfigureAwait(false)) != 0) { await memory.WriteAsync(buffer, 0, bytesRead).ConfigureAwait(false); NetworkSpeed.AddInfo(bytesRead); } return memory.ToArray(); } }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

The only thing really added is NetworkSpeed.AddInfo



. And the only thing we transmit is the number of bytes downloaded.

The code itself for downloading looks like this:

  var request = WebRequest.Create(uri); var response = await request.GetResponseAsync(); var array = await response.GetResponseStream().GetBytesAsync();
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

  var client = new WebClient(); var lastRecorded = 0L; client.DownloadProgressChanged += (sender, eventArgs) => { NetworkSpeed.AddInfo(eventArgs.BytesReceived - lastRecorded); lastRecorded = eventArgs.BytesReceived; }; var array = await client.DownloadDataTaskAsync(uri);
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

  var httpClient = new HttpClient(); var content = await httpClient.GetStreamAsync(uri); var array = await content.GetBytesAsync();
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Well, half the problem is solved - we know how much we downloaded. We turn to speed.

According to wikipedia :

Data transfer rate - the amount of data transmitted per unit of time.

First naive approach

We have a volume. Time can be taken literally from startup and get the difference with DateTime.Now



. Take and share?

For console utilities like curl, this is possible and makes sense.

But if your application is a little more complicated, then literally the pause button will dramatically complicate your life.

A little about the pause

Maybe I'm very naive, or maybe the question is really not so simple - but the pause makes me think constantly. A pause while downloading can behave in at least three ways:

• interrupt the download of files, start again after
• just do not download the file further, hope that the server will continue after
• download already started files, do not download new ones, download new ones after

Since the first two lead to the loss of already downloaded information, I use the third.

A little higher, I noticed that speed is needed precisely at a point in time. So, a pause complicates this matter:

• you can’t properly calculate what the average speed was, just taking the volume for a while
• a pause may have external reasons that will change the speed and channel (reconnecting to the provider's network, switching to VPN, terminating uTorrent which takes the entire channel), which will lead to a change in the real speed
  
  In fact, a pause divides any indicators into before and after it. This does not particularly affect the code below, just a minute of fun information to think about.

Second naive approach

Add a timer. The timer each time period will take all the latest information about the downloaded volume and recalculate the speed indicator. And if you set the timer per second, then all the information received for this second about the downloaded volume will be equal to the speed for this second:

  public class NetworkSpeed { public static double TotalSpeed { get { return totalSpeed; } } private static double totalSpeed = 0; private const uint TimerInterval = 1000; private static Timer speedTimer = new Timer(state => { var now = 0L; while (ReceivedStorage.TryDequeue(out var added)) now += added; totalSpeed = now; }, null, 0, TimerInterval); private static readonly ConcurrentQueue<long> ReceivedStorage = new ConcurrentQueue<long>(); public static void Clear() { while (ReceivedStorage.TryDequeue(out _)) { } totalSpeed = 0; } public static void AddInfo(long received) { ReceivedStorage.Enqueue(received); } }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Compared with the first option, such an implementation begins to respond to a pause - the speed drops to 0 in the next second after the data outside arrives.

But there are also disadvantages. We are working with a buffer of 80kb, which means that the download started in one second will be displayed only in the next. And with a large stream of parallel downloads, such measurement errors will display anything - I had a spread of up to 30% of the real numbers. I might not have noticed, but exceeding 100 Mbit looked too suspicious .

Third approach

The second option is already close enough to the truth, plus his error was observed more at the start of the download, and not throughout the life cycle.

Therefore, a simple solution is to take as an indicator not the figure per second, but the average over the last three seconds. Three here is a magic constant matched by eye. On the one hand, I wanted a pleasant display of the growth and decline of speed, on the other - so that the speed was closer to the truth.

The implementation is a little complicated, but in general, nothing like this:

  public class NetworkSpeed { public static double TotalSpeed { get { return totalSpeed; } } private static double totalSpeed = 0; private const uint Seconds = 3; private const uint TimerInterval = 1000; private static Timer speedTimer = new Timer(state => { var now = 0L; while (ReceivedStorage.TryDequeue(out var added)) now += added; LastSpeeds.Enqueue(now); totalSpeed = LastSpeeds.Average(); OnUpdated(totalSpeed); }, null, 0, TimerInterval); private static readonly LimitedConcurrentQueue<double> LastSpeeds = new LimitedConcurrentQueue<double>(Seconds); private static readonly ConcurrentQueue<long> ReceivedStorage = new ConcurrentQueue<long>(); public static void Clear() { while (ReceivedStorage.TryDequeue(out _)) { } while (LastSpeeds.TryDequeue(out _)) { } totalSpeed = 0; } public static void AddInfo(long received) { ReceivedStorage.Enqueue(received); } public static event Action<double> Updated; private class LimitedConcurrentQueue<T> : ConcurrentQueue<T> { public uint Limit { get; } public new void Enqueue(T item) { while (Count >= Limit) TryDequeue(out _); base.Enqueue(item); } public LimitedConcurrentQueue(uint limit) { Limit = limit; } } private static void OnUpdated(double obj) { Updated?.Invoke(obj); } }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

A couple of points:

• at the time of implementation, I did not find a ready-made queue with a limit on the number of elements and took it on the Internet, in the code above it is LimitedConcurrentQueue
  
  
  
  .
• instead of implementing INotifyPropertyChanged
  
  
  
  for some reason, Action
  
  
  
  , the use is practically the same, I don’t remember the reasons. The logic is simple - the indicator is changing, users need to be notified about this. The implementation can be any, even IObservable
  
  
  
  , to whom it is more convenient.

And a little readability

The API gives the speed in bytes, a simple one (taken on the Internet) is useful for readability

  public static string HumanizeByteSize(this long byteCount) { string[] suf = { "B", "KB", "MB", "GB", "TB", "PB", "EB" }; //Longs run out around EB if (byteCount == 0) return "0" + suf[0]; long bytes = Math.Abs(byteCount); int place = Convert.ToInt32(Math.Floor(Math.Log(bytes, 1024))); double num = Math.Round(bytes / Math.Pow(1024, place), 1); return Math.Sign(byteCount) * num + suf[place]; } public static string HumanizeByteSize(this double byteCount) { if (double.IsNaN(byteCount) || double.IsInfinity(byteCount) || byteCount == 0) return string.Empty; return HumanizeByteSize((long)byteCount); }
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Let me remind you that the speed in bytes, i.e. per 100mbit channel should issue no more than 12.5MB.

How it ultimately looks like:

Conclusion

It was interesting to deal with a seemingly banal task of counting speed. And even though the code works and gives out some numbers, I want to listen to critics - what I missed, how could I do better, maybe there are some ready-made solutions.

I want to say thanks to Stack Overflow in Russian and specifically VladD-exrabbit - although there is half the answer in a good question, any hints and any help always move you forward.

I want to remind you that this is a pet-project - that's why the class is static and one at all, so the accuracy is not really. I see a lot of little things that could be done better, but ... there is always something else to do, so for now I think that's the speed and I think that this is not a bad option.