SSEGWSW: Server-Sent Events Gateway by Service Workers

Hello!



My name is Sasha and I work as an architect at Tinkoff Business.



In this article I want to talk about how to overcome the restriction of browsers on the number of open long-lived HTTP connections within the same domain using service worker.



If you want, feel free to skip the background, description of the problem, search for a solution, and immediately proceed to the result.

Background

Once upon a time in Tinkoff Business there was a chat that worked on Websocket .



After some time, he ceased to fit into the design of his personal account, and in general he was long asking for a rewrite from angular 1.6 to angular 2+. I decided that it was time to start updating it. A colleague-backender found out that the chat frontend will change, and suggested at the same time redo the API, in particular - change the transport from websocket to SSE (server-sent events) . He suggested this because when updating the NGINX config all connections were broken, which then was painful to restore.



We discussed the architecture of the new solution and came to the conclusion that we will receive and send data using ordinary HTTP requests. For example, send a POST: / api / send-message message , get a list of GET dialogs : / api / conversations-list, and so on. And asynchronous events like "a new message from the interlocutor" will be sent via SSE. So we will increase the fault tolerance of the application: if the SSE connection falls off, the chat will still work, only it will not receive realtime notifications.



In addition to the chat in websocket, we also chased events for the “thin notifications” component. This component allows you to send various notifications to the user's personal account, for example, that the import of accounts, which may take several minutes, has been completed successfully. To completely abandon websocket, we moved this component to a separate SSE connection.

Problem

When you open one browser tab, two SSE connections are created: one for chat and one for subtle notifications. Well, let them be created. Sorry, or what? We don’t feel sorry, but browsers feel sorry! They have a limit on the number of concurrent persistent connections for a domain . Guess how much is in Chrome? Right, six! I opened three tabs - I scored the entire connection pool and you can no longer make HTTP requests. This is true for the HTTP / 1.x protocol. In HTTP / 2 there is no such problem due to multiplexing.



There are a couple of ways to solve this problem at the infrastructure level:

1. Domain sharding.
2. HTTP / 2.

Both of these methods seemed expensive, since a lot of infrastructure would have to be affected.



Therefore, for starters, we tried to solve the problem on the browser side. The first idea was to make some kind of transport between the tabs, for example, through the LocalStorage or Broadcast Channel API .



The meaning is this: we open SSE connections in only one tab and send the data to the rest. This solution also did not look optimal, since it would require the release of all 50 SPA, which make up the Tinkoff Business personal account. Releasing 50 applications is also expensive, so I continued to look for other ways.

Decision

I recently worked with service workers and thought: is it possible to apply them in this situation?



To answer this question, you first need to understand what service workers can do at all? They can proxy requests, it looks something like this:

self.addEventListener('fetch', event => { const response = self.caches.open('example') .then(caches => caches.match(event.request)) .then(response => response || fetch(event.request)); event.respondWith(response); });
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

We listen to events for HTTP requests and respond as we like. In this case, we are trying to respond from the cache, and if it does not work out, then we make a request to the server.



Ok, let's try to intercept the SSE connection and answer it:

 self.addEventListener('fetch', event => { const {headers} = event.request; const isSSERequest = headers.get('Accept') === 'text/event-stream'; if (!isSSERequest) { return; } event.respondWith(new Response('Hello!')); });
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

In network requests we see the following picture:







And in the console, this:







Already not bad. The request was intercepted, but the SSE does not want a response in the form of text / plain , but wants text / event-stream . How to create a stream now? But can I even respond with a stream from a service worker? Well let's see:







Fine! The Response class takes as a body ReadableStream . After reading the documentation , you can find out that ReadableStream has a controller that has an enqueue () method - with its help you can stream data. Suitable, take it!

 self.addEventListener('fetch', event => { const {headers} = event.request; const isSSERequest = headers.get('Accept') === 'text/event-stream'; if (!isSSERequest) { return; } const responseText = 'Hello!'; const responseData = Uint8Array.from(responseText, x => x.charCodeAt(0)); const stream = new ReadableStream({start: controller => controller.enqueue(responseData)}); const response = new Response(stream); event.respondWith(response); });
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

There is no error, the connection hangs in pending status and no data arrives on the client side. Comparing my request with a real server request, I realized that the answer was in the headers. For SSE requests, the following headers must be specified:

 const sseHeaders = { 'content-type': 'text/event-stream', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', };
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

When you add these headers, the connection will open successfully, but the data will not be received on the client side. This is obvious, because you can’t just send random text - there must be some format.



At javascript.info, the data format in which you want to send data from the server is well described . It can be easily described with one function:

 const sseChunkData = (data: string, event?: string, retry?: number, id?: number): string => Object.entries({event, id, data, retry}) .filter(([, value]) => ![undefined, null].includes(value)) .map(([key, value]) => `${key}: ${value}`) .join('\n') + '\n\n';
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

To comply with the SSE format, the server must send messages separated by a double line break \ n \ n .



The message consists of the following fields:

• data - message body, several data in a row are interpreted as one message, separated by line breaks \ n;
• id - updates the lastEventId property that is sent in the Last-Event-ID header when reconnecting;
• retry - the recommended delay before reconnecting in milliseconds, cannot be set using JavaScript;
• event - the name of the user event, indicated before data.

Add the necessary headers, change the answer to the desired format and see what happens:

 self.addEventListener('fetch', event => { const {headers} = event.request; const isSSERequest = headers.get('Accept') === 'text/event-stream'; if (!isSSERequest) { return; } const sseChunkData = (data, event, retry, id) => Object.entries({event, id, data, retry}) .filter(([, value]) => ![undefined, null].includes(value)) .map(([key, value]) => `${key}: ${value}`) .join('\n') + '\n\n'; const sseHeaders = { 'content-type': 'text/event-stream', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', }; const responseText = sseChunkData('Hello!'); const responseData = Uint8Array.from(responseText, x => x.charCodeAt(0)); const stream = new ReadableStream({start: controller => controller.enqueue(responseData)}); const response = new Response(stream, {headers: sseHeaders}); event.respondWith(response); });
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Oh my glob! Yes, I made an SSE connection without a server!

Result

Now we can successfully intercept the SSE request and respond to it without going beyond the browser.



Initially, the idea was to establish a connection with the server, but only one thing - and from it to send data to tabs. Let's do it!

 self.addEventListener('fetch', event => { const {headers, url} = event.request; const isSSERequest = headers.get('Accept') === 'text/event-stream'; //   SSE- if (!isSSERequest) { return; } //    SSE const sseHeaders = { 'content-type': 'text/event-stream', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', }; // ,    SSE const sseChunkData = (data, event, retry, id) => Object.entries({event, id, data, retry}) .filter(([, value]) => ![undefined, null].includes(value)) .map(([key, value]) => `${key}: ${value}`) .join('\n') + '\n\n'; //    ,   — url,  — EventSource const serverConnections = {}; //   url             const getServerConnection = url => { if (!serverConnections[url]) serverConnections[url] = new EventSource(url); return serverConnections[url]; }; //          const onServerMessage = (controller, {data, type, retry, lastEventId}) => { const responseText = sseChunkData(data, type, retry, lastEventId); const responseData = Uint8Array.from(responseText, x => x.charCodeAt(0)); controller.enqueue(responseData); }; const stream = new ReadableStream({ start: controller => getServerConnection(url).onmessage = onServerMessage.bind(null, controller) }); const response = new Response(stream, {headers: sseHeaders}); event.respondWith(response); });
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

The same code on github.

I got a pretty simple solution for a not-so-trivial task. But, of course, there are still many nuances. For example, you need to close the connection to the server when closing all the tabs, fully support the SSE protocol, and so on.



We have successfully decided all this - I’m sure it will not be difficult for you!