Problems and nuances when developing for SmartTV using React.js
I want to share my experience in developing an application with viewing video content for SmartTV (Tizen and WebOS) and what problems we encountered.
On modern TVs, as we know, you can install different applications to facilitate the work with some resources and content. In most cases, these applications under the hood are a normal browser, on telly it is Chromium.
Since this is a browser, nothing prevented us from using React.js for development, which affected some performance issues.
I will not talk about exactly how you need to do with the code examples, I will talk about the nuances and decisions that have been made.
Let's go through the main points:
There is not as much memory and performance on the TV as on our computers, so the first problem is resource limitation.
For example, only about 250 megabytes for use are allocated for the application and we encountered such a problem that when displaying a grid with content (picture, description, rating), memory could be spent at the limit, when there are only 1000 elements in the grid and we scroll down, then the image size increases and increases, and at some point the application crashes with the inscription that the memory has run out.
The solution to this react-window problem
It does not store all elements outside the window in the DOM.
If on a computer we are used to clicking on elements on the site, then on televisions we use a remote control that can also be used as a mouse, but most of these are buttons. The navigation problem comes out of here, that we must process the button presses of the remote control and navigate the entire application without the cursor, but at the same time we need to process the click on the elements with a “click”.
Some button codes on the LG and Samsung remotes are different, as well as different from the usual keyboard, so for each platform we had hardcoded button codes that we process.
Tizen example
Tizen
WebOS
Keyboard
As you can see, on the keyboard these are buttons with letters (indicated in the comments).
In order to handle navigation through clicking buttons and clicking with the mouse, we always had the result of one event - mouseClick, thereby we processed all the events the same way.
Another feature of navigation is focus, on each screen there should be an element in focus, because having lost focus, we will no longer be able to navigate the screen. The focus and id properties of the focus were assigned to each element. When navigating it was always necessary to take this into account, but also, sometimes there is a requirement when returning back to restore the state of the completely previous screen, so the navigation was written completely custom.
Animation on CSS works slowly on TVs, especially when there are a lot of DOM elements that fall into the animation, you can see not a smooth change, but a slide show. One solution to this problem is canvas. Drawing on it, animation, lighting during navigation speed up the work by several times, but if you have automation, they may not like this implementation, because it is difficult for them to check the content from the picture.
A lot of page rendering is another pain, everything starts to slow down, twitch. It was necessary to control this process as much as possible, after which shouldComponentUpdate of the components looks like this:
As a result, we rewrote sensitive parts of the application on vanilla js, it became faster.
LG and Samsung have different video players, which also creates additional development difficulties. For LG, a video tag is used, for Samsung AVPlay sdk. Therefore, there are differences in functionality, it must be borne in mind that not all features are the same and may differ from version to OS version.
There were also problems with the load when starting the video - it was solved by caching, parallelization and delaying requests, processing.
The speed of the backend, the speed of processing requests and their number affects the speed of work. The FE side should have a minimal amount of data processing. If you need to load large amounts of data, you can use web workers.
On modern TVs, as we know, you can install different applications to facilitate the work with some resources and content. In most cases, these applications under the hood are a normal browser, on telly it is Chromium.
Since this is a browser, nothing prevented us from using React.js for development, which affected some performance issues.
I will not talk about exactly how you need to do with the code examples, I will talk about the nuances and decisions that have been made.
Let's go through the main points:
- Limited resources
- Navigation
- Performance Styles and Rendering
- Video players
- Backend
Limited resources
There is not as much memory and performance on the TV as on our computers, so the first problem is resource limitation.
For example, only about 250 megabytes for use are allocated for the application and we encountered such a problem that when displaying a grid with content (picture, description, rating), memory could be spent at the limit, when there are only 1000 elements in the grid and we scroll down, then the image size increases and increases, and at some point the application crashes with the inscription that the memory has run out.
The solution to this react-window problem
It does not store all elements outside the window in the DOM.
Navigation
If on a computer we are used to clicking on elements on the site, then on televisions we use a remote control that can also be used as a mouse, but most of these are buttons. The navigation problem comes out of here, that we must process the button presses of the remote control and navigate the entire application without the cursor, but at the same time we need to process the click on the elements with a “click”.
Some button codes on the LG and Samsung remotes are different, as well as different from the usual keyboard, so for each platform we had hardcoded button codes that we process.
Tizen example
Tizen
export default { KEY_0: 48, KEY_1: 49, KEY_2: 50, KEY_3: 51, KEY_4: 52, KEY_5: 53, KEY_6: 54, KEY_7: 55, KEY_8: 56, KEY_9: 57, KEY_UP: 38, KEY_DOWN: 40, KEY_LEFT: 37, KEY_RIGHT: 39, KEY_OK: 13, KEY_BACK: 10009, KEY_CHANNEL_UP: 427, KEY_CHANNEL_DOWN: 428, KEY_MEDIA_FAST_FORWARD: 417, KEY_MEDIA_PAUSE: 19, KEY_MEDIA_PLAY: 415, KEY_MEDIA_PLAY_PAUSE: 10252, KEY_MEDIA_REWIND: 412, KEY_MEDIA_STOP: 413, KEY_DEBUG_TOGGLE_CONSOLE: 403, KEY_DEBUG_TOGGLE_QUICK_EDIT: 404, KEY_DEBUG_SET_FAVOURITES: 405, KEY_DEBUG_CLEAR_FAVOURITES: 406, KEY_SHOW_REMOTE_POINTER: 7777777, // not applicable KEY_HIDE_REMOTE_POINTER: 7777777, // not applicable };
WebOS
export default { KEY_0: 48, KEY_1: 49, KEY_2: 50, KEY_3: 51, KEY_4: 52, KEY_5: 53, KEY_6: 54, KEY_7: 55, KEY_8: 56, KEY_9: 57, KEY_UP: 38, KEY_DOWN: 40, KEY_LEFT: 37, KEY_RIGHT: 39, KEY_OK: 13, KEY_BACK: 461, KEY_MEDIA_FAST_FORWARD: 417, KEY_MEDIA_PAUSE: 19, KEY_MEDIA_PLAY: 415, KEY_MEDIA_PLAY_PAUSE: 10252, KEY_MEDIA_REWIND: 412, KEY_MEDIA_STOP: 413, KEY_CHANNEL_UP: 33, KEY_CHANNEL_DOWN: 34, KEY_DEBUG_SET_EMAIL: 403, KEY_DEBUG_TOGGLE_CONSOLE: 404, KEY_DEBUG_TOGGLE_QUICK_EDIT: 405, KEY_DEBUG_SET_FAVOURITES: 406, KEY_DEBUG_CLEAR_FAVOURITES: 407, KEY_SHOW_REMOTE_POINTER: 1536, KEY_HIDE_REMOTE_POINTER: 1537, };
Keyboard
export default { KEY_0: 48, KEY_1: 49, KEY_2: 50, KEY_3: 51, KEY_4: 52, KEY_5: 53, KEY_6: 54, KEY_7: 55, KEY_8: 56, KEY_9: 57, KEY_UP: 38, KEY_DOWN: 40, KEY_LEFT: 37, KEY_RIGHT: 39, KEY_OK: 13, KEY_BACK: 8, // backspace KEY_MEDIA_FAST_FORWARD: 70, // f KEY_MEDIA_PLAY_PAUSE: 80, // p KEY_MEDIA_REWIND: 66, // b KEY_MEDIA_STOP: 83, // s KEY_CHANNEL_UP: 70, // f KEY_CHANNEL_DOWN: 71, // g KEY_DEBUG_SET_EMAIL: 81, // q KEY_DEBUG_TOGGLE_CONSOLE: 87, // w KEY_DEBUG_TOGGLE_QUICK_EDIT: 69, // e KEY_DEBUG_SET_FAVOURITES: 88, // w KEY_DEBUG_CLEAR_FAVOURITES: 82, // r KEY_SHOW_REMOTE_POINTER: 7777777, // not applicable KEY_HIDE_REMOTE_POINTER: 7777777, // not applicable };
As you can see, on the keyboard these are buttons with letters (indicated in the comments).
In order to handle navigation through clicking buttons and clicking with the mouse, we always had the result of one event - mouseClick, thereby we processed all the events the same way.
Another feature of navigation is focus, on each screen there should be an element in focus, because having lost focus, we will no longer be able to navigate the screen. The focus and id properties of the focus were assigned to each element. When navigating it was always necessary to take this into account, but also, sometimes there is a requirement when returning back to restore the state of the completely previous screen, so the navigation was written completely custom.
Performance Styles and Renders
Animation on CSS works slowly on TVs, especially when there are a lot of DOM elements that fall into the animation, you can see not a smooth change, but a slide show. One solution to this problem is canvas. Drawing on it, animation, lighting during navigation speed up the work by several times, but if you have automation, they may not like this implementation, because it is difficult for them to check the content from the picture.
A lot of page rendering is another pain, everything starts to slow down, twitch. It was necessary to control this process as much as possible, after which shouldComponentUpdate of the components looks like this:
As a result, we rewrote sensitive parts of the application on vanilla js, it became faster.
Video players
LG and Samsung have different video players, which also creates additional development difficulties. For LG, a video tag is used, for Samsung AVPlay sdk. Therefore, there are differences in functionality, it must be borne in mind that not all features are the same and may differ from version to OS version.
There were also problems with the load when starting the video - it was solved by caching, parallelization and delaying requests, processing.
Backend
The speed of the backend, the speed of processing requests and their number affects the speed of work. The FE side should have a minimal amount of data processing. If you need to load large amounts of data, you can use web workers.
All Articles