Story about solving the performance problem Moment.js
Moment.js is one of the most popular JavaScript libraries for parsing and formatting dates. WhereTo uses Node.js, so for them, using this library was a completely natural move. Problems with server usage of Moment.js were not expected. In the end, from the very beginning they used this library in the frontend to display dates and were pleased with its work. However, the fact that the library performed well on the client did not mean that there would be no problems with the server either.
The material, the translation of which we publish today, is dedicated to the story of solving the performance problem Moment.js.
Recently, the number of airline flight records returned by the WhereTo system has grown approximately tenfold. Then we faced a very strong drop in performance. It turned out that the rendering cycle, which took less than 100 milliseconds, now takes more than 3 seconds to display about 5,000 search results. Our team has undertaken research. After several profiling sessions, we noticed that more than 99% of this time is spent in a single function called
.
The createInZone function takes about 3.3 seconds to complete.
Continuing our investigation of the situation, we found that this function is called by the Moment.js
function. Why is she so slow? We had the feeling that the Moment.js library was designed for common use cases, and as a result, it will try to process the input string in various ways. Maybe you should limit it? After we read the documentation, we found out that the
function accepts an optional argument specifying the date format:
The first thing we did was try to use the
function with passing information about the date format to it, but this, as the performance tests showed, did not lead to anything:
Although now the
function worked a little faster, for our needs this speed was clearly not enough.
We used Moment.js to parse dates retrieved from our provider’s API (Travelport). We realized that it always returns data in the same format:
Knowing this, we began to understand the internal structure of Moment.js in order to (as we hoped) write a much more efficient function that produces the same results.
First, we needed to figure out how the Moment.js objects look. This was pretty easy to understand:
The next step was to instantiate Moment without using a constructor:
Now it seemed like we had a lot of Moment instance properties that we could just set (I don’t go into the details of how we found out about this, but if you look at the source code of Moment.js you will understand):
The last step of our work was to find out how to parse the
value of the timestamp. It turned out that this is always the same position in the line. As a result, we were able to optimize this:
Here's what happened after we put it all together:
We tested the performance of the resulting solution using the benchmark npm module. Here is the benchmark code:
Here are the results of our performance research:
As it turned out, we managed to speed up the analysis of timestamps by about 64 times. But how did this affect the actual operation of the system? Here's what happened as a result of profiling.
The total runtime of parseTravelportTimestamp is less than 40 ms.
The results were simply amazing: we started with 3.3 seconds, going into parsing dates, and came to less than 40 milliseconds.
When we started working on our platform, we had to solve just a terrible amount of problems. We only knew that we were repeating to ourselves: “Let it work first, but you can do optimization later”.
Over the past few years, the complexity of our project has grown tremendously. Fortunately, now we have come to the place where we can move on to the second part of our “mantra” - optimization.
Library solutions helped the project get to where it is today. But we are faced with a "library" problem. Solving it, we learned that by creating our own mechanism focused on our needs, we can make the code “easier” in terms of consuming system resources and save valuable time for our users.
Dear readers! Have you encountered problems similar to the one discussed in this article?
The material, the translation of which we publish today, is dedicated to the story of solving the performance problem Moment.js.
Project growth and productivity decline
Recently, the number of airline flight records returned by the WhereTo system has grown approximately tenfold. Then we faced a very strong drop in performance. It turned out that the rendering cycle, which took less than 100 milliseconds, now takes more than 3 seconds to display about 5,000 search results. Our team has undertaken research. After several profiling sessions, we noticed that more than 99% of this time is spent in a single function called
createInZone
.
The createInZone function takes about 3.3 seconds to complete.
Continuing our investigation of the situation, we found that this function is called by the Moment.js
parseZone
function. Why is she so slow? We had the feeling that the Moment.js library was designed for common use cases, and as a result, it will try to process the input string in various ways. Maybe you should limit it? After we read the documentation, we found out that the
parseZone
function accepts an optional argument specifying the date format:
moment.parseZone(input, [format])
The first thing we did was try to use the
parseZone
function with passing information about the date format to it, but this, as the performance tests showed, did not lead to anything:
$ node bench.js moment#parseZone x 22,999 ops/sec ±7.57% (68 runs sampled) moment#parseZone (with format) x 30,010 ops/sec ±8.09% (77 runs sampled)
Although now the
parseZone
function worked a little faster, for our needs this speed was clearly not enough.
Project-specific optimization
We used Moment.js to parse dates retrieved from our provider’s API (Travelport). We realized that it always returns data in the same format:
"2019-12-03T14:05:00.000-07:00"
Knowing this, we began to understand the internal structure of Moment.js in order to (as we hoped) write a much more efficient function that produces the same results.
Creating a faster alternative to parseZone
First, we needed to figure out how the Moment.js objects look. This was pretty easy to understand:
> const m = moment() > console.log(m) Moment { _isAMomentObject: true, _i: '2019-12-03T14:05:00.000-07:00', _f: 'YYYY-MM-DDTHH:mm:ss.SSSSZ', _tzm: -420, _isUTC: true, _pf: { ...snip }, _locale: [object Locale], _d: 2019-12-03T14:05:00.000Z, _isValid: true, _offset: -420 }
The next step was to instantiate Moment without using a constructor:
export function parseTravelportTimestamp(input: string) { const m = {} // $FlowIgnore m.__proto__ = moment.prototype return m }
Now it seemed like we had a lot of Moment instance properties that we could just set (I don’t go into the details of how we found out about this, but if you look at the source code of Moment.js you will understand):
const FAKE = moment() const TRAVELPORT_FORMAT = 'YYYY-MM-DDTHH:mm:ss.SSSSZ' export function parseTravelportTimestamp(input: string) { const m = {} // $FlowIgnore m.__proto__ = moment.prototype const offset = 0 // TODO const date = new Date(input.slice(0, 23)) m._isAMomentObject = true m._i = input m._f = TRAVELPORT_FORMAT m._tzm = offset m._isUTC = true m._locale = FAKE._locale m._d = date m._isValid = true m._offset = offset return m }
The last step of our work was to find out how to parse the
offset
value of the timestamp. It turned out that this is always the same position in the line. As a result, we were able to optimize this:
function parseTravelportDateOffset(input: string) { const hrs = +input.slice(23, 26) const mins = +input.slice(27, 29) return hrs * 60 + (hrs < 0 ? -mins : mins) }
Here's what happened after we put it all together:
const FAKE = moment() const TRAVELPORT_FORMAT = 'YYYY-MM-DDTHH:mm:ss.SSSSZ' function parseTravelportDateOffset(input: string) { const hrs = +input.slice(23, 26) const mins = +input.slice(27, 29) return hrs * 60 + (hrs < 0 ? -mins : mins) } /** * ISO-8601, : * - "2019-12-03T12:30:00.000-07:00" */ export function parseTravelportTimestamp(input: string): moment { const m = {} // $FlowIgnore m.__proto__ = moment.prototype const offset = parseTravelportDateOffset(input) const date = new Date(input.slice(0, 23)) m._isAMomentObject = true m._i = input m._f = TRAVELPORT_FORMAT m._tzm = offset m._isUTC = true m._locale = FAKE._locale m._d = date m._isValid = true m._offset = offset return m }
Performance tests
We tested the performance of the resulting solution using the benchmark npm module. Here is the benchmark code:
const FAKE = moment() const TRAVELPORT_FORMAT = 'YYYY-MM-DDTHH:mm:ss.SSSSZ' function parseTravelportDateOffset(input: string) { const hrs = +input.slice(23, 26) const mins = +input.slice(27, 29) return hrs * 60 + (hrs < 0 ? -mins : mins) } /** * ISO-8601, : * - "2019-12-03T12:30:00.000-07:00" */ export function parseTravelportTimestamp(input: string): moment { const m = {} // $FlowIgnore m.__proto__ = moment.prototype const offset = parseTravelportDateOffset(input) const date = new Date(input.slice(0, 23)) m._isAMomentObject = true m._i = input m._f = TRAVELPORT_FORMAT m._tzm = offset m._isUTC = true m._locale = FAKE._locale m._d = date m._isValid = true m._offset = offset return m }
Here are the results of our performance research:
$ node fastMoment.bench.js moment#parseZone x 21,063 ops/sec ±7.62% (73 runs sampled) moment#parseZone (with format) x 24,620 ops/sec ±6.11% (71 runs sampled) fast#parseTravelportTimestamp x 1,357,870 ops/sec ±5.24% (79 runs sampled) Fastest is fast#parseTravelportTimestamp
As it turned out, we managed to speed up the analysis of timestamps by about 64 times. But how did this affect the actual operation of the system? Here's what happened as a result of profiling.
The total runtime of parseTravelportTimestamp is less than 40 ms.
The results were simply amazing: we started with 3.3 seconds, going into parsing dates, and came to less than 40 milliseconds.
Summary
When we started working on our platform, we had to solve just a terrible amount of problems. We only knew that we were repeating to ourselves: “Let it work first, but you can do optimization later”.
Over the past few years, the complexity of our project has grown tremendously. Fortunately, now we have come to the place where we can move on to the second part of our “mantra” - optimization.
Library solutions helped the project get to where it is today. But we are faced with a "library" problem. Solving it, we learned that by creating our own mechanism focused on our needs, we can make the code “easier” in terms of consuming system resources and save valuable time for our users.
Dear readers! Have you encountered problems similar to the one discussed in this article?
All Articles