Study: fitness trackers can't cope with energy consumption calculations

Today, there are several dozen types of fitness trackers on the wearable device market that are used by millions of people. Devices help monitor physical activity and monitor health, and some users come with information from fitness trackers to a doctor. But how much correct information do these devices provide?



According to data from a new study conducted by scientists at the Stanford University School of Medicine, wearable fitness trackers like the Apple Watch and Fitbit do a good job with pulse tracking, but the accuracy in measuring calories is far from ideal. Scientists decided to evaluate the effectiveness of wearable trackers when they found that in any scientific journal there is no data on the operation of these devices.



“At any time when we receive patient data through the device, we have questions about the accuracy of this information,” notes Ewan Ashley, an associate professor at the Stanford School of Medicine specializing in cardiovascular medicine, and part-time lead author.



There is no close interest on the part of medical regulators to these devices, since they are aimed at optimizing health, and not at identifying diseases. Therefore, they do not adhere to the same standards as specialized medical devices. But despite this, many patients use and will continue to use fitness trackers on an ongoing basis.



The Ashley team tested seven devices — Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn, and Samsung Gear S2 — on a group of 60 volunteers. Six of them measured heart rate with an error rate of less than 5%. Apple Watch made a mistake by 2%, and Samsung Gear S2 by 6.8%. Some devices were more accurate than others, and factors such as skin color and body mass index influenced measurements.



At the same time, none of the seven devices measured the exact cost of energy. The most accurate of them was the Fitbit Surge bracelet, which was 27% wrong, and the most inaccurate was PulseOn (92.6%). People make decisions based on data from consumer devices, but they do not meet the same standards as medical devices. Therefore, doctors can not figure out what to do with reports on heart rate and other data that patients give to fitness trackers.



Researchers claim that to solve these problems, manufacturers can very quickly test the accuracy of devices for operation, but it is difficult for consumers to understand how accurate this information or the process that the company used to test trackers. Therefore, Ashley and his colleagues decided to independently evaluate the activity of trackers that met such criteria as measuring heart rate and burning calories, as well as their commercial availability. Among the volunteers were 31 women and 29 men of different ages, races, height and weight. They wore seven devices while walking, exercising on treadmills or exercise bikes.



Scientists did not compare gadgets with each other: the accuracy of each fitness tracker was compared with the indicators of two control instruments. The heart rate of each participant in the experiment was measured by a medical electrocardiograph, and the metabolic rate was assessed using an inhale oxygen and carbon dioxide measuring device, which is considered optimal for calculating the energy expenditure. The Ashley team considered that for non-medical devices the error should not exceed 10%.



Heart rate measurements exceeded the expectations of researchers. However, the same cannot be said about energy consumption data. According to Ashley, users can largely rely on heart rate measurements by a fitness tracker.



The researchers note that each device uses its own algorithm for calculating energy costs, so now they can not establish the exact cause of this result. Probably, algorithms make assumptions that are not well suited to people. All researchers can do is see how the devices work according to clinical standards. They are also sure that it is rather difficult to prepare an algorithm that will be accurate for a wide range of people, because energy costs depend on the level of physical activity, height and weight of a person and many other factors. Heart rate indicators are measured directly, while energy costs are measured indirectly by several indications.



The Ashley team decided to make their assessments of wearable devices open to the research community, so I developed a website where information on fitness trackers data is available. Scientists also suggested that users download data related to device performance. At the next stage of the study, it is planned to evaluate the operation of the devices in natural conditions: volunteers will wear them throughout the day, and not just in training.



doi: 10.3390 / jpm7020003