How Much Can We Demand of Consumer Connected Health?

A few years back, when I first became fascinated by self-tracking, I committed to use and understand all of the trackers on the market at the time.  It would be hard to make that kind of commitment today as the field has widened dramatically. But in 2009, it was much more confined.

A few years back, when I first became fascinated by self-tracking, I committed to use and understand all of the trackers on the market at the time.  It would be hard to make that kind of commitment today as the field has widened dramatically. But in 2009, it was much more confined. I had a BodyMedia armband, a Withings scale and a FitLinxx ActiPed pedometer.  One thing I learned early on was that their readings don’t always hang together well.

For instance, we know that much of daily weight fluctuation is about salt intake and fluid retention, and the body composition function on the Withings scale has trouble with that.  Here are two real readings from me on recent successive days:  weight 186.6, fat mass 25.59, then 184.6 and 24.03.  Logic would dictate that if I lost 2 lbs in one day it would have to be water weight and my body fat composition should go up not down.

I had the same early consternation with the BodyMedia armband and the ActiPed.  FitLinxx is on the record as saying their technology measures steps most accurately. BodyMedia uses a proprietary algorithm that includes a number of inputs.  My personal data recorded on these two devices were routinely different.  When Fitbit came out, I say that device gave a different number still.

It gets even more peculiar when you use MyFitnessPal to count calories in and try to map the calories/calories out (based on an activity monitor) with the weight readings from Withings.

You get the picture.  Quantified selfers, like me, are asking too much of these simple technologies.

What I learned over time is that these tracking devices are directionally correct.  They are most helpful when the data from each sensor is viewed longitudinally, in its own context.  For a time I wore both a Jawbone UP and a Fitbit and I found that though the step counts registered were quite different on each activity tracker, the percentage variability of each from day to day was pretty consistent. In my experience, they are internally consistent.

This is all background to comment on an article by Nick Bilton that was featured in last Monday’s New York Times, entitled For Fitness Bands, Slick Marketing but Suspect Results.  In this article, Bilton makes the case that activity trackers are really inaccurate. He mentions personal experiences where he and a friend were both wearing the same monitor, at the same time doing the same thing, and got wildly different results. He also reported another incident when he was using his smartphone’s GPS tracker to measure his activity and, though he was sedentary, it showed him to have been active.

First off, I have not reproduced experiences like either of these.  However, it’s worth noting that the technologies are nowhere near perfect.  The standard technology to measure activity is an accelerometer and it is only as good as the software algorithm guiding it.  Almost all activity trackers know when you are driving a car or riding a bike and don’t count that.  However, bouncing your leg up and down while at rest or swinging your arm back and forth can yield false positive results.

Sleep trackers are even worse.  Since they track sleep indirectly by measuring activity, they are all over the map.

Expecting these consumer devices to have scientific accuracy is unrealistic.  Expecting them to help you keep your activity level top of mind and measured in context from day to day is realistic and in most cases helpful.

Our experience using activity monitoring in programs at the Center for Connected Health has demonstrated this.  The objective feedback is helpful in setting a tone for the individual and as a data stream to guide motivational tools such as automated coaching.

For example, one of our physicians at Mass General Hospital gave nine of her patients with diabetes a Fitbit Zip  and met with them, as a group, weekly.  After six weeks, nearly 80% of these 60 plus year old patients increased their amount of weekly exercise, in particular walking, and 22% lost a significant amount of weight, with one patient losing over 10 lbs.  More than half of these patients are still using their Fitbits, eight months after the program was completed, and all reported that they felt more confident in their ability to care for themselves, and everyone was moving more.

We never expected consumer devices to be as accurate as laboratory calculations of activity level or calories burned.

One of the reasons we launched Wellocracy was to help consumers understand nuances like this.  So far it’s been a great success.

Does it matter if every step is counted, every calorie recorded, or is it more important that these personal health technologies are making us more aware, more motivated and more likely to make healthier lifestyle choices?