Many of the people I see complain of poor quality sleep, and a number of them have tired using various sleep apps and wearable devices to try to get figure out why they are sleeping poorly and what they can do about it.
One of my techie colleagues says she thinks that this is a good thing and praises one app, which is the most popular sleep app in the iPhone App Store, in particular…
“About a third of my clients are using Sleep Cycle. I like it because it marks the 4 sleep stages and doesn’t smooth the splines on the waves, so you can see roughness if sleep is fragmented. The “sleep quality” measure is bogus because it includes time in bed in its nebulous calculation, but the rest of the app I like a lot. A friend turned me onto it a couple years ago. I’d had a sleep problem for several years and when I saw it on the little monitor and guessed at the etiology I was able to correct the issue in 2 days. I’ve tweaked regimens, ordered sleep studies, and caught a lot of late night drinking with this thing.”
This sounded very positive. However many of my patients report different experiences with sleep apps. Sometimes when I look at the graphs on their smart phones it’s hard to know what the data shows and how reliable it is.
In this brief review I will try to summarize the existing data about smart phone sleep apps as well as wearable devices for monitoring sleep and activity.
Sleep Apps
The boom in smart phone apps designed to improve and monitor sleep is phenomenal and it’s impossible to keep track of all of them.
Some apps detect movement on a bed. These apps use the movements registered by the phone, tucked under a pillow, to estimate whether the user is in a state of wakefulness or of light or deep sleep. They offer varying degrees of analysis of sleep patterns, and some propose to wake the user at a moment that is most opportune according to the app’s algorithms. Other apps claim to track sleep parameters by measuring the extent to which the user is snoring (Stippig et al., 2014), or the levels of sleep talking, or (as with my friends app, Sleep Cycle) claim to use a “patented, proprietary technology” which can distinguish the sounds of movement during sleep from other sounds, and can even tell whether it is you moving in the bed or your partner… Fascinating and exciting stuff…
It is a lot easier to keep track of the studies that have been published in the research literature comparing sleep apps with the “gold standard” for measuring sleep duration and quality: polysomnography, than the various apps being brought to market, because there is hardly any published data on these apps.
“In the only study comparing a smartphone app to PSG in adults, Bhat et al. evaluated the Sleep Time™ app that provides users with a graph detailing wakefulness and light and deep sleep and, in addition, claims to help users wake up only during light sleep.
In this study, the authors compared the PSG sleep data from 20 healthy adult subjects with no previously diagnosed sleep disorders with data obtained from the app. There was poor correlation between the app and PSG in terms of sleep efficiency, light sleep, and deep sleep. In addition, there was no correlation between app and PSG sleep latency. The app was poor in terms of detecting wakefulness. Finally, there was no evidence that the app consistently awakened subjects only during light sleep (p = 0.159).”
That’s it. Only one published study on the accuracy of the data in smart phone apps and that study found that the data was very inaccurate.
Wearable Devices
The Technology
Wearable devices rely on actigraphy (movement monitoring) to try to determine a person’s pattern of wakefulness, activity and sleep. The fundamental idea is a sound one which is actually used in clinical settings.
“Actigraphy, a portable wrist-worn sleep monitoring device, is used in clinical sleep medicine for assessing certain sleep disorders, such as circadian rhythm sleep–wake disorders, and for characterizing day-to-day patterns or sleep disturbances in insomnia []. It is also used to assess response to treatment in insomniacs and as an adjunct source of information in patients who are unable to provide a clear history ].
Wrist actigraphy is based on the principle that physical movements are increased during wakefulness and reduced during sleep []. It has been found to have a reasonable degree of agreement with PSG, with reported agreement rates of 78.8–99.7% for sleep and 48.5–79.8% for wake []. Actigraphy has been shown to be sensitive to changes in sleep patterns in response to pharmacologic and nonpharmacologic interventions []. However, its validity in special populations such as the elderly, in subjects with poor sleep quality, or in those with major health problems is not well established [.”
The Studies
There are more studies looking at the accuracy of wearable devices and the findings from the studies are more reassuring.
The table below lists details of these summaries. First, though, a word of caution, these studies used somewhat different methodologies and so it may not be possible to directly compare the outcomes.
The studies looked at a few devices – two Fitbit devices and the Jawbone UP.
One study of the current Fitbit model, when compared with the gold standard of polysomnography (PSG), found that the device had a sensitivity of 0.87 and specificity of 0.52 in the ‘normal’ mode and 0.70 and 0.79, respectively, in the ‘sensitive’ mode.
This means that in the normal mode, 90% of the time when someone was asleep it categorized that time period as sleep, but only about half of the time that the device said someone was asleep were they actually asleep. The device overestimated sleep time significantly in the normal mode. In the sensitive mode the device overestimated sleep time less.
The Fitbit Ultra device was also compared to a stand alone actigraphy device in that study and was found to similarly overestimate sleep time when compared with a professional actigraphy device.
In summary, Fitbit was fairly good at detecting sleep but poor at detecting wakefulness. Furthermore, results varied according to age group and sleep apnea status. The older the patient and the more difficulty they had with insomnia, the poorer the correlation with either PSG or actigraphy.
Jawbone™ UP is an activity tracker that also claims to track sleep utilizing bioimpedance sensors.
Three studies looked at the accuracy of the Jawbone.
- De Zambotti et al. compared the accuracy of Jawbone in measuring nighttime sleep to PSG in a sample of 65 healthy adolescents and young adults (ages 12–22 years) with no prior sleep problems.
- De Zambotti et al. also studied the same device, Jawbone, in a sample of 28 adult women (mean age 50.1 ± 3.9 years).
- Toon et al. compared the Jawbone UP device and MotionX 24/7, a smartphone-based app against PSG and actigraphy in a pediatric sample with suspected sleep-disordered breathing (N = 78; mean age 8.4 ± 4.0 years, range 3–18 years).
Across all of these studies, the Jawbone was relatively poor at determining wakefulness, but did a better job at identifying sleep (sensitivity for sleep was high but specificity was not great). Light and deep sleep measured by Jawbone did not correspond with light (stages N1 and N2) and deep sleep (stages N3 and REM) measured by PSG.
However, overall the studies suggested that the Jawbone may be slightly more accurate than Fitbit and Fitbit Ultra in determining sleep parameters and both devices were much more accurate than smartphone apps.
Apps for Sleep Apnea Detection
A final role for smartphone apps is as an inexpensive way of identifying possible sleep apnea. There seems to be better data suggesting that sleep apnea detecting smartphone apps are reasonably accurate than the data looking at general sleep apps.
Nandakumar et al. evaluated a smartphone-based application, ApneaApp™, designed to detect sleep-related respiratory events.
The app uses the smartphone’s microphone to emit an inaudible wave, which functions similar to a sonar system to detect amplitude changes during breathing. It uses a sophisticated algorithm to detect and calculate hypopneas (partial sleep apnea episodes) and obstructive and central apneas. The app measures sleep time by identifying non-breathing body movements and subtracting them from the total recording time. In this study, where the smartphones were used alongside PSG in a sleep laboratory setting, the events as measured by the app showed good correlation with the total number of events recorded on PSG. The ApneaApp correctly classified 32 of 37 subjects with regard to their sleep apnea severity status, and correctly identified those requiring treatment.
The ApneaApp is designed for Android phones with at least two microphones. This includes Samsung Galaxy and HTC One.
Unfortunately, the app is not currently available. The website says that it has been submitted to the FDA for approval –
We can not currently release the app prior to getting the FDA approvals. Please email us at apnea@cs.washington.edu to be on our waiting list so that we can notify you once we get the approval.
There are a number of apps currently available that claim to measure and track snoring. A proof of concept study showed that a smartphone strapped to the anterior chest wall during PSG can detect snoring with reasonable accuracy.
One study of seven commercially available snoring apps suggested that they were not accurate enough to replace current diagnostic standards and worked only in soundproof environments. In real-life environments with background noise, their ability to detect snoring deteriorated considerably.
Another study was more positive. Camacho, et al searched Apple iTunes app store for snoring apps that allow recording and playback. Snoring apps were downloaded, evaluated and rated independently by four authors. Two patients underwent polysomnography, and the data were compared with simultaneous snoring app recordings, and one patient used the snoring app at home.
In this study, the Quit Snoring app received the highest overall rating. When this app’s recordings were compared with in-laboratory polysomnography data, app snoring sensitivities ranged from 64 to 96 per cent, and snoring positive predictive values ranged from 93 to 96 per cent.
REFERENCES
Camacho M, Robertson M, Abdullatif J, Certal V, Kram YA, Ruoff CM, Brietzke SE, Capasso R. Smartphone apps for snoring. J Laryngol Otol. 2015 Oct;129(10):974-9. doi: 10.1017/S0022215115001978. Epub 2015 Sep 3. PubMed PMID: 26333720.
De Zambotti M, Baker FC, Colrain IM. Validation of sleep-tracking technology compared with polysomnography in adolescents. Sleep. 2015;38(9):1461–1468
De Zambotti M, Claudatos S, Inkelis S, et al. Evaluation of a consumer fitness-tracking device to assess sleep in adults. Chronobiol Int. 2015;32(7):1024–1028
Grigsby-Toussaint DS, Shin JC, Reeves DM, Beattie A, Auguste E, Jean-Louis G. Sleep apps and behavioral constructs: A content analysis. Prev Med Rep. 2017 Feb 21;6:126-129. doi: 10.1016/j.pmedr.2017.02.018. eCollection 2017 Jun. PubMed PMID: 28316907; PubMed Central PMCID: PMC5350571.
S. Bhat, A. Ferraris, D. Gupta, et al. Is there a clinical role for smartphone sleep apps? Comparison of sleep cycle detection by a smartphone application to polysomnography. J. Clin. Sleep Med., 11 (7) (2015), pp. 709–715 http://doi.org.ucsf.idm.oclc.org/10.5664/jcsm.4840
Kolla BP, Mansukhani S, Mansukhani MP. Consumer sleep tracking devices: a review of mechanisms, validity and utility. Expert Rev Med Devices. 2016 May;13(5):497-506. doi: 10.1586/17434440.2016.1171708. Epub 2016 Apr 18. Review. PubMed PMID: 27043070.
M.T. Bianchi. Consumer sleep apps: when it comes to the big picture, it’s all about the frame. J. Clin. Sleep Med., 11 (7) (2015), p. 695
Nakano H, Hirayama K, Sadamitsu Y, et al. Monitoring sound to quantify snoring and sleep apnea severity using a smartphone: proof of concept. J Clin Sleep Med. 2014;10(1):73–78
P.T. Ko, J.A. Kientz, E.K. Choe, M. Kay, C.A. Landis, N.F. Watson. Consumer sleep technologies: a review of the landscape. J. Clin. Sleep Med., 11 (2015), pp. 1455–1461
Stippig A, Hübers U, Emerich M. Apps in sleep medicine. Sleep Breath. 2015;19(1):411–417.
Toon E, Davey MJ, Hollis SL, et al. Comparison of commercial wrist-based and smartphone accelerometers, actigraphy, and PSG in a clinical cohort of children and adolescents. J Clin Sleep Med. 2016;12(3):343–350.
Van den Bulck J. Sleep apps and the quantified self: blessing or curse? J Sleep Res. 2015 Apr;24(2):121-3. doi: 10.1111/jsr.12270. Epub 2015 Jan 5. PubMed PMID: 25558955.