Analysing SleepTracker data for R.E.M. phases

I'm collecting sleep data from my Sleeptracker watch for above 3 months. It's a good time to check what is interesting in there. My goal is to check pattern of my sleep R.E.M. phases.

First, I get my sleep data in csv format stored on www.sleeptracker.net site. Then using simple python script I've converted all night/time data into relative values. Now I have all SleepTracker movement occurrence time saved in "hours after going sleep".

After looking at initial data I was a little bit disappointed. I have not seen clean gaps for non R.E.M phases. Picture was so cluttered. There were many possible reasons. I've choosen one - sometimes I go sleep at different time, but sleep patterns may be anchored to usual hours.

I decided to make more preprocessing to move out "outstanding" data. I removed data instances with non standard "going to bed" times, movements it the first hour of sleep and some made some minor tweakings. When I viewed data again the picture was much clearer.



I rerun data clustering software using standard kmeans method and get some results:

Cluster 1
Mean/Mode: 0.8458
Std Devs: 0.2783

Cluster 2
Mean/Mode: 1.5433
Std Devs: 0.2069

Cluster 3
Mean/Mode: 2.6117
Std Devs: 0.2857

Cluster 4
Mean/Mode: 3.6685
Std Devs: 0.236

Cluster 5
Mean/Mode: 4.8086
Std Devs: 0.2848

Cluster 6
Mean/Mode: 5.9405
Std Devs: 0.4276


That means that probably REM phases occurs about 0.85, 1.5, 2.6, 3.6, 4.8 and 5.9 hours after falling asleep. Common value found in literature is about 1.5 hour between R.E.M. phases.
Bigger standard deviation in later vs earlier phases reflects longer R.E.M. phases at each cycle, what is also found in sleep research. First cycle must be treated in special way - the sleep pattern is different sudden after falling asleep.
The last one can be also less reliable because of standard waking time after about 6 hours.

I decided to pick R.E.M. time values by my self using "smoothed" histogram of sleep data. Local modalities are clearly visible on picture.


Now middle of phase values seems like: 0.8, 1.6 , 2.3 , 2.8, 3.4 , 4 , 4.9. Comparing these results with those from automatic clustering method it seems that "k means" method joined 2 close groups together.
I assume that manually picked results are better.

Time for conclusion.

To get more reliable results I need more data and regular sleep :) . SleepTracker is quite effective by using simple accelerometer movement detection method. Unfortunately many movement events aren't recorded (for example the subtle ones or when the arm is blocked under the pillow). I will try to catch more good data and check results again.