Physica A 527 (2019) 121421

Contents lists available at ScienceDirect

Physica A

journal homepage: www.elsevier.com/locate/physa

Automatic identification of rapid eyemovement sleep basedon random forest using heart rate variabilityYitian Wang a,1, DaiYan Wang b,1, Lulu Zhang a, Cong Liu a, Jin Li c,Fengzhen Hou a,∗, Chung-Kang Peng d

a Key Laboratory of Biomedical Functional Materials, School of Science, China Pharmaceutical University, Nanjing 210009, Chinab College of Computer, National University of Defense Technology, Changsha 410073, Chinac College of Physics and Information Technology, Shaanxi Normal University, Xi’an 710062, Chinad Division of Interdisciplinary Medicine and Biotechnology, Department of Medicine, Beth Israel Deaconess Medical Center /Harvard Medical School, Boston, MA 02215, USA

h i g h l i g h t s

• Two strategies for the segmentation of HRV signals were proposed and compared.• The Surrounding strategy overwhelms the Truncating one in RF based REM classifier.• A symmetric surrounding window with 390 RR intervals is optimal for REM detection.

a r t i c l e i n f o

Article history:Received 5 November 2018Received in revised form 6 March 2019Available online 8 May 2019

Keywords:Heart rate variabilityRandom forestRapid eye movement sleepSegmentation

a b s t r a c t

There is broad evidence that the abnormality of rapid eye movement (REM) sleep may bean indicator of some diseases. The scientific identification of REM sleep thus plays a vitalrole in sleep medicine. Since the activity of autonomic nervous system (ANS) which canbe reflected in heart rate variability (HRV) was associated with sleep states, we aimedto develop an automatic REM detecting system based on HRV analysis and machinelearning. HRV signals which derived from 45 healthy participants were adopted and 69HRV features were extracted and fed into a random forest (RF) classifier. We compareddifferent strategies for the segmentation of HRV time series. The results showed a rel-ative good classification performance by segmenting the whole record into overlappingsections, suggesting that the Surrounding Strategy overwhelms the Truncating one inRF based REM identification. Moreover, the classification performance exhibited a non-monotonic trend along with the length of the symmetric surrounding window. Whenthere was 390 data points in such a window, we got the best performance to distinguishREM and non-REM sleeps with an accuracy of 0.84, a sensitivity of 0.80, a specificity of0.88, a positive predictive value of 0.90, a negative predictive value of 0.85 and a kappacoefficient of 0.68. Our study showed the promising application of HRV-based methodsin REM detecting, and furthermore, we threw light on the scientific segmentation ofHRV signals in sleep staging. As the Surrounding strategy proposed in this study makesit possible to produce enough learning samples, our results may bring more impetus onmachine learning-based algorithm, such as deep learning in this field.

© 2019 Elsevier B.V. All rights reserved.

∗ Correspondence to: The School of Science, China Pharmaceutical University, Nanjing 210009, China.E-mail address: houfz@cpu.edu.cn (F.Z. Hou).

1 Those authors contributed equally to this work.

https://doi.org/10.1016/j.physa.2019.1214210378-4371/© 2019 Elsevier B.V. All rights reserved.

2 Y. Wang, D. Wang, L. Zhang et al. / Physica A 527 (2019) 121421

1. Introduction

Sleep is an active and complex process of behaviour, and one-third of a person’s life is spent in sleeping. The qualityof sleep directly affects people’s physical health. Poor sleep can damage the mental and physical recovery process of ourbrain, affect cognitive and memory ability and lead to emotional deterioration and low work efficiency all day long [1].With the acceleration of modern life, sleep disorders, such as chronic insomnia and lethargy, are not only found in patientswith cognitive impairment and mental illness, but also in healthy population [2].

Generally speaking, sleep can be classified into two distinct states, namely, rapid eye movement (REM) sleep andnon-rapid eye movement (NREM) sleep which can be further divided into four stages according to the Rechtschaffenand Kales (R&K) rule [3]. In particular, some studies have been observed a relevance between the abnormality of REMsleep and diseases [4]. Peterson and Benca et al. found an increasing percentage of REM sleep and a decreasing REM sleeplatency in patients with primary depressive disorder [5]. Baglioni et al. demonstrated that patients suffering from primaryinsomnia were characterized with a decline in REM duration and their REM latency was shortened [6]. Moreover, it wasnoticed that apnea has become more frequent during REM sleep than in NREM sleep [7,8] while a long-term sleep apneasyndrome is an independent risk factor for cardiovascular disease such as hypertension, heart failure, arrhythmia andmyocardial infarction. Therefore, the scientific identification of REM state plays a vital role in sleep medicine and couldhelp the monitoring of personal health.

Nowadays, the gold standard for sleep evaluation and REM identification is on the manual scoring of polysomnography(PSG), which often records multiple biological signals synchronously including electroencephalogram (EEG), electromyo-gram (EMG), electrooculogram (EOG), electrocardiographic (ECG), respiratory-related signals, oxygen saturation andetc [9]. However, there exist several limitations in such a method. Firstly, the monitoring of PSG is usually carried out ina specific sleep laboratory or a medical centre which is expensive and hard to popularize. Secondly, the attachment ofnumerous sensors could make participants feel uncomfortable and disturb their nature sleep. Thirdly, the manual scoringis time-consuming and often results in considerable inter-scorer variability [10].

Enormous evidence shows that sleep state associates with the activity of autonomic nervous system (ANS) [11]. Heartrate variability (HRV), describing the variation in duration between adjacent heartbeats, is well recognized as a measure ofthe balance between the sympathetic and parasympathetic activities [12]. Usually, HRV can be derived from the intervalsbetween successive peaks of R-wave in ECG signals and called as RR intervals instead. Nowadays, HRV can be recordedby various wearable devices such as the sensorized T-shirt and sensorized bed [13]. As the non-invasive and convenientnature in HRV collecting, it could be a promising alternative to use HRV analysis in automatic REM detecting [14].

A wealth of HRV indices have been emerging in the quest for better measures of the change in parasympathetic andsympathetic activity since the 1980s [15]. Usually, HRV analysis is conducted either on short-term (about 5 min) or long-term (over 18 h) RR intervals. However, with respect to sleep medicine, sleep staging was generally performed on every30-s epoch following the R&K rules or the American Academy of Sleep Medicine (AASM) Manual. There exist two directionsby using very short-term HRV indices and by combining several 30-s epochs into a long segment for subsequent analysisto solve this issue. Although Anne–Louise Smith et al. have recently surveyed published methods of HRV analysis searchingfor indices that could be applied to very short-term HRV (30 beats, <60-s) analysis and validated the use of 69 HRV indicesin such case [16,17], these indices have never been used in the automatic REM detecting to our knowledge. Furthermore,Xiao et al. have proposed an overlapping and surrounding strategy to combine epochs into short-term (5 min) sections insleep staging [14,18]. However, the duration of the sections should be determined in a more rational manner as involvingepochs with different states may impose an opposite effect in sleep staging.

Drawing inspiration from the previous work, the current study was aimed to investigate whether REM sleep could becorrectly detected by the analysis of HRV signals and find out a suitable approach for the segmentation of sleeping HRV inautomatic REM identification. For this purpose, the whole-night HRV signals derived from 45 participants were obtainedfrom an open access database and to be analysed retrospectively. 69 HRV indices, whose usage in very short-term HRVanalysis were validated by Anne–Louise Smith et al. [16], were applied to extract features from the HRV segments andthen forwarded into a Random Forest (RF) based model for REM/NREM classification.

2. Methods

2.1. Participants

The experimental data was obtained from the online public database called Sleep Heart Rate and Stroke Volume DataBank and provided by the Lithuanian Medical University [19]. The normal sinus RR intervals derived from 45 healthyparticipants, including 28 males (aged from 17 to 61 years old) and 17 females (aged from 16 to 48 years old), wereincluded in the present study. All RR intervals were extracted from ECG signals with a sampling rate of 500 Hz and a periodof 6–8 h during the nocturnal sleep. In this study, only the RR intervals labelled by the data distributor as ‘‘stationary’’ weretaken into consideration as they were suitable for analysis according to the data distributor. Moreover, the RR intervalswere further staged according to the R&K rules by the data provider. The percentages of REM and NREM sleep were listedin Table 1, which shows the imbalance between the two states.

Y. Wang, D. Wang, L. Zhang et al. / Physica A 527 (2019) 121421 3

Table 1Distribution of epochs in NREM and REM sleep for each recording(mean ± standard deviation).Sleep state Percentage of epochs (%) Quantity of epochs

REM Sleep 17.39 ± 8.77 111.76 ± 3.55NREM Sleep 82.61 ± 9.54 531.02 ± 6.21

Fig. 1. Scheme of strategies for the segmentation of epoch series: (a) Truncating; (b) Symmetric Surrounding; (c) a generalized Surrounding window.N represents for an epoch labelled as NREM sleep and R for an epoch labelled as REM sleep.

2.2. Preprocessing

There was no intrinsic sampling frequency for such series as the RR time series was not obtained by sampling. Toapproximate the 30-s epoch in R&K rules, we redefined the term epoch with a specific sleep state (REM or NREM) byusing 30 successive RR intervals which have the same state. In this way, for each subject, all selected ‘‘stationary’’ datapoints in the HRV signal were used to construct epochs from the beginning of the RR intervals without overlapping, whichresults in a series of 30-points epochs labelling as REM or NREM sleep.

2.3. Signal segmentation

Two strategies for the segmentation of epoch series were investigated in the present study. The first one combinedthe successive epochs with the same staging label without overlapping. In this way, the obtained segment can be labelledas the same stage with all the epochs in it. This strategy was called as ‘‘Truncating’’ in the following part and Fig. 1(a)illustrated the construction of segments with 3 epochs (i.e., 90 RR intervals) via Truncating.

The other proposed strategy here was called as ‘‘Surrounding’’. The main idea was to make use of the epochssurrounding an epoch called target epoch. The scheme of symmetric Surrounding with 3 epochs in a segment wasillustrated in Fig. 1(b). As shown in Fig. 1(b), regardless of their labels, 3 successive epochs (a surrounding window) werecombined into a segment, within which the target epoch located in the centre. The label of the segment was set as thesame as that of the target epoch. Such a procedure repeated by sliding the surrounding window backwards one epocheach time until there were no enough epochs in the window. Compared with the Truncating strategy, the Surroundingapproach can result in much more segments for use. As shown in Fig. 1, for the same series, 3 segments were obtainedby using Truncating method while 10 segments by Surrounding.

Furthermore, a generalized Surrounding strategy was introduced. Given that there were m epochs preceded and nepochs followed a target epoch in a surrounding window, as illustrated in Fig. 1(c), the value of m and n was not necessaryto be equal. Once the value of m and n were settled, the generating and staging of the segments could be fulfilled similarlyto that in the symmetric Surrounding strategy.

4 Y. Wang, D. Wang, L. Zhang et al. / Physica A 527 (2019) 121421

Table 2Categories and abbreviations of 69 HRV indices.Categories Abbreviation Quantity

Time domain SDNN, SDSD, rMSSD, NN50, PNN50, PNN20, PNN30PNN6.25, Triang8, TINN8, rMSSDmc, SDNNmc

12

Frequency domain LombHF, LombLF, Lomb LF/HF, LombTotal, LombHFnu, LombLFnu 6Respiratory sinusarrhythmia (RSA)

RSA meanAD, RSA medianAD, RSA PeakValleyRSA Pvtone, RSA 5RR, RSA 5RRnu

6

Poincaré plot Accel, Decel, acv0x, Area, Assym(R/L), CTMdRR, pQa, pQb, pQc, pQd, r(RR),mean r(L1–6),r(1)–r(25), r max, r min, SD1, SD2, SD1nu, SD2nu, SDLD4, SDLD8, SDLD10,SDNN/rMSSD, SDNN/SDSD, SDratio

25

Heart rate characteristicsand other indices

CVdRR, CVRR, gradRR, grad5max, grad5min, kurt.dRR, kurtRR, magn(dRR),meanRR, medRR, normRR, norm dRR, PolVar20, RMS, sign(dRR), skew.dRR,skewRR, TACI(10), TACI(20), VarIndex

20

total 69

Fig. 2. The framework of the proposed REM sleep detecting model based on HRV analysis and RF algorithm.

2.4. Feature extraction

For each RR segment, 69 HRV indices were computed based on the survey of Smith [16]. In Table 2, the categories andabbreviations of those indices were summarized. Please see Ref. [16] for more detailed descriptions.

2.5. Random Forest (RF) based classification algorithm

HRV indices with different dynamic ranges may have influence on the classifier. Thus, for each participant, the values ofeach HRV index over all segments were normalized into the range of [−1, 1]. The normalized HRV indices were then usedto construct the feature vector of each segment and further fed into a RF based REM and NREM classifier, as illustratedin Fig. 2.

RF, which has enormous applications in pattern classifications [20], is based on the ensemble learning algorithm andinvolving a large amount of classification and regression trees (CARTs). At the heart of RF algorithm is Breiman’s ‘‘bagging’’idea [21] and the random selection of features. As shown in Fig. 2, a bootstrap sample of the training data is selected atthe beginning of RF algorithm. For each node of each CART, the selection of the features is random and subsets of featuresare searched over to find the optimal split. An unpruned CART is shaped by repeating this process at each node. At thesame time, the data out of the bootstrap sample, which is called out of bag (OOB) sample, can be used as a test set tomeasure the model performance and estimate the feature importance. A full forest is grown by repeating this procedureat each tree. In this study, in order to achieve a satisfying model performance, five primary parameters in the RF modelwere carefully tuned, including the number of trees in the forest (denoted as n_estimators), the max number of levels ineach tree (denoted as max_depth), the min number of data points placed in a node before the node was split (denoted asmin_samples_split), the min number of data points allowed in a leaf node (denoted as min_samples_leaf), and the maxnumber of features considered for splitting a node (denoted as max_features).

As there was much higher percentage of NREM sleep than of REM sleep in normal human sleep (as shown in Table 1),technique based on under-sampling was adopted in training and testing of the RF model [22,23]. Only a randomly selectedsubset of the NREM samples was used and this procedure of under-sampling was conducted on each participant.

Furthermore, a subject-independent scheme was adopted in the partition of the training and testing datasets, whichmeans that the training and testing data were taken from different participants. Meanwhile, leave-one-out cross-validation(LOO-CV) [24] was employed in the present study by training the model on 44 participants and testing it on the left onerepeatedly until all the participants have been tested.

Y. Wang, D. Wang, L. Zhang et al. / Physica A 527 (2019) 121421 5

Table 3The fine-tuned parameters of RF classifiers.Segmentation strategy Data points n_estimaors max_depth min_samples_split min_samples_leaf max_features

Truncating30 950 20 12 1 3090 610 12 10 1 44

150 130 10 4 1 52

Surrounding

90 890 28 4 1 12150 180 28 2 1 26210 520 22 3 1 58269 169 17 2 1 24330 125 22 10 1 25390 382 26 8 1 30450 145 24 7 1 28510 160 29 6 1 27

2.6. Performance evaluation

Performance of the model was evaluated on each participant in the testing dataset and then averaged. Accordingto formulas (1)–(5), multiple metrics were used to evaluate the performance of REM identification, including Accuracy,Sensitivity, Specificity, positive predictive value (PPV) and negative predictive value (NPV) [25].

Accuracy = (TP + TN)/(TP + FP + FN + TN) (1)

Sensitivity = TP/(TP + FN) (2)

Specificity = TN/(TN + FP) (3)

PPV = TP/(TP + FP) (4)

NPV = TN/(TN + FN) (5)

In formulas (1)–(5), TP, TN, FP and FN were represented for the numbers of items classified as true positive, truenegative, false positive and false negative respectively.

Cohen’s Kappa coefficient, a statistical measure of inter-rater agreement for categorical items, was also calculatedto assess the agreement between the proposed automatic and traditional manual score [26]. According to Landis andKoch [27], classifiers for which the Kappa value is greater than 0.80 are considered almost perfect, while for which theKappa value is between 0.61 and 0.80 are considered substantial.

3. Results

3.1. Comparison between truncating and surrounding strategies

We first investigated which segmentation strategy, Surrounding or Truncating, was more suitable for REM identificationwith HRV time series based on RF algorithm. Comparisons were conducted on two kinds of length of the segments, i.e., 90and 150 RR intervals, as there were insufficient segments for the model training if we include more epochs in a segmentin the case of Truncating. Noting that only the symmetric Surrounding was considered here. Table 3 gave the fine-tunedparameters of the used RF classifiers and Fig. 3 illustrated the performances of the trained classifiers. Compared with theTruncating approach (Fig. 3), the application of the Surrounding strategy led to higher performance of the classificationin both segmentation lengths.

In order to determine whether a symmetric Surrounding segmentation with longer length led to a higher performanceof REM identification, we considered different surrounding windows of 3, 5, 7, 9, 11, 13, 15, 17 epochs, resulted in datasetswith various length of RR segments, from 90 data points to 510 data points. The RF classifiers for different datasetswere fine-tuned, respectively, as shown in Table 3. The classification performances were illustrated in Fig. 4. As it canbe observed from in Fig. 4, the performance metrics altered non-monotonously with the data length. When there is 13epochs in a surrounding window (390 data points in each segment), the RF classifiers behave best to distinguish REMsleep from NREM sleep with an Accuracy of 0.84, a Kappa value of 0.68, a Sensitivity of 0.80, a NPV of 0.85, a PPV of 0.90and a Specificity of 0.88. The performance of classification showed a descending trend when the segments became longerthan 390 data points.

3.2. Results of the generalized surrounding strategy

Considering that epochs preceded or followed the target epoch might have different impacts on the sleep scoring ofthe target epoch, we investigated the classification performance when the generalized Surrounding strategy was used.The value of m and n, i.e., the number of epochs preceded and followed the target epoch, were set from zero to sixteen

6 Y. Wang, D. Wang, L. Zhang et al. / Physica A 527 (2019) 121421

Fig. 3. Performance of REM identification based on RF algorithm when different segmentation strategies were used.

Fig. 4. Performance of REM identification based on RF algorithm when different data length was used with symmetric Surrounding segmentation.Performances (mean ± standard error) were calculated on all the testing datasets using LOO-CV.

respectively, increasing one step at a time. Moreover, the maximal length of a surrounding window was limited to 510data points by controlling the sum of m and n less than seventeen. It is worth noting that in this way both the asymmetricand the symmetric surrounding window were included.

Fig. 5 illustrated the performance of RF classifiers when different combinations ofm and nwere used. It can be observedfrom Fig. 5 that the optimal combination of (m, n) appeared to be (6, 6), suggesting that a symmetric surrounding windowwas sufficient for REM detection with RR intervals. Moreover, the result obtained here was in consistent with that shown

Y. Wang, D. Wang, L. Zhang et al. / Physica A 527 (2019) 121421 7

Fig. 5. Performance of REM identification based on RF algorithm when different data length was used with generalized Surrounding segmentation.Performances (mean ± standard error) were calculated on all the testing datasets using LOO-CV.

in Fig. 4, indicating that 13 epochs in a symmetric surrounding window should be a suitable choice for the segmentationof RR intervals in REM identification.

3.3. Analysis of feature importance

In this study, 69 HRV features including time domain, frequency domain, respiratory sinus arrhythmia (RSA), Poincareand other indicators were extracted from the RR segments. We further investigated the feature importance of the RFmodel when the symmetric Surrounding segmentation (390 data points) was used by the gain which is the most relevantattribute to interpret the relative importance of each feature. The gains were all relative and hence all sum up to one.The features with higher gains play more important roles in the model [28]. As LOO-CV was used when training the RFclassifiers, for each feature, we used the averaged gain in the trained 45 RF models as its importance.

As shown in Fig. 6, features played different roles in the RF model for REM detection. LombHF, a frequency measure,had a much higher importance than other features. Measures from Poincare analysis, mean r(L1–6), r(RR), SDNN/SDSD,and SDratio were also quite important for the classifier. For features of Time domain, SDNNmc contributed a lot to theclassification task. However, features from RSA analysis contributed relatively less to the performance of classification.

4. Discussion

In this study, we investigated how to properly segment the sleep HRV signals for a better performance in automaticREM detecting based on random forest classifiers. Two segmentation strategy, which were called Truncating andSurrounding, were compared. The results demonstrated that Surrounding performs better than Truncating. Moreover, theclassification performance of Surrounding exhibited a non-monotonic trend along with the length of the HRV segment.When there was 390 RR points in a symmetric Surrounding window, we got the best performance to detect REM sleepwith an Accuracy of 0.84, a Sensitivity of 0.80, a Specificity of 0.88, a PPV of 0.90, a NPV of 0.85, and a Kappa coefficientof 0.68. As we mentioned in the background, in sleep medicine, sleep staging was generally performed on every 30-sepoch PSG signals. However, in general, 30-s duration was insufficient for the analysis of HRV as there was only about30 beats in such a duration. Although in the present study, we employed the HRV indices whose usage in RR time serieswith 30 data points were validated, the results showed that the classification performance got it minimum when eachsegment only contained 30 data points. A 30-s duration was thus demonstrated not the best choice for the automaticREM detecting based on HRV analysis and combining several 30-s epochs into a longer segment in an overlapping way(called Surrounding) was more promising. In this way, sleep staging could be done on each epoch with 30 RR data points(approximately to 30-s time duration), except for the first and last several epochs. Furthermore, it was estimated thatnearly 1000 HRV segments could be obtained for a participant who spends 8 h in sleep during the whole night. With thesufficient learning samples, more machine learning algorithms, such as deep learning [29], could be considered in thisfield. That might be an attracting work in future.

In the Surrounding approach, successive epochs with different stages could be merged into a longer segment whichwas labelled as same as the centred epoch. As sleep was a continuous process, when labelling an epoch, it should

8 Y. Wang, D. Wang, L. Zhang et al. / Physica A 527 (2019) 121421

Fig. 6. The feature importance for REM identification in RF model.

provide additional information and become more reliable to take its prior and posterior epochs into consideration [30].However, as distant epochs should be irrelevant to the sleep state of the current epoch, it was reasonably to speculatethat including those epochs would adversely influence the staging Accuracy. There was a question remaining that howmany epochs around should account for the accurately staging? In general, the longer the data being extended, the moreirrelevant information will be introduced. So there should be an optimal data length to provide the optimal classificationperformance. The answer might be underlie the observed non-monotonic trend of the classification performance alongwith the length of the Surrounding segment. Taking six epochs preceded and followed respectively, it should be a suitablechoice for the segmentation of HRV signals in the sleep staging.

In the present study, only the HRV indices validated by Smith et al. [17] were used in the construction of model features.As demonstrated in Fig. 6, the most important six features were LombHF, mean r(L1–6), r(RR), SDNNmc, SDNN/SDSDand SDratio. LombHF is a measure of the power of high frequency (HF, 0.15 – 0.4 Hz) band based on the Lomb powerspectral density estimates [31]. In healthy participants, HF band power has been widely regarded as a marker of vagalactivity [32] and was reported to be significantly lower in REM sleep than in NREM sleep [33,34]. SDNNmc, representingthe mean corrected standard deviation of NN intervals (SDNN), was also used by Antelmi et al. to investigate the influenceof age, gender, body mass index, and functional capacity on HRV in a cohort of subjects without heart disease [35]. Theother four HRV indexes, mean r(L1–6), r(RR), SDNN/SDSD and SDratio were all derived from the analysis of Poincaréplot, which is an intuitive and commonly used method to explore the complex non-linear behaviour of physiologicalsignals. One index, r(RR), the interbeat autocorrelation coefficient, has been reported to be able to track sympathovagalchanges during sleep [36] and indicated as a measure of sympathovagal balance in sleep [37]. The index SDNN/SDSDwas proposed to quantify the vagal modulation of the heart rate [38] while SDratio could be used as an indicator ofsympathetic activity [39]. In all, the current study of feature importance should convince the tight association betweenthe ANS activity and sleep.

Compared to previous studies using a single HRV signal to detect REM sleep (Table 4), a relative good performance wasachieved in the current study. In our study, more choices of the segmentation of HRV signals were tried and we foundthe epochs surrounding an epoch do have impacts on its staging. Such a scientific segmentation of HRV signals shoulddirectly contribute to the better performance of REM identification observed in this study. As the present study focusedon the comparison of different segmentation strategies, only the HRV indices suitable for very-short term HRV analysiswere adopted. Actually, more approaches, such as nonlinear dynamics analysis, could be applicable in the case of HRV

Y. Wang, D. Wang, L. Zhang et al. / Physica A 527 (2019) 121421 9

Table 4Results of existing HRV automatic sleep staging model.First author Data source Model Accuracy Kappa Specificity Sensitivity

Mendez MO [11] 24 healthypeople/1-epoch

Hidden MarkovModels

79.3% – 85.1% 70.2%

Ebrahimi F [30] 30 people/95710-epoch and9570 1-epoch

linear discriminant/quadraticdiscriminant

(71.19%,1-epoch;80.67%,10-epoch)

– – –

Aktaruzzaman M [41] 20 patients/2,6,10-epoch

feedforwardneural network

(71.19%,2-epoch;80.50%,6-epoch;83.78%,10-epoch)

(0.43,2-epoch;0.61,6-epoch;0.68,10-epoch)

(69.38%,2-epoch;81.16%,6-epoch;85.15%,10-epoch)

(74.08%,2-epoch;79.40%,6-epoch;82.41%,10-epoch)

J.M. Kortelainen [42] 18 recordingsfrom healthysubjects

Hidden MarkovModels

79% 0.43 – –

The proposed method 45 healthyparticipants

Random Forest 84% 0.68 88.0% 80.0%

segments with 390 data points. As those approaches have been demonstrated to be sensitive in capturing the alterationsin ANS [40], their involvement might greatly promoted the performance in automatic REM identification.

In conclusion, scientific segmentation of HRV signals plays an important role in the automatic detecting of REM sleepbased on HRV analysis. Surrounding strategy is proposed as a promising segmentation as it has the advantages of keepingenough segments and utilizing additional information from successive epochs. Our results may help break the boundaryof data length and sample size in HRV analysis based REM sleep identification and further inspire a broader work, such asdeep learning based classifiers in this area. However, there were some limitations in the current work. In the comparisonof the Truncating and Surrounding, we only adopted two kinds of length, i.e., 90 data points and 150 data points due tothe restriction of the data amount with the Truncating. Moreover, only healthy adults were included in the investigation.Further studies were encouraged to examine the application of the proposed method in subjects suffering from sleepdisorders.

Declaration of competing interest

None of the authors have potential conflicts of interest to be disclosed.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 61401518, 31671006 and61771251), Jiangsu Provincial Key R & D Program (Social Development), China (Grant No. BE2015690 and BE2016773)and the Double First-Class University project, China.

References

[1] D.J. Buysse, Sleep health: can we define it? Does it matter? Sleep 37 (2014) 9–17.[2] F. Auld, E.L. Maschauer, I. Morrison, D.J. Skene, R.L. Riha, Evidence for the efficacy of melatonin in the treatment of primary adult sleep disorders,

Sleep Med. Rev. 34 (2017) 10–22.[3] D. Moser, P. Anderer, G. Gruber, S. Parapatics, E. Loretz, M. Boeck, G. Kloesch, E. Heller, A. Schmidt, H. Danker-Hopfe, B. Saletu, J. Zeitlhofer, G.

Dorffner, Sleep classification according to AASM and Rechtschaffen & Kales: Effects on sleep scoring parameters, Sleep 32 (2009) 139–149.[4] H. Yoon, S.H. Hwang, J.W. Choi, Y.J. Lee, D.U. Jeong, K.S. Park, REM sleep estimation based on autonomic dynamics using R-R intervals, Physiol.

Meas. 38 (2017) 631–651.[5] E.C. Landsness, M.R. Goldstein, M.J. Peterson, G. Tononi, R.M. Benca, Antidepressant effects of selective slow wave sleep deprivation in major

depression: A high-density EEG investigation, J. Psychiatr. Res. 45 (2011) 1019–1026.[6] C. Baglioni, W. Regen, A. Teghen, K. Spiegelhalder, B. Feige, C. Nissen, D. Riemann, Sleep changes in the disorder of insomnia: a meta-analysis

of polysomnographic studies, Sleep Med. Rev. 18 (2014) 195–213.[7] L.J. Findley, S.C. Wilhoit, P.M. Suratt, Apnea duration and hypoxemia during REM sleep in patients with obstructive sleep apnea, Chest 87

(1985) 432–436.[8] S. Okabe, W. Hida, Y. Kikuchi, O. Taguchi, T. Takishima, K. Shirato, Upper airway muscle activity during REM and non-REM sleep of patients

with obstructive apnea, Chest 106 (1994) 767–773.[9] C.M. Held, J.E. Heiss, P.A. Estévez, C.A. Perez, M. Garrido, C. Algarín, P. Peirano, Extracting fuzzy rules from polysomnographic recordings for

infant sleep classification, IEEE Trans. Biomed. Eng. 53 (2006) 1954–1962.[10] M. Younes, The case for using digital EEG analysis in clinical sleep medicine, Sleep Sci. Pract. 1 (2017) 2.[11] M.O. Mendez, M. Matteucci, V. Castronovo, L. Ferini-Strambi, S. Cerutti, A. Bianchi, Sleep staging from heart rate variability: time-varying

spectral features and hidden markov models, Biomed. Eng. Technol. 3 (2010) 246–263.[12] J.S. Floras, Clinical aspects of sympathetic activation and parasympathetic withdrawal in heart failure, J. Am. Coll. Cardiol. 22 (1993) A72–A84.[13] A.M. Bianchi, M.O. Mendez, S. Cerutti, Processing of signals recorded through smart devices: sleep-quality assessment, IEEE Trans. Inf. Technol.

Biomed. 14 (2010) 741–747.[14] M. Malik, J.T. Bigger, A.J. Camm, R.E. Kleiger, A. Malliani, A.J. Moss, P. Schwartz, Heart rate variability: Standards of measurement, physiological

interpretation, and clinical use, Eur. Heart J. 17 (1996) 354–381.

10 Y. Wang, D. Wang, L. Zhang et al. / Physica A 527 (2019) 121421

[15] Y. İşler, M. Kuntalp, Combining classical HRV indices with wavelet entropy measures improves to performance in diagnosing congestive heartfailure, Comput. Biol. Med. 37 (2007) 1502–1510.

[16] A.L. Smith, H. Owen, K.J. Reynolds, Heart rate variability indices for very short-term (30 beat) analysis. Part 1: survey and toolbox, J. Clin.Monit. Comput. 27 (2013) 569–576.

[17] A.L. Smith, H. Owen, K.J. Reynolds, Heart rate variability indices for very short-term (30 beat) analysis. Part 2: validation, J. Clin. Monit. Comput.27 (2013) 577–585.

[18] M. Xiao, H. Yan, J. Song, Y. Yang, X. Yang, Sleep stages classification based on heart rate variability and random forest, Biomed. Signal Process.Control 8 (2013) 624–633.

[19] http://www.pri.kmu.lt/datbank/archiv.php, Sleep Heart Rate and Stroke Volume Data Bank, Institute of Psychophysiology & Rehabilitation,Kaunas University of Medicine.

[20] E.E. Tripoliti, D.I. Fotiadis, G. Manis, Modifications of the construction and voting mechanisms of the Random Forests Algorithm, Data Knowl.Eng. 87 (2013) 41–65.

[21] L. Breiman, Random forests, Mach. Learn. 45 (2001) 5–32.[22] S.-J. Yen, Y.-S. Lee, Cluster-based under-sampling approaches for imbalanced data distributions, Expert Syst. Appl. 36 (2009) 5718–5727.[23] D. Rasch, F. Teuscher, V. Guiard, How robust are tests for two independent samples? J. Statist. Plann. Inference 137 (2007) 2706–2720.[24] M. Dong, N. Wang, Adaptive network-based fuzzy inference system with leave-one-out cross-validation approach for prediction of surface

roughness, Appl. Math. Model. 35 (2011) 1024–1035.[25] C. Liu, J. Oster, E. Reinertsen, Q. Li, L. Zhao, S. Nemati, G.D. Clifford, A comparison of entropy approaches for AF discrimination, Physiol. Meas.

39 (2018) 074002.[26] N.C. Smeeton, Early history of the kappa statistic, Biometrics 41 (1985) 795.[27] J.R. Landis, G.G. Koch, The measurement of observer agreement for categorical data, Biometrics 33 (1977) 159–174.[28] R. Genuer, J.-M. Poggi, C. Tuleau-Malot, Variable selection using random forests, Pattern Recognit. Lett. 31 (2010) 2225–2236.[29] Q.V. Le, J. Ngiam, A. Coates, A. Lahiri, B. Prochnow, A.Y. Ng, On optimization methods for deep learning, in: Proceedings of the 28th International

Conference on International Conference on Machine Learning, Omnipress, 2011, pp. 265–272.[30] F. Ebrahimi, S.K. Setarehdan, J. Ayala-Moyeda, H. Nazeran, Automatic sleep staging using empirical mode decomposition, discrete wavelet

transform, time-domain, and nonlinear dynamics features of heart rate variability signals, Comput. Methods Programs Biomed. 112 (2013)47–57.

[31] G.B. Moody, Spectral analysis of heart rate without resampling, in: Proceedings of Computers in Cardiology Conference, 1993, pp. 715–718.[32] A. Malliani, F. Lombardi, M. Pagani, Power spectrum analysis of heart rate variability: a tool to explore neural regulatory mechanisms, Br. Heart

J. 71 (1994) 1–2.[33] S. Elsenbruch, M. Harnish, W.C. Orr, S. Elsenbruch, M.J. Harnish, W.C. Orr, Heart rate variability during waking and sleep in healthy males and

females, Sleep 22 (1999) 1067–1071.[34] J. P.V. Busek, J. Opavsky, J. Salinger, S. Nevsimalova, Spectral analysis of heart rate variability in sleep, Physiol. Res. 54 (2005) 369–376.[35] I. Antelmi, R.S. De Paula, A.R. Shinzato, C.A. Peres, A.J. Mansur, C.J. Grupi, Influence of age gender body mass index and functional capacity on

heart rate variability in a cohort of subjects without heart disease, Am. J. Cardiol. 93 (2004) 381–385.[36] H.J. Burgess, P.D. Penev, R. Schneider, C.E. Van, Estimating cardiac autonomic activity during sleep: impedance cardiography, spectral analysis,

and Poincaré plots, Clin. Neurophysiol. Off. J. Int. Fed. Clin. Neurophysiol. 115 (2004) 19–28.[37] H. Otzenberger, C. Gronfier, C. Simon, A. Charloux, J. Ehrhart, F. Piquard, G. Brandenberger, Dynamic heart rate variability: a tool for exploring

sympathovagal balance continuously during sleep in men, Am. J. Physiol. 275 (1998) H946.[38] M. Hirose, H. Imai, M. Ohmori, Y. Matsumoto, F. Amaya, T. Hosokawa, Y. Tanaka, Heart rate variability during chemical thoracic sympathectomy,

Anesthesiol. 89 (1998) 666–670.[39] L. Bergfeldt, Y. Haga, Power spectral and Poincaré plot characteristics in sinus node dysfunction, J. Appl. Physiol. 94 (2003) 2217–2224.[40] B.K. Koichubekov, M.A. Sorokina, Y.M. Laryushina, L.G. Turgunova, I.V. Korshukov, Nonlinear analyses of heart rate variability in hypertension,

Ann. Cardiol. Angeiol. (Paris) 67 (2018) 174–179.[41] M. Aktaruzzaman, M. Migliorini, M. Tenhunen, S.L. Himanen, A.M. Bianchi, R. Sassi, The addition of entropy-based regularity parameters improves

sleep stage classification based on heart rate variability, Med. Biol. Eng. Comput. 53 (2015) 415–425.[42] J.M. Kortelainen, M.O. Mendez, A.M. Bianchi, M. Matteucci, S. Cerutti, Sleep staging based on signals acquired through bed sensor, IEEE Trans.

Inf. Technol. Biomed. 14 (2010) 776–785.