Fujitsu Develops wearable sensors based human activity recognition system for elderly healthcare

Fujitsu R&D Center Co., Ltd. (FRDC) announced the development of an advanced human activity recognition (HAR) technology using wearable sensors, and this technology, augmented by indoor positioning, can provide healthcare service for elderly with higher energy efficiency without loss of recognition accuracy. This technology extracts pairwise correlation features in hybrid sliding window, and uses location information by indoor positioning system to enhance the recognition accuracy. To reduce the computational complexity, two proposals, i.e., simplified random forest (RF) and multi-layer classifiers, respectively, are used to recognize 10 types of daily activities. This technology could reduce energy consumption by 74.9% while maintaining 93.01% of recognition accuracy, which means the wearable device can have 4 times of battery life. Furthermore, the falling detection accuracy can reach up to 99%.

Details of this technology are also presented at the 2018 IEEE Wireless Communications and Networking Conference, opening on 15-18 April 2018 in Barcelona, Spain, and the 5th IEEE International Symposium on Inertial Sensors & Systems, opening on 26-29 March 2018 in Lake Como, Italy.

【Background 】

Wearable sensors have been grown remarkably for healthcare applications in recent years. HAR is one of the critical applications, especially the detection of elderly falling activity which is a dangerous activity that should be alarmed to take emergent action on time. For wearable sensors based HAR system, energy efficiency is as important as accuracy to the practical use. However, higher accuracy could be obtained at the expense of energy cost by more complicated pattern matching algorithms. Hence, effective solutions should be proposed to reduce energy cost without loss of accuracy.

【 Issues 】

Generally, there are two main challenges for wearable sensors based HAR system. One is the energy cost caused by large dimension of features and huge complexity of pattern matching algorithms. In existing works, most HAR systems need to extraction more than three hundreds of features to do the classification, and use huge training samples to get a high complicated classifier, such as random forest (RF) classifier with more than hundreds of trees. Another is the confusion error between different activities. In HAR, some activities have similar properties and have strong correlation with each other, such as walking, upstairs and downstairs, but other activities have little similarity, e.g. walking and standing. In this case, the confusion errors occur mostly between those with high similarity, resulting in the degradation of the whole system accuracy.

【 Technology 】

1. Pairwise correlation features in hybrid sliding windows

In general system, the features, such as mean value and variance, are extracted from one sliding window of one sensor’s axis. These features are absolute measurements that people with different height and weight will have different values. In this case, the system will be not suitable for more users, or the system needs to collect more training data or have more complex classifiers to achieve a certain of recognition accuracy. Hence, FRDC extracts the pairwise correlation features in hybrid sliding window to get the relative measurements. The technology extracts the relationship between two different segments of raw data, such as the ratio of mean values of two axes or two different sliding windows. The total number of feature in this system is only 64, much less than other existing works. The training data for classifier is collected from only one people, so the offline sample collection workload is reduced dramatically.

2. Simplified random forest (RF) and multi-layer classifiers

To further reduce the complexity of the classifiers, two candidate methods are used, as shown in Fig.1, the first is simplified RF classifier, and second is multi-layer classifier. In the first classifier, mutual information theory is used to maximize the classification ability of the activities with high similarity, and a hybrid feature set selection is used for the training of the RF model. In this case, the trained RF could reach the error threshold with less number of trees. The second classifier is to divide the activities into dynamic and static classes. Hence, the number of activities in each class is reduced, and the classifier for each class of activities could be easy to get convergence with less complexity.

Fig.1 Two kinds of classifiers for HAR

3. Location augmentation for recognition

In elderly-healthcare system, the positioning system is generally integrated to provide location information when emergency cases occur. Hence, FRDC uses a coarse location, such as room area, aisle area, and stairwell area as shown in Fig.2, to distinguish the activities with high similarity, i.e. distinguish walking from upstairs and downstairs. The coarse location can be obtained by a low-energy Bluetooth based positioning technique, which is also developed by FRDC [1].

【 Results and effects 】

Fig.3 shows the performance comparison between neural network (NN), classification and regression tree (CART), traditional RF and Fujitsu developed technology. The energy cost is measured to be proportional to the total computational time (unit: us). As shown in Fig.3, our developed system could also achieve 93.01% of accuracy with 74.9% of energy reduction over traditional RF. In this case, the battery lifetime of the wearable device can extend to 4 times of that without using the proposed technologies. Fig. 4 shows the accuracy of proposed technology in terms of confusion matrix. Compared with Fig.4a, our proposal, as shown in Fig.4b, can distinguished well the walking from the upstairs and downstairs, and the falling detection accuracy can reach up to 99%. In these experiments, the training data are collected from only one object, but the testing data are collected from other 6 objects with different height and different weight. The accuracy performance, as shown in Fig.3 and Fig.4, demonstrates that the developed technology has good generalization performance.

Fig.3 performance comparison in terms of accuracy and energy cost

a) Results without location augmentation

b) Results with location augmentation

Fig.4 Confusion matrix of 10 type of activities

【 Future plan 】

Although wearable sensor based HAR system has advantages in low cost, good privacy protection, and immunity to obstacle or clutter environments, it is still a long way from commercialization because of the inconvenience of wearing for elderly. Hence, FRDC will focus on the healthcare system based on non-contact wireless sensing technology, such as microwave radar based HAR system.

[1] Genming Ding, Jun Tian, Qian Zhao, Lili Xie, “Indoor tracking with fusion of wireless positioning, motion recognition and map matching”, IEEE VTC-2017 Spring, 1-5, 2017.