Wearable Network for Physical Context Detection                 

Objective: This project explores a networked proximity sensing and Hidden Markov Model (HMM) based mechanism that can be applied for stochastic identification of body postures using a wearable sensor network. The idea is to collect relative proximity information between wireless sensors that are strategically placed over a subject’s body to monitor the relative movements of the body segments, and then to process that using HMM in order to identify the subject’s body postures. The key novelty of this approach is a departure from the traditional accelerometry based approaches in which the individual body segment movements, rather than their relative proximity, is used for activity monitoring and posture detection. Through experiments with body mounted sensors we demonstrate that while the accelerometry based approaches can be used for differentiating activity intensive postures such as walking and running, they are not very effective for identification and differentiation between low activity postures such as sitting and standing. We develop a wearable sensor network prototype that monitors relative proximity using Radio Signal Strength indication (RSSI), and then construct a HMM system for posture identification in the presence of sensing errors. Controlled experiments using human subjects are carried out for evaluating the accuracy of the HMM identified postures compared to a naïve threshold based mechanism, and its variations over different human subjects. 

Video Demo: Please watch this video to see our preliminary working prototype of a wearable sensor network. To learn more about the project please download this paper and read the rest of this page.

Wearable Sensor Network: A sensor network, as shown in the figure above is constructed by mounting multiple sensors in different parts of the body, so that the individual sensors movement can reflect the displacement of the corresponding body segment. Mica2Dot mote radio nodes, operating with 433MHz radio, and its sensor card MTS510 from the Crossbow Inc. are used as the wearable sensor nodes. The Mica2Dot nodes run from a 570mAH button cell with a total node weight of 5.9 grams. In our experiments, each sensor is worn with an elastic band so that the sensor mount orientation does not change during experiments involving heavy activities including walking and running. As shown in the diagram, the wearable sensor nodes form an ad hoc sensor network with a topology that is dynamically determined based not only on the relative locations of the subject’s limbs but also on the wireless link qualities determined by the transmission power and instantaneous link attenuation. Wireless links are also available to transport raw data or processed events from the body network to an external processing server. A MICA2 radio node with custom-built serial interface to run RS232 protocol has been used for collecting data and events from the body network and send it to a Windows based PC used as the processing server. 

Sensor Modalities: Two sensor modalities, namely, acceleration and relative proximity are used for the developed system. A two-axes piezoelectric accelerometer is used for detecting the changes in body movements. Since acceleration is proportional to the net external force, it provides a very direct indication of the energy expenditure due to the involving physical activities. The proximity between the sensor nodes, shown in the diagram, is the second sensor modality that is measured using received signal strength indication (RSSI) of the radio signal.

Posture Identification using Multi-modality Sensing : As a preliminary target we attempt to identify four body postures, namely, SIT, STAND, WALK and RUN, which represent both activity intensive and non-intensive scenarios. We develop controlled experiments in which human subjects are given pre-determined sequence of those four postures to follow, and the wearable sensor network is used for collecting both accelerometer and proximity data for out-of-body and off-line post processing. Postures, identified using our proposed detection algorithms, are then temporally correlated with the actual sequence given to the subjects for evaluating the identification accuracy.  

The diagram above outlines the sensing scope for the four targeted postures. It is shown that while accelerometry is capable of identifying WALK and RUN, it is not sufficient for the low activity postures. The key idea in this project is to add a second sensing modality, namely, physical proximity, and its associated analysis techniques for enabling a wearable sensor network to identify all four targeted postures. Since accelerometry based posture recognition has already been a subject of investigation in prior literature, the preliminary part of this project explores the sensing and processing issues involving the identification of the non activity-intensive postures, namely SIT and STAND using physical proximity as the sensor modality, and HMM as the statistical pattern identification mechanism.    

Posture Modeling and Generation: The sit-stand behavior of a subject in our experiments is modeled as a Markov process in which the subject’s posture transitions assume to follow a memory less process. The transition probabilities across the postures (see the diagram above) represent the subject’s behavior that remains stationary for certain time intervals. The transition matrix A remains fixed during a stationary interval and can vary across such intervals based on specific subject individual and his or her nature and place of work. We generate a sequence of 50 states of SIT and STAND using the probability transition matrix:

where state-1 is SIT and state-2 is STAND. A subject is handed out the resulting posture sequence (S, S, S, S, T, T, S, S, T, T, T, S, T, T, T, S, S, S, S, S, S, T, S, S, S, S, S, S, S, S, S, S, S, S,  S, S, S, S, T, T, S, T, T, T, T, T, S, S, S, T), where S represents SIT and T represents STAND, and is instructed to follow the sequence with 10sec being spend in each posture, thus the entire experiment lasting for 500 sec.

Sensor Placement: Four RSSI based proximity sensors are mounted on the subject's body – two on the two upper arms and two on the two front thighs. Each sensor maintains information about its current connectivity and RSSI values with all other sensors. The resulting RSSI values are then collected on an out-of-body compute server and averaged to represent the mean instantaneous relative distance between the subject’s body segments those are mounted with the sensors.

Threshold based Posture Identification: The average RSSI values (in dB) are plotted in in the diagram above for the entire 500 sec duration of the experiment. The figure also shows the actual posture state that the subject was in during each 10 sec. slot. In these reading, high RSSI dB values indicate low received radio signal strength, and a low RSSI value indicates high received radio signal strength. The following observations should be made. First, the average RSSI has an overall trend to be high for the SIT postures and low for the STAND posture. This is consistent since the body parts are generally closely situated during sitting, and further apart while standing. Second, while generally maintaining this trend, there are lots of anomalies observed. These anomalies are found to be caused by several factors including radio blockage by the clothing material, unintentional change of sensor and antenna orientations, and other imperfections in sensor mounting.

In spite of these anomalies in the average RSSI data, it is possible to identify the SIT and STAND postures by using a carefully chosen RSSI threshold somewhere between the minimum and the maximum values of the observed data. In the figure below we report the effectiveness of such posture identification using different threshold values. The identified posture for a given RSSI threshold is compared with the subject’s actual state for computing the success rate as reported in the Figure. Such success or match rates are presented for different threshold values and for different individuals participated as experimental subjects. All three individuals in these experiments were asked to follow the same controlled posture sequence used in Figure 6 for several rounds, before the identification performance were computed.

Observe that in spite of the errors contributed by sensor and antenna mis-orientation, and radio signal blockage by clothing material, this simple threshold based mechanism can detect the SIT and STAND postures with up to approximately 90% accuracy. However, since the identification success rate is heavily sensitive to the RSSI threshold, choosing the right threshold is an important design step for this mechanism to work. A potentially more restricting aspect is that the optimal threshold is also sensitive to the individual subjects’ physical and motor aspects during his or her postures. For example, while an RSSI threshold of 70 yields the best identification accuracy for subject-1, the performance for subject-3 maximizes at 86% for a threshold 90. In fact, at the RSSI threshold of 70, for subject-3 the system delivers a poor posture identification rate of only 75%.These results allude to a practical limitation of the threshold based posture identification in terms of the need for person specific threshold dimensioning. Other experiments further indicated that the optimal threshold value can change even for an individual based on his or her behavioral changes over time. In the next section we develop a Hidden Markov Model (HMM) based mechanism for adaptive and person-independent posture detection. 

Capturing Stationary Behavior using Hidden Markov Model: The inability of the simple threshold based mechanism to handle the degraded quality of proximity sensor data stems from the fact that the identification process does not leverage the stationary nature of human behavior over certain time intervals. To address this limitation, we adopt a stochastic posture identification solution that attempts to leverage the stationary nature of the human behavior by modeling the posture state machine as a Hidden Markov Model (HMM). The key concept of the HMM in this context are as follows. A stochastic process is represented by a discrete time Markov Chain consisting of multiple states which are hidden from an observer in the sense that an observer cannot directly determine which state the system is in at any given point in time. However, a number of observable parameters that stochastically represent the states are visible to the observer. The idea of HMM formulation is that if the state transition probability matrix and the observation generation probabilities are known (or measurable) to the observer, the latter can estimate the current state of the Markov Chain. Using HMM it is also possible to compute the probability of occurrence of a specific state sequence.

HMM Mapping: The posture identification problem with our novel proximity sensing framework is mapped as an HMM formulation as follows.

Posture State Space: As shown in the figure above, N postures are modeled as N hidden states with the entire state space represented by S = {S₁, S_{2, …..} S_N}. In this specific case N = 2 for postures SIT and STAND.

Observation: At each state there are M distinct observable parameters that are represented by a vector O = {v₁, v₂, ….. v_M}.  Each v_m (m=1,2, …,M) can take one of multiple possible values. Observation made at time instant t is represented as O_t. At ant given time, the radio RSSI values from the sensor are used to construct the O_t vector, in which each v_m is a binary variable which can be either ‘0’ or ‘1’. The peak-to-peak RSSI range is divided into M equal windows, and then depending on which window the current RSSI value falls in, the corresponding v_m is set to ‘1’. The rest of the v_m‘s are set to ’0’. Note that the value of M determines the granularity of observation, which in turn, is expected to influence the quality of the hidden state identification. We have experimented with M ranging from a small value of 2 (very coarse granularity) to a large value of 10 (very fine granularity observation). As indicated in the figure above, the parameter O_t represents the observation vector at time slot t, with T as the final observations in an experiment. In all our experiments the value of T is 50. In other words, 50 observations, each corresponds a state lasting for 10 seconds, are generated to feed into the HMM estimation system.

Transition Probability Matrix: The posture transition probability matrix is represented as A = [a_ij], where

A is an N x N matrix, and q_t denotes the actual posture at time t. The parameter a_ij represents the probability that the next posture is j, given the current posture of the subject is i.

Observation Probability Matrix: This is represented by B = [b_jm], in which

B is an N x M matrix, and O_t represents the observation vector at time slot t. The parameter b_jm represents the probability that in posture state j, the element v_m in the observation vector O_t is ‘1’ and the rest of the elements are all zero.

Initial State Distribution: This is represented by a vector π = [π_i] of length N, so that:

The quantityrepresents the probability that the posture Markov chain is initialized at state i., by definition,.

Based on the above definitions, a system, modeled using HMM, can be fully specified by the parameters A, B and π which are represented together as a tuple:

Using the HMM derivations shown in Appendix A.1, we first compute the individual probabilities of the system being in each possible posture state at a given time. As shown in the derivation, these probabilities depend on the system’s, and the observation sequence {O₁ O₂ O₃ … O_T}. After the probabilities are computed, the posture state identification is accomplished by finding the most likely state, which is the one with the highest current probability.

Experimental Results: In this section we describe the performance of HMM based posture identification and its performance comparison with the threshold based approach. The same transition probability matrix, as used for the previous experiments, is used for generating a posture sequence to be followed by the subjects. Like in the previous experiments, only two posture states SIT and STAND are used. The observation probability matrix B is constructed by computing the b_jm probabilities (see Equation 2) from observed RSSI values for known posture states. During an initial set of known states, the B matrix is first computed, and then the actual posture identification process was initiated. This initial period is referred to as an observation calibration phase. As for the Initial State Distribution matrix π, we have used [1, 0] for all the results presented here. This means that all the experiments are initiated with the SIT state. These A, B and π matrices constitute the HMM system parameter.The observation sequence {O₁ O₂ O₃ … O_T} is constructed by first collecting the RSSI values from on-body proximity sensors, and then quantizing them into one of M  windows to form the v_m (m = 1, 2, .., M) elements within each observation vector. We have experimented with different observation granularity vector M ranging from 2 to 10.   

After the observation sequence is constructed, standard HMM derivations are used to identify posture states using the system parameterand the observation sequence {O₁ O₂ O₃ … O_T}. State identification performance with HMM in comparison with the threshold based mechanism is presented in the figure above. As done before, the success rates are measured by comparing the detected states with the actual states from the generated posture sequence using transition probability matrix A. The success rate for posture identification using HMM is reported with three different observation granularities corresponding to M = 2, 8 and 10.

The following observations are to be made from the figure above. First, the HMM approach delivers better state match rates (e.g. 92% to 94% identification success for subject-2) compared to the best case performance (88% identification success for subject-2) using the threshold based mechanism, that is with an optimal RSSI threshold of 90 dB. Second, higher observation granularity (larger M) for HMM provides better posture detection rate, with performance saturation occurring beyond around M = 8. Third, once a sufficiently large observation granularity (e.g. M = 10) is chosen for HMM, unlike in the threshold based scheme, there is no optimal parameter dimensioning is needed. This is a significant advantage in terms of implementation feasibility. Finally, with similarly large observation granularities, the HMM continues to provide superior posture identification performance in a subject-independent manner. This further reinforces the practicality of the mechanism in not having to dimension any individual specific parameter which may cause significant performance variation as observed for the threshold based mechanism.

Summary: In this project we are developing an experimental framework for a wearable sensor network to be used for human posture detection. A novel radio frequency based proximity sensing, couple with Hidden Markov Model (HMM) based detection techniques, are being used for detecting low-activity postures that are shown to be not differentiable using the traditional accelerometry based approaches. It was first demonstrated that although a naïve threshold (radio signal) based mechanism can be used for reasonable detection performance, the variation in the proximity information caused by clothing and person-specific behavioral differences require a delicate dimensioning of the used threshold values for consistent posture detection performance across various individuals. To avoid this, an HMM based detection process is applied with observation self-calibration using the Baum-Welch algorithm. It is shown that the HMM method with our novel proximity sensing modality is able to consistently deliver significantly better detection performance than the threshold based mechanism in a more individual-independent manner.

Current Work: Ongoing work on this project includes: 1) development of a real-time and on-body posture detection system, 2) integrating proximity and acceleration data for detecting a wider range of both high-activity and low-activity postures, 3) adjusting the HMM and processing mechanism to adapt for different base behavior (the A matrix).