This paper proposes a multi-level feature learning framework for human action recognition using body-worn inertial sensors. The framework consists of three phases, respectively designed to analyze signal-based (low-level), components (mid-level) and semantic (high-level) information. Low-level features, extracted from raw signals, capture the time and frequency domain property while mid-level representations, obtained through the dictionary learning method, learn the composition of the action. The Max-margin Latent Pattern Learning (MLPL) method is proposed and implemented on the concatenation of low- and mid-level features to learn high-level semantic descriptions of latent action patterns as the output of our framework. Various experiments on Opp, Skoda and WISDM datasets show that the semantic feature learned by this framework possesses higher representation ability than low- and mid-level features. Compared with existing methods, the proposed method achieves state-of-the-art performances.
@cite_18: Permutation modeling is challenging because of the combinatorial nature of the problem. However, such modeling is often required in many real-world applications, including activity recognition where subactivities are often permuted and partially ordered. This paper introduces a novel Hidden Permutation Model (HPM) that can learn the partial ordering constraints in permuted state sequences. The HPM is parameterized as an exponential family distribution and is flexible so that it can encode constraints via different feature functions. A chain-flipping Metropolis-Hastings Markov chain Monte Carlo (MCMC) is employed for inference to overcome the O(n!) complexity. Gradient-based maximum likelihood parameter learning is presented for two cases when the permutation is known and when it is hidden. The HPM is evaluated using both simulated and real data from a location-based activity recognition domain. Experimental results indicate that the HPM performs far better than other baseline models, including the naive Bayes classifier, the HMM classifier, and Kirshner's multinomial permutation model. Our presented HPM is generic and can potentially be utilized in any problem where the modeling of permuted states from noisy data is needed.
@cite_41: In principle, activity recognition can be exploited to great societ al benefits, especially in real-life, human centric applications such as elder care and healthcare. This article focused on recognizing simple human activities. Recognizing complex activities remains a challenging and active area of research and the nature of human activities poses different challenges. Human activity understanding encompasses activity recognition and activity pattern discovery. The first focuses on accurate detection of human activities based on a predefined activity model. An activity pattern discovery researcher builds a pervasive system first and then analyzes the sensor data to discover activity patterns.
@cite_29: 
@cite_55: 
@cite_54: This paper presents an approach for recognition of Activities of Daily Living (ADLs) in smart environments. Our approach is based on the frequent pattern mining principle to extract frequent patterns in the datasets collected from different sensors disseminated in a smart environment. In contrast with existing intrusive activity recognition approaches that have been proposed in the literature, where the datasets are basically composed of audio-visual or images files recorded during experiments, our approach is fully non-intrusive and it is based on the analysis of event sequences collected from heterogenous sensors. Our approach consists of two main phases, (1) frequent pattern mining to extract frequent patterns, and (2) activity recognition using a mapping function between the extracted frequent patterns and the activity models. We show through experiments how our approach accurately recognizes tasks as well as activities and outperforms the HMM model.
@cite_39: Monitoring daily activities of a person has many potential benefits in pervasive computing. These include providing proactive support for the elderly and monitoring anomalous behaviors. A typical approach in existing research on activity detection is to construct sequence-based models of low-level activity features based on the order of object usage. However, these models have poor accuracy, require many parameters to estimate, and demand excessive computational effort. Many other supervised learning approaches have been proposed but they all suffer from poor scalability due to the manual labeling involved in the training process. In this paper, we simplify the activity modeling process by relying on the relevance weights of objects as the basis of activity discrimination rather than on sequence information. For each activity, we mine the web to extract the most relevant objects according to their normalized usage frequency. We develop a KeyExtract algorithm for activity recognition and two algorithms, MaxGap and MaxGain, for activity segmentation with linear time complexities. Simulation results indicate that our proposed algorithms achieve high accuracy in the presence of different noise levels indicating their good potential in real-world deployment.
@cite_27: Complex activities typically consist of multiple primitive events happening in parallel or sequentially over a period of time. Understanding such activities requires recognizing not only each individual event but, more importantly, capturing their spatiotemporal dependencies over different time intervals. Most of the current graphical model-based approaches have several limitations. First, time--sliced graphical models such as hidden Markov models (HMMs) and dynamic Bayesian networks are typically based on points of time and they hence can only capture three temporal relations: precedes, follows, and equals. Second, HMMs are probabilistic finite-state machines that grow exponentially as the number of parallel events increases. Third, other approaches such as syntactic and description-based methods, while rich in modeling temporal relationships, do not have the expressive power to capture uncertainties. To address these issues, we introduce the interval temporal Bayesian network (ITBN), a novel graphical model that combines the Bayesian Network with the interval algebra to explicitly model the temporal dependencies over time intervals. Advanced machine learning methods are introduced to learn the ITBN model structure and parameters. Experimental results show that by reasoning with spatiotemporal dependencies, the proposed model leads to a significantly improved performance when modeling and recognizing complex activities involving both parallel and sequential events.
@cite_50: As compared to actions, activities are much more complex, but semantically they are more representative of a human's real life. Techniques for action recognition from sensor-generated data are mature. However, few efforts have targeted sensor-based activity recognition. In this paper, we present an efficient algorithm to identify temporal patterns among actions and utilize the identified patterns to represent activities for automated recognition. Experiments on a real-world dataset demonstrated that our approach is able to recognize activities with high accuracy from temporal patterns, and that temporal patterns can be used effectively as a mid-level feature for activity representation.
High-level recognition tasks mainly focus on obtaining intuitive and semantic descriptions of actions. Pattern-mining methods @cite_55 @cite_54 @cite_29 @cite_41 @cite_39 @cite_50 and probabilistic graphical models @cite_27 @cite_18 are the most prevalent approaches. Pattern-mining methods explore the diversity in human actions through learning discriminative action patterns and motion primitives. @cite_55 applied probabilistic topic models which stemmed from the text processing community to automatically extract action patterns from sensor data. They described the recognition of daily routines as a probabilistic combination of learned patterns. @cite_50 presented an algorithm capable of identifying temporal patterns from low-level actions and utilized these temporal patterns to further represent high-level human actions. Various methods have also been proposed based on probabilistic graphical models @cite_27 @cite_18 to capture the temporal, sequential, interleaved or concurrent relationship among motion primitives. However, graphical models are limited to capturing rich temporal relationships in complex actions and also suffer from the exponential increase in calculation when the number of involved actions grows @cite_50 . Instead of modeling temporal relationships, we propose an efficient pattern-mining approach, which takes advantage of being compact in representation, intuitive in understanding and efficient in calculation.