A person’s behavior significantly influences their health and well-being. It also contributes to the social environment in which humans interact, with cascading impacts to the health and behaviors of others. During social interactions, our understanding and awareness of vital nonverbal messages expressing beliefs, emotions, and intentions can be obstructed by a variety of factors including greatly flawed self-awareness. For these reasons, human behavior is a very important topic to study using the most advanced technology. Moreover, technology offers a breakthrough opportunity to improve people’s social awareness and self-awareness through machine-enhanced recognition and interpretation of human behaviors.
This research focuses on how to effectively combine a variety of sensor modalities, such as physiological and audio sensors, how to effectively synchronize the acquired information, and how to optimize machine learning algorithmic implementations to identify human behaviors of interest. We have designed a multi-sensor infrastructure that allows us to perform human studies, collect data for algorithm design, and test real-time implementations.