Multimodal Neurocognitive & Neurohormonal Assessment of Human Behavior in Human-Robot Interactions

Thursday, August 22, 2024

10:00 AM-12:00 PM

BIOMED PhD Research Proposal

Title:
Multimodal Neurocognitive and Neurohormonal Assessment of Human Behavior During Naturalistic Human-Robot Interactions

Speaker:
Yigit Topoglu, PhD Candidate
School of Biomedical Engineering, Science and Health Systems
Drexel University

Advisor:
Hasan Ayaz, PhD
Associate Professor
School of Biomedical Engineering, Science and Health Systems
Drexel University

Details:
Advancing the understanding of real-world human behavior through the lens of neurophysiological mechanisms is a primary goal of neuroergonomics, a field that converges in the fields of cognitive science, neuroscience, biomedical engineering, psychology, and human factors. The neuroergonomic approach utilizes mobile and wearable neuroimaging to record biomedical signals alongside behavior in unencumbered everyday settings to provide the opportunity for understanding the continuum of the brain, body, behavior, and environment during complex scenarios.

The objective of this proposal is to apply the neuroergonomic approach by combining wearable neuroimaging using functional near-infrared spectroscopy (fNIRS) with hormonal measures using oxytocin (OT), self-reported surveys, and behavioral measures for the assessment of human social mechanisms towards humanoid robots during naturalistic face-to-face human-robot interaction (HRI), such as social conversations and teaming scenarios. The growth of integrating autonomous agents in everyday settings is exponentially increasing, influencing individuals and society. As a result of this, the need for designing robots that are socially adept with human users has been one of the challenges of the HRI field. To be able to find the factors and features for robots to optimize the interaction with users, there is a need to understand the human social mechanisms while interacting with a sociable robot. However, gaps persist in our understanding of the neural mechanisms in immersive real HRI settings, as the traditional HRI assessment methods, such as behavioral and self-reported measures, lack the ability to capture the user’s behavior in the moment of interaction. Incorporating biomarkers such as cortical activity using mobile and wearable brain imaging and OT measurements to the traditional methods can provide extensive insights into how human social mechanisms during naturalistic complex HRI, as prefrontal cortex (PFC) activity is related to processing social information and collaborative decision-making, and OT is associated with social bonding and trust. The joint integration of such complementary measures offers the potential to enhance the understanding of social and teaming dynamics of HRI beyond using either approach individually and aligns well with the emerging field of neuroergonomics, which aims to study the brain and body in everyday settings. This multimodal approach can provide valuable insights towards the design and development of more socially adept robots in the future.

The proposed thesis will provide several novel contributions to a knowledge base that can further both human-robot conversations and teaming scenarios. In the first aim, we explore conversational naturalistic HRI, where the robot acts as a dialogue partner, and look at the effects of the robot’s activeness (features such as backchanneling, eye gaze, and gestures) and performance (congruent vs erroneous) on the user’s neural response and behavior towards the robot using the neural, physiological and behavioral correlates. In the second aim, we investigate the short-term collaborative HRI, where the user teams up with the robot on a series of collaborative tasks, examining the how robot’s performance and task difficulty affect the user’s neural response and behavior toward the robot alongside overall team performance. In the third aim, we expand our approach to a longitudinal collaborative HRI scenario to evaluate how the robot’s performance during teaming impacts the task performance, the user’s neural response, and behavior towards the robot over time. Collectively, this thesis will serve as a guideline for the assessment of human social behavior and future social robot design that can be utilized in real-world HRI outside of a lab and add to the field of neuroergonomics in terms of expanding the application of wearable and mobile neuroimaging technologies even further.

Contact Information

Natalia Broz
njb33@drexel.edu