A unified multimodal control framework for human–robot interaction - LIRMM - Laboratoire d’Informatique, de Robotique et de Microélectronique de Montpellier Access content directly
Journal Articles Robotics and Autonomous Systems Year : 2015

A unified multimodal control framework for human–robot interaction

Andrea Cherubini
Robin Passama
Philippe Fraisse
André Crosnier
  • Function : Author
  • PersonId : 938570


In human–robot interaction, the robot controller must reactively adapt to sudden changes in the environment (due to unpredictable human behaviour). This often requires operating different modes, and managing sudden signal changes from heterogeneous sensor data. In this paper, we present a multimodal sensor-based controller, enabling a robot to adapt to changes in the sensor signals (here, changes in the human collaborator behaviour). Our controller is based on a unified task formalism, and in contrast with classical hybrid visicn–force–position control, it enables smooth transitions and weighted combinations of the sensor tasks. The approach is validated in a mock-up industrial scenario, where pose, vision (from both traditional camera and Kinect), and force tasks must be realized either exclusively or simultaneously, for human–robot collaboration.
Fichier principal
Vignette du fichier
J8-multimodal control for human-robot interaction.pdf (5.8 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

lirmm-01222976 , version 1 (09-11-2015)



Andrea Cherubini, Robin Passama, Philippe Fraisse, André Crosnier. A unified multimodal control framework for human–robot interaction. Robotics and Autonomous Systems, 2015, 70, pp.106-115. ⟨10.1016/j.robot.2015.03.002⟩. ⟨lirmm-01222976⟩
326 View
1160 Download



Gmail Mastodon Facebook X LinkedIn More