The Role of Audio-Visual Feedback in a Thought-Based Control of a Humanoid Robot: A BCI Study in Healthy and Spinal Cord Injured People

Emmanuele Tidoni 1 Pierre Gergondet 2 Gabriele Fusco 3 Abderrahmane Kheddar 2 Salvatore Aglioti 1
1 Fondazione Santa Lucia [IRCCS]
2 IDH - Interactive Digital Humans
LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier
3 Department of Psychology [University of Roma "La Sapienza"]
Abstract : The efficient control of our body and successful interaction with the environment are possible through the integration of multisensory information. Brain-computer interface (BCI) may allow people with sensorimotor disorders to actively interact in the world. In this study, visual information was paired with auditory feedback to improve the BCI control of a humanoid surrogate. Healthy and spinal cord injured (SCI) people were asked to embody a humanoid robot and complete a pick-and-place task by means of a visual evoked potentials BCI system. Participants observed the remote environment from the robot's perspective through a head mounted display. Human-footsteps and computer-beep sounds were used as synchronous/asynchronous auditory feedback. Healthy participants achieved better placing accuracy when listening to human footstep sounds relative to a computer-generated sound. SCI people demonstrated more difficulty in steering the robot during asynchronous auditory feedback conditions. Importantly, subjective reports highlighted that the BCI mask overlaying the display did not limit the observation of the scenario and the feeling of being in control of the robot. Overall, the data seem to suggest that sensorimotor-related information may improve the control of external devices. Further studies are required to understand how the contribution of residual sensory channels could improve the reliability of BCI systems.
Document type :
Journal articles
Complete list of metadatas
https://hal-lirmm.ccsd.cnrs.fr/lirmm-02155061
Contributor : Isabelle Gouat <>
Submitted on : Thursday, June 13, 2019 - 12:32:55 PM
Last modification on : Tuesday, June 18, 2019 - 12:06:37 PM

Identifiers

Collections

LIRMM | IDH | MIPS | UNIV-MONTPELLIER

Citation

Emmanuele Tidoni, Pierre Gergondet, Gabriele Fusco, Abderrahmane Kheddar, Salvatore Aglioti. The Role of Audio-Visual Feedback in a Thought-Based Control of a Humanoid Robot: A BCI Study in Healthy and Spinal Cord Injured People. IEEE Transactions on Neural Systems and Rehabilitation Engineering, Institute of Electrical and Electronics Engineers, 2017, 25 (6), pp.772-781. ⟨10.1109/TNSRE.2016.2597863⟩. ⟨lirmm-02155061⟩

Export

BibTeX TEI DC DCterms EndNote

Share

Metrics

Record views

21

Contact

support.ccsd.cnrs.fr
hal.support@ccsd.cnrs.fr
Logo CCSD