Convolutional Neural Network-Based Robot Control for an Eye-in-Hand Camera - LIRMM - Laboratoire d’Informatique, de Robotique et de Microélectronique de Montpellier
Journal Articles IEEE Transactions on Systems, Man, and Cybernetics: Systems Year : 2023

Convolutional Neural Network-Based Robot Control for an Eye-in-Hand Camera

Abstract

In past decades, much progress has been obtained in vision-based robot control theories with traditional image processing methods. With the advances in deep learning based methods, Convolutional Neural Network (CNN) has now replaced the traditional image processing methods for object detection and recognition. However, it is not clear how the CNN-based methods can be integrated into robot control theories in a stable and predictable manner for object detection and tracking, especially when the aspect ratio of the object is unknown and also varies during manipulation. In this paper, we develop a vision-based control method for robots with an eye-in-hand configuration, which can be directly integrated with existing CNN-based object detectors. The task variables are generated based on parameters of the bounding box from the output of any real-time CNN object detector such as You Only Look Once (Yolo). To address the chattering problem of bounding box, Long Short-Term Memory (LSTM) is used to provide smoothed bounding box information. A vision-based controller is then proposed following task-space motion control design formulation in order to keep the object of unknown aspect ratio in the center of field of view of the camera. The stability of the overall closed-loop control system is analyzed rigorously using Lyapunov-like approach. Experimental results are presented to illustrate the performance of the proposed CNNbased robot controller.
Fichier principal
Vignette du fichier
revised.pdf (12.78 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

lirmm-04102324 , version 1 (22-05-2023)

Identifiers

Cite

Jia Guo, Huu-Thiet Nguyen, Chao Liu, Chien Chern Cheah. Convolutional Neural Network-Based Robot Control for an Eye-in-Hand Camera. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53 (8), pp.4764-4775. ⟨10.1109/TSMC.2023.3257416⟩. ⟨lirmm-04102324⟩
26 View
123 Download

Altmetric

Share

More