Browse > Article
http://dx.doi.org/10.3837/tiis.2022.08.004

Robot Manipulator Visual Servoing via Kalman Filter- Optimized Extreme Learning Machine and Fuzzy Logic  

Zhou, Zhiyu (School of Information Science and Technology, Zhejiang Sci-Tech University)
Hu, Yanjun (School of Information Science and Technology, Zhejiang Sci-Tech University)
Ji, Jiangfei (School of Information Science and Technology, Zhejiang Sci-Tech University)
Wang, Yaming (Lishui University)
Zhu, Zefei (School of Mechanical Engineering, Hangzhou Dianzi University)
Yang, Donghe (School of Information Science and Technology, Zhejiang Sci-Tech University)
Chen, Ji (School of Information Science and Technology, Zhejiang Sci-Tech University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.16, no.8, 2022 , pp. 2529-2551 More about this Journal
Abstract
Visual servoing (VS) based on the Kalman filter (KF) algorithm, as in the case of KF-based image-based visual servoing (IBVS) systems, suffers from three problems in uncalibrated environments: the perturbation noises of the robot system, error of noise statistics, and slow convergence. To solve these three problems, we use an IBVS based on KF, African vultures optimization algorithm enhanced extreme learning machine (AVOA-ELM), and fuzzy logic (FL) in this paper. Firstly, KF online estimation of the Jacobian matrix. We propose an AVOA-ELM error compensation model to compensate for the sub-optimal estimation of the KF to solve the problems of disturbance noises and noise statistics error. Next, an FL controller is designed for gain adaptation. This approach addresses the problem of the slow convergence of the IBVS system with the KF. Then, we propose a visual servoing scheme combining FL and KF-AVOA-ELM (FL-KF-AVOA-ELM). Finally, we verify the algorithm on the 6-DOF robotic manipulator PUMA 560. Compared with the existing methods, our algorithm can solve the three problems mentioned above without camera parameters, robot kinematics model, and target depth information. We also compared the proposed method with other KF-based IBVS methods under different disturbance noise environments. And the proposed method achieves the best results under the three evaluation metrics.
Keywords
Extreme learning machine; fuzzy logic; image-based visual servoing; Kalman filter;
Citations & Related Records
연도 인용수 순위
  • Reference
1 W. Li, C. Song, Z. Li, "An accelerated recurrent neural network for visual servo control of a robotic flexible endoscope with joint limit constraint," IEEE Transactions on Industrial Electronics, vol. 67, no. 12, pp. 10787-10797, Dec. 2020.   DOI
2 Z. Zhou, R. Zhang, Z. Zhu, "Robust Kalman filtering with LSTM for image-based visual servo control," Multimedia Tools and Applications, vol. 78, no. 18, pp. 26341-26371, Sep. 2019.   DOI
3 G. Q. Dong, Z. H. Zhu, "Position-based visual servo control of autonomous robotic manipulators," Acta Astronautica, vol. 115, pp. 291-302, Oct-Nov. 2015.   DOI
4 Gossaye Mekonnen, Sanjeev Kumar, P.M. Pathak, "Wireless hybrid visual servoing of omnidirectional wheeled mobile robots," Robotics and Autonomous Systems, vol. 75, pp. 450-462,Jan, 2016,   DOI
5 O. Araar, N. Aouf, "A new hybrid approach for the visual servoing of VTOL UAVs from unknown geometries," in Proc. of the IEEE 22nd Mediterranean Conference on Control and Automation, pp. 1425-1432, 2014.
6 R. E. Kalman, "A new approach to linear filtering and prediction problems," J. Basic Eng, vol. 82, pp. 35-45, 1960.   DOI
7 S. Y. Chen, "Kalman filter for robot vision: a survey," IEEE Trans. Ind. Electron, vol. 59, no. 11, pp. 4409-4420, Nov. 2012.   DOI
8 G. B. Huang, Q. Y. Zhu, C. K. Siew, "Extreme learning machine: Theory and applications," Neurocomputing, vol. 70, pp. 489-501, Dec. 2006.   DOI
9 X. Lv, X. Huang, "Fuzzy adaptive Kalman filtering based estimation of image Jacobin for uncalibrated visual servoing," in Proc. of the IEEE/RSJ/GI Conference Intelligent Robots and Systems, pp. 2167-2172, Oct. 2006.
10 Tolga Yuksel, "Intelligent visual servoing with extreme learning machine and fuzzy logic," Expert Systems with Applications, vol. 72, pp. 344-356, 2017.   DOI
11 X. G. Zhong, X. Y. Zhong, X. F. Peng, "Robots visual servo control with features constraint employing Kalman-neural-network filtering scheme," Neurocomputing, vol. 151, pp. 268-277, May. 2015.   DOI
12 Z. Miljkovic, M. Mitic, M. Lazarevic, B. Babic, "Neural network reinforcement learning for visual control of robot manipulators," Expert Systems with Applications, vol. 40, no. 5, pp. 1721-1736, Apr. 2013.   DOI
13 Maxwell Hwang, Yu-Jen Chen, Ming-Yi Ju, Wei-Cheng Jiang, "A fuzzy CMAC learning approach to image based visual servoing system," Information Sciences, vol.576, pp.187-203, 2021,   DOI
14 Jinhui Wu, Zhehao Jin, Andong Liu, Li Yu, Fuwen Yang, "A survey of learning-based control of robotic visual servoing systems," Journal of the Franklin Institute, vol. 359, pp. 556-577, 2022.   DOI
15 G. B. Huang, L. Chen, C. K. Siew, "Universal Approximation Using Incremental Constructive Feedforward Networks with Random Hidden Nodes," IEEE Transactions on Neural Networks, vol. 17, no. 4, pp. 879-892, 2006.   DOI
16 Y. Zhang, G. Zhao, G., J. Sun, "Smart pathological brain detection by synthetic minority oversampling technique, extreme learning machine, and jaya algorithm," Multimedia Tools and Applications, vol. 77, no. 17, pp. 22629-22648, Sep. 2018.   DOI
17 G. B. Huang, H. M. Zhu, X. J. Ding, R. Zhang, "Extreme learning machine for regression and multiclass classification," IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, vol. 42 no. 2, pp. 513-529, Apr. 2012.   DOI
18 Wilson, W.J., Hulls, C.C.W. and Bell, G.S., "Relative end-effector control using cartesian positionbased visual servoing," IEEE Transactions on Robotics and Automation, vol. 12, pp. 684-696, 1996.   DOI
19 J. Qian, J. B. Su, "On-line estimation of image Jacobian matrix based on Kalman filter," Control and Decision, vol. 18, no. 1, pp. 77-80, Oct. 2003.   DOI
20 F. Lizarralde, A. C. Leite, L. Hsu, R. R. Costa, "Adaptive visual servoing scheme free of image velocity measurement for uncertain robot manipulators," Automatica, vol. 49, no. 5, pp. 1304-1309, May. 2013.   DOI
21 H. A. Junaid, "ANN based robotic arm visual servoing nonlinear system," Procedia Computer Science, vol. 62, pp. 23-30, 2015.   DOI
22 Abdollahzadeh, B., F. S. Gharehchopogh, S. Mirjalili, "African vultures optimization algorithm: a new nature-inspired metaheuristic algorithm for global optimization problems," Computers & Industrial Engineering, vol. 158, Aug. 2021.
23 Bassoma Diallo, Jie Hu, Tianrui Li, Ghufran Ahmad Khan and Ahmed Saad Hussein, "Multiview document clustering based on geometrical similarity measurement," International Joural of Machine Learning and Cybernetics, vol 13, pp.663-675, 2022.   DOI
24 H. Sutanto, R. Sharma, V. Varma, "Image based autodocking without calibration," in Proc. of IEEE International Conference on Robotics and Automation, pp. 974-979, Apr. 1997.
25 S. R. Jang, C. T. Sun, "Neuro-fuzzy and soft computing: A computational approach to learning and machine intelligence," IEEE Transactions on Automatic Control, vol. 42, no. 10, pp. 1482-1484, Oct. 1997.   DOI
26 B. Armstrong, O. Khatib, J. Burdick, "The explicit dynamic model and inertial parameters of the PUMA 560 arm," IEEE international conference on robotics and automation, pp. 510-518, Apr. 1986.
27 G. Chesi, Y. S. Hung, "Global path-planning for constrained and optimal visual servoing," IEEE Transactions on Robotics, vol. 23, no. 5, pp. 1050-1060, Oct. 2007.   DOI
28 P. I. Corke, "Robotics, vision and control: Fundamental algorithms in MATLAB," Springer, 2011.
29 Cui L, Wang H, Liang X, Wang J, Chen W, "Visual servoing of a flexible aerial refueling boom with an eye-in-hand camera," IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, no. 10, pp. 6282-6292, Oct. 2021.   DOI
30 Ghufran Ahmad Khan, Jie Hu, Tianrui Li, Bassoma Diallo and Hongjun Wang, "Multi-view data clustering via non-negative matrix factorization with manifold regularization," International Joural of Machine Learning and Cybernetics, vol.13, pp. 677-689, 2022.   DOI
31 Xianxia Zhang, Jinqiang Zhang, Zhiyuan Li, Shiwei Ma and Banghua Yang, "Visual feedback fuzzy control for a robot manipulator based on SVR learning," Journal of System Simulation, vol. 32, no. 10, 2020.
32 J. Dong, J Zhang, "A new imaged-based visual servoing method with velocity direction control," J. Franklin. Inst, vol. 357, no. 7, pp. 3993-4007, 2020.   DOI
33 X. Ren, H. Li, Y. Li, "Image-based visual servoing control of robot manipulators using hybrid algorithm with feature constraints," IEEE Access, vol. 8, pp. 223495-223508, 2020.   DOI
34 S. Kagami, K. Omi, K. Hashimoto, "Alignment of a flexible sheet object with position- based and image-based visual servoing," Advanced Robotics, vol. 30, no. 15, pp. 965-978, 2016.   DOI
35 Z. Zhou, B. Wu, "Adaptive sliding mode control of manipulators based on fuzzy random vector function links for friction compensation," OPTIK, vol. 227, Feb. 2021.
36 Y. Wang, G. L. Zhang, H. X. Lang, B. S. Zuo, C. W. Silva, "A modified image-based visual servo controller with hybrid camera configuration for robust robotic grasping," Robotics and Autonomous Systems, vol. 62, no. 10, pp. 1398-1407, Oct. 2014.   DOI
37 Yaozhen He, Jian Gao, Yimin Chen, "Deep learning-based pose prediction for visual servoing of robotic manipulators using image similarity," Neurocomputing, vol. 491, pp. 343-352, 2022.   DOI
38 T. Drummond, R. Cipolla, "Real-time tracking of complex structures with on-line camera calibration," Image & Vision Computing, vol. 20, pp. 427-433, Apr. 2002.   DOI