Journal of the Semiconductor & Display Technology (반도체디스플레이기술학회지)

The Korean Society Of Semiconductor & Display Technology (한국반도체디스플레이기술학회)
권호 표지

A Study on Vision-based Calibration Method for Bin Picking Robots for Semiconductor Automation

반도체 자동화를 위한 빈피킹 로봇의 비전 기반 캘리브레이션 방법에 관한 연구

  • Kyo Mun Ku (Department of IT Semiconductor Engineering, Tech University of Korea) ;
  • Ki Hyun Kim (Department of Mechatronics Engineering, Tech University of Korea) ;
  • Hyo Yung Kim (Department of Mechatronics Engineering, Tech University of Korea) ;
  • Jae Hong Shim (Department of Mechatronics Engineering, Tech University of Korea)
  • 구교문 (한국공학대학교 IT반도체융합공학부) ;
  • 김기현 (한국공학대학교 메카트로닉스공학부) ;
  • 김효영 (한국공학대학교 메카트로닉스공학부) ;
  • 심재홍 (한국공학대학교 메카트로닉스공학부)
  • Received : 2023.03.03
  • Accepted : 2023.03.22
  • Published : 2023.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

In many manufacturing settings, including the semiconductor industry, products are completed by producing and assembling various components. Sorting out from randomly mixed parts and classification operations takes a lot of time and labor. Recently, many efforts have been made to select and assemble correct parts from mixed parts using robots. Automating the sorting and classification of randomly mixed components is difficult since various objects and the positions and attitudes of robots and cameras in 3D space need to be known. Previously, only objects in specific positions were grasped by robots or people sorting items directly. To enable robots to pick up random objects in 3D space, bin picking technology is required. To realize bin picking technology, it is essential to understand the coordinate system information between the robot, the grasping target object, and the camera. Calibration work to understand the coordinate system information between them is necessary to grasp the object recognized by the camera. It is difficult to restore the depth value of 2D images when 3D restoration is performed, which is necessary for bin picking technology. In this paper, we propose to use depth information of RGB-D camera for Z value in rotation and movement conversion used in calibration. Proceed with camera calibration for accurate coordinate system conversion of objects in 2D images, and proceed with calibration of robot and camera. We proved the effectiveness of the proposed method through accuracy evaluations for camera calibration and calibration between robots and cameras.

Keywords

Acknowledgement

이 논문은 경기도의 경기도협력연구센터(GRRC)사업 [(GRRC TU Korea2020-B02), 이종소재 접합 제조공정 자동화를 위한 로봇 응용기술 개발]과 2022년도 정부(산업통상자원부)와 한국산업기술진흥원의 '한/체코 국제공동 기술개발사업(No. P0019623)'으로 수행된 연구 결과입니다.

References

  1. Torres, Pedro, et al. "Bin-Picking Solution for Randomly Placed Automotive Connectors Based on Machine Learning Techniques." Electronics 11.3 (2022): 476.
  2. Lee, Sukhan, and Yeonho Lee. "Real-Time Industrial Bin-Picking with a Hybrid Deep Learning-Engineering Approach." 2020 IEEE International Conference on Big Data and Smart Computing (BigComp). IEEE, 2020.
  3. Ellekilde, Lars-Peter, and Henrik Gordon Petersen. "Motion planning efficient trajectories for industrial binpicking." The International Journal of Robotics Research 32.9-10 (2013): 991-1004. https://doi.org/10.1177/0278364913487237
  4. Rahardja, Krisnawan, and Akio Kosaka. "Vision-based bin-picking: Recognition and localization of multiple complex objects using simple visual cues." Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS'96. Vol. 3. IEEE, 1996.
  5. Ha, Eun-Hyeon, and Goon-Ho Choi. "Position Control of an Object Using Vision Sensor." Journal of the Semiconductor & Display Technology 10.2 (2011): 49- 56.
  6. Lee, Yong-Hwan, and Youngseop Kim. "Implementation of object feature extraction within image for object tracking." Journal of the Semiconductor & Display Technology 17.3 (2018): 113-116.
  7. Park, Frank C., and Bryan J. Martin. "Robot sensor calibration: solving AX= XB on the Euclidean group." IEEE Transactions on Robotics and Automation 10.5 (1994): 717-721. https://doi.org/10.1109/70.326576
  8. Open-Source Computer Vision 4.6.0, "Camera Calibration and 3D Reconstruction", Generated on Sun Jun 5, 2022, 16:19:58 for OpenCV, 2022.
  9. Fassi, Irene, and Giovanni Legnani. "Hand to sensor calibration: A geometrical interpretation of the matrix equation AX= XB." Journal of Robotic Systems 22.9 (2005): 497-506. https://doi.org/10.1002/rob.20082
  10. Li, Haiyuan, et al. "Simultaneous hand-eye and robot-world calibration by solving the $ AX= YB $ problem without correspondence." IEEE Robotics and Automation Letters 1.1 (2015): 145-152.
  11. Condurache, D., and A. Burlacu. "Orthogonal dual tensor method for solving the AX= XB sensor calibration problem." Mechanism and Machine Theory 104 (2016): 382-404. https://doi.org/10.1016/j.mechmachtheory.2016.06.002
  12. Horaud, Radu, and Fadi Dornaika. "Hand-eye calibration." The international journal of robotics research 14.3 (1995): 195-210. https://doi.org/10.1177/027836499501400301
  13. Tabb, Amy, and Khalil M. Ahmad Yousef. "Solving the robot-world hand-eye (s) calibration problem with iterative methods." Machine Vision and Applications 28.5 (2017): 569-590. https://doi.org/10.1007/s00138-017-0841-7
  14. Park, Frank C., and Bryan J. Martin. "Robot sensor calibration: solving AX= XB on the Euclidean group." IEEE Transactions on Robotics and Automation 10.5 (1994): 717-721. https://doi.org/10.1109/70.326576
  15. Lee, Yong-Hwan, and Youngseop Kim. "Comparative Analysis of Cost Aggregation Algorithms in Stereo Vision." Journal of the Semiconductor & Display Technology 15.1 (2016): 47-51.