Browse > Article
http://dx.doi.org/10.7746/jkros.2021.16.1.049

Real-Time Tomato Instance Tracking Algorithm by using Deep Learning and Probability Model  

Ko, KwangEun (Robot Applied R&D Group, Korea Institute of Industrial Technology)
Park, Hyun Ji (Robot Applied R&D Group, Korea Institute of Industrial Technology)
Jang, In Hoon (Robot Applied R&D Group, Korea Institute of Industrial Technology)
Publication Information
The Journal of Korea Robotics Society / v.16, no.1, 2021 , pp. 49-55 More about this Journal
Abstract
Recently, a smart farm technology is drawing attention as an alternative to the decline of farm labor population problems due to the aging society. Especially, there is an increasing demand for automatic harvesting system that can be commercialized in the market. Pre-harvest crop detection is the most important issue for the harvesting robot system in a real-world environment. In this paper, we proposed a real-time tomato instance tracking algorithm by using deep learning and probability models. In general, It is hard to keep track of the same tomato instance between successive frames, because the tomato growing environment is disturbed by the change of lighting condition and a background clutter without a stochastic approach. Therefore, this work suggests that individual tomato object detection for each frame is conducted by YOLOv3 model, and the continuous instance tracking between frames is performed by Kalman filter and probability model. We have verified the performance of the proposed method, an experiment was shown a good result in real-world test data.
Keywords
Tomato Instance Tracking Algorithm; Object Detection; Kalman Filter; Probability Robot; Automatic Harvesting Robot; Smart Farm;
Citations & Related Records
연도 인용수 순위
  • Reference
1 O. O. Arjenaki, P. A. Moghaddam, and A. M. Motlagh, "Online tomato sorting based on shape, maturity, size, and surface defects using machine vision," Turkish J. Agric. For., vol. 37, pp. 62-68, 2013, DOI: 10.3906/tar-1201-10.   DOI
2 M. P. Arakeri and Lakshmana, "Computer Vision Based Fruit Grading System for Quality Evaluation of Tomato in Agriculture industry," Procedia Comput. Sci., vol. 79, pp. 426-433, 2016, DOI: 10.1016/j.procs.2016.03.055.   DOI
3 C. Li, Q. Cao, and F. Guo, "A method for color classification of fruits based on machine vision," WSEAS Trans. Syst., vol. 8, no. 2, 2009, [Online], https://dl.acm.org/doi/10.5555/1558836.1558849.
4 N. El-Bendary, E. El Hariri, A. E. Hassanien, and A. Badr, "Using machine learning techniques for evaluating tomato ripeness," Expert Syst. Appl., vol. 42, no. 4, pp. 1892-1905, 2015, DOI: 10.1016/j.eswa.2014.09.057.   DOI
5 Y.-Y. Zheng, J.-L. Kong, X.-B. Jin, X.-Y. Wang, T.-L. Su, and M. Zuo, "CropDeep: The Crop Vision Dataset for Deep-Learning-Based Classification and Detection in Precision Agriculture," Sensors, vol. 19, no. 5, p. 1058, Mar. 2019, DOI: 10.3390/s19051058.   DOI
6 K. E. Ko, S.-H. Yang, and I. Jang, "Real-time tomato ripeness classification system based on deep learning model for object detection," J. Inst. Control. Robot. Syst., vol. 24, no. 11, pp. 999-1004, 2018, DOI: 10.5302/J.ICROS.2018.18.0166.   DOI
7 N. Goel and P. Sehgal, "Fuzzy classification of pre-harvest tomatoes for ripeness estimation {\textendash} An approach based on automatic rule learning using decision tree," Appl. Soft Comput. J., vol. 36, pp. 45-56, 2015, DOI: 10.1016/j.asoc.2015.07.009.   DOI
8 G. L. Grinblat, L. C. Uzal, M. G. Larese, and P. M. Granitto, "Deep learning for plant identification using vein morphological patterns," Comput. Electron. Agric., vol. 127, no. C, pp. 418-424, 2016, DOI: 10.1016/j.compag.2016.07.003.   DOI
9 G. Liu, S. Mao, and J. H. Kim, "A mature-tomato detection algorithm using machine learning and color analysis," Sensors (Switzerland), vol. 19, no. 9, May, 2019, DOI: 10.3390/s19092023.   DOI
10 A. Singh, B. Ganapathysubramanian, A. K. Singh, and S. Sarkar, "Machine Learning for High-Throughput Stress Phenotyping in Plants," Trends Plant Sci., vol. 21, no. 2, pp. 110-124, 2016, DOI: 10.1016/j.tplants.2015.10.015.   DOI
11 J. Redmon and A. Farhadi, "YOLOv3: An Incremental Improvement," arXiv:1804.02767, 2018, [Online], https://arxiv.org/abs/1804.02767.
12 G. Welch and G. Bishop, "The Discrete Kalman Filter," An Introduction to the Kalman Filter, University of North Carolina at Chapel Hill, 1995, pp. 2-7, [Online], https://courses.cs.washington.edu/courses/cse571/03wi/notes/welch-bishop-tutorial.pdf
13 S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: towards real-time object detection with region proposal networks," IEEE Trans. Pattern Anal. Mach. Intell., no. 6, pp. 1137-1149, 2017, DOI: 10.1109/TPAMI.2016.2577031.   DOI
14 J. Redmon and A. Farhadi, "YOLO9000: better, faster, stronger," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 7263-7271, 2017, DOI: 10.1109/CVPR.2017.690.   DOI
15 K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp. 770-778, DOI: 10.1109/CVPR.2016.90.   DOI
16 J. Redmon, Darknet: Open source neural networks in c, 2013, [Online], https://pjreddie.com/darknet.
17 S. Thrun, "Probabilistic robotics," Commun. ACM, vol. 45, no. 3, pp. 52-57, Mar., 2002, DOI: 10.1145/504729.504754.   DOI