Journal of information and communication convergence engineering

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Real-Time CCTV Based Garbage Detection for Modern Societies using Deep Convolutional Neural Network with Person-Identification

  • Syed Muhammad Raza (Department of IT Convergence Engineering, Kumoh National Institute of Technology) ;
  • Syed Ghazi Hassan (Department of Computing and Information Sciences, Karachi Institute of Economics and Technology) ;
  • Syed Ali Hassan (Department of Bio Robotics, Scuola Superiore Sant'Anna) ;
  • Soo Young Shin (Department of IT Convergence Engineering, Kumoh National Institute of Technology)
  • Received : 2023.03.08
  • Accepted : 2023.09.27
  • Published : 2024.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Trash or garbage is one of the most dangerous health and environmental problems that affect pollution. Pollution affects nature, human life, and wildlife. In this paper, we propose modern solutions for cleaning the environment of trash pollution by enforcing strict action against people who dump trash inappropriately on streets, outside the home, and in unnecessary places. Artificial Intelligence (AI), especially Deep Learning (DL), has been used to automate and solve issues in the world. We availed this as an excellent opportunity to develop a system that identifies trash using a deep convolutional neural network (CNN). This paper proposes a real-time garbage identification system based on a deep CNN architecture with eight distinct classes for the training dataset. After identifying the garbage, the CCTV camera captures a video of the individual placing the trash in the incorrect location and sends an alert notice to the relevant authority.

Keywords

Acknowledgement

This work was supported by the Priority Research Centers Program through the National Research Foundation of Republic of Korea (NRF), funded by the Ministry of Education, Science, and Technology (2018R1A6A1A03024003). This research was supported by the MSIT (Ministry of Science and ICT), Republic of Korea, under the ITRC (Information Technology Research Center) support program (IITP-2023-RS-2023-00259061) supervised by the IITP (Institute for Information and Communications Technology Planning and Evaluation (IITP).

References

  1. Air Index Quality Report for annual garbage collection, IQAir [Online]. Available: https://www.iqair.com/pakistan. (Accessed 08 December 2021)
  2. Y. Yu, "A computer vision-based detection system for trash bins identification during trash classification," in Journal of Physics: Conference Series, Lanzhou, China, p. 012015, 2020. DOI: 10.1088/1742-6596/1617/1/012015.
  3. S. A. Hassan, T. Rahim, and S. Y. Shin, "Real-time uav detection based on deep learning network," in 2019 International Conference on Information and Communication Technology Convergence (ICTC), Jeju, Korea, pp. 630-632, 2019. DOI: 10.1109/ICTC46691.2019.8939564.
  4. S. A. Hassan, T. Rahim, and S. Y. Shin, "An improved deep convolutional neural network-based autonomous road inspection scheme using unmanned aerial vehicles," Electronics, vol. 10, no. 22, p. 2764, Nov. 2021. DOI: 10.3390/electronics10222764.
  5. M. Yang and G. Thung, "Classification of trash for recyclability status," CS229 project report, vol. 2016, no. 1, 2016.
  6. J. J. Philipps, I. Bonninger, M. Weigert, and J. Vasquez, "Automatic tracking and counting of moving objects," in 3rd IEEE International Work-Conference on Bioinspired Intelligence, Liberia, Costa Rica, pp. 93-97, 2014. DOI: 10.1109/IWOBI.2014.6913945.
  7. R. A. M. Budiman, B. Achmad, A. Arif, and L. Zharif, "Localization of white blood cell images using Haar cascade classifiers," in 2016 1st International Conference on Biomedical Engineering (IBIOMED), Yogyakarta, Indonesia, pp. 1-5, 2016. DOI: 10.1109/IBIOMED.2016.7869822.
  8. J. Donovan, "Auto-trash sort's garbage automatically at the-techcrunch- drisrupt hackathon," techcrunch [Online]. Available: https://techcrunch.com/2016/09/13/auto-trash-sorts-garbage-automaticallyat-the-techcrunch-disrupt-hackathon/40.
  9. U. Ozkaya and L. Seyfi, "Fine-tuning models comparisons on garbage classification for recyclability," arXiv preprint arXiv:1908.04393, DOI: 10.48550/arXiv.1908.0439.
  10. Y. Wang, and X. Zhang, "Autonomous garbage detection for intelligent urban management", in MATEC Web of Conferences, Vol. 232, p. 01056, 2018. DOI: 10.1051/matecconf/201823201056.
  11. K. Yun, Y. Kwon, S. Oh, J. Moon, and J. Park, "Vision-based garbage dumping action detection for real-world surveillance platform", ETRI Journal, 41(4), 494-505. DOI: 10.4218/etrij.2018-0520.
  12. S. Hulyalkar, R. Deshpande, K. Makode, and S. Kajale, "Implementation of smartbin using convolutional neural networks", International Research Journal of Engineering and Technology (IRJET), 5(4), 2018.
  13. K. Tarun, K. Sreelakshmi, and K. P. Peeyush, "Segregation of plastic and non-plastic waste using convolutional neural network", in 2019 IOP Conference Series: Materials Science and Engineering, Vol. 561, No. 1, p. 012113, 2019. DOI: 10.1088/1757-899X/561/1/012113.
  14. M. Valente, H. Silva, J. M. Caldeira, V. N. Soares, and P. D Gaspar, "Detection of waste containers using computer vision." Applied System Innovation 2, no. 1, vol 2019: 11.12. DOI: 10.3390/asi2010011.
  15. A. Pervez, Y. Mao, and D. Lee, "Learning deep movement primitives using convolutional neural networks," 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), Birmingham, UK, 2017, pp. 191-197, DOI: 10.1109/HUMANOIDS.2017.8246874.
  16. J. Bai, S. Lian, Z. Liu, K. Wang, and D. Liu, "Deep Learning Based Robot for Automatically Picking Up Garbage on the Grass," in IEEE Transactions on Consumer Electronics, vol. 64, no. 3, pp. 382-389, Aug. 2018, DOI: 10.1109/TCE.2018.2859629.
  17. S. L. Rabano, M. K. Cabatuan, E. Sybingco, E. P. Dadios and E. J. Calilung, "Common Garbage Classification Using MobileNet," 2018 IEEE 10th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management (HNICEM), Baguio City, Philippines, 2018, pp. 1-4, doi: 10.1109/HNICEM.2018.8666300.
  18. Y. Chu, C. Huang, X. Xie, B. Tan, S. Kamal, and X. Xiong, "Multilayer hybrid deep-learning method for waste classification and recycling". Computational Intelligence and Neuroscience, 2018. DOI: 10.1155/2018/5060857.
  19. J. Sousa, A. Rebelo and J. S. Cardoso, "Automation of Waste Sorting with Deep Learning," 2019 XV Workshop de Visao Computacional (WVC), Sao Bernardo do Campo, Brazil, 2019, pp. 43-48, DOI: 10.1109/WVC.2019.8876924.
  20. S. Meng and W. -T. Chu, "A Study of Garbage Classification with Convolutional Neural Networks," 2020 Indo-Taiwan 2nd International Conference on Computing, Analytics and Networks (Indo-Taiwan ICAN), Rajpura, India, 2020, pp. 152-157, DOI: 10.1109/Indo-TaiwanICAN48429.2020.9181311.
  21. D. Zeng, S. Zhang, F. Chen and Y. Wang, "Multi-Scale CNN Based Garbage Detection of Airborne Hyperspectral Data," in IEEE Access, vol. 7, pp. 104514-104527, 2019, DOI: 10.1109/ACCESS.2019.2932117.
  22. P. Zhang, Q. Zhao, J. Gao, W. Li and J. Lu, "Urban Street Cleanliness Assessment Using Mobile Edge Computing and Deep Learning," in IEEE Access, vol. 7, pp. 63550-63563, 2019, DOI: 10.1109/ACCESS.2019.2914270.
  23. M. Tharani, A. W. Amin, M. Maaz, and M. Taj, "Attention neural network for trash detection on water channels," arXiv preprint arXiv:2007.04639, DOI: 10.48550/arXiv.2007.04639.
  24. O. Awe, R. Mengistu, and V. Sreedhar, "Smart trash net: Waste localization and classification," arXiv preprint.
  25. Y. H. Liao, and J. G. Juang, "Real-time UAV trash monitoring system," Applied Sciences, 12, no. 4 (2022): 183. DOI: 10.3390/app12041838.
  26. I. Salimi, B. S. Bayu Dewantara and I. K. Wibowo, "Visual-based trash detection and classification system for smart trash bin robot," 2018 International Electronics Symposium on Knowledge Creation and Intelligent Computing (IES-KCIC), Bali, Indonesia, 2018, pp. 378-383, DOI: 10.1109/KCIC.2018.8628499.
  27. P. F. Proenca, and P. T. Simoes, "Trash annotations in context for litter detection," arXiv preprint arXiv:2003.06975, DOI: 10.48550/arXiv.2003.06975.
  28. Pedropro/TACO, Trash Annotations in Context Dataset Toolkit., [Online]. Available: https://github.com/pedropro/TACO (accessed on 21 March 2021).
  29. Y. Liu, Z. Ge, G. Lv, and S. Wang, "Research on automatic garbage detection system based on deep learning and narrowband internet of things," in Journal of Physics: Conference Series, vol. 1069, no. 1, p. 012032. IOP Publishing, 2018, August 2018. DOI: 10.1088/1742-6596/1069/1/012032.
  30. J. Redmon and A. Farhadi, "YOLO9000: Better, Faster, Stronger," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 6517-6525, DOI: 10.1109/CVPR.2017.690.
  31. G. Niu, J. Li, S. Guo, M. -O. Pun, L. Hou and L. Yang, "SuperDock: A Deep Learning-Based Automated Floating Trash Monitoring System," 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), Dali, China, 2019, pp. 1035-1040, DOI: 10.1109/ROBIO49542.2019.8961509.
  32. J. Redmon and A. Farhadi, "Yolov3: An incremental improvement", arXiv preprint arXiv:1804.02767, DOI: 10.48550/arXiv.1804.02767
  33. P. Viola, and M. J. Jones, "Robust real-time face detection," International journal of computer vision, 57, 137-154, 2004. DOI: 10.1023/B: VISI.0000013087. 49260.fb.
  34. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Fu, and A.C. Berg, "Ssd: Single shot multibox detector", in Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I 14 pp. 21-37. DOI: 10.1007/978-3-319-46448-0_2.
  35. J. Redmon and A. Farhadi, "Yolov3: An incremental improvement", arXiv preprint arXiv:1804.02767, DOI: 10.48550/arXiv.1804.02767
  36. J. Redmon, S. Divvala, R. Girshick and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 779-788, DOI: 10.1109/CVPR.2016.91.
  37. R. J. Wang, X. Li, C. X. Ling, "Pelee: A real-time object detection system on mobile devices", Advances in neural information processing systems, 31, 2018.
  38. R. A. Aral, S. R. Keskin, M. Kaya, and M. Haciomeroglu, "Classification of TrashNet Dataset Based on Deep Learning Models," 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA, 2018, pp. 2058-2062, DOI: 10.1109/BigData.2018.8622212.
  39. G. White, C. Cabrera, A. Palade, F.Li, S. Clarke, "WasteNet: Waste classification at the edge for smart bins", arXiv preprint arXiv: 2006.05873, DOI: 10.48550/arXiv.2006.05873.
  40. H. N. Kulkarni, N. K. S. Raman, "Waste object detection and classification", CS230 Stanford, 2019.
  41. A. V. Seredkin, M. P. Tokarev, I. A. Plohih, O. A. Gobyzov and D. M. Markovich, "Development of a method of detection and classification of waste objects on a conveyor for a robotic sorting system", in Journal of Physics: Conference Series, vol. 1359, No. 1, p. 012127. DOI: 10.1088/1742-6596/1359/1/012127.
  42. O. Awe, R. Menugistu, V. Sreedhar, "Smart trash net: Waste localization and classification", arXiv preprint, 2017.
  43. J. Knowles, S. Kennedy, T. Kennedy, "OscarNet: Using transfer learning to classify disposable waste", CS230 Report: Deep Learning. Stanford University, CA, Winter, 2018.
  44. H. N. Kulkarni, N. K. S. Raman, "Waste object detection and classification", CS230 Stanford, 2019.
  45. C. Bircanoglu, M. Atay, F. Beser, O. Genc and M. A. Kizrak, "RecycleNet: Intelligent Waste Sorting Using Deep Neural Networks," 2018 Innovations in Intelligent Systems and Applications (INISTA), Thessaloniki, Greece, 2018, pp. 1-7, DOI: 10.1109/INISTA.2018.8466276.
  46. B. S. Costa, A. C. Bernardes, J. V. Pereira, V. H. Zampa, V. A. Pereira, G. F. Matos, ... and A. F. Silva, "Artificial intelligence in automated sorting in trash recycling", in Anais do XV Encontro Nacional de Inteligencia Artificial e Computacional, pp. 198-205, 2018, October. DOI: 10.5753/eniac.2018.4416.
  47. O. Adedeji, and Z. Wang, "Intelligent waste classification system using deep learning convolutional neural network", Procedia Manufacturing, 35, 607-612, 2019. DOI: 10.1016/j.promfg.2019.05.086.
  48. J. Sousa, A. Rebelo and J. S. Cardoso, "Automation of Waste Sorting with Deep Learning," 2019 XV Workshop de Visao Computacional (WVC), Sao Bernardo do Campo, Brazil, 2019, pp. 43-48, DOI: 10.1109/WVC.2019.8876924.
  49. X. Xu, X. Qi, and X. Diao, "Reach on waste classification and identification by transfer learning and lightweight neural network", Preprints 2020, 2020020327.
  50. S. H. Lee, C. H. Yeh, T. W. Hou, and C. S. Yang, "A lightweight neural network based on AlexNet-SSD model for garbage detection", in Proceedings of the 2019 3rd High Performance Computing and Cluster Technologies Conference, pp. 274-278, June 2019. DOI: 10.1145/3341069.3341087.
  51. Y. Chu, C. Huang, X. Xie, B. Tan, S. Kamal, and X. Xiong, "Multilayer hybrid deep-learning method for waste classification and recycling", Computational Intelligence and Neuroscience, 2018, 2018. DOI: 10.1155/2018/5060857.
  52. P. L. Mazzeo, A. Argentieri, F. De Luca, P. Spagnolo, C. Distante, M. Leo, and P. Carcagni, "Convolutional neural networks for recognition and segmentation of aluminum profiles", in Multimodal Sensing: Technologies and Applications, vol. 11059, pp. 219-229, June 2019. DOI: 10.1117/12.2525687.