International Journal of Computer Science & Network Security

International Journal of Computer Science & Network Security (국제컴퓨터통신보호논문지학회)
권호 표지
DOI QR코드

DOI QR Code

Image-based Soft Drink Type Classification and Dietary Assessment System Using Deep Convolutional Neural Network with Transfer Learning

  • Rubaiya Hafiz (Dept. of Computer Science & Engineering, Daffodil International University) ;
  • Mohammad Reduanul Haque (Dept. of Computer Science & Engineering, Daffodil International University) ;
  • Aniruddha Rakshit (Dept. of Computer Science & Engineering, Daffodil International University) ;
  • Amina khatun (Dept. of Computer Science & Engineering, Jahangirnagar University) ;
  • Mohammad Shorif Uddin (Dept. of Computer Science & Engineering, Jahangirnagar University)
  • Received : 2024.02.05
  • Published : 2024.02.29
Download PDF
⟨ Previous Next ⟩

Abstract

There is hardly any person in modern times who has not taken soft drinks instead of drinking water. The rate of people taking soft drinks being surprisingly high, researchers around the world have cautioned from time to time that these drinks lead to weight gain, raise the risk of non-communicable diseases and so on. Therefore, in this work an image-based tool is developed to monitor the nutritional information of soft drinks by using deep convolutional neural network with transfer learning. At first, visual saliency, mean shift segmentation, thresholding and noise reduction technique, collectively known as 'pre-processing' are adopted to extract the location of drinks region. After removing backgrounds and segment out only the desired area from image, we impose Discrete Wavelength Transform (DWT) based resolution enhancement technique is applied to improve the quality of image. After that, transfer learning model is employed for the classification of drinks. Finally, nutrition value of each drink is estimated using Bag-of-Feature (BoF) based classification and Euclidean distance-based ratio calculation technique. To achieve this, a dataset is built with ten most consumed soft drinks in Bangladesh. These images were collected from imageNet dataset as well as internet and proposed method confirms that it has the ability to detect and recognize different types of drinks with an accuracy of 98.51%.

Keywords

References

  1. Health Risks of Drinking Soft Drinks, 2019 (Accessed June 10, 2019). http://www.historyofsoftdrinks.com/soft-drinks-facts/health-e ects-of-soft-drinks/.
  2. V. S. Malik, Y. Li, A. Pan, L. De Koning, E. Schernhammer, W. C. Willett, and F. B. Hu, "Long-term consumption of sugar-sweetened and arti cially sweetened beverages and risk of mortality in us adults," Circulation, 2019.
  3. V. Shukla, Top 10 Most Obese Countries In The World According To WHO And OECD, January 14, 2019 (Accessed June 3, 2019). https://www.valuewalk.com/2019/01/top-10-most-obese-countries-oecd-who/.
  4. T. Lobstein, Reducing consumption of sugar-sweetened beverages to reduce the risk of childhood overweight and obesity, September 2014 (Accessed June 9, 2019). https://www.who.int/elena/titles/commentary/ssbschildhoodobesity=en/.
  5. D. Hyde, Remove sugary drinks from children's diets, health officials say, Jul 2015 (Accessed June 9, 2019). https://www.telegraph.co.uk/news/health/11745 806/Remove-sugary-drinks-from-childrens-diets-health-o cials-say.html.
  6. D. Moza arian, Food is medicine: How US policy is shifting toward nutrition for better health, January 2019 (Accessed June 9, 2019). https://theconversation.com/food-is-medicine-how-us-policy-is-shifting-toward-nutrition-for-better-health-107650.
  7. M. Haque et al., A Survey on Soft Drinks Intake Behaviour among University Going Students, PhD thesis, East West University, 2018.
  8. C. K. Martin, H. Han, S. M. Coulon, H. R. Allen, C. M. Champagne, and S. D. Anton, "A novel method to remotely measure food intake of free-living individuals in real time: the remote food photography method," British Journal of Nutrition, vol. 101, no. 3, pp. 446-456, 2008. https://doi.org/10.1017/S0007114508027438
  9. Y. He, C. Xu, N. Khanna, C. J. Boushey, and E. J. Delp, "Analysis of food images: Features and classification," in 2014 IEEE International Conference on Image Processing (ICIP), pp. 2744-2748, IEEE, 2014.
  10. S. Yang, M. Chen, D. Pomerleau, and R. Sukthankar, "Food recognition using statistics of pairwise local features," in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2249-2256, IEEE, 2010.
  11. M. Bosch, F. Zhu, N. Khanna, C. J. Boushey, and E. J. Delp, "Combining global and local features for food identification in dietary assessment," in 2011 18th IEEE International Conference on Image Processing, pp. 1789-1792, IEEE, 2011.
  12. F. Zhu, M. Bosch, N. Khanna, C. J. Boushey, and E. J. Delp, "Multilevel segmentation for food classi cation in dietary assessment," in 2011 7th International Symposium on Image and Signal Processing and Analysis (ISPA), pp. 337-342, IEEE, 2011.
  13. R. Hafiz, S. Islam, R. Khanom, and M. S. Uddin, "Image based drinks identi cation for dietary assessment," in 2016 International Workshop on Computational Intelligence (IWCI), pp. 192-197, IEEE, 2016.
  14. W. Rawat and Z. Wang, "Deep convolutional neural networks for image classi cation: A comprehensive review," Neural computation, vol. 29, no. 9, pp. 2352-2449, 2017. https://doi.org/10.1162/neco_a_00990
  15. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, pp. 1097-1105, 2012.
  16. A. Kolsch, M. Z. Afzal, M. Ebbecke, and M. Liwicki, "Real-time document image classification using deep CNN and extreme learning machines," in 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 1318-1323, IEEE, 2017.
  17. H. Kagaya, K. Aizawa, and M. Ogawa, "Food detection and recognition using convolutional neural network," in Proceedings of the 22nd ACM international conference on Multimedia, pp. 1085-1088, ACM, 2014.
  18. H. Kagaya and K. Aizawa, "Highly accurate food/non-food image classi cation based on a deep convolutional neural network," in International Conference on Image Analysis and Processing, pp. 350-357, Springer, 2015.
  19. S. Mezgec and B. Korousic Seljak, "Nutrinet: A deep learning food and drink image recognition system for dietary assessment," Nutrients, vol. 9, no. 7, p. 657, 2017.
  20. S. Mezgec, T. Eftimov, T. Bucher, and B. K. Seljak, "Mixed deep learning and natural language processing method for fake-food image recognition and standardization to help automated dietary assessment," Public health nutrition, vol. 22, no. 7, pp. 1193-1202, 2019.
  21. J. Harel, C. Koch, and P. Perona, "Graph-based visual saliency," in Advances in neural information processing systems, pp. 545-552, 2007.
  22. G. E. Suji, Y. Lakshmi, and G. W. Jiji, "Comparative study on image segmentation algo-rithms," International Journal of Advanced Computer Research, vol. 3, no. 3, pp. 400-405, 2013.
  23. M. Huang, L. Men, and C. Lai, "Accelerating mean shift segmentation algorithm on hybrid cpu/gpu platforms," in Modern Accelerator Technologies for Geographic Information Science, pp. 157-166, Springer, 2013.
  24. H. H. A. Kadouf and Y. M. Mustafah, "Colour-based object detection and tracking for autonomous quadrotor uav," in IOP Conference Series: Materials Science and Engineering, vol. 53, p. 012086, IOP Publishing, 2013. https://doi.org/10.1088/1757-899X/53/1/012086
  25. T. Rahman, M. R. Haque, L. J. Rozario, and M. S. Uddin, "Gaussian noise reduction in digital images using a modified fuzzy filter," in 2014 17th International Conference on Computer and Information Technology (ICCIT), pp. 217-222, IEEE, 2014.
  26. M. G. Khaire and R. Shelkikar, "Resolution enhancement of images with interpolation and dwt-swt wavelet domain components," International Journal of Application or Innovation in Engineering and Management, Vol2, 2013.
  27. P. Karunakar, V. Praveen, and O. R. Kumar, "Discrete wavelet transform-based satellite image resolution enhancement," Advance in Electronic and Electric Engineering, vol. 3, no. 4, pp. 405-412, 2013.
  28. J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, "How transferable are features in deep neural networks?," in Advances in neural information processing systems, pp. 3320-3328, 2014.
  29. K. Nogueira, O. A. Penatti, and J. A. dos Santos, "Towards better exploiting convolutional neural networks for remote sensing scene classification," Pattern Recognition, vol. 61, pp. 539- 556, 2017. https://doi.org/10.1016/j.patcog.2016.07.001
  30. S. Thrun, "Is learning the n-th thing any easier than learning the first?," in Advances in neural information processing systems, pp. 640-646, 1996.
  31. X. Tian, D. Tao, and Y. Rui, "Sparse transfer learning for interactive video search reranking," ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), vol. 8, no. 3, p. 26, 2012.
  32. N. Tajbakhsh, J. Y. Shin, S. R. Gurudu, R. T. Hurst, C. B. Kendall, M. B. Gotway, and J. Liang, "Convolutional neural networks for medical image analysis: Full training or ne tuning?," IEEE transactions on medical imaging, vol. 35, no. 5, pp. 1299-1312, 2016. https://doi.org/10.1109/TMI.2016.2535302
  33. H. Kaya, F. Gurp nar, and A. A. Salah, "Video-based emotion recognition in the wild using deep transfer learning and score fusion," Image and Vision Computing, vol. 65, pp. 66-75, 2017. https://doi.org/10.1016/j.imavis.2017.01.012
  34. K. Nguyen, C. Fookes, A. Ross, and S. Sridharan, "Iris recognition with o -the-shelf cnn features: A deep learning perspective," IEEE Access, vol. 6, pp. 18848-18855, 2018. https://doi.org/10.1109/ACCESS.2017.2784352
  35. Z. Chen, T. Zhang, and C. Ouyang, "End-to-end airplane detection using transfer learning in remote sensing images," Remote Sensing, vol. 10, no. 1, p. 139, 2018.
  36. M. Xie, N. Jean, M. Burke, D. Lobell, and S. Ermon, "Transfer learning from deep features for remote sensing and poverty mapping," in Thirtieth AAAI Conference on Artificial Intelligence, 2016.
  37. H. Bay, T. Tuytelaars, and L. Van Gool, "Surf: Speeded up robust features," in European conference on computer vision, pp. 404-417, Springer, 2006.
  38. S. O'Hara and B. A. Draper, "Introduction to the bag of features paradigm for image classification and retrieval," arXiv preprint arXiv:1101.3354, 2011.
  39. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image database," in 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255, Ieee, 2009.