KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Digital Mirror System with Machine Learning and Microservices

머신 러닝과 Microservice 기반 디지털 미러 시스템

  • Received : 2020.05.21
  • Accepted : 2020.07.30
  • Published : 2020.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Mirror is a physical reflective surface, typically of glass coated with a metal amalgam, and it is to reflect an image clearly. They are available everywhere anytime and become an essential tool for us to observe our faces and appearances. With the advent of modern software technology, we are motivated to enhance the reflection capability of mirrors with the convenience and intelligence of realtime processing, microservices, and machine learning. In this paper, we present a development of Digital Mirror System that provides the realtime reflection functionality as mirror while providing additional convenience and intelligence including personal information retrieval, public information retrieval, appearance age detection, and emotion detection. Moreover, it provides a multi-model user interface of touch-based, voice-based, and gesture-based. We present our design and discuss how it can be implemented with current technology to deliver the realtime mirror reflection while providing useful information and machine learning intelligence.

거울은 일반적으로 아말감으로 코팅된 물리적 반사 표면으로 거울 앞의 상을 선명하게 반사한다. 이것은 언제 어디서나 사용이 가능하며 사용자의 얼굴이나 외모를 확인하기 위한 필수적인 도구이다. 현대 소프트웨어 기술의 출현으로 사람들은 실시간 처리, Microservice 및 머신 러닝이 적용된 편의성과 지능성을 통해 거울 반사 기능을 향상시킬 수 있다. 본 논문에서는 거울로써 실시간 반영과 동시에 사용자 맞춤 정보 조회, 공공 정보 조회, 외모를 통한 나이와 감정 탐지 등의 기능을 가진 디지털 거울 시스템 개발을 제안한다. 더불어, 본 시스템은 터치 기반, 음성 인식 기반, 제스처 기반의 Multi-Modal 사용자 인터페이스를 제공한다. 본 논문에서는 이 시스템에 대한 디자인을 제시하고 현재 기술을 이용하여 실시간 거울 반영과 동시에 유용한 정보 제공 및 지능형 머신 러닝 기술을 제공하는 구현 방법을 제안한다.

Keywords

References

  1. B. Cvetkoska, N. Marina, D. C. Bogatinoska, and Z. Mitreski, "Smart mirror E-health assistant - Posture analyze algorithm proposed model for upright posture," In proceedings of IEEE EUROCON 2017. -17th International Conference on Smart Technologies, Ohrid, Macedonia, pp. 507-512, Jul. 2017.
  2. K. Jin, X. Deng, Z. Huang, and S. Chen, "Design of the smart mirror based on raspberry PI," In Proceedings of 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC 2018), Xi'an, China, pp. 1919-1923, May 2018.
  3. Y. Sun, L. Geng, and K. Dan, "Design of smart mirror based on raspberry PI," In Proceedings of International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS 2018), Xiamen, China, pp. 77-80, Jan. 2018.
  4. M. M. Yusri, S. Kasim, R. Hassan, Z. Abdullah, H. Ruslai, K. Jahidin, and M. S. Arshad, "Smart mirror for smart life," 6th ICT International Student Project Conference (ICTISPC 2017), Skudai, Malaysia, pp. 1-5, May 2017.
  5. V. Singh and D. Singh, "Smart Interactive Mirror Display," In Proceedings of International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon 2019), Faridabad, India, pp. 140-145, Feb. 2019.
  6. P. Mathivanan, G. Anbarasan, A. Sakthivel, and G. Selvam, "Home automation using smart mirror," In Proceedings of IEEE International Conference on System, Computation, Automation and Networking (ICSCAN), Pondicherry, India, pp. 1-4, Mar. 2019.
  7. R. M. A. Latif, L. B. Imran, M. Farhan, M. J. Bah, G. Ali, and Y. A. Abid, "Real-time simulation of IoT based smart home live mirror using WSN," In Proceedings of International Conference on Frontiers of Information Technology (FIT 2019), Islamabad, Pakistan, pp. 49-54, Dec. 2019.
  8. O. Gomez-Carmona and D. Casado-Mansilla, "SmiWork: An interactive smart mirror platform for workplace health promotion," In Proceedings of 2nd International Multidisciplinary Conference on Computer and Energy Science (SpliTech 2017), Split, Croatia, pp. 1-6, Jul. 2017.
  9. W. Dai, P. Wang, W. Sun, X. Wu, H. Zhang, V. Vyatkin, and G. Yang, "Semantic integration of plug-and-play software components for industrial edges based on microservices," IEEE Access, Vol. 7, pp. 125882-125892, Aug. 2019. https://doi.org/10.1109/ACCESS.2019.2938565
  10. J. Innerbichler, S. Gonul, V. Damjanovic-Behrendt, B. Mandler, and F. Strohmeier, "NIMBLE collaborative platform: Microservice architectural approach to federated IoT," In Proceedings of Global Internet of Things Summit (GIoTS 2017), Geneva, Switzerland, pp. 1-6, Jun. 2017.
  11. S. K. Datta and . Bonnet, "Next-generation, data centric and end-to-end iot architecture based on microservices," In Proceedings of IEEE International Conference on Consumer Electronics - Asia (ICCE-Asia 2018), Jeju, South Korea, pp. 1-4, Jun. 2018.
  12. M. A. Jarwar, S. Ali, and I. Chong, "Exploring Web Objects enabled Data-Driven Microservices for E-Health Service Provision in IoT Environment," In Proceedings of International Conference on Information and Communication Technology Convergence (ICTC 2018), pp. 112-117, Oct. 2018.
  13. M. De Sanctis, H. Muccini, and K. Vaidhyanathan, "Datadriven adaptation in microservice-based IoT architectures," In Proceedings of IEEE International Conference on Software Architecture Companion (ICSA-C 2020), Salvador, Brazil, pp. 59-62, Mar. 2020.
  14. M. A. A. Mostafa and A. M. Khater, "Horizontal offloading mechanism for IoT application in fog computing using microservices case study: Traffic management system," In Proceedings of IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT 2019), Amman, Jordan, pp. 640-647, Apr. 2019.
  15. E. Al-Masri, "Enhancing the microservices architecture for the internet of things," In Proceedings of IEEE International Conference on Big Data (Big Data 2018), WA, USA, pp. 5119-5125, Dec. 2018.
  16. C. Lai, F. Boi, A. Buschettu, and R. Caboni, "IoT and microservice architecture for multimobility in a smart city," In Proceedings of 7th International Conference on Future Internet of Things and Cloud (FiCloud 2019), Istanbul, Turkey, pp. 238-242, Aug. 2019.
  17. W. Kang and D. Kim, "DeepRT: A predictable deep learning inference framework for IoT devices," In Proceedings of IEEE/ACM Third International Conference on Internet-of-Things Design and Implementation (IoTDI 2018), FL, USA, pp. 279-280, Apr. 2018.
  18. J. Tang, D. Sun, S. Liu, and J. L. Gaudiot, “Enabling deep learning on IoT devices,” Computer, Vol. 50, No. 10, pp. 92-96, Oct. 2017. https://doi.org/10.1109/MC.2017.3641648
  19. S. Durga, R. Nag, and E. Daniel, "Survey on machine learning and deep learning algorithms used in internet of things (IoT) healthcare," In Proceedings of 3rd International Conference on Computing Methodologies and Communication (ICCMC 2019), Erode, India, pp. 1018-1022, Mar. 2019.
  20. M. Song, K. Zhong, J. Zhang, Y. Hu, D. Liu, W. Zhang, J. Wang, and T. Li, "In-Situ AI: Towards autonomous and incremental deep learning for IoT systems," IEEE International Symposium on High Performance Computer Architecture (HPCA 2018), Vienna, Austria, pp. 92-10, Feb. 2018.
  21. M., Jin, K., Park, and M. Song, "An intelligent CMOS image sensor with a deep learning algorithm for smart internet of things," In Proceedings of 34th International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC 2019), Jeju, South Korea, pp. 1-2, Jun. 2019.
  22. S. Yao, Y. Zhao, A. Zhang, S. Hu, H. Shao, C. Zhang, L. Su, and T. Abdelzaher, “Deep learning for the internet of things,” Computer, Vol. 51, No. 5, pp. 32-41, May 2018. https://doi.org/10.1109/mc.2018.2381131
  23. J. S. Shin, U. T. Kim, D. K. Lee, S. J. Park, S. J. Oh, and T. J. Yun, "Real-time vehicle detection using deep learning scheme on embedded system," In Proceedings of Ninth International Conference on Ubiquitous and Future Networks (ICUFN 2017), Milan, Italy, pp. 272-274, Jul. 2017.
  24. D. Gold and D. Sollinger, "SmartReflect: A modular smart mirror application platform," In Proceedigs of IEEE 7th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON 2016), Vancouver, Canada, pp. 1-7, Oct. 2016.
  25. M. Ghazal, T. Al Hadithy, Y. Al Khalil, M. Akmal, and H. Hajjdiab, "A mobile-programmable smart mirror for ambient IoT environments," In Proceedings of 5th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW 2017), Prague, Czech Republic, pp. 240-245, Aug. 2017.
  26. A. Johri, S. Jafri, R. N. Wahi, and D. Pandey, "Smart mirror: A time-saving and affordable assistant," In Proceedings of 4th International Conference on Computing Communication and Automation (ICCCA 2018), Greater Noida, India, pp. 1-4, Dec. 2018.
  27. A. C. Njaka, N. Li, and L. Li, "Voice controlled smart mirror with multifactor authentication," In Proceedings of IEEE International Smart Cities Conference (ISC2 2018), MO., USA, pp. 1-8, Sept. 2018.
  28. R. Nadaf, and V. Bonal, "Smart mirror using raspberry pi as a security and vigilance system," In Proceedings of 3rd International Conference on Trends in Electronics and Informatics (ICOEI 2019), Tirunelveli, India, pp. 360-365, Apr. 2019.
  29. P. Henriquez, B. J. Matuszewski, Y. Andreu-Cabedo, L. Bastiani, S. Colantonio, G. Coppini, M. D'Acunto, R. Favilla, D. Germanese, D. Giorgi, P. Marraccini, M. Martinelli, M.-A. Morales, M. A. Pascali, M. Righi, O. Salvetti, M. Larsson, T. Stromberg, L. Randeberg, A. Bjorgan, G. Giannakakis, M. Pediaditis, F. Chiarugi, E. Christinaki, K. Marias, and M. Tsiknakis, "Mirror mirror on the wall... an unobtrusive intelligent multisensory mirror for well-being status self-assessment and visualization," IEEE Transactions on Multimedia, Vol. 19, No. 7, pp. 1467-1481, Jul. 2017. https://doi.org/10.1109/TMM.2017.2666545
  30. K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint face detection and alignment using multi-task cascaded convolutional networks,” IEEE Signal Processing Letters, Vol. 23, No. 10, pp. 1499-1503, Oct. 2016. https://doi.org/10.1109/LSP.2016.2603342
  31. F. Schroff, D. Kalenichenko, and J. Philbin, "FaceNet: A unified embedding for face recognition and clustering," In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), MA, USA, pp. 815-823, Jun. 2015.
  32. R. Rothe, R. Timofte, and L. Van Gool, "Deep expectation of real and apparent age from a single image without facial landmarks," International Journal of Computer Vision, Vol. 126, pp. 144-157, Apr. 2018. https://doi.org/10.1007/s11263-016-0940-3
  33. Z. Hu, Y. Wen, J. Wang, M. Wang, R. Hong, and S. Yan, “Facial Age Estimation With Age Difference,” IEEE Transactions on Image Processing, Vol. 26, No. 7, pp. 3087-3097, Jul. 2017. https://doi.org/10.1109/TIP.2016.2633868
  34. K. H. Liu and T. J. Liu, “A structure-based human facial age estimation framework under a constrained condition,” IEEE Transactions on Image Processing, Vol. 28, No. 10, pp. 5187-5200, Oct. 2019. https://doi.org/10.1109/TIP.2019.2916768
  35. R., Jana and A. Basu, "Automatic age estimation from face image," In Proceedings of International Conference on Innovative Mechanisms for Industry Applications (ICIMIA 2017), Bangalore, India, pp. 87-90, Feb. 2017.
  36. M. T. B. Iqbal, M. Shoyaib, B. Ryu, M. Abdullah-Al-Wadud, and O. Chae, “Directional age-primitive pattern (DAPP) for human age group recognition and age estimation,” IEEE Transactions on Information Forensics and Security, Vol. 12, No. 11, pp. 2505-2517, Nov. 2017. https://doi.org/10.1109/TIFS.2017.2695456
  37. A. M. Cadayona, N. M. S. Cerilla, D. M. M. Jurilla, A. K. D. Balan, and J. C. de Goma, "Emotional state classification: An additional step in emotion classification through face detection," In Proceedings of IEEE 6th International Conference on Industrial Engineering and Applications (ICIEA 2019), Tokyo, Japan, pp. 667-671, Apr. 2019.
  38. H. Zhang, A. Jolfaei, and M. Alazab, "A face emotion recognition method using convolutional neural network and image edge computing," IEEE Access, Vol. 7, pp. 159081-159089, Oct. 2019. https://doi.org/10.1109/ACCESS.2019.2949741
  39. M. S. Hossain and G. Muhammad, "An emotion recognition system for mobile applications," IEEE Access, Vol. 5, pp. 2281-2287, Feb. 2017. https://doi.org/10.1109/ACCESS.2017.2672829
  40. K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), NV, USA, pp. 770-778, Jun. 2016.