Journal of the Korea Academia-Industrial cooperation Society (한국산학기술학회논문지)

The Korea Academia-Industrial cooperation Society (한국산학기술학회)
권호 표지
DOI QR코드

DOI QR Code

Display of Irradiation Location of Ultrasonic Beauty Device Using AR Scheme

증강현실 기법을 이용한 초음파 미용기의 조사 위치 표시

  • Kang, Moon-Ho (Department of Information and Communication Engineering, Sunmoon University)
  • 강문호 (선문대학교 정보통신학과)
  • Received : 2020.05.27
  • Accepted : 2020.09.04
  • Published : 2020.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, for the safe use of a portable ultrasonic skin-beauty device, an android app was developed to show the irradiation locations of focused ultrasound to a user through augmented reality (AR) and enable stable self-surgery. The utility of the app was assessed through testing. While the user is making a facial treatment with the beauty device, the user's face and the ultrasonic irradiation location on the face are detected in real-time with a smart-phone camera. The irradiation location is then indicated on the face image and shown to the user so that excessive ultrasound is not irradiated to the same area during treatment. To this end, ML-Kit is used to detect the user's face landmarks in real-time, and they are compared with a reference face model to estimate the pose of the face, such as rotation and movement. After mounting a LED on the ultrasonic irradiation part of the device and operating the LED during irradiation, the LED light was searched to find the position of the ultrasonic irradiation on the smart-phone screen, and the irradiation position was registered and displayed on the face image based on the estimated face pose. Each task performed in the app was implemented through the thread and the timer, and all tasks were executed within 75 ms. The test results showed that the time taken to register and display 120 ultrasound irradiation positions was less than 25ms, and the display accuracy was within 20mm when the face did not rotate significantly.

본 연구에서는 휴대용 초음파 피부 미용기 사용시 집속 초음파의 조사 위치를 증강현실 (Augmented Reality: AR) 기법을 통해 사용자에게 보여주어 안전하게 셀프시술을 하도록 하는 안드로이드 앱을 개발하고 시험을 통해 유용성을 보인다. 사용자가 초음파 미용기로 얼굴 부위를 시술하는 동안에 스마트폰 카메라를 통해 사용자의 얼굴과 얼굴위의 초음파 조사 위치를 실시간 검지한 후, 얼굴 영상위에 조사 위치를 표시하여 사용자에게 보여줌으로서, 초음파가 동일한 부위에 과도하게 중복되어 조사되지 않도록 한다. 이를 위해 ML-Kit를 이용하여 사용자 얼굴의 랜드마크(landmark)들을 실시간 검지하고 얼굴형상 기준 모델과 비교하여 얼굴의 회전과 이동 등의 자세를 추정한다. 미용기의 초음파 조사부위에 LED를 장착하고 조사 중에 작동시킨 후, LED의 불빛의 위치를 탐색하여 스마트폰 화면상의 초음파 조사 위치를 알아내고, 추정된 자세정보를 토대로 얼굴 영상위에 조사 위치를 정합시켜 표시한다. 앱에서 수행되는 각 작업들을 스레드와 타이머를 통해 구현하여 전체 작업이 75ms 이내에서 실행된다. 시험 결과, 120개의 초음파 조사 위치를 정합하고 표시하는 데 걸린 시간은 25ms 이하이고, 얼굴이 크게 회전하지 않는 경우 표시 정확도는 20mm 이내 임을 알 수 있다.

Keywords

References

  1. G. T. Haar and C. Coussios, "High intensity focused ultrasound: Physical principles and devices", International Journal of Hyper-thermia, vol. 23, no. 2, pp. 89-104, 2007. DOI: https://doi.org/10.1109/LSP.2014.2338911
  2. D. Schmalstieg and T. Hollerer, Augmented Reality, Principles and Practice, Addison-Wesley, pp. 78-84, 2016.
  3. S. Aukstakalnis, Practical Augmented Reality: A Guide to the Technologies, Applications, and Human Factors for AR and VR, Addison-Wesley, pp. 227-329, 2016.
  4. Karthkeyan NG, Machine Learning Projects for Mobile Applications, pp. 85-108, Packt, 2018.
  5. A. Kumar et al., "Face detection techniques: a review", Artificial Intelligence Review, 52, pp. 927-948, 2019. DOI: https://doi.org/10.1007/s10462-018-9650-2
  6. J. Shin and D. Kim, "Hybrid Approach for Facial Feature Detection and Tracking under Occlusion", IEEE Signal Processing Letters, vol. 21, no. 12, pp. 1486-1490, Dec. 2014. DOI: https://doi.org/10.1109/LSP.2014.2338911
  7. D. Han et al., "Design and Implementation of Real-time High Performance Face Detection Engine", The Institute of Electronics Engineers of Korea-SP, vol. 47, no. 2, pp. 33-44, 2010.
  8. P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features", Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 511-518, 2001. DOI: https://doi.org/10.1109/LSP.2014.2338911
  9. D. Garg et al., "A Deep Learning Approach for Face Detection using YOLO", IEEE Punecon, pp. 1-4, 2018. DOI: https://doi.org/10.1109/LSP.2014.2338911
  10. A. Rosebrock, Facial landmarks with dlib, OpenCV, and Python, https://www.pyimagesearch.com/2017/04/03/facial-la ndmarks-dlib-opencv-python/ 2017.
  11. R. Laganiere, OpenCV Computer Vision Application Programming Cookbook, 2nd Edition, Packt, pp. 281-313, 2014.