• Title/Summary/Keyword: multi-modal

Search Result 631, Processing Time 0.023 seconds

A Conversational Interactive Tactile Map for the Visually Impaired (시각장애인의 길 탐색을 위한 대화형 인터랙티브 촉각 지도 개발)

  • Lee, Yerin;Lee, Dongmyeong;Quero, Luis Cavazos;Bartolome, Jorge Iranzo;Cho, Jundong;Lee, Sangwon
    • Science of Emotion and Sensibility
    • /
    • v.23 no.1
    • /
    • pp.29-40
    • /
    • 2020
  • Visually impaired people use tactile maps to get spatial information about their surrounding environment, find their way, and improve their independent mobility. However, classical tactile maps that make use of braille to describe the location within the map have several limitations, such as the lack of information due to constraints on space and limited feedback possibilities. This study describes the development of a new multi-modal interactive tactile map interface that addresses the challenges of tactile maps to improve the usability and independence of visually impaired people when using tactile maps. This interface adds touch gesture recognition to the surface of tactile maps and enables the users to verbally interact with a voice agent to receive feedback and information about navigation routes and points of interest. A low-cost prototype was developed to conduct usability tests that evaluated the interface through a survey and interview given to blind participants after using the prototype. The test results show that this interactive tactile map prototype provides improved usability for people over traditional tactile maps that use braille only. Participants reported that it was easier to find the starting point and points of interest they wished to navigate to with the prototype. Also, it improved self-reported independence and confidence compared with traditional tactile maps. Future work includes further development of the mobility solution based on the feedback received and an extensive quantitative study.

Integration of Component Image Information and Design Information by Graph to Support Product Design Information Reuse (제품 설계 정보 재사용을 위한 그래프 기반의 부품 영상 정보와 설계 정보의 병합)

  • Lee, Hyung-Jae;Yang, Hyung-Jeong;Kim, Kyoung-Yun;Kim, Soo-Hyung;Kim, Sun-Hee
    • The KIPS Transactions:PartD
    • /
    • v.13D no.7 s.110
    • /
    • pp.1017-1026
    • /
    • 2006
  • Recently, distributed collaborative development environment has been recognized an alternative environment for product development in which multidisciplinary participants are naturally involving. Reuse of Product design information has long been recognized as one of core requirements for efficient product development. This paper addresses an image-based retrieval system to support product design information reuse. In the system, product images obtained from multi-modal devices are utilized to reuse design information. The proposed system conducts the segmentation of a product image by using a labeling method and generates an attributed relational graph (ARG) that represents properties of segmented regions and their relationships. The generated ARG is extended by integrating corresponding part/assembly information. In this manner, the reuse of assembly design information using a product image has been realized. The main advantages of the presented system are following. First, the system is not dependent to specific design tools, because it utilizes multimedia images that can be obtained easily from peripheral devices. Second ratio-based features extracted from images enable image retrievals that contain various sizes of parts. Third, the system has shown outstanding search performance, because we applied various information of segmented part regions and their relationships between parts.

Compact Design and Fabrication of 'Improved QS-MMI' Demultiplexer (Improved QS-MMI' 1.31/1.55μm 파장분리기의 최적화 설계 및 제작)

  • Kim, Nam-Kook;Kim, Jang-Kyum;Choi, Chul-Hyun;O, Beom-Hoan;Lee, Seung-Gol;Park, Se-Gun;Lee, El-Hang
    • Korean Journal of Optics and Photonics
    • /
    • v.16 no.3
    • /
    • pp.248-253
    • /
    • 2005
  • We designed and fabricated a compact multi-mode interference (MMI) wavelength demultiplexer using the concept of 'Improved Quasi-State' modes. The output power and extinction ratio were improved by utilizing modal phase error which is specially occurred in low-index contrast. For a designed demultiplexer, the mode propagation analysis with effective index approximation shows significant improvement of extinction ratio to -25 dB for both $1.31{\mu}m\;and\;1.51{\mu}m$ wavelength region and the split-length was reduced about 1/5 of other MMI devices. The fabricated device shows successful characteristics for both 1.31 and $1.55{\mu}m$ wavelengths. These results demonstrate the potential of low-index materials system and the embossing process for photonic integrated circuits.

The Impact of Argumentation-based General Chemistry Laboratory Programs on Multimodal Representation and Embeddedness in University Students' Science Writing (논의가 강조된 일반화학실험이 대학생들의 글쓰기에서 나타난 다중 표상 및 다중 표상의 내재성에 미치는 영향)

  • Nam, Jeong-Hee;Cho, Dong-Won;Lee, Hye-Sook
    • Journal of The Korean Association For Science Education
    • /
    • v.31 no.6
    • /
    • pp.931-941
    • /
    • 2011
  • This study aimed to examine the effects of argument-based chemistry laboratory investigations using the Science Writing Heuristic (SWH) approach on students' use and embedding of multimodal representations in summary writing. Participants of this study were thirty-nine freshman students majoring in science education at a National University in Korea. Argument-based chemistry laboratory investigations using the SWH approach were implemented for twenty-three students enrolled in one cohort, and the traditional chemistry laboratory teaching was implemented for 16 students enrolled in the other cohort. Summary writing samples were collected from students before and after the implementation. Summary writing samples produced by students were examined using an analysis framework for examining the use and embeddedness of multimodal representations. Summary writing was categorized into one of verbal mode, symbolic mode, and visual mode. With regard to the embedding of multi-modal representations, summary writing samples were analyzed in terms of 'constructing understanding,' 'integrating multiple modes,' 'providing valid claims and evidence,' and 'representing multiple modes.' Data analysis shows that the students of the SWH group were better at utilizing and embedding multimodal representations in summary writing as they provided evidence supporting their claims. This study provides important implications on pre-service science teacher education.

Measurement of two-dimensional vibration and calibration using the low-cost machine vision camera (저가의 머신 비전 카메라를 이용한 2차원 진동의 측정 및 교정)

  • Kim, Seo Woo;Ih, Jeong-Guon
    • The Journal of the Acoustical Society of Korea
    • /
    • v.37 no.2
    • /
    • pp.99-109
    • /
    • 2018
  • The precision of the vibration-sensors, contact or non-contact types, is usually satisfactory for the practical measurement applications, but a sensor is confined to the measurement of a point or a direction. Although the precision and frequency span of the low-cost camera are inferior to these sensors, it has the merits in the cost and in the capability of simultaneous measurement of a large vibrating area. Furthermore, a camera can measure multi-degrees-of-freedom of a vibrating object simultaneously. In this study, the calibration method and the dynamic characteristics of the low-cost machine vision camera as a sensor are studied with a demonstrating example of the two-dimensional vibration of a cantilever beam. The planar image of the camera shot reveals two rectilinear and one rotational motion. The rectilinear vibration motion of a single point is first measured using a camera and the camera is experimentally calibrated by calculating error referencing the LDV (Laser Doppler Vibrometer) measurement. Then, by measuring the motion of multiple points at once, the rotational vibration motion and the whole vibration motion of the cantilever beam are measured. The whole vibration motion of the cantilever beam is analyzed both in time and frequency domain.

Exploratory Study of Characterizing Scholarly Communication Patterns in Humanities for Facilitating Consilience in Cyberscholarship Environment: Based on Historians' Research Activities (사이버스칼러쉽 환경에서의 융복합 연구 촉진을 위한 인문학 분야 학술 커뮤니케이션 특성 파악에 관한 연구 - 역사학 분야를 중심으로 -)

  • Yu, So-Young
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.50 no.1
    • /
    • pp.331-351
    • /
    • 2016
  • Digitalized data and literature in scholarly community has developed the concept of digital humanities and cyberscholarship which indicate the characteristics of a new aspect and approach in scholarly activities with digitalized resources or new media. This study was performed in order to identify the changes in national research activities of art and humanities by using a multi-modal approach. The combined methodology of in-depth interview and content analysis on publishing and citing behaviors in literature was executed. The steps of research process is identified as a non-linear combination of 3 parts: developing research idea, developing the research idea to write, and submitting manuscript to publish. Prominent implementations of cyberscholarship were found in the 2nd step for accessing and using research data and literatures. Understanding the characteristics of scholar communication using cyberscholarhip factors in humanities for interdisciplinarity, sophisticating the environment of cyberscholarhip for data sharing, investing and developing archivist and archives, and providing a various platform for accelerating scholarly communication were derived by the panel discussion for developing interdisciplinary research for humanities.

Pharmacotherapy for Patients Complaining With Somatic Symptoms (신체증상을 호소하는 환자의 약물치료)

  • Lee, Kyung-Kyu
    • Korean Journal of Psychosomatic Medicine
    • /
    • v.29 no.2
    • /
    • pp.95-101
    • /
    • 2021
  • Patients complaining with somatic symptoms are very common in clinical practice, and are often consulted to medical department. But it is difficult to treat well. The treatment of somatic symptom disorder is multi-modal as none of the methods on their own provide a satisfactory outcome. The treatment of somatic symptoms disorders is complicated by lack of boundary, conceptual clarity, and overemphasis on psychosocial causation and effectiveness of psychological treatments. In clinical practice all classes of psychotropics are used to treat somatic symptoms disorder. Drugs such as tricyclic antidepressants, serotonin reuptake inibitors(SSRI), serotonin and noradrenalin reuptake inhibitors (SNRI), atypical antipsychotics are studied. The evidence indicates that these drugs are effective in somatic symptom disorders. All classes of antidepressants seem to be effective against somatic symptom disorders. SSRIs are more effective against hypochondriasis and body dysmorphic disorder, and SNRIs appear to be more effective than other antidepressants when pain is predominant. The author suggest that psychiatrists should know how to treat patients complaining with somatic symptoms by using not only psychotherapeutic approach but also pharmacological treatment. It will be helpful to reduce suffering and increase quality of life of these patients.

Multi-modal Image Processing for Improving Recognition Accuracy of Text Data in Images (이미지 내의 텍스트 데이터 인식 정확도 향상을 위한 멀티 모달 이미지 처리 프로세스)

  • Park, Jungeun;Joo, Gyeongdon;Kim, Chulyun
    • Database Research
    • /
    • v.34 no.3
    • /
    • pp.148-158
    • /
    • 2018
  • The optical character recognition (OCR) is a technique to extract and recognize texts from images. It is an important preprocessing step in data analysis since most actual text information is embedded in images. Many OCR engines have high recognition accuracy for images where texts are clearly separable from background, such as white background and black lettering. However, they have low recognition accuracy for images where texts are not easily separable from complex background. To improve this low accuracy problem with complex images, it is necessary to transform the input image to make texts more noticeable. In this paper, we propose a method to segment an input image into text lines to enable OCR engines to recognize each line more efficiently, and to determine the final output by comparing the recognition rates of CLAHE module and Two-step module which distinguish texts from background regions based on image processing techniques. Through thorough experiments comparing with well-known OCR engines, Tesseract and Abbyy, we show that our proposed method have the best recognition accuracy with complex background images.

Anomaly Detection Methodology Based on Multimodal Deep Learning (멀티모달 딥 러닝 기반 이상 상황 탐지 방법론)

  • Lee, DongHoon;Kim, Namgyu
    • Journal of Intelligence and Information Systems
    • /
    • v.28 no.2
    • /
    • pp.101-125
    • /
    • 2022
  • Recently, with the development of computing technology and the improvement of the cloud environment, deep learning technology has developed, and attempts to apply deep learning to various fields are increasing. A typical example is anomaly detection, which is a technique for identifying values or patterns that deviate from normal data. Among the representative types of anomaly detection, it is very difficult to detect a contextual anomaly that requires understanding of the overall situation. In general, detection of anomalies in image data is performed using a pre-trained model trained on large data. However, since this pre-trained model was created by focusing on object classification of images, there is a limit to be applied to anomaly detection that needs to understand complex situations created by various objects. Therefore, in this study, we newly propose a two-step pre-trained model for detecting abnormal situation. Our methodology performs additional learning from image captioning to understand not only mere objects but also the complicated situation created by them. Specifically, the proposed methodology transfers knowledge of the pre-trained model that has learned object classification with ImageNet data to the image captioning model, and uses the caption that describes the situation represented by the image. Afterwards, the weight obtained by learning the situational characteristics through images and captions is extracted and fine-tuning is performed to generate an anomaly detection model. To evaluate the performance of the proposed methodology, an anomaly detection experiment was performed on 400 situational images and the experimental results showed that the proposed methodology was superior in terms of anomaly detection accuracy and F1-score compared to the existing traditional pre-trained model.

Recent Spatial and Temporal Changes in Means and Extreme Events of Temperature and Precipitation across the Republic of Korea (최근 우리나라 기온 및 강수 평균과 극한 사상의 시.공간적 변화)

  • Choi, Gwang-Yong;Kwon, Won-Tae;Boo, Kyung-On;Cha, Yu-Mi
    • Journal of the Korean Geographical Society
    • /
    • v.43 no.5
    • /
    • pp.681-700
    • /
    • 2008
  • In this study, the spatial and temporal patterns of changes in means and extreme events of temperature and precipitation across the Republic of Korea over the last 35 years (1973-2007) are examined. Over the study period, meteorological winter (December-February) mean minimum (maximum) temperature has increased by $+0.54^{\circ}C$/decade ($+0.6^{\circ}C$/decade), while there have been no significant changes in meteorological summer (June-August) mean temperatures. According to analyses of upper or lower $10^{th}$ percentile-based extreme temperature indices, the annual frequency of cool nights (days) has decreased by -9.2 days/decade (-3.3 days/decade), while the annual frequency of warm nights (days) has increased by +4.9 days/decade (+6.8 days/decade). In contrast, the increase rates of summer warm nights (+8.0 days/$^{\circ}C$) and days (+6.6 days/$^{\circ}C$) relative to changes in summer means minimum and maximum temperatures means are greater than the decreasing rates of winter nights (-5.2 days/$^{\circ}C$) and days (-4.3 days/$^{\circ}C$) relative to changes in winter temperatures. These results demonstrate that seasonal and diurnal asymmetric changes in extreme temperature events have occurred. Moreover, annual total precipitation has increased by 85.5 mm/decade particularly in July and August, which led to the shift of a bimodal behavior of summer precipitation into a multi-modal structure. These changes have resulted from the intensification of heavy rainfall events above 40mm in recent decades, and spatially the statistically-significant increases in these heavy rainfall events are observed around the Taebaek mountain region.