• Journal of the Korea Convergence Society
• /
• v.11 no.11
• /
• pp.175-182
• /
• 2020

In this study, we introduced the technique of matching classical music using deep learning to design soundscape that can help the viewer appreciate painting and proposed an evaluation index to evaluate how well matching painting and music. The evaluation index was conducted with suitability evaluation through the Likeard 5-point scale and evaluation in a multimodal aspect. The suitability evaluation score of the 13 test participants for the deep learning based best match between painting and music was 3.74/5.0 and band the average cosine similarity of the multimodal evaluation of 13 participants was 0.79. We expect multimodal evaluation to be an evaluation index that can measure a new user experience. In addition, this study aims to improve the experience of multisensory artworks by proposing the interaction between visual and auditory. The proposed matching of painting and music method can be used in multisensory artwork exhibition and furthermore it will increase the accessibility of visually impaired people to appreciate artworks.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.16 no.4
• /
• pp.430-435
• /
• 2006

In this paper, we propose a method based on multimodal biometrics system using the face and signature under ubiquitous computing environments. First, the face and signature images are obtained by PDA and then these images with user ID and name are transmitted via WLAN(Wireless LAN) to the server and finally the PDA receives verification result from the server. The multimodal biometrics recognition system consists of two parts. In client part located in PDA, user interface program executes the user registration and verification process. The server consisting of the PCA and LDA algorithm shows excellent face recognition performance and the signature recognition method based on the Kernel PCA and LDA algorithm for signature image projected to vertical and horizontal axes by grid partition method. The proposed algorithm is evaluated with several face and signature images and shows better recognition and verification results than previous unimodal biometrics recognition techniques.

• Journal of Internet Computing and Services
• /
• v.13 no.5
• /
• pp.9-19
• /
• 2012

Virtual human used as HCI in digital contents expresses his various emotions across modalities like facial expression and body posture. However, few studies considered combinations of such nonverbal multimodal in emotion perception. Computational engine models have to consider how a combination of nonverbal modal like facial expression and body posture will be perceived by users to implement emotional virtual human, This paper proposes the impacts of nonverbal multimodal in design of emotion expressed virtual human. First, the relative impacts are analysed between different modals by exploring emotion recognition of modalities for virtual human. Then, experiment evaluates the contribution of the facial and postural congruent expressions to recognize basic emotion categories, as well as the valence and activation dimensions. Measurements are carried out to the impact of incongruent expressions of multimodal on the recognition of superposed emotions which are known to be frequent in everyday life. Experimental results show that the congruence of facial and postural expression of virtual human facilitates perception of emotion categories and categorical recognition is influenced by the facial expression modality, furthermore, postural modality are preferred to establish a judgement about level of activation dimension. These results will be used to implementation of animation engine system and behavior syncronization for emotion expressed virtual human.

• Journal of the Optical Society of Korea
• /
• v.18 no.5
• /
• pp.551-557
• /
• 2014

In this study, we demonstrated multimodal nonlinear optical (NLO) microscopy integrated simultaneously with two-photon excitation fluorescence (TPEF), second-harmonic generation (SHG), and coherent anti-Stokes Raman scattering (CARS) in order to obtain targeted cellular and label-free images in an immunofluorescence assay of the atherosclerotic aorta from apolipoprotein E-deficient mice. The multimodal NLO microscope used two laser systems: picosecond (ps) and femtosecond (fs) pulsed lasers. A pair of ps-pulsed lights served for CARS (817 nm and 1064 nm) and SHG (817 nm) images; light from the fs-pulsed laser with the center wavelength of 720 nm was incident into the sample to obtain autofluorescence and targeted molecular TPEF images for high efficiency of fluorescence intensity without cross-talk. For multicolor-targeted TPEF imaging, we stained smooth-muscle cells and macrophages with fluorescent dyes (Alexa Fluor 350 and Alexa Fluor 594) for an immunofluorescence assay. Each depth-sectioned image consisted of $512{\times}512$ pixels with a field of view of $250{\times}250{\mu}m^2$, a lateral resolution of $0.4{\mu}m$, and an axial resolution of $1.3{\mu}m$. We obtained composite multicolor images with conventional label-free NLO images and targeted TPEF images in atherosclerotic-plaque samples. Multicolor 3-D imaging of atherosclerotic-plaque structural and functional composition will be helpful for understanding the pathogenesis of cardiovascular disease.

• Korean Journal of Fisheries and Aquatic Sciences
• /
• v.29 no.3
• /
• pp.386-392
• /
• 1996

A numerical method based on genetic algorithms was introduced to characterize the grain-size distribution more effectively. This technique was proved significant particularly for multimodal size distributions, as was verified for samples from Korea Strait continental shelf. Sediment samples collected from the Korea Strait continental shelf revealed that $96\%$ of the grain-size distributions were multimodal. Therefore, the use of grain-size parameters was not the ideal method. As an alternative method, the decomposition of sue curves into elementary normal component curves was used. Means and standard deviations of 593 decomposed normal components were calculated by a numerical method from 268 size curves of Korea Strait sediments. The mean values of decomposed normal components showed peaks at $1\~3\phi\;and\;7\~9\phi$ size classes. The plot of mean and standard deviation values of the coarse fraction normal components on the map showed a characteristic areal distribution. The characteristic distribution was found to derive from underlying Pleistocene sediment on the basis of sea bottom geologic distribution of the area. The method of decomposition into normal components was found to be more effective than the analysis using traditional grain-size parameters in investigation of multimodal size distribution of Korea Strait shelf sediment.

• Clinics in Orthopedic Surgery
• /
• v.10 no.4
• /
• pp.455-461
• /
• 2018

Background: Surgical-site, multimodal drug injection has recently evolved to be a safe and useful method for multimodal pain management even in patients with musculoskeletal trauma. Methods: Fifty consecutive patients who underwent plating for mid-shaft and distal clavicular fractures were included in the study. To evaluate whether surgical-site injections (SIs) have pain management benefits, the patients were divided into two groups (SI and no-SI groups). The injection was administered between the deep and superficial tissues prior to wound closure. The mixture of anesthetics consisted of epinephrine hydrochloride (HCL), morphine sulfate, ropivacaine HCL, and normal saline. The visual analogue scale (VAS) pain scores were measured at 6-hour intervals until postoperative hour (POH) 72; stress biomarkers (dehydroepiandrosterone sulfate [DHEA-S], insulin, and fibrinogen) were measured preoperatively and at POH 24, 48, and 72. In patients who wanted further pain control or had a VAS pain score of 7 points until POH 72, 75 mg of intravenous tramadol was administered, and the intravenous tramadol requirements were also recorded. Other medications were not used for pain management. Results: The SI group showed significantly lower VAS pain scores until POH 24, except for POH 18. Tramadol requirement was significantly lower in the SI group until POH 24, except for POH 12 and 18. The mean DHEA-S level significantly decreased in the no-SI group ($74.2{\pm}47.0{\mu}g/dL$) at POH 72 compared to that in the SI group ($110.1{\pm}87.1{\mu}g/dL$; p = 0.046). There was no significant difference in the insulin and fibrinogen levels between the groups. The correlation values between all the biomarkers and VAS pain scores were not significantly different between the two groups (p > 0.05). Conclusions: After internal fixation of the clavicular fracture, the surgical-site, multimodal drug injection effectively relieved pain on the day of the surgery without any complications. Therefore, we believe that SI is a safe and effective method for pain management after internal fixation of a clavicular fracture.

• Journal of Convergence for Information Technology
• /
• v.11 no.3
• /
• pp.174-184
• /
• 2021

The present study intends to understand how multimodality can be implemented in a content course curriculum and how students perceive multimodal tasks. Twenty-eight students majoring in English were engaged in a digital storytelling assignment as a part of the content curriculum. Findings from the questionnaire and reflective essays that investigated students perceptions of digital storytelling showed that students felt that the assignment helped them engage in the task and felt motivated. In comparison to traditional writing tasks, students perceived digital storytelling to be more engaging and motivating, but felt that the assignment required more mental effort and caused more anxiety. By supporting students to explore technology and implement multimodal aspects in the learning process, digital storytelling can encourage engagement and autonomous learning to create meaningful works that are purposeful and enjoyable.

• Archives of Plastic Surgery
• /
• v.49 no.5
• /
• pp.633-641
• /
• 2022

Introduction In chronic facial palsy, synkinetic muscle overactivity and shortening causes muscle stiffness resulting in reduced movement and functional activity. This article studies the role of multimodal therapy in improving outcomes. Methods Seventy-five facial palsy patients completed facial rehabilitation before being successfully discharged by the facial therapy team. The cohort was divided into four subgroups depending on the time of initial attendance post-onset. The requirement for facial therapy, chemodenervation, or surgery was assessed with East Grinstead Grade of Stiffness (EGGS). Outcomes were measured using the Facial Grading Scale (FGS), Facial Disability Index, House-Brackmann scores, and the Facial Clinimetric Evaluation scale. Results FGS composite scores significantly improved posttherapy (mean-standard deviation, 60.13 ± 23.24 vs. 79.9 ± 13.01; confidence interval, -24.51 to -14.66, p < 0.0001). Analysis of FGS subsets showed that synkinesis also reduced significantly (p < 0.0001). Increasingly, late clinical presentations were associated with patients requiring longer durations of chemodenervation treatment (p < 0.01), more chemodenervation episodes (p < 0.01), increased doses of botulinum toxin (p < 0.001), and having higher EGGS score (p < 0.001). Conclusions This study shows that multimodal facial rehabilitation in the management of facial palsy is effective, even in patients with chronically neglected synkinesis. In terms of the latency periods between facial palsy onset and treatment initiation, patients presenting later than 2 years were still responsive to multimodal treatment albeit to a lesser extent, which we postulate is due to increasing muscle contracture within their facial muscles.

• SUVANNABHUMI
• /
• v.15 no.2
• /
• pp.91-129
• /
• 2023

This paper reports the findings of a multimodal study conducted on 10 travel blog posts about Vietnam by seven British professional travel bloggers. The study takes a sociolinguistic view to tourism by seeing travel blogs as a source for linguistic and other semiotic materials while considering language as situated practice for the social construction of fundamental categories such as "human," "society," and "nation." It borrows concepts from Halliday's Systemic Functional Linguistics for interpersonal metafunction to develop an analytical framework to study how the co-occurrence of text and still images in these travel blog posts formulated the portrayal of Vietnam as a tourism destination and indicated the main sociolinguistic features of the blogs. The analysis of appreciation values and interactive qualities encoded in evaluative adjectives and still images show that Vietnam is generally portrayed as a country of identity and diversity. It provides tourists with positive experiences in terms of places of interest, food and local lifestyles and is cost-competitive. Strangerhood and authenticity are two outstanding sociolinguistic features exhibited in these travel blog posts. The findings of this study also underline the co-contribution of the linguistic sign, in this case evaluative adjectives, and the visual sign, in this case still images, as interpersonal meaning-making resources. To portray Vietnam, still images served as integral elements to evidence the credibility of verbal narrations. To unveil sociolinguistic characteristics of travel blogs, still images supported the linguistic realizations of authenticity and strangerhood on the posts, and in some case delivered an even stronger message than words. Not only does the study present a source of feedback from international travelers to tourism practice in Vietnam, but it also provides insights into multimodal analysis of tourism discourse which remains an under-researched area in Vietnam.

• Journal of Korean Society of Transportation
• /
• v.26 no.3
• /
• pp.233-243
• /
• 2008

Recognition for the importance and roles of public transportation is increasing because of traffic problems in many cities. In spite of this paradigm change, previous researches related with public transportation trip assignment have limits in some aspects. Especially, in case of multimodal public transportation networks, many characters should be considered such as transfers. operational time schedules, waiting time and travel cost. After metropolitan integrated transfer discount system was carried out, transfer trips are increasing among traffic modes and this takes the variation of users' route choices. Moreover, the advent of high-technology public transportation card called smart card, public transportation users' travel information can be recorded automatically and this gives many researchers new analytical methodology for multimodal public transportation networks. In this paper, it is suggested that the methodology for establishment of brand new multimodal public transportation networks based on computer programming methods using transportation card data. First, we propose the building method of integrated transportation networks based on bus and urban railroad stations in order to make full use of travel information from transportation card data. Second, it is offered how to connect the broken transfer links by computer-based programming techniques. This is very helpful to solve the transfer problems that existing transportation networks have. Lastly, we give the methodology for users' paths finding and network establishment among multi-modes in multimodal public transportation networks. By using proposed methodology in this research, it becomes easy to build multimodal public transportation networks with existing bus and urban railroad station coordinates. Also, without extra works including transfer links connection, it is possible to make large-scaled multimodal public transportation networks. In the end, this study can contribute to solve users' paths finding problem among multi-modes which is regarded as an unsolved issue in existing transportation networks.