• Journal of the Korean Chemical Society
• /
• v.56 no.4
• /
• pp.500-508
• /
• 2012

The purpose of this study was to investigate the impact of multimodal representation-based lesson on embeddedness of multimodal representation in high school students' writing. The participants in this study were two groups of second-year science-track students (74 students) at an academic high school in a metropolitan city. One group (41 students) was assigned to the experimental group, the other group (33 students) was assigned to the comparative group. Data analysis showed that the students of the experimental group were better at utilizing and embedding multimodal representations. Thus, the conclusion was drawn that multimodal representation-based lesson had an effect on high school students' embeddedness of multimodal representation.

• Journal of Intelligence and Information Systems
• /
• v.29 no.1
• /
• pp.203-214
• /
• 2023

The Korea Customs Service is efficiently handling business with an electronic customs system that can effectively handle one-stop business. This is the case and a more effective method is needed. Import and export require HS Code (Harmonized System Code) for classification and tax rate application for all goods, and item classification that classifies the HS Code is a highly difficult task that requires specialized knowledge and experience and is an important part of customs clearance procedures. Therefore, this study uses various types of data information such as product name, product description, and product image in the item classification request form to learn and develop a deep learning model to reflect information well based on Multimodal representation learning. It is expected to reduce the burden of customs duties by classifying and recommending HS Codes and help with customs procedures by promptly classifying items.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.1
• /
• pp.131-146
• /
• 2021

Multimodal biometric-based recognition has been an active topic because of its higher convenience in recent years. Due to high user convenience of finger, finger-based personal identification has been widely used in practice. Hence, taking Finger-Print (FP), Finger-Vein (FV) and Finger-Knuckle-Print (FKP) as the ingredients of characteristic, their feature representation were helpful for improving the universality and reliability in identification. To usefully fuse the multimodal finger-features together, a new robust representation algorithm was proposed based on hierarchical model. Firstly, to obtain more robust features, the feature maps were obtained by Gabor magnitude feature coding and then described by Local Binary Pattern (LBP). Secondly, the LGBP-based feature maps were processed hierarchically in bottom-up mode by variable rectangle and circle granules, respectively. Finally, the intension of each granule was represented by Local-invariant Gray Features (LGFs) and called Hierarchical Local-Gabor-based Gray Invariant Features (HLGGIFs). Experiment results revealed that the proposed algorithm is capable of improving rotation variation of finger-pose, and achieving lower Equal Error Rate (EER) in our homemade database.

• ETRI Journal
• /
• v.44 no.2
• /
• pp.183-193
• /
• 2022

Smart cities are expected to provide residents with convenience via various agents such as CCTV, delivery robots, security robots, and unmanned shuttles. Environmental data collected by various agents can be used for various purposes, including advertising and security monitoring. This study suggests a surveillance map data framework for efficient and integrated multimodal data representation from multi-agents. The suggested surveillance map is a multilayered global information grid, which is integrated from the multimodal data of each agent. To confirm this, we collected surveillance map data for 4 months, and the behavior patterns of humans and vehicles, distribution changes of elevation, and temperature were analyzed. Moreover, we represent an anomaly detection algorithm based on a surveillance map for security service. A two-stage anomaly detection algorithm for unusual situations was developed. With this, abnormal situations such as unusual crowds and pedestrians, vehicle movement, unusual objects, and temperature change were detected. Because the surveillance map enables efficient and integrated processing of large multimodal data from a multi-agent, the suggested data framework can be used for various applications in the smart city.

• Journal of The Korean Association For Science Education
• /
• v.34 no.6
• /
• pp.583-592
• /
• 2014

The purpose of this study is to investigate the impact of argument-based modeling strategy using scientific writing on student's modeling ability. For this study, 66 students (three classes) from the 7th grade were selected and of these, 43 students (two classes) were assigned to two experimental groups while the other 23 students (one class) were assigned to comparative group. In the experimental groups, one group (22 students) was Argument-based multimodal Representation and Modeling (AbRM), and the other group (21 students) was Argument-based Modeling (AbM). Modeling ability consisted of identifying the problem, structuring of scientific concepts, adequacy of claim and evidence and index of multimodal representation. As for the modeling ability, AbRM group scored significantly higher than the other groups, AbM group was significantly higher than comparative group. The four sub-elements of modeling ability in the AbRM group was significantly higher than the other groups statistically and AbM group scored significantly higher than comparative group. From these results, the argument-based modeling strategy using scientific writing was effective on students' modeling ability. Students organized or expressed the model and evaluated or modified it through the process of argument-based modeling using scientific writing and the exchange of opinions with others by scientific language as argument and writing.

• Journal of The Korean Association For Science Education
• /
• v.34 no.5
• /
• pp.479-490
• /
• 2014

The purpose of this study is to develop an argument-based modeling strategy, utilizing writing and argumentation for communication in science education. We need to support students and teachers who have difficulty in modeling in science education, this strategy focuses on development of four kinds of factors as follows: First, awareness of problems, recognizing in association with problems by observing several problematic situations. Second is science concept structuralization suggesting enough science concepts by organization for scientific explanation. The third is claim-evidence appropriateness that suggests appropriate representation as evidence for assertions. Last, the use of various representations and multimodal representations that converts and integrates these representations in evidence suggestion. For the development of these four factors, this study organized three stages. 'Recognition process' for understanding of multimodal representations, and 'Interpretation process' for understanding of activity according to multimodal representations, 'Application process' for understanding of modeling through argumentation. This application process has been done with eight stages of 'Asking questions or problems - Planning experiment - Investigation through observation on experiment - Analyzing and interpreting data - Constructing pre-model - Presenting model - Expressing model using multimodal representations - Evaluating model - Revising model'. After this application process, students could have opportunity to form scientific knowledge by making their own model as scientific explanation system for the phenomenon of the natural world they observed during a series of courses of modeling.

• Journal of The Korean Association For Science Education
• /
• v.31 no.6
• /
• pp.931-941
• /
• 2011

This study aimed to examine the effects of argument-based chemistry laboratory investigations using the Science Writing Heuristic (SWH) approach on students' use and embedding of multimodal representations in summary writing. Participants of this study were thirty-nine freshman students majoring in science education at a National University in Korea. Argument-based chemistry laboratory investigations using the SWH approach were implemented for twenty-three students enrolled in one cohort, and the traditional chemistry laboratory teaching was implemented for 16 students enrolled in the other cohort. Summary writing samples were collected from students before and after the implementation. Summary writing samples produced by students were examined using an analysis framework for examining the use and embeddedness of multimodal representations. Summary writing was categorized into one of verbal mode, symbolic mode, and visual mode. With regard to the embedding of multi-modal representations, summary writing samples were analyzed in terms of 'constructing understanding,' 'integrating multiple modes,' 'providing valid claims and evidence,' and 'representing multiple modes.' Data analysis shows that the students of the SWH group were better at utilizing and embedding multimodal representations in summary writing as they provided evidence supporting their claims. This study provides important implications on pre-service science teacher education.

• Journal of Digital Convergence
• /
• v.13 no.6
• /
• pp.151-156
• /
• 2015

In this paper, we have proposed personal multimodal biometric authentication system based on face detection, recognition and speaker verification for smart-phone environment. Proposed system detect the face with Modified Census Transform algorithm then find the eye position in the face by using gabor filter and k-means algorithm. Perform preprocessing on the detected face and eye position, then we recognize with Linear Discriminant Analysis algorithm. Afterward in speaker verification process, we extract the feature from the end point of the speech data and Mel Frequency Cepstral Coefficient. We verified the speaker through Dynamic Time Warping algorithm because the speech feature changes in real-time. The proposed multimodal biometric system is to fuse the face and speech feature (to optimize the internal operation by integer representation) for smart-phone based real-time face detection, recognition and speaker verification. As mentioned the multimodal biometric system could form the reliable system by estimating the reasonable performance.

• Journal of Information Processing Systems
• /
• v.16 no.1
• /
• pp.6-29
• /
• 2020

Biometrics identification using multiple modalities has attracted the attention of many researchers as it produces more robust and trustworthy results than single modality biometrics. In this paper, we present a novel multimodal recognition system that trains a deep learning network to automatically learn features after extracting multiple biometric modalities from a single data source, i.e., facial video clips. Utilizing different modalities, i.e., left ear, left profile face, frontal face, right profile face, and right ear, present in the facial video clips, we train supervised denoising auto-encoders to automatically extract robust and non-redundant features. The automatically learned features are then used to train modality specific sparse classifiers to perform the multimodal recognition. Moreover, the proposed technique has proven robust when some of the above modalities were missing during the testing. The proposed system has three main components that are responsible for detection, which consists of modality specific detectors to automatically detect images of different modalities present in facial video clips; feature selection, which uses supervised denoising sparse auto-encoders network to capture discriminative representations that are robust to the illumination and pose variations; and classification, which consists of a set of modality specific sparse representation classifiers for unimodal recognition, followed by score level fusion of the recognition results of the available modalities. Experiments conducted on the constrained facial video dataset (WVU) and the unconstrained facial video dataset (HONDA/UCSD), resulted in a 99.17% and 97.14% Rank-1 recognition rates, respectively. The multimodal recognition accuracy demonstrates the superiority and robustness of the proposed approach irrespective of the illumination, non-planar movement, and pose variations present in the video clips even in the situation of missing modalities.

• Journal of Computing Science and Engineering
• /
• v.6 no.2
• /
• pp.168-177
• /
• 2012

The search for relevant and actionable information is a key to achieving clinical and research goals in biomedicine. Biomedical information exists in different forms: as text and illustrations in journal articles and other documents, in images stored in databases, and as patients' cases in electronic health records. This paper presents ways to move beyond conventional text-based searching of these resources, by combining text and visual features in search queries and document representation. A combination of techniques and tools from the fields of natural language processing, information retrieval, and content-based image retrieval allows the development of building blocks for advanced information services. Such services enable searching by textual as well as visual queries, and retrieving documents enriched by relevant images, charts, and other illustrations from the journal literature, patient records and image databases.