• Proceedings of the Korea Information Processing Society Conference
• /
• 2022.11a
• /
• pp.656-658
• /
• 2022

The purpose of infrared and visible image fusion is to integrate images of different modes with different details into a result image with rich information, which is convenient for high-level computer vision task. Considering many deep networks only work in a single scale, this paper proposes a novel image fusion based on three-scale dense network to preserve the content and key target features from the input images in the fused image. It comprises an encoder, a three-scale block, a fused strategy and a decoder, which can capture incredibly rich background details and prominent target details. The encoder is used to extract three-scale dense features from the source images for the initial image fusion. Then, a fusion strategy called l₁-norm to fuse features of different scales. Finally, the fused image is reconstructed by decoding network. Compared with the existing methods, the proposed method can achieve state-of-the-art fusion performance in subjective observation.

• Journal of Biosystems Engineering
• /
• v.26 no.1
• /
• pp.21-28
• /
• 2001

Major deficiencies of current automation scheme including various robots for bioproduction include the lack of task adaptability and real time processing, low job performance for diverse tasks, and the lack of robustness of take results, high system cost, failure of the credit from the operator, and so on. This paper proposed a scheme that could solve the current limitation of task abilities of conventional computer controlled automatic system. The proposed scheme is the man-machine hybrid automation via tele-operation which can handle various bioproduction processes. And it was classified into two categories. One category was the efficient task sharing between operator and CCM(computer controlled machine). The other was the efficient interface between operator and CCM. To realize the proposed concept, task of the object identification and extraction of 3D coordinate of an object was selected. 3D coordinate information was obtained from camera calibration using camera as a measurement device. Two stereo images were obtained by moving a camera certain distance in horizontal direction normal to focal axis and by acquiring two images at different locations. Transformation matrix for camera calibration was obtained via least square error approach using specified 6 known pairs of data points in 2D image and 3D world space. 3D world coordinate was obtained from two sets of image pixel coordinates of both camera images with calibrated transformation matrix. As an interface system between operator and CCM, a touch pad screen mounted on the monitor and remotely captured imaging system were used. Object indication was done by the operator’s finger touch to the captured image using the touch pad screen. A certain size of local image processing area was specified after the touch was made. And image processing was performed with the specified local area to extract desired features of the object. An MS Windows based interface software was developed using Visual C++6.0. The software was developed with four modules such as remote image acquisiton module, task command module, local image processing module and 3D coordinate extraction module. Proposed scheme shoed the feasibility of real time processing, robust and precise object identification, and adaptability of various job and environments though selected sample tasks.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.17 no.5
• /
• pp.1413-1432
• /
• 2023

Recommender systems aim to recommend items to users by taking into account their probable interests. This study focuses on creating a model that utilizes multiple sources of information about users and items by employing a multimodality approach. The study addresses the task of how to gather information from different sources (modalities) and transform them into a uniform format, resulting in a multi-modal feature description for users and items. This work also aims to transform and represent the features extracted from different modalities so that the information is in a compatible format for integration and contains important, useful information for the prediction model. To achieve this goal, we propose a novel multi-modal recommendation model, which involves extracting latent features of users and items from a utility matrix using matrix factorization techniques. Various transformation techniques are utilized to extract features from other sources of information such as user reviews, item descriptions, and item categories. We also proposed the use of Principal Component Analysis (PCA) and Feature Selection techniques to reduce the data dimension and extract important features as well as remove noisy features to increase the accuracy of the model. We conducted several different experimental models based on different subsets of modalities on the MovieLens and Amazon sub-category datasets. According to the experimental results, the proposed model significantly enhances the accuracy of recommendations when compared to SVD, which is acknowledged as one of the most effective models for recommender systems. Specifically, the proposed model reduces the RMSE by a range of 4.8% to 21.43% and increases the Precision by a range of 2.07% to 26.49% for the Amazon datasets. Similarly, for the MovieLens dataset, the proposed model reduces the RMSE by 45.61% and increases the Precision by 14.06%. Additionally, the experimental results on both datasets demonstrate that combining information from multiple modalities in the proposed model leads to superior outcomes compared to relying on a single type of information.

• International journal of advanced smart convergence
• /
• v.12 no.4
• /
• pp.434-442
• /
• 2023

The relationship between acute kidney injury (AKI) prediction and nephrotoxic drugs, or drugs that adversely affect kidney function, is one that has yet to be explored in the critical care setting. One contributing factor to this gap in research is the limited investigation of drug modalities in the intensive care unit (ICU) context, due to the challenges of processing prescription data into the corresponding drug representations and a lack in the comprehensive understanding of these drug representations. This study addresses this gap by proposing a novel approach that leverages patient prescription data as a modality to improve existing models for AKI prediction. We base our research on Electronic Health Record (EHR) data, extracting the relevant patient prescription information and converting it into the selected drug representation for our research, the extended-connectivity fingerprint (ECFP). Furthermore, we adopt a unique multimodal approach, developing machine learning models and 1D Convolutional Neural Networks (CNN) applied to clinical drug representations, establishing a procedure which has not been used by any previous studies predicting AKI. The findings showcase a notable improvement in AKI prediction through the integration of drug embeddings and other patient cohort features. By using drug features represented as ECFP molecular fingerprints along with common cohort features such as demographics and lab test values, we achieved a considerable improvement in model performance for the AKI prediction task over the baseline model which does not include the drug representations as features, indicating that our distinct approach enhances existing baseline techniques and highlights the relevance of drug data in predicting AKI in the ICU setting.

• Journal of the Ergonomics Society of Korea
• /
• v.29 no.2
• /
• pp.203-210
• /
• 2010

For the development of a better product which fits to the target user population, physical workloads such as reach and visibility are evaluated using digital human simulation in the early stage of product development; however, ergonomic workload assessment mainly relies on visual observation of reach envelopes and view cones generated in a 3D graphic environment. The present study developed a quantitative assessment method of physical workload in a digital environment and applied to the evaluation of a Korean utility helicopter (KUH) cockpit design. The proposed assessment method quantified physical workloads for the target user population by applying a 3-step process and identified design features requiring improvement based on the quantified workload evaluation. The scores of physical workloads were quantified in terms of posture, reach, visibility, and clearance, and 5-point scales were defined for the evaluation measures by referring to existing studies. The postures of digital humanoids for a given task were estimated to have the minimal score of postural workload by finding all feasible postures that satisfy task constraints such as a contact between the tip of the index finger and a target point. The proposed assessment method was applied to evaluate the KUH cockpit design in the preliminary design stage and identified design features requiring improvement. The proposed assessment method can be utilized to ergonomic evaluation of product designs using digital human simulation.

• Journal of Biosystems Engineering
• /
• v.29 no.1
• /
• pp.71-78
• /
• 2004

Real time and robust algorithm to extract the features of watermelon was developed from the remotely transmitted image of the watermelon. Features of the watermelon at the cultivation site such as size and shape including position are crucial to the successful tole-robotic operation and development of the cultivation data base. Algorithm was developed based on the concept of task sharing between the computer and the operator utilizing man-computer interface. Task sharing was performed based on the functional characteristics of human and computer. Identifying watermelon from the image transmitted from the cultivation site is very difficult because of the variable light condition and the complex image contents such as soil, mulching vinyl, straws on the ground, irregular leaves and stems. Utilizing operator's teaching through the touch screen mounted on the image monitor, the complex time consuming image processing process and instability of processing results in the watermelon identification has been avoided. Color and brightness characteristics were analyzed from the image area specified by the operator's teaching. Watermelon segmentation was performed using the brightness and color distribution of the specified imae processing area. Modified general Hough transform was developed to extract the shape, major and minor axes, and the position, of the watermelon. It took less than 100 msec of the image processing time, and was a lot faster than conventional approach. The proposed method showed the robustness and practicability in identifying watermelon from the wireless transmitted color image of the cultivation site.

• Journal of Internet Computing and Services
• /
• v.14 no.5
• /
• pp.49-57
• /
• 2013

A novel approach for the feature selection is proposed, which is the important preprocessing task of on-line document classification. In previous researches, the features based on information from their single population for feature selection task have been selected. In this paper, a mixed feature set is constructed by selecting features from multi-population as well as single population based on various information. The mixed feature set consists of two feature sets: the original feature set that is made up of words on documents and the transformed feature set that is made up of features generated by LSA. The hybrid feature selection method using both filter and wrapper method is used to obtain optimal features set from the mixed feature set. We performed classification experiments using the obtained optimal feature sets. As a result of the experiments, our expectation that our approach makes better performance of classification is verified, which is over 90% accuracy. In particular, it is confirmed that our approach has over 90% recall and precision that have a low deviation between categories.

• Korean Journal of Cognitive Science
• /
• v.32 no.2
• /
• pp.55-73
• /
• 2021

Psychological essentialism states that people believe some categories to have hidden and defining essential features which cause other features of the category (Gelman, 2003; Hirschfeld, 1996; Medin & Ortony, 1989). Essentialist belief on categories questions the Roschian argument (Rosch, 1973, 1978) that categories merely consist of clusters of correlated features. Unlike family resemblance categories, essentialized categories are likely to have clear between-category boundaries and high within-category coherence (Gelman, 2003; Prentice & Miller, 2007). Two experiments were conducted to test the effects of essentialist belief on category representation (i.e., between-category boundary, within-category coherence). Participants learned family resemblance and essentialized categories in their assigned conditions and then performed categorization task (Expt. 1) and frequency estimation task of category exemplars (Expt. 2). The results showed, in essentialized categories, both boundary intensification and greater category coherence. Theses results are likely to have arisen due to increased cue and category validity in essentialized categories and suggest that essentialist belief influences macroscopic representation of category structure.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.18 no.6
• /
• pp.133-137
• /
• 2018

Arduino is a small microcomputer that is used in a variety of industry fields and especially is widely used as an open source hardware IoT device. The multi-tasking method of Arduino is divided into super loop timing and RTOS thread method. The super loop timing method is simple and easy to understand. However, when one task is long, it affects the execution of the next task. In addition, RTOS threading has the advantage of being able to run without being influenced by other work time. However, Arduino, a small microcomputer, has a disadvantage in that, when the number of threads increases, the context switching time of the thread causes additional time not included in the super loop timing method have. In this paper, we use Arduino Uno R3 and FreeRTOS to analyze these different features, and the task for the experiment is to send 8000 digital signals to the built-in LED port. If two tasks of the same size are executed, the super loop method executes 3 ms faster than FreeRTOS multitasking. If multiple tasks are executed simultaneously, superloop type task is sequential execution and difference in execution time between first task and last task is large. FreeRTOS method can be executed concurrently, but execution time delay of about 30 ms occurs in context switching time.

• KIPS Transactions on Software and Data Engineering
• /
• v.12 no.12
• /
• pp.505-518
• /
• 2023

3D point cloud semantic segmentation is a computer vision task that involves dividing the point cloud into different objects and regions by predicting the class label of each point. Existing 3D semantic segmentation models have some limitations in performing sufficient fusion of multi-modal features while ensuring both characteristics of 2D visual features extracted from RGB images and 3D geometric features extracted from point cloud. Therefore, in this paper, we propose MMCA-Net, a novel 3D semantic segmentation model using 2D-3D multi-modal features. The proposed model effectively fuses two heterogeneous 2D visual features and 3D geometric features by using an intermediate fusion strategy and a multi-modal cross attention-based fusion operation. Also, the proposed model extracts context-rich 3D geometric features from input point cloud consisting of irregularly distributed points by adopting PTv2 as 3D geometric encoder. In this paper, we conducted both quantitative and qualitative experiments with the benchmark dataset, ScanNetv2 in order to analyze the performance of the proposed model. In terms of the metric mIoU, the proposed model showed a 9.2% performance improvement over the PTv2 model using only 3D geometric features, and a 12.12% performance improvement over the MVPNet model using 2D-3D multi-modal features. As a result, we proved the effectiveness and usefulness of the proposed model.