• IEMEK Journal of Embedded Systems and Applications
• /
• v.17 no.2
• /
• pp.93-100
• /
• 2022

In this paper, we propose a machine learning-based method for supporting resource management of IoT software platforms in a multi-modal sensing scenario. We assume that an IoT device installed with a oneM2M-compatible software platform is connected with various sensors such as PIR, sound, dust, ambient light, ultrasonic, accelerometer, through different embedded system interfaces such as general purpose input output (GPIO), I2C, SPI, USB. Based on a collected dataset including CPU usage and user-defined priority, a machine learning model is trained to estimate the level of nice value required to adjust according to the resource usage patterns. The proposed method is validated by comparing with a rule-based control strategy, showing its practical capability in a multi-modal sensing scenario of IoT devices.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.6 no.4
• /
• pp.1090-1105
• /
• 2012

A multi-modal sensing M2M healthcare monitoring system for the continuous monitoring of patients under their natural physiological states or elderly persons with chronic diseases is summarized. The system is designed for homecare or the monitoring of the elderly who live in country side or small rest home without enough support from caregivers or doctors, instead of patient monitoring in big hospital environment. Further insights into the natural cause and progression of diseases are afforded by context-aware sensing, which includes the use of accelerometers to monitor patient activities, or by location-aware indoor tracking based on ultrasonic and RF sensing. Moreover, indoor location tracking provides information about the location of patients in their physical environment and helps the caregiver in the provision of appropriate support.

• Journal of Construction Engineering and Project Management
• /
• v.5 no.4
• /
• pp.16-22
• /
• 2015

This paper presents an automated computer vision-based system to update BIM data by leveraging multi-modal visual data collected from existing buildings under inspection. Currently, visual inspections are conducted for building envelopes or mechanical systems, and auditors analyze energy-related contextual information to examine if their performance is maintained as expected by the design. By translating 3D surface thermal profiles into energy performance metrics such as actual R-values at point-level and by mapping such properties to the associated BIM elements using XML Document Object Model (DOM), the proposed method shortens the energy performance modeling gap between the architectural information in the as-designed BIM and the as-is building condition, which improve the reliability of building energy analysis. Several case studies were conducted to experimentally evaluate their impact on BIM-based energy analysis to calculate energy load. The experimental results on existing buildings show that (1) the point-level thermography-based thermal resistance measurement can be automatically matched with the associated BIM elements; and (2) their corresponding thermal properties are automatically updated in gbXML schema. This paper provides practitioners with insight to uncover the fundamentals of how multi-modal visual data can be used to improve the accuracy of building energy modeling for retrofit analysis. Open research challenges and lessons learned from real-world case studies are discussed in detail.

• International conference on construction engineering and project management
• /
• 2015.10a
• /
• pp.532-536
• /
• 2015

This paper presents an automated computer vision-based system to update BIM data by leveraging multi-modal visual data collected from existing buildings under inspection. Currently, visual inspections are conducted for building envelopes or mechanical systems, and auditors analyze energy-related contextual information to examine if their performance is maintained as expected by the design. By translating 3D surface thermal profiles into energy performance metrics such as actual R-values at point-level and by mapping such properties to the associated BIM elements using XML Document Object Model (DOM), the proposed method shortens the energy performance modeling gap between the architectural information in the as-designed BIM and the as-is building condition, which improve the reliability of building energy analysis. The experimental results on existing buildings show that (1) the point-level thermography-based thermal resistance measurement can be automatically matched with the associated BIM elements; and (2) their corresponding thermal properties are automatically updated in gbXML schema. This paper provides practitioners with insight to uncover the fundamentals of how multi-modal visual data can be used to improve the accuracy of building energy modeling for retrofit analysis. Open research challenges and lessons learned from real-world case studies are discussed in detail.

• Science of Emotion and Sensibility
• /
• v.14 no.4
• /
• pp.627-636
• /
• 2011

The purpose of this study is to develop the multi-modal interaction system. This system provides realistic and an immersive experience through multi-modal interaction. The system recognizes user behavior, intention, and attention, which overcomes the limitations of uni-modal interaction. The multi-modal interaction system is based upon gesture interaction methods, intuitive gesture interaction and attention evaluation technology. The gesture interaction methods were based on the sensors that were selected to analyze the accuracy of the 3-D gesture recognition technology using meta-analysis. The elements of intuitive gesture interaction were reflected through the results of experiments. The attention evaluation technology was developed by the physiological signal analysis. This system is divided into 3 modules; a motion cognitive system, an eye gaze detecting system, and a bio-reaction sensing system. The first module is the motion cognitive system which uses the accelerator sensor and flexible sensors to recognize hand and finger movements of the user. The second module is an eye gaze detecting system that detects pupil movements and reactions. The final module consists of a bio-reaction sensing system or attention evaluating system which tracks cardiovascular and skin temperature reactions. This study will be used for the development of realistic digital entertainment technology.

• International Journal of Computer Science & Network Security
• /
• v.23 no.3
• /
• pp.31-48
• /
• 2023

The popularization of smart devices and subsequent optimization of their sensing capacity has resulted in a novel mobile crowdsensing (MCS) pattern, which employs smart devices as sensing nodes by recruiting users to develop a sensing network for multiple-task performance. This technique has garnered much scholarly interest in terms of sensing range, cost, and integration. The MCS is prevalent in various fields, including environmental monitoring, noise monitoring, and road monitoring. A complete MCS life cycle entails task allocation, data collection, and data aggregation. Regardless, specific drawbacks remain unresolved in this study despite extensive research on this life cycle. This article mainly summarizes single-task, multi-task allocation, and space-time multi-task allocation at the task allocation stage. Meanwhile, the quality, safety, and efficiency of data collection are discussed at the data collection stage. Edge computing, which provides a novel development idea to derive data from the MCS system, is also highlighted. Furthermore, data aggregation security and quality are summarized at the data aggregation stage. The novel development of multi-modal data aggregation is also outlined following the diversity of data obtained from MCS. Overall, this article summarizes the three aspects of the MCS life cycle, analyzes the issues underlying this study, and offers developmental directions for future scholars' reference.

• Proceedings of the KSRS Conference
• /
• 2003.11a
• /
• pp.1433-1435
• /
• 2003

Very decisive progress was made in advancing fundamental POL-IN-SAR theory and algorithm development during the past decade. This was accomplished with the aid of airborne & shuttle platforms supporting single -to-multi-band multi-modal POL-SAR and also some POL-IN-SAR sensor systems, which will be compared and assessed with the aim of establishing the hitherto not completed but required missions such as tomographic and holographic imaging. Because the operation of airborne test-beds is extremely expensive, aircraft platforms are not suited for routine monitoring missions which is better accomplished with the use drones or UAVs. Such unmanned aerial vehicles were developed for defense applications, however lacking the sophistic ation of implementing advanced forefront POL-IN-SAR technology. This shortcoming will be thoroughly scrutinized resulting in the finding that we do now need to develop most rapidly POL-IN-SAR drone-platform technology especially for environmental stress-change monitoring with a great variance of applications beginning with flood, bush/forest-fire to tectonic-stress (earth-quake to volcanic eruptions) for real-short-time hazard mitigation. However, for routine global monitoring purposes of the terrestrial covers neither airborne sensor implementation - aircraft and/or drones - are sufficient; and there -fore multi-modal and multi-band space-borne POL-IN-SAR space-shuttle and satellite sensor technology needs to be further advanced at a much more rapid phase. The existing ENVISAT with the forthcoming ALOSPALSAR, RADARSAT-2, and the TERRASAT will be compared, demonstrating that at this phase of development the fully polarimetric and polarimetric-interferometric modes of operation must be viewed and treated as preliminary algorithm verification support modes and at this phase of development are still not to be viewed as routine modes.

• Transactions of the Korean Society for Noise and Vibration Engineering
• /
• v.25 no.11
• /
• pp.787-797
• /
• 2015

In this paper, a cochlea-inspired artificial filter bank(CAFB) was developed in order to efficiently acquire dynamic response of structure, and it was also evaluated via dynamic response experiments. To sort out signals containing significant modal information from all the dynamic responses of structure, it was made to adopt a band-pass filter optimizing algorithm(BOA) and a peak-picking algorithm (PPA). Optimally designed on the basis of El-centro and Kobe earthquake signals, it was then embedded into the wireless multi-measurement system(WiMMS). In order to evaluate the performance of the developed CAFB, a vibration test was conducted using the El-centro and Kobe earthquake signals, and structural responses of a two-span bridge were obtained and analyzed simultaneously by both a wired measurement system and a CAFB-based WiMMS. The test results showed that the compressed dynamic responses acquired by the CAFB-based WiMMS matched with those of the wired system, and they included significant modal information of the two-span bridge. Therefore this study showed that the developed CAFB could be used as a new, economic, and efficient measurement device for wireless sensor networks(WSNs) based real time structural health monitoring because it could reconstruct the whole dynamic response using the compressed dynamic response with significant modal information.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.3 no.5
• /
• pp.423-443
• /
• 2009

We present the design and implementation of a unique sensing and actuation application -- the Illuminator: a sensor network-based intelligent light control system for entertainment and media production. Unlike most sensor network applications, which focus on sensing alone, a distinctive aspect of the Illuminator is that it closes the loop from light sensing to lighting control. We describe the Illuminator's design requirements, system architecture, algorithms, implementation and experimental results. The system uses the Illumimote, a multi-modal and high fidelity light sensor module well-suited for wireless sensor networks, to satisfy the high-performance light sensing requirements of entertainment and media production applications. The Illuminator system is a toolset to characterize the illumination profile of a deployed set of fixed position lights, generate desired lighting effects for moving targets (actors, scenic elements, etc.) based on user constraints expressed in a formal language, and to assist in the set up of lights to achieve the same illumination profile in multiple venues. After characterizing deployed lights, the Illuminator computes optimal light settings at run-time to achieve a user-specified actuation profile, using an optimization framework based on a genetic algorithm. Uniquely, it can use deployed sensors to incorporate changing ambient lighting conditions and moving targets into actuation. Experimental results demonstrate that the Illuminator handles various high-level user requirements and generates an optimal light actuation profile. These results suggest that the Illuminator system supports entertainment and media production applications.

• Proceedings of the Korean Information Science Society Conference
• /
• 2004.04a
• /
• pp.124-126
• /
• 2004

본 논문에서는 유비쿼터스 헬스케어 시스템을 위한 센싱 단말기 구현 내용을 설명한다. 본 연구에서 설계한 유비쿼터스 헬스케어 시스템은 센싱 단말기, 처리 단말기, DB 서버, 전문가 서버로 구성된다. 센싱 단말기는 사용자의 생체 신호를 검출하고 처리한 후 그 결과를 무선 통신을 통해 처리 단말기로 전달한다. 본 연구에서 구현한 센싱 단말기는 8MHz로 동작하는 AVR MicroController 처리기와 맥박 센서, 그리고 Bluetooth 무선 통신 모듈로 구성되어 있다. 또한 5개의 포트별로 8개의 입력 라인이 있어 다중 센싱 (Multi-Modal Sensing)이 가능하며, Bluetooth를 지원하는 다양한 처리 단말기와 연동이 가능하다는 특징을 갖는다. 실험결과 사용자의 생체 신호를 제대로 검출하여 무선 통신으로 전달함을 알 수 있었다.