• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.16 no.5
• /
• pp.1516-1539
• /
• 2022

This article shows a set of physical information fusion IoT systems that we designed for smart buildings. Its essence is a computer system that combines physical quantities in buildings with quantitative analysis and control. In the part of the Internet of Things, its mechanism is controlled by a monitoring system based on sensor networks and computer-based algorithms. Based on the design idea of the agent, we have realized human-machine interaction (HMI) and machine-machine interaction (MMI). Among them, HMI is realized through human-machine interaction, while MMI is realized through embedded computing, sensors, controllers, and execution. Device and wireless communication network. This article mainly focuses on the function of wireless sensor networks and MMI in environmental monitoring. This function plays a fundamental role in building security, environmental control, HVAC, and other smart building control systems. The article not only discusses various network applications and their implementation based on agent design but also demonstrates our collaborative information fusion strategy. This strategy can provide a stable incentive method for the system through collaborative information fusion when the sensor system is unstable in the physical measurements, thereby preventing system jitter and unstable response caused by uncertain disturbances and environmental factors. This article also gives the results of the system test. The results show that through the CPS interaction of HMI and MMI, the intelligent building IoT system can achieve comprehensive monitoring, thereby providing support and expansion for advanced automation management.

• Journal of the Ergonomics Society of Korea
• /
• v.16 no.1
• /
• pp.107-117
• /
• 1997

It is necessary to evaluate HMI inaspects of human factors in the design stage of MMIS(man machine interface system) and feedback the result of evaluation because operators performance is mainly influenced by the HMI. Therefore, the MMIS design should be reflected the operators psychological, behavioral and physiological characteristics in the interaction with human machine interface(HMI) in order to improve the safety and availability of the MMIS of a nuclear power plant(NPP) by reduction of human error. The development of human factors experimental evaluation techniques and integrated test facility(ITF) for the human factors evaluation become an important research field to resolve hi,am factors issues on the design of an advanced control room(ACR). We developed am ITF, which is aimed to experiment with the design of the ACR and the human machine interaction as it relates to the control of NPP. This paper presents the development of an ITF that consists of three rooms such as main test room(MTR), supporting test room(STR) and experiment control room(ECR). And, the ITF has a various facilities such as a human machine simulator(HMS), experimental measurement systems and data analysis and experiment evaluation supporting system(DAEXESS). The HMS consists of full-scope simulation model of Korean standard NPP and advanced HMI based on visual display nits (VDUS) such as touch color CRT, large scale display panel(LSDP), flat panel display unit and so on.

• Korean Journal of Computational Design and Engineering
• /
• v.12 no.2
• /
• pp.87-94
• /
• 2007

The functional behavior of a consumer electronic product is nearly all expressed with human-machine interaction (HMI) tasks. Although physical prototyping and computer aided design (CAD) software can show the appearance of the product, they cannot properly reflect its functional behavior. In this paper, we propose an approach to virtual prototyping (VP) that incorporates HMI functional simulation into virtual reality techniques in order to enables users to capture not only the realistic look of a consumer electronic product but also its functional behavior. We adopt state transition methodology to capture the HMI functional behavior of the product into a state transition chart, which is later used to construct a finite state machine (FSM) for the functional simulation of the product. The FSM plays an important role to control the transition between states of the product. We have developed a VP system based on the proposed approach. The system receives input events such as mouse clicks on buttons and switches of the virtual prototype model, and it reacts to the events based on the FSM by activating associated activities. The system provides the realistic visualization of the product and the vivid simulation of its functional behavior using head-mounted displays (HMD) and stereo speakers. It can easily allow users to perform functional evaluation and usability testing. A case study about the virtual prototyping of an MP3 player is given to show the usefulness of the proposed approach.

• The Transactions of The Korean Institute of Electrical Engineers
• /
• v.62 no.5
• /
• pp.717-722
• /
• 2013

The wastewater reclamation and reusing system has been rising as an alternative of water resource exhaustion that the whole world is experiencing. In order to be able to bring about improvement of the existing wastewater reclamation and reusing system, this research has developed of Conversion-Human Machine Interaction (C-HMI) based real-time control and monitoring system such as a sensor module and gate module, web monitoring system. This system was communication almost-error-free in various environment and situation. As a result, we have achieved our goal that has to doing work correctly as a sensor and gateway module that communication error is less than 0.2% throughout the embodied system and add that it can be easily controled and configured as an interface equipment to a complex sensor of water quality. According to this, the construction of a database capable of analyzing and assessing collection, storage and various elements of reliable water quality and flow rate data can be possible.

• Proceedings of the Korean Operations and Management Science Society Conference
• /
• 2005.05a
• /
• pp.854-861
• /
• 2005

The functional behavior of a portable consumer electronic (PCE) product is nearly all expressed with human-machine interaction (HMI) tasks. Although physical prototyping and computer aided design (CAD) software can show the appearance of the product, they cannot properly reflect its functional behavior. In this paper, we propose a virtual prototyping (VP) system that incorporates virtual reality and HMI functional simulation in order to enables users to capture not only the realistic look of a PCE product but also its functional behavior. We obtain geometric part models of the product and their assembly and kinematics information with the help of CAD and reverse engineering tools, and visualize them with various display tools. We adopt state transition methodology to capture the HMI functional behavior of the product into a state transition chart, which is later used to construct a finite state machine (FSM) for the functional simulation of the product. The FSM plays an important role to control the transition between states of the product. The proposed VP system receives input events such as mouse clicks on buttons and switches of the virtual prototype model, and it reacts to the events based on the FSM by activating associated activities. The VP system provides the realistic visualization of the product and the vivid simulation of its functional behavior. It can easily allow users to perform functional evaluation and usability testing. Moreover, it can greatly reduce communication errors occurring in a typical product development process. A case study about VP of an MP3 player is given to show the usefulness of the proposed VP system.

• Korean Journal of Computational Design and Engineering
• /
• v.20 no.1
• /
• pp.75-83
• /
• 2015

In the virtual prototyping (VP) of digital products, it is important to provide the people involved in product development with the visualization and interaction of the products, and the simulation of their human machine interaction (HMI) behaviors in interactive 3D virtual environments. Especially, for the HMI behavior simulation, it is necessary to represent them properly and to play them back effectively according to user interaction in the virtual environments. In a conventional approach to HMI behavior simulation, user interface (UI) designers use UI design software tools to generate the HMI behavior of a digital product of interest. Due to lack of reusability of the HMI behavior, VP developers need to analyze and integrate it into a VP system for its simulation in a 3D virtual environment. As this approach hinders the effective communication between the UI designers and the VP developers, it is easy to create errors and thereby it takes significant time and effort especially when it is required to represent the HMI behavior to the finest level of detail. In order to overcome the shortcomings of the conventional approach, we propose an approach for representing the HMI behavior of a digital product using XML (eXtensible Markup Language) and for reusing it to perform the HMI behavior simulation in 3D virtual environments. Based on the approach, a VP system has been developed and applied for the design evaluation of various products. A case study about the design evaluation is given to show the usefulness of the proposed approach.

• Journal of Sensor Science and Technology
• /
• v.21 no.1
• /
• pp.28-33
• /
• 2012

The visual sensing of human hands plays an important part in many man-machine interaction/interface systems. Most existing visionbased hand detection techniques depend on the color cues of human skin. The RGB color image from a vision sensor is often transformed to another color space as a preprocessing of hand detection because the color space transformation is assumed to increase the detection accuracy. However, the actual effect of color space transformation has not been well investigated in literature. This paper discusses a comparative evaluation of the pixel classification performance of hand skin detection in four widely used color spaces; RGB, YIQ, HSV, and normalized rgb. The experimental results indicate that using the normalized red-green color values is the most reliable under different backgrounds, lighting conditions, individuals, and hand postures. The nonlinear classification of pixel colors by the use of a multilayer neural network is also proposed to improve the detection accuracy.

• Journal of the HCI Society of Korea
• /
• v.14 no.2
• /
• pp.49-60
• /
• 2019

With the appearance of self-driving cars and electric cars, the automobile industry is rapidly changing. In the midst of these changes, HMI studies are becoming more important as to how the driver obtains safety and convenience with controlling the vehicle. This study sought to understand how automobile manufacturers understand the driving situation, and how they define and limit driver interaction. For this, prior studies about HMI were reviewed and 15 participants performed an on-road study to drive vehicles from five manufacturers with using their interfaces. The results of the study confirmed that buttons and switches that are easily controlled by the user while driving were different from manufacturer to manufacturer. And there are some buttons that are more intensively controlled and others that are difficult to control while driving. It was able to derive 'selection and concentration' from Audi's vehicle, 'optimization of the driving ' from BMW's, 'simple and minimize' from Benz's vehicle, 'remove the manual distraction' from the vehicle of Lexus, and 'visual stability' from KIA's vehicle as the distinctive keywords for the HMI. This shows that each manufacturer has a different definition and interpretation of the driver's driving control area. This study has a distinct value in that it has identified the characteristics of vehicle-specific HMI in actual driving conditions, which is not apparent in appearance. It is expected that this research approach can be useful to see differences in interaction through actual driving despite changes in driving environment such as vehicle platooning and self-driving technology.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.50 no.6
• /
• pp.212-220
• /
• 2013

Recently, there has been tremendous increase in human-computer/machine interaction system, where the goal is to provide with an appropriate service to the user at the right time with minimal human inputs for human augmented cognition system. To develop an efficient human augmented cognition system based on human computer/machine interaction, it is important to interpret the user's implicit intention, which is vague, in addition to the explicit intention. According to cognitive visual-motor theory, human eye movements and pupillary responses are rich sources of information about human intention and behavior. In this paper, we propose a novel approach for the identification of human implicit visual search intention based on eye movement pattern and pupillary analysis such as pupil size, gradient of pupil size variation, fixation length/count for the area of interest. The proposed model identifies the human's implicit intention into three types such as navigational intent generation, informational intent generation, and informational intent disappearance. Navigational intent refers to the search to find something interesting in an input scene with no specific instructions, while informational intent refers to the search to find a particular target object at a specific location in the input scene. In the present study, based on the human eye movement pattern and pupillary analysis, we used a hierarchical support vector machine which can detect the transitions between the different implicit intents - navigational intent generation to informational intent generation and informational intent disappearance.

• Journal of Navigation and Port Research
• /
• v.46 no.5
• /
• pp.458-470
• /
• 2022

As the 4th industrial revolution progresses, the application of several cutting-edge technologies such as the Internet of Things, big data, and mixed reality (MR) in relation to autonomous ships is being considered in the maritime logistics field. The aim of this study was to apply the concept of a digital twin model based on Human Machine Interaction (HMI) including a digital twin model and the role of an operator to a ship. The role of the digital twin is divided into information provision, support, decision, and implementation. The role of the operator is divided into operation, decision-making, supervision, and standby. The system constituting the ship was investigated. The digital twin system that could be applied to the ship was also investigated. The cloud-based digital twin system architecture that could apply investigated applications was divided into ship data collection (part 1), cloud system (part 2), analysis system/ application (part 3), and MR/mobile system (part 4). A Mixed Reality device HoloLens was used as an HMI equipment to perform a simulation test of a digital twin system of an 8 m battery-based electric propulsion ship.