Human Mental Condition Monitoring through Measurement of Physiological Signals

Abstract

Nowadays, one of the most common diseases is chronic mental fatigue syndrome. This can be caused by many factors, such as busy life, heavy workload, high population density, and adverse technological impact. Most office workers and students who are sitting all day long while being exposed to this kind of environments are likely to be involved in the mental illness. Therefore, to prevent the illness, it has been highly required to design a device that enables mental fatigue to be monitored continuously without human intervention. This paper proposes a linear regression method to reliably estimating the level of human mental fatigue using wearable physiological sensors, with an estimation error of 0.852. Also, this paper presents an Android application that is able to check mental health conditions in daily life.

1. INTRODUCTION

One of the biggest problem we are facing in the 21st century is the accumulation of mental fatigue. It can negatively impact our performance at work, family life, and social relationships. But it is difficult to detect the mental fatigue level without any medical equipment. Mental fatigue has been mainly assessed by using electroencephalographic (EEG) measures [1, 2]. So many people with it do not know they got mental fatigue.

Mental fatigue is the result of brain over-activity. It can happen when a person expends too much mental effort on a project or task. Basically, it makes our brain overwork, leaving us exhausted, hampering our productivity and overall cognitive function. The most common symptoms include mental block, lack of motivation, irritability, stress eating, loss of appetite and insomnia [3].

Research shows that mental fatigue results in an inability to concentrate and an increase in simple mistakes. Mental fatigue may lead to feeling stressed, irritated and even depressed. What's more, being in a state of mental fatigue not only affects people's well-being, it also spills over into their interactions with family and others they associate with. It's draining for them to be around someone who is continuously mentally exhausted. If left unchecked, mental fatigue can also lead to all sorts of serious health problems, including anxiety and burnout. So it has been highly required to develop a system that enables people's mental fatigue to be monitored continuously without any medical expert or equipment.

In this paper, we design an Arduino-based system that would measure the mental fatigue, which is easy to measure, light and body fit. When people have mental fatigue, the value of blood pressure, heart rate, and body temperature changes because of the reaction of heart, kidney and liver. Therefore, we use Arduino-compatible wearable sensors to measure those physiological signals. Then, we calculate mental fatigue levels using these the physiological values by linear regression. Finally, we present an Android application that is implemented to check mental health conditions in daily life.

This paper is an extension of our previous work [4] and includes more details for system implementation and extended experimental results.

2. MENTAL FATIGUE MEASUREMENT

The hypothalamus is a small region of the brain (Fig. 1) and plays a crucial role on keeping the body in a healthful, balanced state as much as possible. The body is always trying to achieve this balance. Feelings of hunger, for example, are the brain's way of letting its owner know that they need more nutrients [5]. This hypothalamus acts as the connector between the endocrine and nervous systems in many essential functions of the body such as: body temperature, thirst, appetite, weight control, emotions, sleep cycles, sex drive, childbirth, blood pressure, heart rate, production of digestive juices and balancing bodily fluids [6]. Because of the activity of hypothalamus, when people get mental fatigue, lots of symptoms appear in their body, including changes in vital signs such as blood pressure, heart rate, and body temperature. Therefore if we have sensors that can measure the physiological signals, we can detect our mental fatigue level [3].

Fig. 1. Location of the human hypothalamus.

We measure three physiological signals (heart rate, blood pressure and body temperature) and formularize the relationship between the physiological measurements and the mental fatigue levels as follows.

\(F=\Psi(H R, B P, T)\)       (1)

Here, F, HR, BR and T denote mental fatigue level, heart rate, blood pressure and body temperature, respectively. The ground-truth stress levels are obtained from survey results using a mental fatigue indicator questionnaire (described in the subsequent subsection). Thus, the unknown function Ψ can be estimated by regression.

We use Arduino-compatible sensors to measure the physiological signals. The details for the sensors will be explained in the subsequent subsections.

2.1 Blood pressure sensor

To measure blood pressure, the wrist blood pressure monitor, HBP-700, is used. The automatic blood pressure monitor has its own pressure sensor which is 2SMPP-02 (Fig. 2). It inflates a cuff surrounding an arm with sufficient pressure to prevent blood flow in the local main artery. This pressure is gradually released until the moment that the blood begins to flow through the artery, the measurement of which determines the systolic pressure. Pulse rate is also sensed at the same time with blood pressure. The measurement taken when the blood flow is no longer restricted determines the diastolic pressure. This measurement cycle is performed automatically with a pressure pump and pressure sensor [7].

Fig. 2. Pressure sensor, 2SMPP-02.

2.2 Heart rate sensor

The HBP-700 device does not have its own heart rate sensor. But it detects heart beats from the blood pressure graph (Fig. 3). When our heart beats, it squeezes and pushes blood through our arteries to the rest of body. This force creates pressure on those blood vessels, and that is systolic blood pressure. So if we count each systolic (or diastolic) blood pressure we can detect heart rate in beats per minute (BPM) [8].

\(H R=60 / t_{\text {arg }}, \text { where } t_{\text {arg }}=\frac{1}{K} \sum_{k=1}^{K}\left(t_{\text {peak }}^{k+1}-t_{\text {peak }}^{k}\right)\)       (2)

Here, K is the number of systolic blood pressure in a certain length of time.

Fig. 3. Calculating heart rate in blood pressure graph.

2.3 Body temperature sensor

Body temperature is a measure of human body's ability to make and get rid of heat. The average normal temperature is 37°C (= 98.6°F). It can be measured on several body parts, such as the for head, mouth, rectum, armpit and ear [8].

With an analog thermometer, temperature variations cause a change in the volume of the liquid, which in turn causes it to rise or fall in a small tube. A card bearing some numbers is placed near the tube, and by comparing the height of the column to the nearest number, a human can “digitize” the temperature and turn it into a string of digits. A digital thermometer works in somewhat the same way. There are things that change their electrical resistance with temperature, or generate a voltage proportional to temperature in several different ways. But still, the result is an analog representation of temperature. The signal can then be sent to some sort of “analog to digital converter” which outputs a string of digits representing a best approximation to the continuous analog signal. The string of digits is then “scaled” — divided or multiplied as needed — to get the result in the expected “Celsius” units.

In this paper, we use DS18B20 human body temperature sensor (Fig. 4) whose electrical resistance changes exponentially with change in temperature. The sensor can be placed on armpit and connected with Arduino micro-controller directly [9].

Fig. 4. Temperature sensor, DS18B20.

2.4 Mental fatigue indicator questionnaire

To get the ground truth for the mental fatigue level, we use a mental fatigue indicator questionnaire which was developed by Loviband [10]. By answering mental fatigue indicator questions, we can find out how tired people are mentally. There are four choices to answer for each question. By selecting the choices that best describe how often one experiences each mental fatigue indicator during a week, the mental fatigue is scaled to 10 levels [11]. The questionnaire consists of ten questions and four answers [12] (Table 1).

Table 1. Mental fatigue indicator questionnaire

3. SYSTEM CONFIGURATION

In order to monitor the human mental fatigue/ condition using the proposed method, various devices are required, which includes the sensors for measuring the physiological signals, a micro-controller for controlling the sensors, a smartphone for interacting with the user, a relay switch for turning on/off the system and a device for supporting the communication between the micro-controller and the smartphone. Table 2 shows the hardware components we chose to build our system. The main reasons why to choose these components were because they were accurate, fast, small, and light. Also these components are easy to use and cheap to build our system.

Table 2. Hardware components used to build our system

All the components including sensors are wired or wirelessly connected to Arduino micro-controller (Fig. 5) [13]. After collecting the physiological signals from the sensors, the data is sent to the Android phone via Bluetooth wireless communication. Then the data is processed and the mental fatigue level is determined in an Android application.

Fig. 5. System configuration and process flow.

Our system should work automatically to measure about 3-5 times per day. However, the HBP700 has only start/stop button and need to push the button manually to run it. That is why we used a relay switch to turn on/off the device automatically. In other words, we used the 5V relay instead of the start and stop button of the HBP-700 device to measure blood pressure and heart rate. To make a connection we need to connect the relay pins NO and NC to the A and B pins of start button in the HBP-700 device (Fig. 6).

Fig. 6. Adding a relay to turn on/off the system automatically.

4. EXPERIMENTS AND DISCUSSION

For experiment, heart rate, blood pressure and body temperature were measured from 80 participants who are of different genders and ages and have different mental fatigue levels. Measurements were taken when the participants were at rest sitting on the chair as shown in Fig. 7. Measurement For experiment, heart rate, blood pressure and body temperature were measured from 80 participants who are of different genders and ages and have different mental fatigue levels. Measurements were taken when the participants were at rest sitting on the chair as shown in Fig. 7. Measurements may be affected by the unknown measuring environments or the physical condition of the participants. In this paper, we assume that such a variation is ignorable. The participants also filled out the mental fatigue questionnaire after the measurement. The participants were divided into two groups: 50 for estimating the unknown function model in Eq. (1) and 30 for testing the estimated model.

Fig. 7. Measuring mental fatigue.​​​​​​​

Firstly, we tried to find out the relationship (ψ in Eq. (1)) between mental fatigue levels and the individual physiological measurements. As shown in Fig. 8, it turned out that, as one of physiological values increased, the mental fatigue level also increased. That means all the physiological values are proportional to mental fatigue levels (this has also been confirmed in a number of previous studies [3]). In this paper, we assumed that the relationships are linear (most samples are well fitted to a straight line in Fig. 8). So, we defined the function ψ in Eq. (1) as a linear function as follows.

\(F=w_{1} H R+w_{2} B P+w_{3} T\)       (3)

Fig. 8. Correlation of heart rate, blood pressure and body temperature to mental fatigue level. The straight line represents the asymptotic line of data points.​​​​​​​

From the physiological measurements and questionnaire results collected from the participants, the weights w₁, w₂ and w₃ were calculated by linear regression as follows.

\(\left[\begin{array}{l} w_{1} \\ w_{2} \\ w_{3} \end{array}\right]=\left[\begin{array}{c} \sum_{i=1}^{N} H R_{i} \cdot H R_{i} \sum_{i=1}^{N} H R_{i} \cdot B P_{i} \sum_{i=1}^{N} H R_{i} \cdot T_{i} \\ \sum_{i=1}^{N} H R_{i} \cdot B P_{i} \sum_{i=1}^{N} B P_{i} B P_{i} \sum_{i=1}^{N} B P_{i} T_{i} \\ \sum_{i=1}^{N} H R_{i} \cdot T_{i} \sum_{i=1}^{N} B P_{i} T_{i} \sum_{i=1}^{N} T_{i} \cdot T_{i} \end{array}\right]^{-1}\left[\begin{array}{c} \sum_{i=1}^{N} F_{i} \cdot H R_{i} \\ \sum_{i=1}^{N} F_{i} B P_{i} \\ \sum_{i=1}^{N} F_{i} T_{i} \end{array}\right] .\)       (4)

Here, N is the number of measurements (50 in our experiments). Before the regression, the measurements and fatigue levels were normalized to [0, 1]. In our experiment, the results are that: w1 = 0.69, w₂ = 0.28 and w₃ = 0.038. The human mental fatigue was more strongly correlated to heart rate and blood pressure than body temperature. This may be because the range of changes in body temperature is relatively small (at most 3°C) and the temperature sensor is not sufficiently accurate. With these weights, we evaluated the accuracy of estimating the human mental fatigue of 30 participants using Eq. (3). The mean absolute error was 0.852 and its standard deviation was 0.571. Here, the error means the difference between the estimated fatigue level and that obtained from the questionnaire method. Therefore, our system can be used as a reliable mental fatigue measurement device.

To monitor mental fatigue continuously, an Android application that communicates with the Arduino was implemented. As shown in Fig. 9, the application is designed easy to use and understand [14]. It consists of three screens (activities) that appear sequentially. In the screen ‘A’, there is a Bluetooth connection button to connect the smartphone with the Arduino Nano. And, there are a memory button to save measurements and a time setup button to allow measuring mental fatigue in a set time. In the screen ‘B’, the physiological measurement values received from the Arduino Nano are displayed. And the last screen shows the results of the mental fatigue level and the mental condition.

5. CONCLUSION

In this paper, we proposed a linear regression method for estimating human mental fatigue levels from physiological signals measured using wearable sensors. Then, we implemented an Android application wirelessly connected with an Arduinobased sensing system for daily-monitoring of human mental fatigue/condition. By using the application, we confirmed that people's mental fatigue/condition can be monitored easily without any medical expert or equipment.

In this paper, we assumed the linear relationship between physiological signals and mental fatigue levels. However, to monitor human mental condition more accurately, we will attempt to find out a more sophisticated model by learning the unknown relationship using a neural network.