• Journal of Intelligence and Information Systems
• /
• v.19 no.1
• /
• pp.95-110
• /
• 2013

Recently, the amount of unstructured data being generated through a variety of social media has been increasing rapidly, resulting in the increasing need to collect, store, search for, analyze, and visualize this data. This kind of data cannot be handled appropriately by using the traditional methodologies usually used for analyzing structured data because of its vast volume and unstructured nature. In this situation, many attempts are being made to analyze unstructured data such as text files and log files through various commercial or noncommercial analytical tools. Among the various contemporary issues dealt with in the literature of unstructured text data analysis, the concepts and techniques of opinion mining have been attracting much attention from pioneer researchers and business practitioners. Opinion mining or sentiment analysis refers to a series of processes that analyze participants' opinions, sentiments, evaluations, attitudes, and emotions about selected products, services, organizations, social issues, and so on. In other words, many attempts based on various opinion mining techniques are being made to resolve complicated issues that could not have otherwise been solved by existing traditional approaches. One of the most representative attempts using the opinion mining technique may be the recent research that proposed an intelligent model for predicting the direction of the stock index. This model works mainly on the basis of opinions extracted from an overwhelming number of economic news repots. News content published on various media is obviously a traditional example of unstructured text data. Every day, a large volume of new content is created, digitalized, and subsequently distributed to us via online or offline channels. Many studies have revealed that we make better decisions on political, economic, and social issues by analyzing news and other related information. In this sense, we expect to predict the fluctuation of stock markets partly by analyzing the relationship between economic news reports and the pattern of stock prices. So far, in the literature on opinion mining, most studies including ours have utilized a sentiment dictionary to elicit sentiment polarity or sentiment value from a large number of documents. A sentiment dictionary consists of pairs of selected words and their sentiment values. Sentiment classifiers refer to the dictionary to formulate the sentiment polarity of words, sentences in a document, and the whole document. However, most traditional approaches have common limitations in that they do not consider the flexibility of sentiment polarity, that is, the sentiment polarity or sentiment value of a word is fixed and cannot be changed in a traditional sentiment dictionary. In the real world, however, the sentiment polarity of a word can vary depending on the time, situation, and purpose of the analysis. It can also be contradictory in nature. The flexibility of sentiment polarity motivated us to conduct this study. In this paper, we have stated that sentiment polarity should be assigned, not merely on the basis of the inherent meaning of a word but on the basis of its ad hoc meaning within a particular context. To implement our idea, we presented an intelligent investment decision-support model based on opinion mining that performs the scrapping and parsing of massive volumes of economic news on the web, tags sentiment words, classifies sentiment polarity of the news, and finally predicts the direction of the next day's stock index. In addition, we applied a domain-specific sentiment dictionary instead of a general purpose one to classify each piece of news as either positive or negative. For the purpose of performance evaluation, we performed intensive experiments and investigated the prediction accuracy of our model. For the experiments to predict the direction of the stock index, we gathered and analyzed 1,072 articles about stock markets published by "M" and "E" media between July 2011 and September 2011.

• Korean Journal of Soil Science and Fertilizer
• /
• v.44 no.5
• /
• pp.887-894
• /
• 2011

To evaluate the effects of abiotic factors of paddy fields on greenhouse gases (GHGs) emissions from rice paddy fields, $CH_4$ emission amounts were investigated from rice paddy fields by different soil organic matter contents, paddy types, and agricultural climatic zone in Yeongnam area during 3 years. $CH_4$ emission amounts according to soil organic matter contents in paddy field were conducted at having different contents of 5 soil organic matters fields (23.6, 28.7, 31.0, 34.5, and $38.0g\;kg^{-1}$), The highest $CH_4$ emission amount was recorded in the highest soil organic matters plot of $38.0g\;kg^{-1}$. High correlation coefficient (r=$0.963^{**}$) was obtained between $CH_4$ emissions from paddy fields and their soil organic matter contents. According to paddy field types, $CH_4$ emission amounts were investigated at 4 different paddy fields as wet paddy, sandy paddy, immature paddy, and mature paddy. The highest $CH_4$ emissions was recorded in wet paddy (100%) and followed as immature paddy 64.0%, mature paddy 46.8%, and sandy paddy 23.8%, respectively. For the effects of temperature on $CH_4$ emissions from paddy fields, 4 agricultural climatic zones were investigated, which were Yeongnam inland zone (YIZ), eastern coast of central zone (ECZ), plain area of Yeongnam inland mountainous zone (PMZ), and mountainous area of Yeongnam inland mountainous zone (MMZ). The order of $CH_4$ emission amounts from paddy fields by agricultural climatic zone were YIZ (100%) > ECZ (94.6%) > PMZ (91.6%) > MMZ (78.9%). The regression equation between $CH_4$ emission amounts from paddy fields and average air temperature of Jul. to Sep. of agricultural climatic zone was y = 389.7x-4,287 (x means average temperature of Jul. to Sep. of agricultural climatic zone, $R^2=0.906^*$)

• Korean Journal of Environment and Ecology
• /
• v.32 no.6
• /
• pp.698-708
• /
• 2018

The purpose of this study was to investigate differences in the breeding call characteristics of cicada species found in urban and forest areas in the central region of Korea by examining the interspecific effects and environmental factors affecting the breeding calls and breeding call patterns. The selected research sites were Gyungnam Apartment in Bangbae-dong, Seoul for the urban area and Chiak Mountain National Park in Wonju for the forest area. The research method for both sites was to record cicada breeding calls for 24 hours with a recorder installed at the site and analyze the results. Data from the Korea Meteorological Administration were used for environmental factors. The research period was from June 19, 2017 to September 30, 2017. As a result of the study, there were differences in the emergence of species between the two research sites: while Platypleura kaempferi, Hyalessa fuscata, Meimuna opalifera, Graptopsaltria nigrofuscata, and Suisha coreana were observed at both sites, Cryptotympana atrata was observed in the urban area and Leptosemia takanonis in the forest area only. The emergence periods of cicadas at the two sites were also different. The activities of P. kaempferi and L. takanonis were noticeable in the forest area. In the urban area, however, L. takanonis was not observed and the duration of activity of P. kaempferi was short. In the urban area, C. atrata appeared and sang for a long period; H. fuscata, M. opalifera, and G. nigrofuscata appeared earlier than in the forest area. S. coreana appeared earlier in the forest area than in the urban area. According to the daily call cycle analysis, even cospecific cicada showed a wide variation in their daily cycle depending on the region and the interspecific effects between different cicadas, and the environmental differences between the urban and forest areas affected the calls of cicadas. The results of correlation analysis between each cicada breeding calls and environmental factors of each site showed positive correlation with average temperature of most cicadas except P. kaempferi and C. atrata. The same species of each site showed positive correlations with more diverse weather factors such as solar irradiance. Logistic regression analysis showed that cicadas with overlapping calling times had significant effects on each other's breeding calls. C. atrata, which appeared only in the urban area, had a positive effect on the calling frequency of H. fuscata, M. opalifera, and G. nigrofuscata, which called in the same period. Additionally, L. takanonis, which appeared only in the forest area, and P. kaempferi had a positive effect on each other, and M. opalifera had a positive effect on the calling frequency of H. fuscata and G. nigrofuscata in the forest area. For the environmental factors, the calling frequency of cicadas was affected by the average temperatures of the urban and forest areas, and cicadas that appeared in the forest area were also affected by the amount of solar radiation. According to the results of statistical analysis, urban cicadas with similar activity periods are influenced by species, especially with respect to urban dominant species, C. atrata. Forest cicadas were influenced by species, mainly M. opalifera, which is a forest dominant species. The results of the meteorological impact analysis were similar to those of the correlation analysis, and were influenced mainly by the temperature, and the influence of the insolation was more increased in the forests.

• Journal of Intelligence and Information Systems
• /
• v.25 no.2
• /
• pp.141-166
• /
• 2019

Recently, channels like social media and SNS create enormous amount of data. In all kinds of data, portions of unstructured data which represented as text data has increased geometrically. But there are some difficulties to check all text data, so it is important to access those data rapidly and grasp key points of text. Due to needs of efficient understanding, many studies about text summarization for handling and using tremendous amounts of text data have been proposed. Especially, a lot of summarization methods using machine learning and artificial intelligence algorithms have been proposed lately to generate summary objectively and effectively which called "automatic summarization". However almost text summarization methods proposed up to date construct summary focused on frequency of contents in original documents. Those summaries have a limitation for contain small-weight subjects that mentioned less in original text. If summaries include contents with only major subject, bias occurs and it causes loss of information so that it is hard to ascertain every subject documents have. To avoid those bias, it is possible to summarize in point of balance between topics document have so all subject in document can be ascertained, but still unbalance of distribution between those subjects remains. To retain balance of subjects in summary, it is necessary to consider proportion of every subject documents originally have and also allocate the portion of subjects equally so that even sentences of minor subjects can be included in summary sufficiently. In this study, we propose "subject-balanced" text summarization method that procure balance between all subjects and minimize omission of low-frequency subjects. For subject-balanced summary, we use two concept of summary evaluation metrics "completeness" and "succinctness". Completeness is the feature that summary should include contents of original documents fully and succinctness means summary has minimum duplication with contents in itself. Proposed method has 3-phases for summarization. First phase is constructing subject term dictionaries. Topic modeling is used for calculating topic-term weight which indicates degrees that each terms are related to each topic. From derived weight, it is possible to figure out highly related terms for every topic and subjects of documents can be found from various topic composed similar meaning terms. And then, few terms are selected which represent subject well. In this method, it is called "seed terms". However, those terms are too small to explain each subject enough, so sufficient similar terms with seed terms are needed for well-constructed subject dictionary. Word2Vec is used for word expansion, finds similar terms with seed terms. Word vectors are created after Word2Vec modeling, and from those vectors, similarity between all terms can be derived by using cosine-similarity. Higher cosine similarity between two terms calculated, higher relationship between two terms defined. So terms that have high similarity values with seed terms for each subjects are selected and filtering those expanded terms subject dictionary is finally constructed. Next phase is allocating subjects to every sentences which original documents have. To grasp contents of all sentences first, frequency analysis is conducted with specific terms that subject dictionaries compose. TF-IDF weight of each subjects are calculated after frequency analysis, and it is possible to figure out how much sentences are explaining about each subjects. However, TF-IDF weight has limitation that the weight can be increased infinitely, so by normalizing TF-IDF weights for every subject sentences have, all values are changed to 0 to 1 values. Then allocating subject for every sentences with maximum TF-IDF weight between all subjects, sentence group are constructed for each subjects finally. Last phase is summary generation parts. Sen2Vec is used to figure out similarity between subject-sentences, and similarity matrix can be formed. By repetitive sentences selecting, it is possible to generate summary that include contents of original documents fully and minimize duplication in summary itself. For evaluation of proposed method, 50,000 reviews of TripAdvisor are used for constructing subject dictionaries and 23,087 reviews are used for generating summary. Also comparison between proposed method summary and frequency-based summary is performed and as a result, it is verified that summary from proposed method can retain balance of all subject more which documents originally have.

• Korean Journal of Heritage: History & Science
• /
• v.44 no.3
• /
• pp.178-199
• /
• 2011

The ancient tombs found in Bokcheon-dong, Busan originate from the time between the $4^{th}$ and $5^{th}$ centuries, the period of the Three Nations. They are known as the tombs where the Vertical Plate Armour was mainly buried. In 2006, two units of the Vertical Plate Armour were additionally investigated in the tombs No.164 and No.165 which had been constructed at the end of the eastern slope near the hill of the group of ancient tombs in Bokcheon-dong. Throughout this study, the contents of the two units of the Vertical Plate Armour, whose preservation process has been completed, have been arranged, while the group of constructed ancient tombs in Bokcheon-dong from the $4^{th}$ century has been observed through the consideration of the burial pattern. The units of the Vertical Plate Armour from the tombs No.164 and No.165 can be classified as the IIa-typed armor showing the Gyeongju and Ulsan patterns, according to the attribute of the manufacturing technology. Also, they can be chronologically recorded as those from the early period of Stage II among the three stages regarding the chronological recording of the Vertical Plate Armour. While more than two units of the Vertical Plate Armour were buried in the largesized tomb on the top of the hill of the group of ancient tombs, one unit of the Vertical Plate Armour was buried in the small-sized tomb. By considering such a trend, it can be said that in the stage of burying the armor showing the Gyeongju and Ulsan patterns (I-type and IIa-type), different units of the Vertical Plate Armour were buried according to the size of the tomb. However, as the armor showing the Busan pattern (IIb-type) was settled, only one unit was buried. Meanwhile, the tombs No.164 and No.165 can be included in the wooden chamber tomb showing the Gyeongju pattern, which is a slender rectangular wooden chamber tomb with the aspect ratio of more than 1:3. However, according to the trend shown by the buried earthenware, it can be said that there seem to be common types and patterns shared with the earthenware which has been found in the area of Gimhae and is called the one showing the Geumgwan Gaya pattern. In other words, there seem to be close relationships between the subject tombs and the tomb No.3 in Gujeong-dong and the tomb No.55 in Sara-ri, Gyeongju, regarding the types of armor and tombs and the arrangement of buried artifacts. However, the buried earthenware shows a relationship with the areas of Busan and Gimhae. By considering the combined trend of the Gyeongju and Gimhae elements found in one tomb, it is possible to assume that the group of constructed ancient tombs in Bokcheon-dong used to be actively related with both areas. It has been thought that the Vertical Plate Armour used to be the exclusive property of the upper hierarchy until now, since it was buried in the large-sized tomb located on the top of the hill of the group of ancient tombs in Bokcheondong. However, as shown in case of the tombs No.164 and No.165, it has been verified that the Vertical Plate Armour was also buried in the small-sized tomb in terms of such factors as locations, sizes, the amount of buried artifacts and the qualitative aspect. Therefore, it is impossible to discuss the hierarchical characteristic of the tomb just based on the buried units of the Vertical Plate Armour. Also, it is difficult to assume that armor used to symbolize the domination of the military forces. The hierarchical characteristic of the group of constructed ancient tombs in Bokcheon-dong from the $4^{th}$ century can be verified according to the location and size of each tomb. As are sult, the re seem to be some differences regarding the buried units of the vertical plate armour. However, it would be necessary to carry out amore multilateral examination in order to find out whether the burial of the vertical plate armour could be regarded as the artifact which symbolizes the status or class of the deceased.

• The Journal of Korean Society for Radiation Therapy
• /
• v.31 no.1
• /
• pp.33-41
• /
• 2019

Purpose : To evaluate the effectiveness of Jaw-tracking(JT) technique in Intensity-modulated radiation therapy(IMRT) and Volumetric-modulated arc therapy(VMAT) for radiation therapy of esophageal cancer by analyzing volume dose of perimetrical normal organs along with the low-dose volume regions. Materials and Method: A total of 27 patients were selected who received radiation therapy for esophageal cancer with using $VitalBeam^{TM}$(Varian Medical System, U.S.A) in our hospital. Using Eclipse system(Ver. 13.6 Varian, U.S.A), radiation treatment planning was set up with Jaw-tracking technique(JT) and Non-Jaw-tracking technique(NJT), and was conducted for the patients with T-shaped Planning target volume(PTV), including Supraclavicular lymph nodes(SCL). PTV was classified into whether celiac area was included or not to identify the influence on the radiation field. To compare the treatment plans, Organ at risk(OAR) was defined to bilateral lung, heart, and spinal cord and evaluated for Conformity index(CI) and Homogeneity index(HI). Portal dosimetry was performed to verify a clinical application using Electronic portal imaging device(EPID) and Gamma analysis was performed with establishing thresholds of radiation field as a parameter, with various range of 0 %, 5 %, and 10 %. Results: All treatment plans were established on gamma pass rates of 95 % with 3 mm/3 % criteria. For a threshold of 10 %, both JT and NJT passed with rate of more than 95 % and both gamma passing rate decreased more than 1 % in IMRT as the low dose threshold decreased to 5 % and 0 %. For the case of JT in IMRT on PTV without celiac area, $V_5$ and $V_{10}$ of both lung showed a decrease by respectively 8.5 % and 5.3 % in average and up to 14.7 %. A $D_{mean}$ decreased by $72.3{\pm}51cGy$, while there was an increase in radiation dose reduction in PTV including celiac area. A $D_{mean}$ of heart decreased by $68.9{\pm}38.5cGy$ and that of spinal cord decreased by $39.7{\pm}30cGy$. For the case of JT in VMAT, $V_5$ decreased by 2.5 % in average in lungs, and also a little amount in heart and spinal cord. Radiation dose reduction of JT showed an increase when PTV includes celiac area in VMAT. Conclusion: In the radiation treatment planning for esophageal cancer, IMRT showed a significant decrease in $V_5$, and $V_{10}$ of both lungs when applying JT, and dose reduction was greater when the irradiated area in low-dose field is larger. Therefore, IMRT is more advantageous in applying JT than VMAT for radiation therapy of esophageal cancer and can protect the normal organs from MLC leakage and transmitted doses in low-dose field.

• The Korean Journal of Nuclear Medicine Technology
• /
• v.23 no.1
• /
• pp.35-39
• /
• 2019

Purpose Hepatitis B virus (hepatitis B virus, HBV) infection is a worldwide major public health problem and it is known as a major cause of chronic hepatitis, liver cirrhosis and liver cancer. And serologic tests of hepatitis B virus is essential for diagnosing and treating these diseases. In addition, with the development of molecular diagnostics, the detection of HBV DNA in serum diagnoses HBV infection and is recognized as an important indicator for the antiviral agent treatment response assessment. We performed HBeAg assay using Immunoradiometric assay (IRMA) and Chemiluminescent Microparticle Immunoassay (CMIA) in hepatitis B patients treated with antiviral agents. The detection rate of HBV DNA in serum was measured and compared by RT-PCR (Real Time - Polymerase Chain Reaction) method Materials and Methods HBeAg serum examination and HBV DNA quantification test were conducted on 270 hepatitis B patients undergoing anti-virus treatment after diagnosis of hepatitis B virus infection. Two serologic tests (IRMA, CMIA) with different detection principles were applied for the HBeAg serum test. Serum HBV DNA was quantitatively measured by real-time polymerase chain reaction (RT-PCR) using the Abbott m2000 System. Results The detection rate of HBeAg was 24.1% (65/270) for IRMA and 82.2% (222/270) for CMIA. Detection rate of serum HBV DNA by real-time RT-PCR is 29.3% (79/270). The measured amount of serum HBV DNA concentration is $4.8{\times}10^7{\pm}1.9{\times}10^8IU/mL$($mean{\pm}SD$). The minimum value is 16IU/mL, the maximum value is $1.0{\times}10^9IU/mL$, and the reference value for quantitative detection limit is 15IU/mL. The detection rates and concentrations of HBV DNA by group according to the results of HBeAg serological (IRMA, CMIA)tests were as follows. 1) Group I (IRMA negative, CMIA positive, N = 169), HBV DNA detection rate of 17.7% (30/169), $6.8{\times}10^5{\pm}1.9{\times}10^6IU/mL$ 2) Group II (IRMA positive, CMIA positive, N = 53), HBV DNA detection rate 62.3% (33/53), $1.1{\times}10^8{\pm}2.8{\times}10^8IU/mL$ 3) Group III (IRMA negative, CMIA negative, N = 36), HBV DNA detection rate 36.1% (13/36), $3.0{\times}10^5{\pm}1.1{\times}10^6IU/mL$ 4) Group IV(IRMA positive, CMIA negative, N = 12), HBV DNA detection rate 25% (3/12), $1.3{\times}10^3{\pm}1.1{\times}10^3IU/mL$ Conclusion HBeAg detection rate according to the serological test showed a large difference. This difference is considered for a number of reasons such as characteristics of the Ab used for assay kit and epitope, HBV of genotype. Detection rate and the concentration of the group-specific HBV DNA classified serologic results confirmed the high detection rate and the concentration in Group II (IRMA-positive, CMIA positive, N = 53).

• Korean Journal of Heritage: History & Science
• /
• v.51 no.3
• /
• pp.4-31
• /
• 2018

It is difficult to ascertain exactly when the Hongjuupseong (Town Wall) was first constructed, due to it had undergone several times of repair and maintenance works since it was piled up newly in 1415, when the first year of the reign of King Munjong (the 5th King of the Joseon Dynasty). Parts of its walls were demolished during the Japanese occupation, leaving the wall as it is today. Hongseong region is also susceptible to historical earthquakes for geological reasons. There have been records of earthquakes, such as the ones in 1978 and 1979 having magnitudes of 5.0 and 4.0, respectively, which left part of the walls collapsed. Again, in 2010, heavy rainfall destroyed another part of the wall. The fortress walls of the Hongjuupseong comprise various rocks, types of facing, building methods, and filling materials, according to sections. Moreover, the remaining wall parts were reused in repair works, and characteristics of each period are reflected vertically in the wall. Therefore, based on the vertical distribution of the walls, the Hongjuupseong was divided into type I, type II, and type III, according to building types. The walls consist mainly of coarse-grained granites, but, clearly different types of rocks were used for varying types of walls. The bottom of the wall shows a mixed variety of rocks and natural and split stones, whereas the center is made up mostly of coarse-grained granites. For repairs, pink feldspar granites was used, but it was different from the rock variety utilized for Suguji and Joyangmun Gate. Deterioration types to the wall can be categorized into bulging, protrusion of stones, missing stones at the basement, separation of framework, fissure and fragmentation, basement instability, and structural deformation. Manually and light-wave measurements were used to check the amount and direction of behavior of the fortress walls. A manual measurement revealed the sections that were undergoing structural deformation. Compared with the result of the light-wave measurement, the two monitoring methods proved correlational. As a result, the two measuring methods can be used complementarily for the long-term conservation and management of the wall. Additionally, the measurement system must be maintained, managed, and improved for the stability of the Hongjuupseong. The measurement of Nammunji indicated continuing changes in behavior due to collapse and rainfall. It can be greatly presumed that accumulated changes over the long period reached the threshold due to concentrated rainfall and subsequent behavioral irregularities, leading to the walls' collapse. Based on the findings, suggestions of the six grades of management from 0 to 5 have been made, to manage the Hongjuupseong more effectively. The applied suggested grade system of 501.9 m (61.10%) was assessed to grade 1, 29.5 m (3.77%) to grade 2, 10.4 m (1.33%) to grade 3, 241.2 m (30.80%) and grade 4. The sections with grade 4 concentrated around the west of Honghwamun Gate and the east of the battlement, which must be monitored regularly in preparation for a potential emergency. The six-staged management grade system is cyclical, where after performing repair and maintenance works through a comprehensive stability review, the section returned to grade 0. It is necessary to monitor thoroughly and evaluate grades on a regular basis.

• The Korean Journal of Archival Studies
• /
• no.29
• /
• pp.175-223
• /
• 2011

A university is an organization charged with publicity and has accountability to the community for the operating process. Students account for a majority of members in a university. In universities, numerous creatures are pouring out every year and university students are major producers of these records. However, roles and functions of university students producing enormous amount of records as main agents of universities and focused concentration on produced records have not been made yet. It is reality that from the archival point of view, the importance of produced records of which main agents are university students has been relatively underestimated. In this background, this study attempted approach in archival point of view on records produced by university students, main agents. There are various types of records that university students produce such as records produced in the process of research and teaching as well as records produced in the process of various autonomy activities like clubs, students' associations. This study especially focused on university student autonomy activity process and placed emphasis on accountability securing measures on autonomy activity process of university students. To secure accountability of activities, records management should be based. Therefore, as a way to ensure accountability of unversity students autonomy activity, we tried to present records management systematization and records utilization measures. For this, a student body, a university student autonomy organization was analyzed and a student body of Myongji University Humanities Campus was selected as a specific target. First, to identify records management status, activities and organization and functions of the student body, we conducted an interview with the president of the student body. Through this, we analyzed the activities of the university student body and examined the necessity of accountability accordingly. Also, we derived the types and characteristics of records to be produced at each stage by analyzing the organization and functions of the student body of Myongji University. Like this, after deriving the types of production records according to the necessity, organization and functions of accountability and activities of the student body, we analyzed records management status of the present student body. First, to identify the general process status of activities of the student body, we analyzed activity process by stage of the student body of Myongji University. And we analyzed records management method of the student body and responsibility principal and conducted real condition analysis. Through this analysis, we presented the measures to ensure accountability of a university student body in three categories such as systematization of records management process, establishment of records management infrastructure, accountability guarantee measures. This study discussed accountability on society by analyzing activities and functions of a student body, targeting a student body, an autonomy organization of university students. And as a measure to secure accountability of a student body, we proposed a model for records management environment settlement. But in terms that a student body is an organization operated in one year basis, there is a limit that records management environment is hard to settle. This study pointed out this limit and was to provide clues when more active researches were carried out in the field of student records management in the future through presentation of student body records management model. Also, it is expected that the analysis results derived from this research will have significance in terms of school history arrangement and conservation.

• Journal of Intelligence and Information Systems
• /
• v.25 no.1
• /
• pp.163-177
• /
• 2019

As smartphones are getting widely used, human activity recognition (HAR) tasks for recognizing personal activities of smartphone users with multimodal data have been actively studied recently. The research area is expanding from the recognition of the simple body movement of an individual user to the recognition of low-level behavior and high-level behavior. However, HAR tasks for recognizing interaction behavior with other people, such as whether the user is accompanying or communicating with someone else, have gotten less attention so far. And previous research for recognizing interaction behavior has usually depended on audio, Bluetooth, and Wi-Fi sensors, which are vulnerable to privacy issues and require much time to collect enough data. Whereas physical sensors including accelerometer, magnetic field and gyroscope sensors are less vulnerable to privacy issues and can collect a large amount of data within a short time. In this paper, a method for detecting accompanying status based on deep learning model by only using multimodal physical sensor data, such as an accelerometer, magnetic field and gyroscope, was proposed. The accompanying status was defined as a redefinition of a part of the user interaction behavior, including whether the user is accompanying with an acquaintance at a close distance and the user is actively communicating with the acquaintance. A framework based on convolutional neural networks (CNN) and long short-term memory (LSTM) recurrent networks for classifying accompanying and conversation was proposed. First, a data preprocessing method which consists of time synchronization of multimodal data from different physical sensors, data normalization and sequence data generation was introduced. We applied the nearest interpolation to synchronize the time of collected data from different sensors. Normalization was performed for each x, y, z axis value of the sensor data, and the sequence data was generated according to the sliding window method. Then, the sequence data became the input for CNN, where feature maps representing local dependencies of the original sequence are extracted. The CNN consisted of 3 convolutional layers and did not have a pooling layer to maintain the temporal information of the sequence data. Next, LSTM recurrent networks received the feature maps, learned long-term dependencies from them and extracted features. The LSTM recurrent networks consisted of two layers, each with 128 cells. Finally, the extracted features were used for classification by softmax classifier. The loss function of the model was cross entropy function and the weights of the model were randomly initialized on a normal distribution with an average of 0 and a standard deviation of 0.1. The model was trained using adaptive moment estimation (ADAM) optimization algorithm and the mini batch size was set to 128. We applied dropout to input values of the LSTM recurrent networks to prevent overfitting. The initial learning rate was set to 0.001, and it decreased exponentially by 0.99 at the end of each epoch training. An Android smartphone application was developed and released to collect data. We collected smartphone data for a total of 18 subjects. Using the data, the model classified accompanying and conversation by 98.74% and 98.83% accuracy each. Both the F1 score and accuracy of the model were higher than the F1 score and accuracy of the majority vote classifier, support vector machine, and deep recurrent neural network. In the future research, we will focus on more rigorous multimodal sensor data synchronization methods that minimize the time stamp differences. In addition, we will further study transfer learning method that enables transfer of trained models tailored to the training data to the evaluation data that follows a different distribution. It is expected that a model capable of exhibiting robust recognition performance against changes in data that is not considered in the model learning stage will be obtained.