A Deep Learning Based Approach to Recognizing Accompanying Status of Smartphone Users Using Multimodal Data (스마트폰 다종 데이터를 활용한 딥러닝 기반의 사용자 동행 상태 인식)
-
- Journal of Intelligence and Information Systems
- /
- v.25 no.1
- /
- pp.163-177
- /
- 2019
As smartphones are getting widely used, human activity recognition (HAR) tasks for recognizing personal activities of smartphone users with multimodal data have been actively studied recently. The research area is expanding from the recognition of the simple body movement of an individual user to the recognition of low-level behavior and high-level behavior. However, HAR tasks for recognizing interaction behavior with other people, such as whether the user is accompanying or communicating with someone else, have gotten less attention so far. And previous research for recognizing interaction behavior has usually depended on audio, Bluetooth, and Wi-Fi sensors, which are vulnerable to privacy issues and require much time to collect enough data. Whereas physical sensors including accelerometer, magnetic field and gyroscope sensors are less vulnerable to privacy issues and can collect a large amount of data within a short time. In this paper, a method for detecting accompanying status based on deep learning model by only using multimodal physical sensor data, such as an accelerometer, magnetic field and gyroscope, was proposed. The accompanying status was defined as a redefinition of a part of the user interaction behavior, including whether the user is accompanying with an acquaintance at a close distance and the user is actively communicating with the acquaintance. A framework based on convolutional neural networks (CNN) and long short-term memory (LSTM) recurrent networks for classifying accompanying and conversation was proposed. First, a data preprocessing method which consists of time synchronization of multimodal data from different physical sensors, data normalization and sequence data generation was introduced. We applied the nearest interpolation to synchronize the time of collected data from different sensors. Normalization was performed for each x, y, z axis value of the sensor data, and the sequence data was generated according to the sliding window method. Then, the sequence data became the input for CNN, where feature maps representing local dependencies of the original sequence are extracted. The CNN consisted of 3 convolutional layers and did not have a pooling layer to maintain the temporal information of the sequence data. Next, LSTM recurrent networks received the feature maps, learned long-term dependencies from them and extracted features. The LSTM recurrent networks consisted of two layers, each with 128 cells. Finally, the extracted features were used for classification by softmax classifier. The loss function of the model was cross entropy function and the weights of the model were randomly initialized on a normal distribution with an average of 0 and a standard deviation of 0.1. The model was trained using adaptive moment estimation (ADAM) optimization algorithm and the mini batch size was set to 128. We applied dropout to input values of the LSTM recurrent networks to prevent overfitting. The initial learning rate was set to 0.001, and it decreased exponentially by 0.99 at the end of each epoch training. An Android smartphone application was developed and released to collect data. We collected smartphone data for a total of 18 subjects. Using the data, the model classified accompanying and conversation by 98.74% and 98.83% accuracy each. Both the F1 score and accuracy of the model were higher than the F1 score and accuracy of the majority vote classifier, support vector machine, and deep recurrent neural network. In the future research, we will focus on more rigorous multimodal sensor data synchronization methods that minimize the time stamp differences. In addition, we will further study transfer learning method that enables transfer of trained models tailored to the training data to the evaluation data that follows a different distribution. It is expected that a model capable of exhibiting robust recognition performance against changes in data that is not considered in the model learning stage will be obtained.
The goal of this article is to examine the connection between creativity of unconscious and religion. Jung criticized how Freud's approach in studying the unconscious as a scientific inquiry focuses on the unconscious as reflecting only those which is repressed by the ego. Jung conceived of the unconscious as encompassing not only the repressed but also the variety of other psychic materials that have not reached the threshold of the consciousness in its range. Moreover, since human psyche is as individualistic as is a collective phenomenon, the collective psyche is thought to be pervasive at the bottom of the psychic functioning and the conscious and the personal unconscious comprising the upper level of the psychic functioning. Through clinical and personal experience, Jung had come to a realization that the unconscious has the self-regulatory function. The unconscious can make "demands" and also can retract its demands. Jung saw this as the autonomous function of the unconscious. And this autonomous unconscious creates, through dreams and fantasies, images that include an abundance of ideas and feelings. These creative images the unconscious produces assist and lead the "individuation process" which leads to the discovery of the Self. Because this unconscious process compensates the conscious ego, it has the necessary ingredients for self-regulation and can function in a creative and autonomous fashion. Jung saw religion as a special attitude of human psyche, which can be explained by careful and diligent observation about a dynamic being or action, which Rudolph Otto called the Numinosum. This kind of being or action does not get elicited by artificial or willful action. On the contrary, it takes a hold and dominates the human subject. Jung distinguished between religion and religious sector or denomination. He explained religious sector as reflecting the contents of sanctified and indoctrinated religious experiences. It is fixated in the complex organization of ritualized thoughts. And this ritualization gives rise to a system that is fixated. There is a clear goal in the religious sector to replace intellectual experiences with firmly established dogma and rituals. Religion as Jung experienced is the attitude of contemplation about Numinosum, which is formed by the images of the collective unconscious that is propelled by the creativity and autonomy of the unconscious. Religious sector is a religious community that is formed by these images that are ritualized. Jung saw religion as the relationship with the best or the uttermost value. And this relationship has a duality of being involuntary and reflecting free will. Therefore people can be influenced by one value, overcome with the unconscious being charged with psychic energy, or could accept it on a conscious level. Jung saw God as the dominating psychic element among humans or that psychic reality itself. Although Jung grew up in the atmosphere of the traditional Swiss reformed church, it does not seem that he considered himself to be a devoted Christian. To Jung, Christianity is a habitual, ritualized institution, which lacked vitality because it did not have the intellectual honesty or spiritual energy. However, Jung's encounter with the dramatic religious experience at age 12 through hallucination led him to perceive the existence of living god in his unconscious. This is why the theological questions and religious problems in everyday life became Jung's life-long interest. To this author, the reason why Jung delved into problems with religion has to do with his personal interest and love for the revival of the Christian church which had lost its spiritual vitality and depth and had become heavily ritualized.
At the initial stage of Internet advertising, banner advertising came into fashion. As the Internet developed into a central part of daily lives and the competition in the on-line advertising market was getting fierce, there was not enough space for banner advertising, which rushed to portal sites only. All these factors was responsible for an upsurge in advertising prices. Consequently, the high-cost and low-efficiency problems with banner advertising were raised, which led to an emergence of keyword advertising as a new type of Internet advertising to replace its predecessor. In the beginning of 2000s, when Internet advertising came to be activated, display advertisement including banner advertising dominated the Net. However, display advertising showed signs of gradual decline, and registered minus growth in the year 2009, whereas keyword advertising showed rapid growth and started to outdo display advertising as of the year 2005. Keyword advertising refers to the advertising technique that exposes relevant advertisements on the top of research sites when one searches for a keyword. Instead of exposing advertisements to unspecified individuals like banner advertising, keyword advertising, or targeted advertising technique, shows advertisements only when customers search for a desired keyword so that only highly prospective customers are given a chance to see them. In this context, it is also referred to as search advertising. It is regarded as more aggressive advertising with a high hit rate than previous advertising in that, instead of the seller discovering customers and running an advertisement for them like TV, radios or banner advertising, it exposes advertisements to visiting customers. Keyword advertising makes it possible for a company to seek publicity on line simply by making use of a single word and to achieve a maximum of efficiency at a minimum cost. The strong point of keyword advertising is that customers are allowed to directly contact the products in question through its more efficient advertising when compared to the advertisements of mass media such as TV and radio, etc. The weak point of keyword advertising is that a company should have its advertisement registered on each and every portal site and finds it hard to exercise substantial supervision over its advertisement, there being a possibility of its advertising expenses exceeding its profits. Keyword advertising severs as the most appropriate methods of advertising for the sales and publicity of small and medium enterprises which are in need of a maximum of advertising effect at a low advertising cost. At present, keyword advertising is divided into CPC advertising and CPM advertising. The former is known as the most efficient technique, which is also referred to as advertising based on the meter rate system; A company is supposed to pay for the number of clicks on a searched keyword which users have searched. This is representatively adopted by Overture, Google's Adwords, Naver's Clickchoice, and Daum's Clicks, etc. CPM advertising is dependent upon the flat rate payment system, making a company pay for its advertisement on the basis of the number of exposure, not on the basis of the number of clicks. This method fixes a price for advertisement on the basis of 1,000-time exposure, and is mainly adopted by Naver's Timechoice, Daum's Speciallink, and Nate's Speedup, etc, At present, the CPC method is most frequently adopted. The weak point of the CPC method is that advertising cost can rise through constant clicks from the same IP. If a company makes good use of strategies for maximizing the strong points of keyword advertising and complementing its weak points, it is highly likely to turn its visitors into prospective customers. Accordingly, an advertiser should make an analysis of customers' behavior and approach them in a variety of ways, trying hard to find out what they want. With this in mind, her or she has to put multiple keywords into use when running for ads. When he or she first runs an ad, he or she should first give priority to which keyword to select. The advertiser should consider how many individuals using a search engine will click the keyword in question and how much money he or she has to pay for the advertisement. As the popular keywords that the users of search engines are frequently using are expensive in terms of a unit cost per click, the advertisers without much money for advertising at the initial phrase should pay attention to detailed keywords suitable to their budget. Detailed keywords are also referred to as peripheral keywords or extension keywords, which can be called a combination of major keywords. Most keywords are in the form of texts. The biggest strong point of text-based advertising is that it looks like search results, causing little antipathy to it. But it fails to attract much attention because of the fact that most keyword advertising is in the form of texts. Image-embedded advertising is easy to notice due to images, but it is exposed on the lower part of a web page and regarded as an advertisement, which leads to a low click through rate. However, its strong point is that its prices are lower than those of text-based advertising. If a company owns a logo or a product that is easy enough for people to recognize, the company is well advised to make good use of image-embedded advertising so as to attract Internet users' attention. Advertisers should make an analysis of their logos and examine customers' responses based on the events of sites in question and the composition of products as a vehicle for monitoring their behavior in detail. Besides, keyword advertising allows them to analyze the advertising effects of exposed keywords through the analysis of logos. The logo analysis refers to a close analysis of the current situation of a site by making an analysis of information about visitors on the basis of the analysis of the number of visitors and page view, and that of cookie values. It is in the log files generated through each Web server that a user's IP, used pages, the time when he or she uses it, and cookie values are stored. The log files contain a huge amount of data. As it is almost impossible to make a direct analysis of these log files, one is supposed to make an analysis of them by using solutions for a log analysis. The generic information that can be extracted from tools for each logo analysis includes the number of viewing the total pages, the number of average page view per day, the number of basic page view, the number of page view per visit, the total number of hits, the number of average hits per day, the number of hits per visit, the number of visits, the number of average visits per day, the net number of visitors, average visitors per day, one-time visitors, visitors who have come more than twice, and average using hours, etc. These sites are deemed to be useful for utilizing data for the analysis of the situation and current status of rival companies as well as benchmarking. As keyword advertising exposes advertisements exclusively on search-result pages, competition among advertisers attempting to preoccupy popular keywords is very fierce. Some portal sites keep on giving priority to the existing advertisers, whereas others provide chances to purchase keywords in question to all the advertisers after the advertising contract is over. If an advertiser tries to rely on keywords sensitive to seasons and timeliness in case of sites providing priority to the established advertisers, he or she may as well make a purchase of a vacant place for advertising lest he or she should miss appropriate timing for advertising. However, Naver doesn't provide priority to the existing advertisers as far as all the keyword advertisements are concerned. In this case, one can preoccupy keywords if he or she enters into a contract after confirming the contract period for advertising. This study is designed to take a look at marketing for keyword advertising and to present effective strategies for keyword advertising marketing. At present, the Korean CPC advertising market is virtually monopolized by Overture. Its strong points are that Overture is based on the CPC charging model and that advertisements are registered on the top of the most representative portal sites in Korea. These advantages serve as the most appropriate medium for small and medium enterprises to use. However, the CPC method of Overture has its weak points, too. That is, the CPC method is not the only perfect advertising model among the search advertisements in the on-line market. So it is absolutely necessary that small and medium enterprises including independent shopping malls should complement the weaknesses of the CPC method and make good use of strategies for maximizing its strengths so as to increase their sales and to create a point of contact with customers.
The overall study of Samsaebulhoedo (painting of the Assembly of Buddhas of Three Ages) at Yongjusa Temple has focused on dating it, analyzing the painting style, identifying its painter, and scrutinizing the related documents. However, its greater coherence could be achieved through additional support from empirical evidence and logical consistency. Recent studies on Samsaebulhoedo at Yongjusa Temple that postulate that the painting could have been produced by a monk-painter in the late nineteenth century and that an original version produced in 1790 could have been retouched by a painter in the 1920s using a Western painting style lack such empirical proof and logic. Although King Jeongjo's son was not yet installed as crown prince, the Samsaebulhoedo at Yongjusa Temple contained a conventional written prayer wishing for a long life for the king, queen, and crown prince: "May his majesty the King live long / May her majesty the Queen live long / May his highness the Crown Prince live long" (主上殿下壽萬歲, 王妃殿下壽萬歲, 世子邸下壽萬歲). Later, this phrase was erased using cinnabar and revised to include unusual content in an exceptional order: "May his majesty the King live long / May his highness the King's Affectionate Mother (Jagung) live long / May her majesty the Queen live long / May his highness the Crown Prince live long" (主上殿下壽萬歲, 慈宮邸下壽萬歲, 王妃殿下壽萬歲, 世子邸下壽萬歲). A comprehensive comparison of the formats and contents in written prayers found on late Joseon Buddhist paintings and a careful analysis of royal liturgy during the reign of King Jeongjo reveal Samsaebulhoedo at Yongjusa Temple to be an original version produced at the time of the founding of Yongjusa Temple in 1790. According to a comparative analysis of formats, iconography, styles, aesthetic sensibilities, and techniques found in Buddhist paintings and paintings by Joseon court painters from the eighteenth and nineteenth centuries, Samsaebulhoedo at Yongjusa Temple bears features characteristic of paintings produced around 1790, which corresponds to the result of analysis on the written prayer. Buddhist paintings created up to the early eighteenth century show deities with their sizes determined by their religious status and a two-dimensional conceptual composition based on the traditional perspective of depicting close objects in the lower section and distant objects above. This Samsaebulhoedo, however, systematically places the Buddhist deities within a threedimensional space constructed by applying a linear perspective. Through the extensive employment of chiaroscuro as found in Western painting, it expresses white highlights and shadows, evoking a feeling that the magnificent world of the Buddhas of the Three Ages actually unfolds in front of viewers. Since the inner order of a linear perspective and the outer illusion of chiaroscuro shading are intimately related to each other, it is difficult to believe that the white highlights were a later addition. Moreover, the creative convergence of highly-developed Western painting style and techniques that is on display in this Samsaebulhoedo could only have been achieved by late-Joseon court painters working during the reign of King Jeongjo, including Kim Hongdo, Yi Myeong-gi, and Kim Deuksin. Deungun, the head monk of Yongjusa Temple, wrote Yongjusa sajeok (History of Yongjusa Temple) by compiling the historical records on the temple that had been transmitted since its founding. In Yongjusa sajeok, Deungun recorded that Kim Hongdo painted Samsaebulhoedo as if it were a historical fact. The Joseon royal court's official records, Ilseongnok (Daily Records of the Royal Court and Important Officials) and Suwonbu jiryeong deungnok (Suwon Construction Records), indicate that Kim Hongdo, Yi Myeong-gi, and Kim Deuksin all served as a supervisor (gamdong) for the production of Buddhist paintings. Since within Joseon's hierarchical administrative system it was considered improper to allow court painters of government position to create Buddhist paintings which had previously been produced by monk-painters, they were appointed as gamdong in name only to avoid a political liability. In reality, court painters were ordered to create Buddhist paintings. During their reigns, King Yeongjo and King Jeongjo summoned the literati painters Jo Yeongseok and Kang Sehwang to serve as gamdong for the production of royal portraits and requested that they paint these portraits as well. Thus, the boundary between the concept of supervision and that of painting occasionally blurred. Supervision did not completely preclude painting, and a gamdong could also serve as a painter. In this light, the historical records in Yongjusa sajeok are not inconsistent with those in Ilseongnok, Suwonbu jiryeong deungnok, and a prayer written by Hwang Deok-sun, which was found inside the canopy in Daeungjeon Hall at Yongjusa Temple. These records provided the same content in different forms as required for their purposes and according to the context. This approach to the Samsaebulhoedo at Yongjusa Temple will lead to a more coherent explanation of dating the painting, analyzing its style, identifying its painter, and interpreting the relevant documents based on empirical grounds and logical consistency.