• Journal of Intelligence and Information Systems
• /
• v.25 no.1
• /
• pp.163-177
• /
• 2019

As smartphones are getting widely used, human activity recognition (HAR) tasks for recognizing personal activities of smartphone users with multimodal data have been actively studied recently. The research area is expanding from the recognition of the simple body movement of an individual user to the recognition of low-level behavior and high-level behavior. However, HAR tasks for recognizing interaction behavior with other people, such as whether the user is accompanying or communicating with someone else, have gotten less attention so far. And previous research for recognizing interaction behavior has usually depended on audio, Bluetooth, and Wi-Fi sensors, which are vulnerable to privacy issues and require much time to collect enough data. Whereas physical sensors including accelerometer, magnetic field and gyroscope sensors are less vulnerable to privacy issues and can collect a large amount of data within a short time. In this paper, a method for detecting accompanying status based on deep learning model by only using multimodal physical sensor data, such as an accelerometer, magnetic field and gyroscope, was proposed. The accompanying status was defined as a redefinition of a part of the user interaction behavior, including whether the user is accompanying with an acquaintance at a close distance and the user is actively communicating with the acquaintance. A framework based on convolutional neural networks (CNN) and long short-term memory (LSTM) recurrent networks for classifying accompanying and conversation was proposed. First, a data preprocessing method which consists of time synchronization of multimodal data from different physical sensors, data normalization and sequence data generation was introduced. We applied the nearest interpolation to synchronize the time of collected data from different sensors. Normalization was performed for each x, y, z axis value of the sensor data, and the sequence data was generated according to the sliding window method. Then, the sequence data became the input for CNN, where feature maps representing local dependencies of the original sequence are extracted. The CNN consisted of 3 convolutional layers and did not have a pooling layer to maintain the temporal information of the sequence data. Next, LSTM recurrent networks received the feature maps, learned long-term dependencies from them and extracted features. The LSTM recurrent networks consisted of two layers, each with 128 cells. Finally, the extracted features were used for classification by softmax classifier. The loss function of the model was cross entropy function and the weights of the model were randomly initialized on a normal distribution with an average of 0 and a standard deviation of 0.1. The model was trained using adaptive moment estimation (ADAM) optimization algorithm and the mini batch size was set to 128. We applied dropout to input values of the LSTM recurrent networks to prevent overfitting. The initial learning rate was set to 0.001, and it decreased exponentially by 0.99 at the end of each epoch training. An Android smartphone application was developed and released to collect data. We collected smartphone data for a total of 18 subjects. Using the data, the model classified accompanying and conversation by 98.74% and 98.83% accuracy each. Both the F1 score and accuracy of the model were higher than the F1 score and accuracy of the majority vote classifier, support vector machine, and deep recurrent neural network. In the future research, we will focus on more rigorous multimodal sensor data synchronization methods that minimize the time stamp differences. In addition, we will further study transfer learning method that enables transfer of trained models tailored to the training data to the evaluation data that follows a different distribution. It is expected that a model capable of exhibiting robust recognition performance against changes in data that is not considered in the model learning stage will be obtained.

• Asia Marketing Journal
• /
• v.12 no.2
• /
• pp.155-175
• /
• 2010

Most of the service industries including finance and telecommunications have become matured and saturated. The competitions have become severe while the differences among brands become smaller. Therefore maintaining good relationships with customers has been critical for the service providers. In case of credit card and debit card, the similar patterns are shown. It is important for them to maintain good relationships with customers, and therefore, they have used marketing program which provides customized services to customers and utilizes the membership programs. Not only do they build and maintain good relationships, but also highlight their brands from the emotional aspects. For example, KB Card or Hyundai Card uses well-known designers' works for their credit card design. As well, they differentiate the designs of credit cards to stress on their brand personalities. BC Card introduced the credit card with perfume that a customer would like. Even though the credit card is small and not shown to public easily, it becomes more important for those companies to touch the customers' feelings with the brand personalities and their images. This is partly because of changes in consumers' lifestyles. Y-generations becomes highly likely to express themselves in many different ways and more emotional than X-generations. For the Y-generations, therefore, even credit cards in the wallet should be personalized and well-designed. In line with it, credit cards with good design can be seen as an example of brand identity, where different design for each customer can be used to recognize the membership groups that customers want to belong. On the other hand, these credit card companies offer the special treatment benefits for those customers who are heavy users for the cards. For example, those customers who love sports will receive some special discounts when they use their credit cards for sports related products. Therefore this study attempted to explore the relationships between relational benefits, brand identification and loyalty. It has been well known that relational benefits and brand identification lead to loyalty independently from many other studies, but there has been few study to review all the three variables all together in a research model. Furthermore, as reviewed above, in the card industry, many companies attempt to associate the brand image with their products to fit their customers' lifestyles while relational benefits are still playing an important role for their business. Therefore in our research model, relational benefits, brand identification, and loyalty are all included. We focus on the mediating effect of brand identification. From the relational benefits perspective, only special treatment benefit and confidence benefit are included. Social benefit is not applicable for this credit card industry because not many cases of face-to-face interaction can be found. From the brand identification perspective, personal brand identity and social brand identity are reviewed and included in the model. Overall, the research model emphasizes that the relationships between relational benefits and loyalty will be mediated by the effect of brand identification. The effects of relational benefits which are confidence benefit and special treatment benefits on loyalty will be realized when they fit to the personal brand identity and social brand identity. In the research model, therefore, the relationships between confidence benefit and social brand identity, and between confidence benefit and personal identity are hypothesized while the effects of special treatment benefit on social brand identity and personal brand identity are hypothesized. Loyalty, then, is hypothesized to have positive relationships with personal brand identity and social brand identity. In addition, confidence benefit among the relational benefits is expected to have a direct, positive relationship with loyalty because confidence benefit has been recognized as a critical factor for good relationships and satisfaction. Data were collected from college students who have been using either credit cards or debit cards. College students were regarded good subjects because they are in Y-generation cohorts and have tendency to express themselves more. Total sample size was two hundred three at the beginning, but after deleting those data with many missing values, one hundred ninety-seven data points were remained and used for the model testing. Measurement items were brought from the previous literatures and modified for this research. To test the reliability, using SPSS 14, chronbach's α was examined and all the values were from .874 to .928 exceeding over .7. Using AMOS 7.0, confirmatory factor analysis was conducted to investigate the measurement model. The measurement model was found good fit with χ²(67)=188.388 (p= .000), GFI=.886, AGFI=.821, CFI=.941, RMSEA=.096. Using AMOS 7.0, structural equation modeling has been used to analyze the research model. Overall, the research model fit were χ²(68)=188.670 (p= .000), GFI=.886, AGFI=,824 CFI=.942, RMSEA=.095 indicating good fit. In details, all the paths hypothesized in the research model were found significant except for the path from social brand identity to loyalty. Personal brand identity leads to loyalty while both confidence benefit and special treatment benefit have a positive relationships with personal and social identities. As well, confidence benefit has a direct positive effect on loyalty. The results indicates the followings. First, personal brand identity plays an important role for credit/debit card usage. Therefore even for the products which are not shown to public easy, design and emotional aspect can be important to fit the customers' lifestyles. Second, confidence benefit and special treatment benefit have a positive effects on personal brand identity. Therefore it will be needed for marketers to associate the special treatment and trust and confidence benefits with personal image, personality and personal identity. Third, this study found again the importance of confidence and trust. However interestingly enough, social brand identity was not found to be significantly related to loyalty. It can be explained that the main sample of this study consists of college students. Those strategies to facilitate social brand identity are focused on high social status groups while college students have not been established their status yet.