Design and Implementation of MongoDB-based Unstructured Log Processing System over Cloud Computing Environment (클라우드 환경에서 MongoDB 기반의 비정형 로그 처리 시스템 설계 및 구현)
-
- Journal of Internet Computing and Services
- /
- v.14 no.6
- /
- pp.71-84
- /
- 2013
Log data, which record the multitude of information created when operating computer systems, are utilized in many processes, from carrying out computer system inspection and process optimization to providing customized user optimization. In this paper, we propose a MongoDB-based unstructured log processing system in a cloud environment for processing the massive amount of log data of banks. Most of the log data generated during banking operations come from handling a client's business. Therefore, in order to gather, store, categorize, and analyze the log data generated while processing the client's business, a separate log data processing system needs to be established. However, the realization of flexible storage expansion functions for processing a massive amount of unstructured log data and executing a considerable number of functions to categorize and analyze the stored unstructured log data is difficult in existing computer environments. Thus, in this study, we use cloud computing technology to realize a cloud-based log data processing system for processing unstructured log data that are difficult to process using the existing computing infrastructure's analysis tools and management system. The proposed system uses the IaaS (Infrastructure as a Service) cloud environment to provide a flexible expansion of computing resources and includes the ability to flexibly expand resources such as storage space and memory under conditions such as extended storage or rapid increase in log data. Moreover, to overcome the processing limits of the existing analysis tool when a real-time analysis of the aggregated unstructured log data is required, the proposed system includes a Hadoop-based analysis module for quick and reliable parallel-distributed processing of the massive amount of log data. Furthermore, because the HDFS (Hadoop Distributed File System) stores data by generating copies of the block units of the aggregated log data, the proposed system offers automatic restore functions for the system to continually operate after it recovers from a malfunction. Finally, by establishing a distributed database using the NoSQL-based Mongo DB, the proposed system provides methods of effectively processing unstructured log data. Relational databases such as the MySQL databases have complex schemas that are inappropriate for processing unstructured log data. Further, strict schemas like those of relational databases cannot expand nodes in the case wherein the stored data are distributed to various nodes when the amount of data rapidly increases. NoSQL does not provide the complex computations that relational databases may provide but can easily expand the database through node dispersion when the amount of data increases rapidly; it is a non-relational database with an appropriate structure for processing unstructured data. The data models of the NoSQL are usually classified as Key-Value, column-oriented, and document-oriented types. Of these, the representative document-oriented data model, MongoDB, which has a free schema structure, is used in the proposed system. MongoDB is introduced to the proposed system because it makes it easy to process unstructured log data through a flexible schema structure, facilitates flexible node expansion when the amount of data is rapidly increasing, and provides an Auto-Sharding function that automatically expands storage. The proposed system is composed of a log collector module, a log graph generator module, a MongoDB module, a Hadoop-based analysis module, and a MySQL module. When the log data generated over the entire client business process of each bank are sent to the cloud server, the log collector module collects and classifies data according to the type of log data and distributes it to the MongoDB module and the MySQL module. The log graph generator module generates the results of the log analysis of the MongoDB module, Hadoop-based analysis module, and the MySQL module per analysis time and type of the aggregated log data, and provides them to the user through a web interface. Log data that require a real-time log data analysis are stored in the MySQL module and provided real-time by the log graph generator module. The aggregated log data per unit time are stored in the MongoDB module and plotted in a graph according to the user's various analysis conditions. The aggregated log data in the MongoDB module are parallel-distributed and processed by the Hadoop-based analysis module. A comparative evaluation is carried out against a log data processing system that uses only MySQL for inserting log data and estimating query performance; this evaluation proves the proposed system's superiority. Moreover, an optimal chunk size is confirmed through the log data insert performance evaluation of MongoDB for various chunk sizes.
The popularity of communities on the internet has captured the attention of marketing scholars and practitioners. By adapting to the culture of the internet, however, and providing consumer with the ability to interact with one another in addition to the company, businesses can build new and deeper relationships with customers. The economic potential of online communities has been discussed with much hope in the many popular papers. In contrast to this enthusiastic prognostications, empirical and practical evidence regarding the economic potential of the online community has shown a little different conclusion. To date, even communities with high levels of membership and vibrant social arenas have failed to build financial viability. In this perspective, this study investigates the role of various kinds of influencing factors to online community loyalty and basically suggests the framework that explains the process of building purchase loyalty. Even though the importance of building loyalty in an online environment has been emphasized from the marketing theorists and practitioners, there is no sufficient research conclusion about what is the process of building purchase loyalty and the most powerful factors that influence to it. In this study, the process of building purchase loyalty is divided into three levels; characteristics of community site such as content superiority, site vividness, navigation easiness, and customerization, the mediating variables such as self congruency, consumer experience, and consumer to consumer interactivity, and finally various factors about online community loyalty such as visit loyalty, affect, trust, and purchase loyalty are those things. And the findings of this research are as follows. First, consumer-to-consumer interactivity is an important factor to online community purchase loyalty and other loyalty factors. This means, in order to interact with other people more actively, many participants in online community have the willingness to buy some kinds of products such as music, content, avatar, and etc. From this perspective, marketers of online community have to create some online environments in order that consumers can easily interact with other consumers and make some site environments in order that consumer can feel experience in this site is interesting and self congruency is higher than at other community sites. It has been argued that giving consumers a good experience is vital in cyber space, and websites create an active (rather than passive) customer by their nature. Some researchers have tried to pin down the positive experience, with limited success and less empirical support. Web sites can provide a cognitively stimulating experience for the user. We define the online community experience as playfulness based on the past studies. Playfulness is created by the excitement generated through a website's content and measured using three descriptors Marketers can promote using and visiting online communities, which deliver a superior web experience, to influence their customers' attitudes and actions, encouraging high involvement with those communities. Specially, we suggest that transcendent customer experiences(TCEs) which have aspects of flow and/or peak experience, can generate lasting shifts in beliefs and attitudes including subjective self-transformation and facilitate strong consumer's ties to a online community. And we find that website success is closely related to positive website experiences: consumers will spend more time on the site, interacting with other users. As we can see figure 2, visit loyalty and consumer affect toward the online community site didn't directly influence to purchase loyalty. This implies that there may be a little different situations here in online community site compared to online shopping mall studies that shows close relations between revisit intention and purchase intention. There are so many alternative sites on web, consumers do not want to spend money to buy content and etc. In this sense, marketers of community websites must know consumers' affect toward online community site is not a last goal and important factor to influnece consumers' purchase. Third, building good content environment can be a really important marketing tool to create a competitive advantage in cyberspace. For example, Cyworld, Korea's number one community site shows distinctive superiority in the consumer evaluations of content characteristics such as content superiority, site vividness, and customerization. Particularly, comsumer evaluation about customerization was remarkably higher than the other sites. In this point, we can conclude that providing comsumers with good, unique and highly customized content will be urgent and important task directly and indirectly impacting to self congruency, consumer experience, c-to-c interactivity, and various loyalty factors of online community. By creating enjoyable, useful, and unique online community environments, online community portals such as Daum, Naver, and Cyworld are able to build customer loyalty to a degree that many of today's online marketer can only dream of these loyalty, in turn, generates strong economic returns. Another way to build good online community site is to provide consumers with an interactive, fun, experience-oriented or experiential Web site. Elements that can make a dot.com's Web site experiential include graphics, 3-D images, animation, video and audio capabilities. In addition, chat rooms and real-time customer service applications (which link site visitors directly to other visitors, or with company support personnel, respectively) are also being used to make web sites more interactive. Researchers note that online communities are increasingly incorporating such applications in their Web sites, in order to make consumers' online shopping experience more similar to that of an offline store. That is, if consumers are able to experience sensory stimulation (e.g. via 3-D images and audio sound), interact with other consumers (e.g., via chat rooms), and interact with sales or support people (e.g. via a real-time chat interface or e-mail), then they are likely to have a more positive dot.com experience, and develop a more positive image toward the online company itself). Analysts caution, however, that, while high quality graphics, animation and the like may create a fun experience for consumers, when heavily used, they can slow site navigation, resulting in frustrated consumers, who may never return to a site. Consequently, some analysts suggest that, at least with current technology, the rule-of-thumb is that less is more. That is, while graphics etc. can draw consumers to a site, they should be kept to a minimum, so as not to impact negatively on consumers' overall site experience.
Brand has received much attention from considerable marketing research. When consumers consume product or services, they are exposed to a lot of brand related stimuli. These contain brand personality, brand experience, brand identity, brand communications and so on. A special kind of new crisis occasionally confronting companies' brand management today is the brand related rumor. An important influence on consumers' purchase decision making is the word-of-mouth spread by other consumers and most decisions are influenced by other's recommendations. In light of this influence, firms have reasonable reason to study and understand consumer-to-consumer communication such as brand rumor. The importance of brand rumor to marketers is increasing as the number of internet user and SNS(social network service) site grows. Due to the development of internet technology, people can spread rumors without the limitation of time, space and place. However relatively few studies have been published in marketing journals and little is known about brand rumors in the marketplace. The study of rumor has a long history in all major social science. But very few studies have dealt with the antecedents and consequences of any kind of brand rumor. Rumor has been generally described as a story or statement in general circulation without proper confirmation or certainty as to fact. And it also can be defined as an unconfirmed proposition, passed along from people to people. Rosnow(1991) claimed that rumors were transmitted because people needed to explain ambiguous and uncertain events and talking about them reduced associated anxiety. Especially negative rumors are believed to have the potential to devastate a company's reputation and relations with customers. From the perspective of marketer, negative rumors are considered harmful and extremely difficult to control in general. It is becoming a threat to a company's sustainability and sometimes leads to negative brand image and loss of customers. Thus there is a growing concern that these negative rumors can damage brands' reputations and lead them to financial disaster too. In this study we aimed to distinguish antecedents of brand rumor transmission and investigate the effects of brand rumor characteristics on rumor spread intention. We also found key components in personal acceptance of brand rumor. In contextualist perspective, we tried to unify the traditional psychological and sociological views. In this unified research approach we defined brand rumor's characteristics based on five major variables that had been found to influence the process of rumor spread intention. The five factors of usefulness, source credibility, message credibility, worry, and vividness, encompass multi level elements of brand rumor. We also selected product involvement as a control variable. To perform the empirical research, imaginary Korean 'Kimch' brand and related contamination rumor was created and proposed. Questionnaires were collected from 178 Korean samples. Data were collected from college students who have been experienced the focal product. College students were regarded as good subjects because they have a tendency to express their opinions in detail. PLS(partial least square) method was adopted to analyze the relations between variables in the equation model. The most widely adopted causal modeling method is LISREL. However it is poorly suited to deal with relatively small data samples and can yield not proper solutions in some cases. PLS has been developed to avoid some of these limitations and provide more reliable results. To test the reliability using SPSS 16 s/w, Cronbach alpha was examined and all the values were appropriate showing alpha values between .802 and .953. Subsequently, confirmatory factor analysis was conducted successfully. And structural equation modeling has been used to analyze the research model using smartPLS(ver. 2.0) s/w. Overall, R2 of adoption of rumor is .476 and R2 of intention of rumor transmission is .218. The overall model showed a satisfactory fit. The empirical results can be summarized as follows. According to the results, the variables of brand rumor characteristic such as source credibility, message credibility, worry, and vividness affect argument strength of rumor. And argument strength of rumor also affects rumor intention. On the other hand, the relationship between perceived usefulness and argument strength of rumor is not significant. The moderating effect of product involvement on the relations between argument strength of rumor and rumor W.O.M intention is not supported neither. Consequently this study suggests some managerial and academic implications. We consider some implications for corporate crisis management planning, PR and brand management. This results show marketers that rumor is a critical factor for managing strong brand assets. Also for researchers, brand rumor should become an important thesis of their interests to understand the relationship between consumer and brand. Recently many brand managers and marketers have focused on the short-term view. They just focused on strengthen the positive brand image. According to this study we suggested that effective brand management requires managing negative brand rumors with a long-term view of marketing decisions.