• Journal of Intelligence and Information Systems
• /
• v.23 no.1
• /
• pp.127-141
• /
• 2017

Advances in Internet technologies and the proliferation of mobile devices enabled consumers to approach a wide range of goods and services, while causing an adverse effect that they have hard time reaching their congenial items even if they devote much time to searching for them. Accordingly, businesses are using the recommender systems to provide tools for consumers to find the desired items more easily. Association Rule Mining (ARM) technology is advantageous to recommender systems in that ARM provides intuitive form of a rule with interestingness measures (support, confidence, and lift) describing the relationship between items. Given an item, its relevant items can be distinguished with the help of the measures that show the strength of relationship between items. Based on the strength, the most pertinent items can be chosen among other items and exposed to a given item's web page. However, the diversity of the measures may confuse which items are more recommendable. Given two rules, for example, one rule's support and confidence may not be concurrently superior to the other rule's. Such discrepancy of the measures in distinguishing one rule's superiority from other rules may cause difficulty in selecting proper items for recommendation. In addition, in an online environment where a web page or mobile screen can provide a limited number of recommendations that attract consumer interest, the prudent selection of items to be included in the list of recommendations is very important. The exposure of items of little interest may lead consumers to ignore the recommendations. Then, such consumers will possibly not pay attention to other forms of marketing activities. Therefore, the measures should be aligned with the probability of consumer's acceptance of recommendations. For this reason, this study proposes a model-based approach to combine those measures into one unified measure that can consistently determine the ranking of recommended items. A regression model was designed to describe how well the measures (independent variables; i.e., support, confidence, and lift) explain consumer's acceptance of recommendations (dependent variables, hit rate of recommended items). The model is intuitive to understand and easy to use in that the equation consists of the commonly used measures for ARM and can be used in the estimation of hit rates. The experiment using transaction data from one of the Korea's largest online shopping malls was conducted to show that the proposed model can improve the hit rates of recommendations. From the top of the list to 13th place, recommended items in the higher rakings from the proposed model show the higher hit rates than those from the competitive model's. The result shows that the proposed model's performance is superior to the competitive model's in online recommendation environment. In a web page, consumers are provided around ten recommendations with which the proposed model outperforms. Moreover, a mobile device cannot expose many items simultaneously due to its limited screen size. Therefore, the result shows that the newly devised recommendation technique is suitable for the mobile recommender systems. While this study has been conducted to cover the cross-selling in online shopping malls that handle merchandise, the proposed method can be expected to be applied in various situations under which association rules apply. For example, this model can be applied to medical diagnostic systems that predict candidate diseases from a patient's symptoms. To increase the efficiency of the model, additional variables will need to be considered for the elaboration of the model in future studies. For example, price can be a good candidate for an explanatory variable because it has a major impact on consumer purchase decisions. If the prices of recommended items are much higher than the items in which a consumer is interested, the consumer may hesitate to accept the recommendations.

• Journal of Intelligence and Information Systems
• /
• v.21 no.4
• /
• pp.93-110
• /
• 2015

The recommender system is a system which recommends products to the customers who are likely to be interested in. Based on automated information filtering technology, various recommender systems have been developed. Collaborative filtering (CF), one of the most successful recommendation algorithms, has been applied in a number of different domains such as recommending Web pages, books, movies, music and products. But, it has been known that CF has a critical shortcoming. CF finds neighbors whose preferences are like those of the target customer and recommends products those customers have most liked. Thus, CF works properly only when there's a sufficient number of ratings on common product from customers. When there's a shortage of customer ratings, CF makes the formation of a neighborhood inaccurate, thereby resulting in poor recommendations. To improve the performance of CF based recommender systems, most of the related studies have been focused on the development of novel algorithms under the assumption of using a single profile, which is created from user's rating information for items, purchase transactions, or Web access logs. With the advent of big data, companies got to collect more data and to use a variety of information with big size. So, many companies recognize it very importantly to utilize big data because it makes companies to improve their competitiveness and to create new value. In particular, on the rise is the issue of utilizing personal big data in the recommender system. It is why personal big data facilitate more accurate identification of the preferences or behaviors of users. The proposed recommendation methodology is as follows: First, multimodal user profiles are created from personal big data in order to grasp the preferences and behavior of users from various viewpoints. We derive five user profiles based on the personal information such as rating, site preference, demographic, Internet usage, and topic in text. Next, the similarity between users is calculated based on the profiles and then neighbors of users are found from the results. One of three ensemble approaches is applied to calculate the similarity. Each ensemble approach uses the similarity of combined profile, the average similarity of each profile, and the weighted average similarity of each profile, respectively. Finally, the products that people among the neighborhood prefer most to are recommended to the target users. For the experiments, we used the demographic data and a very large volume of Web log transaction for 5,000 panel users of a company that is specialized to analyzing ranks of Web sites. R and SAS E-miner was used to implement the proposed recommender system and to conduct the topic analysis using the keyword search, respectively. To evaluate the recommendation performance, we used 60% of data for training and 40% of data for test. The 5-fold cross validation was also conducted to enhance the reliability of our experiments. A widely used combination metric called F1 metric that gives equal weight to both recall and precision was employed for our evaluation. As the results of evaluation, the proposed methodology achieved the significant improvement over the single profile based CF algorithm. In particular, the ensemble approach using weighted average similarity shows the highest performance. That is, the rate of improvement in F1 is 16.9 percent for the ensemble approach using weighted average similarity and 8.1 percent for the ensemble approach using average similarity of each profile. From these results, we conclude that the multimodal profile ensemble approach is a viable solution to the problems encountered when there's a shortage of customer ratings. This study has significance in suggesting what kind of information could we use to create profile in the environment of big data and how could we combine and utilize them effectively. However, our methodology should be further studied to consider for its real-world application. We need to compare the differences in recommendation accuracy by applying the proposed method to different recommendation algorithms and then to identify which combination of them would show the best performance.

• Journal of KIISE:Computing Practices and Letters
• /
• v.12 no.3
• /
• pp.192-207
• /
• 2006

Today, the SSL protocol has been used as core part in various computing environments or security systems. But, the SSL protocol has several problems, because of the rigidity on operating. First, SSL protocol brings considerable burden to the CPU utilization so that performance of the security service in encryption transaction is lowered because it encrypts all data which is transferred between a server and a client. Second, SSL protocol can be vulnerable for cryptanalysis due to the key in fixed algorithm being used. Third, it is difficult to add and use another new cryptography algorithms. Finally. it is difficult for developers to learn use cryptography API(Application Program Interface) for the SSL protocol. Hence, we need to cover these problems, and, at the same time, we need the secure and comfortable method to operate the SSL protocol and to handle the efficient data. In this paper, we propose the SSL component which is designed and implemented using CBD(Component Based Development) concept to satisfy these requirements. The SSL component provides not only data encryption services like the SSL protocol but also convenient APIs for the developer unfamiliar with security. Further, the SSL component can improve the productivity and give reduce development cost. Because the SSL component can be reused. Also, in case of that new algorithms are added or algorithms are changed, it Is compatible and easy to interlock. SSL Component works the SSL protocol service in application layer. First of all, we take out the requirements, and then, we design and implement the SSL Component, confidentiality and integrity component, which support the SSL component, dependently. These all mentioned components are implemented by EJB, it can provide the efficient data handling when data is encrypted/decrypted by choosing the data. Also, it improves the usability by choosing data and mechanism as user intend. In conclusion, as we test and evaluate these component, SSL component is more usable and efficient than existing SSL protocol, because the increase rate of processing time for SSL component is lower that SSL protocol's.

• Journal of Intelligence and Information Systems
• /
• v.18 no.2
• /
• pp.157-175
• /
• 2012

It is difficult to evaluate the performance of process innovation of e-procurement which has large scale and complex processes. The existing evaluation methods for measuring the effects of process innovation have been mainly done with statistically quantitative methods by analyzing operational data or with qualitative methods by conducting surveys and interviews. However, these methods have some limitations to evaluate the effects because the performance evaluation of e-procurement process innovation should consider the interactions among participants who are active either directly or indirectly through the processes. This study considers the e-procurement process as a complex system and develops a simulation model based on MAS(Multi-Agent System) to evaluate the effects of e-procurement process innovation. Multi-agent based simulation allows observing interaction patterns of objects in virtual world through relationship among objects and their behavioral mechanism. Agent-based simulation is suitable especially for complex business problems. In this study, we used Netlogo Version 4.1.3 as a MAS simulation tool which was developed in Northwestern University. To do this, we developed a interaction model of agents in MAS environment. We defined process agents and task agents, and assigned their behavioral characteristics. The developed simulation model was applied to G2B system (KONEPS: Korea ON-line E-Procurement System) of Public Procurement Service (PPS) in Korea and used to evaluate the innovation effects of the G2B system. KONEPS is a successfully established e-procurement system started in the year 2002. KONEPS is a representative e-Procurement system which integrates characteristics of e-commerce into government for business procurement activities. KONEPS deserves the international recognition considering the annual transaction volume of 56 billion dollars, daily exchanges of electronic documents, users consisted of 121,000 suppliers and 37,000 public organizations, and the 4.5 billion dollars of cost saving. For the simulation, we analyzed the e-procurement of process of KONEPS into eight sub processes such as 'process 1: search products and acquisition of proposal', 'process 2 : review the methods of contracts and item features', 'process 3 : a notice of bid', 'process 4 : registration and confirmation of qualification', 'process 5 : bidding', 'process 6 : a screening test', 'process 7 : contracts', and 'process 8 : invoice and payment'. For the parameter settings of the agents behavior, we collected some data from the transactional database of PPS and some information by conducting a survey. The used data for the simulation are 'participants (government organizations, local government organizations and public institutions)', 'the number of bidding per year', 'the number of total contracts', 'the number of shopping mall transactions', 'the rate of contracts between bidding and shopping mall', 'the successful bidding ratio', and the estimated time for each process. The comparison was done for the difference of time consumption between 'before the innovation (As-was)' and 'after the innovation (As-is).' The results showed that there were productivity improvements in every eight sub processes. The decrease ratio of 'average number of task processing' was 92.7% and the decrease ratio of 'average time of task processing' was 95.4% in entire processes when we use G2B system comparing to the conventional method. Also, this study found that the process innovation effect will be enhanced if the task process related to the 'contract' can be improved. This study shows the usability and possibility of using MAS in process innovation evaluation and its modeling.

• Asia pacific journal of information systems
• /
• v.19 no.4
• /
• pp.177-200
• /
• 2009

AIn the advanced - economy, the services industry hasbecome a dominant sector. Evidently, the services sector has grown at a much faster rate than any other. For instance, in such developed countries as the U.S., the proportion of the services sector in its GDP is greater than 75%. Even in the developing countries including India and China, the magnitude of the services sector in their GDPs is rapidly growing. The increasing dependence on service gives rise to new initiatives including service science and service-dominant logic. These new initiatives propose a new theoretical prism to promote the better understanding of the changing economic structure. From the new perspectives, service is no longer regarded as a transaction or exchange, but rather co-creation of value through the interaction among service users, providers, and other stakeholders including partners, external environments, and customer communities. The purpose of this study is the following. First, we review previous literature on service, service innovation, and service systems and integrate the studies based on service dominant logic. Second, we categorize the ten propositions of service dominant logic into conceptual propositions and the ones that are directly related to service provision. Conceptual propositions are left out to form the research model. With the selected propositions, we define the research constructs for this study. Third, we develop measurement items for the new service concepts including service provider network, customer network, value co-creation, and convergence of service with product. We then propose a research model to explain the relationship among the factors that affect the value creation mechanism. Finally, we empirically investigate the effects of the factors on firm performance. Through the process of this research study, we want to show the value creation mechanism of service systems in which various participants in service provision interact with related parties in a joint effort to create values. To test the proposed hypotheses, we developed measurement items and distributed survey questionnaires to domestic companies. 500 survey questionnaires were distributed and 180 were returned among which 171 were usable. The results of the empirical test can be summarized as the following. First, service providers' network which is to help offer required services to customers is found to affect customer network, while it does not have a significant effect on value co-creation and product-service convergence. Second, customer network, on the other hand, appears to influence both value co-creation and product-service convergence. Third, value co-creation accomplished through the collaboration of service providers and customers is found to have a significant effect on both product-service convergence and firm performance. Finally, product-service convergence appears to affect firm performance. To interpret the results from the value creation mechanism perspective, service provider network well established to support customer network is found to have significant effect on customer network which in turn facilitates value co-creation in service provision and product-service convergence to lead to greater firm performance. The results have some enlightening implications for practitioners. If companies want to transform themselves into service-centered business enterprises, they have to consider the four factors suggested in this study: service provider network, customer network, value co-creation, and product-service convergence. That is, companies becoming a service-oriented organization need to understand what the four factors are and how the factors interact with one another in their business context. They then may want to devise a better tool to analyze the value creation mechanism and apply the four factors to their own environment. This research study contributes to the literature in following ways. First, this study is one of the very first empirical studies on the service dominant logic as it has categorized the fundamental propositions into conceptual and empirically testable ones and tested the proposed hypotheses against the data collected through the survey method. Most of the propositions are found to work as Vargo and Lusch have suggested. Second, by providing a testable set of relationships among the research variables, this study may provide policy makers and decision makers with some theoretical grounds for their decision making on what to do with service innovation and management. Finally, this study incorporates the concepts of value co-creation through the interaction between customers and service providers into the proposed research model and empirically tests the validity of the concepts. The results of this study will help establish a value creation mechanism in the service-based economy, which can be used to develop and implement new service provision.

• Asia pacific journal of information systems
• /
• v.20 no.1
• /
• pp.81-99
• /
• 2010

Lately, in consumers' markets the number of new items is rapidly increasing at an overwhelming rate while consumers have limited access to information about those new products in making a sensible, well-informed purchase. Therefore, item providers and customers need a system which recommends right items to right customers. Also, whenever new items are released, for instance, the recommender system specializing in new items can help item providers locate and identify potential customers. Currently, new items are being added to an existing system without being specially noted to consumers, making it difficult for consumers to identify and evaluate new products introduced in the markets. Most of previous approaches for recommender systems have to rely on the usage history of customers. For new items, this content-based (CB) approach is simply not available for the system to recommend those new items to potential consumers. Although collaborative filtering (CF) approach is not directly applicable to solve the new item problem, it would be a good idea to use the basic principle of CF which identifies similar customers, i,e. neighbors, and recommend items to those customers who have liked the similar items in the past. This research aims to suggest a hybrid recommendation procedure based on the preference boundary of target customer. We suggest the hybrid recommendation procedure using the preference boundary in the feature space for recommending new items only. The basic principle is that if a new item belongs within the preference boundary of a target customer, then it is evaluated to be preferred by the customer. Customers' preferences and characteristics of items including new items are represented in a feature space, and the scope or boundary of the target customer's preference is extended to those of neighbors'. The new item recommendation procedure consists of three steps. The first step is analyzing the profile of items, which are represented as k-dimensional feature values. The second step is to determine the representative point of the target customer's preference boundary, the centroid, based on a personal information set. To determine the centroid of preference boundary of a target customer, three algorithms are developed in this research: one is using the centroid of a target customer only (TC), the other is using centroid of a (dummy) big target customer that is composed of a target customer and his/her neighbors (BC), and another is using centroids of a target customer and his/her neighbors (NC). The third step is to determine the range of the preference boundary, the radius. The suggested algorithm Is using the average distance (AD) between the centroid and all purchased items. We test whether the CF-based approach to determine the centroid of the preference boundary improves the recommendation quality or not. For this purpose, we develop two hybrid algorithms, BC and NC, which use neighbors when deciding centroid of the preference boundary. To test the validity of hybrid algorithms, BC and NC, we developed CB-algorithm, TC, which uses target customers only. We measured effectiveness scores of suggested algorithms and compared them through a series of experiments with a set of real mobile image transaction data. We spilt the period between 1st June 2004 and 31st July and the period between 1st August and 31st August 2004 as a training set and a test set, respectively. The training set Is used to make the preference boundary, and the test set is used to evaluate the performance of the suggested hybrid recommendation procedure. The main aim of this research Is to compare the hybrid recommendation algorithm with the CB algorithm. To evaluate the performance of each algorithm, we compare the purchased new item list in test period with the recommended item list which is recommended by suggested algorithms. So we employ the evaluation metric to hit the ratio for evaluating our algorithms. The hit ratio is defined as the ratio of the hit set size to the recommended set size. The hit set size means the number of success of recommendations in our experiment, and the test set size means the number of purchased items during the test period. Experimental test result shows the hit ratio of BC and NC is bigger than that of TC. This means using neighbors Is more effective to recommend new items. That is hybrid algorithm using CF is more effective when recommending to consumers new items than the algorithm using only CB. The reason of the smaller hit ratio of BC than that of NC is that BC is defined as a dummy or virtual customer who purchased all items of target customers' and neighbors'. That is centroid of BC often shifts from that of TC, so it tends to reflect skewed characters of target customer. So the recommendation algorithm using NC shows the best hit ratio, because NC has sufficient information about target customers and their neighbors without damaging the information about the target customers.

• Journal of Global Scholars of Marketing Science
• /
• v.18 no.2
• /
• pp.65-88
• /
• 2008

Since the implementation of economic reforms in 1978, the Chinese economy grows rapidly at an average annul growth rate of 9% over the post two decades. Franchising has been widely recognized as an important source of entrepreneurial activity. Trust is important in that it facilitates relational exchanges by permits partners to transcend short-run inequities or risks to concentrate on long-term profits or gains. In the relationship between the franchisors and franchisees, trust has been described as an important source of competitive advantage. However, little research has been done on the factors affecting trust in Chinese franchisor-franchisee relationships. The purpose of this study is to investigate what factors affect the trust in the franchise system in China, and to provide guidelines and insights to franchisors which enter Chinese market. In this study, according to Morgan and Hunt (1994), trust is defined as the extending when one party has confidence in an exchange partner's reliability and integrity. We offered a conceptual model of the empirical study. The model shows that the factors affecting the trust include franchisor's supports, communication, satisfaction with previous outcome and conflict. We also suggested the franchisor's supports and communication like to enhance the franchisee's satisfaction with previous outcome, and the franchisor's supports, communication and he franchisee's satisfaction with previous outcome tend to decrease conflict. Before the formal study, a pretest involving exploratory interviews with owners from three franchisees was conducted to make sure the questionnaire was relevant and clear to the respondents. The data were collected using trained interviewers to carry out personal interviews with the aid of an unidentified, muti-page, structured questionnaire. The respondents comprised of owners, managers, and owner managers of franchisee-owned food service franchises located in Beijing, China. Even though a total of 256 potential franchises were initially contacted, the finally usable sample consisted of 125 respondents. As expected, the sampling method was successful in soliciting respondents with waried personal and firm characteristics. Self-administrated questionnaires were used for all measures. And established scales were used to measure the latent constructs in this study. The measures tapped the franchisees' perceptions of the relationship with the referent franchisor. Five-point Likert-type scales ranging from "strongly disagree" (=1) to "strongly agree" (=7) were used throughout the constructs (trust, eight items; support, five items; communication, four items; satisfaction, six items; conflict, three items). The reliability measurements traditionally employed, such as the Cronbach's alpha, were used. All the reliabilities were greater than.80. The proposed measurement model was estimated using SPSS 12.0 and AMOS 5.0 analysis package. We conducted A series of exploratory factor analyses and confirmatory factor analyses to assess the convergent validity, discriminant validity, and reliability. The results indicate reasonable overall fits between the model and the observed data. The overall fit of measurement model were $X^2$= 159.699, p=0.004, d.f. = 116, GFI =.879, NFI =.898, CFI =.969, IFI =.970, TLI =.959, RMR =.058. The results demonstrated that the data reasonably fitted the model. We also examined construct reliability and reliability and average variance extracted (AVE). The construct reliability of each construct was greater than.80 and the AVE of each construct was greater than.50. According to the analysis of Structure Equation Modeling (SEM), the results of path model indicated an adequate fit of the model: $X^2$= 142.126, p = 0.044, d.f. = 115, GFI =.892, NFI =.909, CFI =.981, IFI =.981, TLI =.974, RMR =.057. As hypothesized, the results showed that it is strategically important to establish trust in a franchise system, and the franchisor's supports, communication and satisfaction with previous outcome tend to reinforce franchisee's trust. The results also showed trust seems to decrease as the experience of conflict episodes increases. And we also noticed that franchisor's supports and communication tend to enhance the franchisee's satisfaction with previous outcome, and communication tend to decrease conflict. If the trust between the franchisor and franchisee can be established in a franchise system, franchising offers many benefits and reduces many costs. To manage a mutual trust of relationship with their franchisees, franchisor's should provide support effectively to their franchisees. Effective assistant services have direct effect on franchisees' satisfaction with previous outcome and trust in franchisor. Especially, franchise sales process, orientation, and training in the start-up period are key elements for success of the franchise system. Franchisor's support is an accumulated separate satisfaction evaluation with different kind of service provided by the franchisor. And providing support definitely can improve the trustworthy image of the franchisor. In the franchise system, conflicts of interests and exertions of different power sources are very common. The experience of conflict episodes seems to negatively relate to trust. Therefore, it is important to reduce the negative side of the relationship conflicts. Communication actually plays a broader role in reducing conflict and establish mutual trust in franchisor-franchisee relationship. And effective communication between franchisors and franchisees can improve franchisees' satisfaction toward the franchise system. As the diversification of Chinese markets, both franchisors and franchisees must keep the relevant, timely, and reliable communication. And it is very important to improve the quality of communication. Satisfaction with precious outcomes seems to positively relate to trust. Franchisors and franchisees that are highly satisfied with the previous outcomes that flow from their relationship will perceive their partner as advancing their goal achievement. Therefore, it is necessary for both franchisor and their franchisees to make the welfare of partner with effort. Little literature has focused on what factors affect the trust between franchisors and their franchisees in China. This study developed the hypotheses regarding the factors affecting trust in the transaction relationship. The results of data analysis supported the hypotheses strongly. There are certain limitations in this study. First, we may point out that some other factors missed in this study could be significantly important. Second, the context of this study, food service industry, limits its potential generalizability for all franchise systems. More studies in different categories of franchise system are needed to broaden its generalizability. Third, the model was tested empirically in a sample in Beijing, more empirical tests of the proposed model in other Chinese areas are needed. Finally, the analysis in this study was solely based on the perception of franchisees and the opinions of franchisors were not included.

• Journal of Intelligence and Information Systems
• /
• v.26 no.4
• /
• pp.111-126
• /
• 2020

Most industries have recently become aware of the importance of customer lifetime value as they are exposed to a competitive environment. As a result, preventing customers from churn is becoming a more important business issue than securing new customers. This is because maintaining churn customers is far more economical than securing new customers, and in fact, the acquisition cost of new customers is known to be five to six times higher than the maintenance cost of churn customers. Also, Companies that effectively prevent customer churn and improve customer retention rates are known to have a positive effect on not only increasing the company's profitability but also improving its brand image by improving customer satisfaction. Predicting customer churn, which had been conducted as a sub-research area for CRM, has recently become more important as a big data-based performance marketing theme due to the development of business machine learning technology. Until now, research on customer churn prediction has been carried out actively in such sectors as the mobile telecommunication industry, the financial industry, the distribution industry, and the game industry, which are highly competitive and urgent to manage churn. In addition, These churn prediction studies were focused on improving the performance of the churn prediction model itself, such as simply comparing the performance of various models, exploring features that are effective in forecasting departures, or developing new ensemble techniques, and were limited in terms of practical utilization because most studies considered the entire customer group as a group and developed a predictive model. As such, the main purpose of the existing related research was to improve the performance of the predictive model itself, and there was a relatively lack of research to improve the overall customer churn prediction process. In fact, customers in the business have different behavior characteristics due to heterogeneous transaction patterns, and the resulting churn rate is different, so it is unreasonable to assume the entire customer as a single customer group. Therefore, it is desirable to segment customers according to customer classification criteria, such as loyalty, and to operate an appropriate churn prediction model individually, in order to carry out effective customer churn predictions in heterogeneous industries. Of course, in some studies, there are studies in which customers are subdivided using clustering techniques and applied a churn prediction model for individual customer groups. Although this process of predicting churn can produce better predictions than a single predict model for the entire customer population, there is still room for improvement in that clustering is a mechanical, exploratory grouping technique that calculates distances based on inputs and does not reflect the strategic intent of an entity such as loyalties. This study proposes a segment-based customer departure prediction process (CCP/2DL: Customer Churn Prediction based on Two-Dimensional Loyalty segmentation) based on two-dimensional customer loyalty, assuming that successful customer churn management can be better done through improvements in the overall process than through the performance of the model itself. CCP/2DL is a series of churn prediction processes that segment two-way, quantitative and qualitative loyalty-based customer, conduct secondary grouping of customer segments according to churn patterns, and then independently apply heterogeneous churn prediction models for each churn pattern group. Performance comparisons were performed with the most commonly applied the General churn prediction process and the Clustering-based churn prediction process to assess the relative excellence of the proposed churn prediction process. The General churn prediction process used in this study refers to the process of predicting a single group of customers simply intended to be predicted as a machine learning model, using the most commonly used churn predicting method. And the Clustering-based churn prediction process is a method of first using clustering techniques to segment customers and implement a churn prediction model for each individual group. In cooperation with a global NGO, the proposed CCP/2DL performance showed better performance than other methodologies for predicting churn. This churn prediction process is not only effective in predicting churn, but can also be a strategic basis for obtaining a variety of customer observations and carrying out other related performance marketing activities.