Efficient Topic Modeling by Mapping Global and Local Topics (전역 토픽의 지역 매핑을 통한 효율적 토픽 모델링 방안)
-
- Journal of Intelligence and Information Systems
- /
- v.23 no.3
- /
- pp.69-94
- /
- 2017
Recently, increase of demand for big data analysis has been driving the vigorous development of related technologies and tools. In addition, development of IT and increased penetration rate of smart devices are producing a large amount of data. According to this phenomenon, data analysis technology is rapidly becoming popular. Also, attempts to acquire insights through data analysis have been continuously increasing. It means that the big data analysis will be more important in various industries for the foreseeable future. Big data analysis is generally performed by a small number of experts and delivered to each demander of analysis. However, increase of interest about big data analysis arouses activation of computer programming education and development of many programs for data analysis. Accordingly, the entry barriers of big data analysis are gradually lowering and data analysis technology being spread out. As the result, big data analysis is expected to be performed by demanders of analysis themselves. Along with this, interest about various unstructured data is continually increasing. Especially, a lot of attention is focused on using text data. Emergence of new platforms and techniques using the web bring about mass production of text data and active attempt to analyze text data. Furthermore, result of text analysis has been utilized in various fields. Text mining is a concept that embraces various theories and techniques for text analysis. Many text mining techniques are utilized in this field for various research purposes, topic modeling is one of the most widely used and studied. Topic modeling is a technique that extracts the major issues from a lot of documents, identifies the documents that correspond to each issue and provides identified documents as a cluster. It is evaluated as a very useful technique in that reflect the semantic elements of the document. Traditional topic modeling is based on the distribution of key terms across the entire document. Thus, it is essential to analyze the entire document at once to identify topic of each document. This condition causes a long time in analysis process when topic modeling is applied to a lot of documents. In addition, it has a scalability problem that is an exponential increase in the processing time with the increase of analysis objects. This problem is particularly noticeable when the documents are distributed across multiple systems or regions. To overcome these problems, divide and conquer approach can be applied to topic modeling. It means dividing a large number of documents into sub-units and deriving topics through repetition of topic modeling to each unit. This method can be used for topic modeling on a large number of documents with limited system resources, and can improve processing speed of topic modeling. It also can significantly reduce analysis time and cost through ability to analyze documents in each location or place without combining analysis object documents. However, despite many advantages, this method has two major problems. First, the relationship between local topics derived from each unit and global topics derived from entire document is unclear. It means that in each document, local topics can be identified, but global topics cannot be identified. Second, a method for measuring the accuracy of the proposed methodology should be established. That is to say, assuming that global topic is ideal answer, the difference in a local topic on a global topic needs to be measured. By those difficulties, the study in this method is not performed sufficiently, compare with other studies dealing with topic modeling. In this paper, we propose a topic modeling approach to solve the above two problems. First of all, we divide the entire document cluster(Global set) into sub-clusters(Local set), and generate the reduced entire document cluster(RGS, Reduced global set) that consist of delegated documents extracted from each local set. We try to solve the first problem by mapping RGS topics and local topics. Along with this, we verify the accuracy of the proposed methodology by detecting documents, whether to be discerned as the same topic at result of global and local set. Using 24,000 news articles, we conduct experiments to evaluate practical applicability of the proposed methodology. In addition, through additional experiment, we confirmed that the proposed methodology can provide similar results to the entire topic modeling. We also proposed a reasonable method for comparing the result of both methods.
Green or an environmental consciousness has been a major issue for businesses and government offices, as well as consumers, worldwide. In response to this movement, the Korean government announced, in the early 2000s, the era of "Green Growth" as a way to encourage green-related business activities. The Korean fashion industry, in various levels of involvement, presents diverse eco-friendly products as a part of the green movement. These apparel products include organic products and recycled clothing. For these companies to be successful, they need information about who are the consumers who consider green issues (e.g., environmental sustainability) as part of their personal values when making a decision for product purchase, use, and disposal. These consumers can be considered as eco-sumers. Previous studies have examined consumers' purchase intention for or with eco-friendly products. In addition, studies have examined influential factors used to identify the eco-sumers or green consumers. However, limited attention was paid to eco-sumers' disposal or recycling behavior of clothes in comparison with their green product purchases. Clothing disposal behaviors are ways that consumer can get rid of unused clothing and in clue temporarily lending the item or permanently eliminating the item by "handing down" (e.g., giving it to a younger sibling), donating, exchanging, selling, or simply throwing it away. Accordingly, examining purchasing behaviors of eco-friendly fashion items in conjunction with clothing disposal behaviors should improve understanding of a consumer's clothing consumption behavior from the environmental perspective. The purpose of this exploratory study is to provide descriptive information about Korean eco-sumers who have ecologically-favorable lifestyles and behaviors when buying and disposing of clothes. The objectives of this study are to (a) categorize Koreans on the basis of clothing disposal behaviors; (b) investigate the differences in demographics, lifestyles, and clothing consumption values among segments; and (c) compare the purchase intention of eco-friendly fashion items and influential factors among segments. A self-administered questionnaire was developed based on previous studies. The questionnaire included 10 items of clothing disposal behavior, 22 items of LOHAS (Lifestyles of Health and Sustainability) characteristics, and 19 items of consumption values, measured by five-point Likert-type scales. In addition, the purchase intention of two eco-friendly fashion items and 11 attributes of each item were measured by seven-point Likert type scales. Two polyester fleece pullovers, made from fabric created from recycled bottles with the PET identification code, were selected from one Korean brand and one US imported brand among outdoor sportswear brands. A brief description of each product with a color picture was provided in the survey. Demographic variables (i.e., gender, age, marital status, education level, income, occupation) were also included. The data were collected through a professional web survey agency during May 2009. A total of 600 final usable questionnaires were analyzed. The age of respondents ranged from 20 to 49 years old with a mean age of 34 years. Fifty percent of the respondents were males and about 58% were married, and 62% reported having earned university degrees. Principal components factor analysis with varimax rotation was used to identify the underlying dimensions of the clothing disposal behavior scale, and three factors were generated (i.e., reselling behavior, donating behavior, non-recycling behavior). To categorize the respondents on the basis of clothing disposal behaviors, k-mean cluster analysis was used, and three segments were obtained. These consumer segments were labeled as 'Resale Group', 'Donation Group', and 'Non-Recycling Group.' The classification results indicated approximately 98 percent of the original cases were correctly classified. With respect to demographic characteristics among the three segments, significant differences were found in gender, marital status, occupation, and age. LOHAS characteristics were reduced into the following five factors: self-satisfaction, family orientation, health concern, environmental concern, and voluntary service. Significant differences were found in the LOHAS factors among the three clusters. Resale Group and Donation Group showed a similar predisposition to LOHAS issues while the Non-Recycling Group presented the lowest mean scores on the LOHAS factors compared to the other segments. The Resale and Donation Groups described themselves as enjoying or being satisfied with their lives and spending spare-time with family. In addition, these two groups cared about health and organic foods, and tried to conserve energy and resources. Principal components factor analysis generated clothing consumption values into the following three factors: personal values, social value, and practical value. The ANOVA test with the factors showed differences primarily between the Resale Group and the other two groups. The Resale Group was more concerned about personal value and social value than the other segments. In contrast, the Non-Recycling Group presented the higher level of social value than did Donation Group. In a comparison of the intention to purchase eco-friendly products, the Resale Group showed the highest mean score on intent to purchase Product A. On the other hand, the Donation Group presented the highest intention to purchase for Product B among segments. In addition, the mean scores indicated that the Korean product (Product B) was more preferable for purchase than the U.S. product (Product A). Stepwise regression analysis was used to identify the influence of product attributes on the purchase intention of eco product. With respect to Product A, design, price and contribution to environmental preservation were significant to predict purchase intention for the Resale Group, while price and compatibility with my image factors were significant for the Donation Group. For the Non-Recycling Group, design, price compatibility with the factors of my image, participation to eco campaign, and contribution to environmental preservation were significant. Price appropriateness was significant for each of the three clusters. With respect to Product B, design, price and compatibility with my image factors were important, but different attributes were associated significantly with purchase intention for each of the three groups. The influence of LOHAS characteristics and clothing consumption values on intention to purchase Products A and B were also examined. The LOHAS factor of health concern and the personal value factor were significant in the relationships with the purchase intention; however, the explanatory powers were low in the three segments. Findings showed that each group as classified by clothing disposal behaviors showed differences in the attributes of a product, personal values, and the LOHAS characteristics that influenced their purchase intention of eco-friendly products. Findings would enable organizations to understand eco-friendly behavior and to design appropriate strategic decisions to appeal eco-sumers.
To discover significant social issues such as unemployment, economy crisis, social welfare etc. that are urgent issues to be solved in a modern society, in the existing approach, researchers usually collect opinions from professional experts and scholars through either online or offline surveys. However, such a method does not seem to be effective from time to time. As usual, due to the problem of expense, a large number of survey replies are seldom gathered. In some cases, it is also hard to find out professional persons dealing with specific social issues. Thus, the sample set is often small and may have some bias. Furthermore, regarding a social issue, several experts may make totally different conclusions because each expert has his subjective point of view and different background. In this case, it is considerably hard to figure out what current social issues are and which social issues are really important. To surmount the shortcomings of the current approach, in this paper, we develop a prototype system that semi-automatically detects social issue keywords representing social issues and problems from about 1.3 million news articles issued by about 10 major domestic presses in Korea from June 2009 until July 2012. Our proposed system consists of (1) collecting and extracting texts from the collected news articles, (2) identifying only news articles related to social issues, (3) analyzing the lexical items of Korean sentences, (4) finding a set of topics regarding social keywords over time based on probabilistic topic modeling, (5) matching relevant paragraphs to a given topic, and (6) visualizing social keywords for easy understanding. In particular, we propose a novel matching algorithm relying on generative models. The goal of our proposed matching algorithm is to best match paragraphs to each topic. Technically, using a topic model such as Latent Dirichlet Allocation (LDA), we can obtain a set of topics, each of which has relevant terms and their probability values. In our problem, given a set of text documents (e.g., news articles), LDA shows a set of topic clusters, and then each topic cluster is labeled by human annotators, where each topic label stands for a social keyword. For example, suppose there is a topic (e.g., Topic1 = {(unemployment, 0.4), (layoff, 0.3), (business, 0.3)}) and then a human annotator labels "Unemployment Problem" on Topic1. In this example, it is non-trivial to understand what happened to the unemployment problem in our society. In other words, taking a look at only social keywords, we have no idea of the detailed events occurring in our society. To tackle this matter, we develop the matching algorithm that computes the probability value of a paragraph given a topic, relying on (i) topic terms and (ii) their probability values. For instance, given a set of text documents, we segment each text document to paragraphs. In the meantime, using LDA, we can extract a set of topics from the text documents. Based on our matching process, each paragraph is assigned to a topic, indicating that the paragraph best matches the topic. Finally, each topic has several best matched paragraphs. Furthermore, assuming there are a topic (e.g., Unemployment Problem) and the best matched paragraph (e.g., Up to 300 workers lost their jobs in XXX company at Seoul). In this case, we can grasp the detailed information of the social keyword such as "300 workers", "unemployment", "XXX company", and "Seoul". In addition, our system visualizes social keywords over time. Therefore, through our matching process and keyword visualization, most researchers will be able to detect social issues easily and quickly. Through this prototype system, we have detected various social issues appearing in our society and also showed effectiveness of our proposed methods according to our experimental results. Note that you can also use our proof-of-concept system in http://dslab.snu.ac.kr/demo.html.
With the fourth industrial revolution and the arrival of the New Normal era due to Corona, the importance of Non-contact technologies such as artificial intelligence and big data research has been increasing. Convergent research is being conducted in earnest to keep up with these research trends, but not many studies have been conducted in the area of nuclear research using artificial intelligence and big data-related technologies such as natural language processing and text mining analysis. This study was conducted to confirm the applicability of data science analysis techniques to the field of nuclear research. Furthermore, the study of identifying trends in nuclear spent fuel recognition is critical in terms of being able to determine directions to nuclear industry policies and respond in advance to changes in industrial policies. For those reasons, this study conducted a media trend analysis of pyroprocessing, a spent nuclear fuel treatment technology. We objectively analyze changes in media perception of spent nuclear fuel dry treatment techniques by applying text mining analysis techniques. Text data specializing in Naver's web news articles, including the keywords "Pyroprocessing" and "Sodium Cooled Reactor," were collected through Python code to identify changes in perception over time. The analysis period was set from 2007 to 2020, when the first article was published, and detailed and multi-layered analysis of text data was carried out through analysis methods such as word cloud writing based on frequency analysis, TF-IDF and degree centrality calculation. Analysis of the frequency of the keyword showed that there was a change in media perception of spent nuclear fuel dry treatment technology in the mid-2010s, which was influenced by the Gyeongju earthquake in 2016 and the implementation of the new government's energy conversion policy in 2017. Therefore, trend analysis was conducted based on the corresponding time period, and word frequency analysis, TF-IDF, degree centrality values, and semantic network graphs were derived. Studies show that before the 2010s, media perception of spent nuclear fuel dry treatment technology was diplomatic and positive. However, over time, the frequency of keywords such as "safety", "reexamination", "disposal", and "disassembly" has increased, indicating that the sustainability of spent nuclear fuel dry treatment technology is being seriously considered. It was confirmed that social awareness also changed as spent nuclear fuel dry treatment technology, which was recognized as a political and diplomatic technology, became ambiguous due to changes in domestic policy. This means that domestic policy changes such as nuclear power policy have a greater impact on media perceptions than issues of "spent nuclear fuel processing technology" itself. This seems to be because nuclear policy is a socially more discussed and public-friendly topic than spent nuclear fuel. Therefore, in order to improve social awareness of spent nuclear fuel processing technology, it would be necessary to provide sufficient information about this, and linking it to nuclear policy issues would also be a good idea. In addition, the study highlighted the importance of social science research in nuclear power. It is necessary to apply the social sciences sector widely to the nuclear engineering sector, and considering national policy changes, we could confirm that the nuclear industry would be sustainable. However, this study has limitations that it has applied big data analysis methods only to detailed research areas such as "Pyroprocessing," a spent nuclear fuel dry processing technology. Furthermore, there was no clear basis for the cause of the change in social perception, and only news articles were analyzed to determine social perception. Considering future comments, it is expected that more reliable results will be produced and efficiently used in the field of nuclear policy research if a media trend analysis study on nuclear power is conducted. Recently, the development of uncontact-related technologies such as artificial intelligence and big data research is accelerating in the wake of the recent arrival of the New Normal era caused by corona. Convergence research is being conducted in earnest in various research fields to follow these research trends, but not many studies have been conducted in the nuclear field with artificial intelligence and big data-related technologies such as natural language processing and text mining analysis. The academic significance of this study is that it was possible to confirm the applicability of data science analysis technology in the field of nuclear research. Furthermore, due to the impact of current government energy policies such as nuclear power plant reductions, re-evaluation of spent fuel treatment technology research is undertaken, and key keyword analysis in the field can contribute to future research orientation. It is important to consider the views of others outside, not just the safety technology and engineering integrity of nuclear power, and further reconsider whether it is appropriate to discuss nuclear engineering technology internally. In addition, if multidisciplinary research on nuclear power is carried out, reasonable alternatives can be prepared to maintain the nuclear industry.
The main purpose of the investigation of an accident caused by aircraft is to be prevented the sudden and casual accidents caused by wilful misconduct and fault from pilots, air traffic controllers, hijack, trouble of engine and machinery of aircraft, turbulence during the bad weather, collision between birds and aircraft, near miss flight by aircrafts etc. It is not the purpose of this activity to apportion blame or liability for offender of aircraft accidents. Accidents to aircraft, especially those involving the general public and their property, are a matter of great concern to the aviation community. The system of international regulation exists to improve safety and minimize, as far as possible, the risk of accidents but when they do occur there is a web of systems and procedures to investigate and respond to them. I would like to trace the general line of regulation from an international source in the Chicago Convention of 1944. Article 26 of the Convention lays down the basic principle for the investigation of the aircraft accident. Where there has been an accident to an aircraft of a contracting state which occurs in the territory of another contracting state and which involves death or serious injury or indicates serious technical defect in the aircraft or air navigation facilities, the state in which the accident occurs must institute an inquiry into the circumstances of the accident. That inquiry will be in accordance, in so far as its law permits, with the procedure which may be recommended from time to time by the International Civil Aviation Organization ICAO). There are very general provisions but they state two essential principles: first, in certain circumstances there must be an investigation, and second, who is to be responsible for undertaking that investigation. The latter is an important point to establish otherwise there could be at least two states claiming jurisdiction on the inquiry. The Chicago Convention also provides that the state where the aircraft is registered is to be given the opportunity to appoint observers to be present at the inquiry and the state holding the inquiry must communicate the report and findings in the matter to that other state. It is worth noting that the Chicago Convention (Article 25) also makes provision for assisting aircraft in distress. Each contracting state undertakes to provide such measures of assistance to aircraft in distress in its territory as it may find practicable and to permit (subject to control by its own authorities) the owner of the aircraft or authorities of the state in which the aircraft is registered, to provide such measures of assistance as may be necessitated by circumstances. Significantly, the undertaking can only be given by contracting state but the duty to provide assistance is not limited to aircraft registered in another contracting state, but presumably any aircraft in distress in the territory of the contracting state. Finally, the Convention envisages further regulations (normally to be produced under the auspices of ICAO). In this case the Convention provides that each contracting state, when undertaking a search for missing aircraft, will collaborate in co-ordinated measures which may be recommended from time to time pursuant to the Convention. Since 1944 further international regulations relating to safety and investigation of accidents have been made, both pursuant to Chicago Convention and, in particular, through the vehicle of the ICAO which has, for example, set up an accident and reporting system. By requiring the reporting of certain accidents and incidents it is building up an information service for the benefit of member states. However, Chicago Convention provides that each contracting state undertakes collaborate in securing the highest practicable degree of uniformity in regulations, standards, procedures and organization in relation to aircraft, personnel, airways and auxiliary services in all matters in which such uniformity will facilitate and improve air navigation. To this end, ICAO is to adopt and amend from time to time, as may be necessary, international standards and recommended practices and procedures dealing with, among other things, aircraft in distress and investigation of accidents. Standards and Recommended Practices for Aircraft Accident Injuries were first adopted by the ICAO Council on 11 April 1951 pursuant to Article 37 of the Chicago Convention on International Civil Aviation and were designated as Annex 13 to the Convention. The Standards Recommended Practices were based on Recommendations of the Accident Investigation Division at its first Session in February 1946 which were further developed at the Second Session of the Division in February 1947. The 2nd Edition (1966), 3rd Edition, (1973), 4th Edition (1976), 5th Edition (1979), 6th Edition (1981), 7th Edition (1988), 8th Edition (1992) of the Annex 13 (Aircraft Accident and Incident Investigation) of the Chicago Convention was amended eight times by the ICAO Council since 1966. Annex 13 sets out in detail the international standards and recommended practices to be adopted by contracting states in dealing with a serious accident to an aircraft of a contracting state occurring in the territory of another contracting state, known as the state of occurrence. It provides, principally, that the state in which the aircraft is registered is to be given the opportunity to appoint an accredited representative to be present at the inquiry conducted by the state in which the serious aircraft accident occurs. Article 26 of the Chicago Convention does not indicate what the accredited representative is to do but Annex 13 amplifies his rights and duties. In particular, the accredited representative participates in the inquiry by visiting the scene of the accident, examining the wreckage, questioning witnesses, having full access to all relevant evidence, receiving copies of all pertinent documents and making submissions in respect of the various elements of the inquiry. The main shortcomings of the present system for aircraft accident investigation are that some contracting sates are not applying Annex 13 within its express terms, although they are contracting states. Further, and much more important in practice, there are many countries which apply the letter of Annex 13 in such a way as to sterilise its spirit. This appears to be due to a number of causes often found in combination. Firstly, the requirements of the local law and of the local procedures are interpreted and applied so as preclude a more efficient investigation under Annex 13 in favour of a legalistic and sterile interpretation of its terms. Sometimes this results from a distrust of the motives of persons and bodies wishing to participate or from commercial or related to matters of liability and bodies. These may be political, commercial or related to matters of liability and insurance. Secondly, there is said to be a conscious desire to conduct the investigation in some contracting states in such a way as to absolve from any possibility of blame the authorities or nationals, whether manufacturers, operators or air traffic controllers, of the country in which the inquiry is held. The EEC has also had an input into accidents and investigations. In particular, a directive was issued in December 1980 encouraging the uniformity of standards within the EEC by means of joint co-operation of accident investigation. The sharing of and assisting with technical facilities and information was considered an important means of achieving these goals. It has since been proposed that a European accident investigation committee should be set up by the EEC (Council Directive 80/1266 of 1 December 1980). After I would like to introduce the summary of the legislation examples and system for aircraft accidents investigation of the United States, the United Kingdom, Canada, Germany, The Netherlands, Sweden, Swiss, New Zealand and Japan, and I am going to mention the present system, regulations and aviation act for the aircraft accident investigation in Korea. Furthermore I would like to point out the shortcomings of the present system and regulations and aviation act for the aircraft accident investigation and then I will suggest my personal opinion on the new and dramatic innovation on the system for aircraft accident investigation in Korea. I propose that it is necessary and desirable for us to make a new legislation or to revise the existing aviation act in order to establish the standing and independent Committee of Aircraft Accident Investigation under the Korean Government.
Social media has become one of the most popular media in web and mobile application. In 2011, social networks and blogs are still the top destination of online users, according to a study from Nielsen Company. In their studies, nearly 4 in 5active users visit social network and blog. Social Networks and Blogs sites rule Americans' Internet time, accounting to 23 percent of time spent online. Facebook is the main social network that the U.S internet users spend time more than the other social network services such as Yahoo, Google, AOL Media Network, Twitter, Linked In and so on. In recent trend, most of the companies promote their products in the Facebook by creating the "Facebook Page" that refers to specific product. The "Like" option allows user to subscribed and received updates their interested on from the page. The film makers which produce a lot of films around the world also take part to market and promote their films by exploiting the advantages of using the "Facebook Page". In addition, a great number of streaming service providers allows users to subscribe their service to watch and enjoy movies and TV program. They can instantly watch movies and TV program over the internet to PCs, Macs and TVs. Netflix alone as the world's leading subscription service have more than 30 million streaming members in the United States, Latin America, the United Kingdom and the Nordics. As the matter of facts, a million of movies and TV program with different of genres are offered to the subscriber. In contrast, users need spend a lot time to find the right movies which are related to their interest genre. Recent years there are many researchers who have been propose a method to improve prediction the rating or preference that would give the most related items such as books, music or movies to the garget user or the group of users that have the same interest in the particular items. One of the most popular methods to build recommendation system is traditional Collaborative Filtering (CF). The method compute the similarity of the target user and other users, which then are cluster in the same interest on items according which items that users have been rated. The method then predicts other items from the same group of users to recommend to a group of users. Moreover, There are many items that need to study for suggesting to users such as books, music, movies, news, videos and so on. However, in this paper we only focus on movie as item to recommend to users. In addition, there are many challenges for CF task. Firstly, the "sparsity problem"; it occurs when user information preference is not enough. The recommendation accuracies result is lower compared to the neighbor who composed with a large amount of ratings. The second problem is "cold-start problem"; it occurs whenever new users or items are added into the system, which each has norating or a few rating. For instance, no personalized predictions can be made for a new user without any ratings on the record. In this research we propose a clustering method according to the users' genre interest extracted from social network service (SNS) and user's movies rating information system to solve the "cold-start problem." Our proposed method will clusters the target user together with the other users by combining the user genre interest and the rating information. It is important to realize a huge amount of interesting and useful user's information from Facebook Graph, we can extract information from the "Facebook Page" which "Like" by them. Moreover, we use the Internet Movie Database(IMDb) as the main dataset. The IMDbis online databases that consist of a large amount of information related to movies, TV programs and including actors. This dataset not only used to provide movie information in our Movie Rating Systems, but also as resources to provide movie genre information which extracted from the "Facebook Page". Formerly, the user must login with their Facebook account to login to the Movie Rating System, at the same time our system will collect the genre interest from the "Facebook Page". We conduct many experiments with other methods to see how our method performs and we also compare to the other methods. First, we compared our proposed method in the case of the normal recommendation to see how our system improves the recommendation result. Then we experiment method in case of cold-start problem. Our experiment show that our method is outperform than the other methods. In these two cases of our experimentation, we see that our proposed method produces better result in case both cases.