Efficient Topic Modeling by Mapping Global and Local Topics (전역 토픽의 지역 매핑을 통한 효율적 토픽 모델링 방안)
-
- Journal of Intelligence and Information Systems
- /
- v.23 no.3
- /
- pp.69-94
- /
- 2017
Recently, increase of demand for big data analysis has been driving the vigorous development of related technologies and tools. In addition, development of IT and increased penetration rate of smart devices are producing a large amount of data. According to this phenomenon, data analysis technology is rapidly becoming popular. Also, attempts to acquire insights through data analysis have been continuously increasing. It means that the big data analysis will be more important in various industries for the foreseeable future. Big data analysis is generally performed by a small number of experts and delivered to each demander of analysis. However, increase of interest about big data analysis arouses activation of computer programming education and development of many programs for data analysis. Accordingly, the entry barriers of big data analysis are gradually lowering and data analysis technology being spread out. As the result, big data analysis is expected to be performed by demanders of analysis themselves. Along with this, interest about various unstructured data is continually increasing. Especially, a lot of attention is focused on using text data. Emergence of new platforms and techniques using the web bring about mass production of text data and active attempt to analyze text data. Furthermore, result of text analysis has been utilized in various fields. Text mining is a concept that embraces various theories and techniques for text analysis. Many text mining techniques are utilized in this field for various research purposes, topic modeling is one of the most widely used and studied. Topic modeling is a technique that extracts the major issues from a lot of documents, identifies the documents that correspond to each issue and provides identified documents as a cluster. It is evaluated as a very useful technique in that reflect the semantic elements of the document. Traditional topic modeling is based on the distribution of key terms across the entire document. Thus, it is essential to analyze the entire document at once to identify topic of each document. This condition causes a long time in analysis process when topic modeling is applied to a lot of documents. In addition, it has a scalability problem that is an exponential increase in the processing time with the increase of analysis objects. This problem is particularly noticeable when the documents are distributed across multiple systems or regions. To overcome these problems, divide and conquer approach can be applied to topic modeling. It means dividing a large number of documents into sub-units and deriving topics through repetition of topic modeling to each unit. This method can be used for topic modeling on a large number of documents with limited system resources, and can improve processing speed of topic modeling. It also can significantly reduce analysis time and cost through ability to analyze documents in each location or place without combining analysis object documents. However, despite many advantages, this method has two major problems. First, the relationship between local topics derived from each unit and global topics derived from entire document is unclear. It means that in each document, local topics can be identified, but global topics cannot be identified. Second, a method for measuring the accuracy of the proposed methodology should be established. That is to say, assuming that global topic is ideal answer, the difference in a local topic on a global topic needs to be measured. By those difficulties, the study in this method is not performed sufficiently, compare with other studies dealing with topic modeling. In this paper, we propose a topic modeling approach to solve the above two problems. First of all, we divide the entire document cluster(Global set) into sub-clusters(Local set), and generate the reduced entire document cluster(RGS, Reduced global set) that consist of delegated documents extracted from each local set. We try to solve the first problem by mapping RGS topics and local topics. Along with this, we verify the accuracy of the proposed methodology by detecting documents, whether to be discerned as the same topic at result of global and local set. Using 24,000 news articles, we conduct experiments to evaluate practical applicability of the proposed methodology. In addition, through additional experiment, we confirmed that the proposed methodology can provide similar results to the entire topic modeling. We also proposed a reasonable method for comparing the result of both methods.
To optimise the efficient use of nutrients in pig slurry is to cultivate friendly environmental crops. This field survey is to investigate the actual conditions of pig slurry utilization for cultivation of crops in the agricultural farm, based on the survey for 407 selected farms in 9 provinces included 78 counties in Korea. The results obtained in this survey were summarized as follow ; The motive which came to use pig slurry in the agricultural farm were production of friendly environmental crops (29.7%), economy of chemical fertilizer (25.1%), spontaneously (19.2%), inducement of neighboring farmhouse (16.0%), increase of soil fertility (9.3%), and the others (0.7%), respectively. The proportions of pig slurry application land were 56.5% for.ice paddy, 22.6% for dry field, 13.3% for orchard, 4.4% for controlled agriculture and 3.2% for other, respectively. The number of times of pig slurry utilization per year were once (48.9%), twice (31.9%), thrice (14.0%), and the others (5.2%), respectively. The controversial points of pig slurry utilization were malodor (54.1%), insufficiency of spread equipment (22.1%), inconvenience (14.5%), over application (3.4%), over cost (2.9%), heavy metal (1.7%), sanitation (1.0%) and the other (0.2%), respectively. The results indicated that pig slurry could be used as fertilizer source of friendly environmental crops, but further studies are needed to determine the application method and value of the pig slurry for crop cultivation.
For the purpose of establishing the systematized technical scheme of the cooperative rice cultivation which has most significant impact to improve rice productivity and the farm management, the author have studied the cultivation practices, and the variation of rice growth and yield between the cooperative rice cultivation and the individual rice cultivation at random selected 18 paddy fields. The author also have investigated through comparative method on the cultivation practices, management, organization and operation scheme of the two different rice cultivation methods at 460 paddy fields. The economic feasibility has been ana lysed and added in this report. The results obtained from this study are summarized as follows; 1. In the nursery, the average amount of fertilizer application, especially, phosphate and potassium, and the frequency of chemicals spray for the disease, insect and pest control at the cooperative rice cultivation are significantly higher than those of the individual rice cultivation. 2. The cultivation techniques of the cooperative rice farming after the transplanting can be characterized by a) the earlier transplanting of rice, b) the denser hills per unit area and the lesser number of seedlings per hill, c) the application of larger quantities of fertilizer including nitrogen, phosphate and potassium, d) more divided application of fertilizers, split doses of the nitrogen and potassium, e) the increased frequencies of the chemicals spray for the prevention of disease, insect and pest damages. 3. The rate of lodging in the cooperative rice cultivation was slightly higher than that of the individual rice cultivation, however, the losses of rice yield owing to the occurrence of rice stem borer and grass leaf roller in the cooperative rice cultivation were lower than that of the individual rice cultivation. 4. The culm length, panicle length, straw weight and grain-straw ratio are respectively higher at the cooperative rice cultivation, moreover, the higher variation of the above factors due to different localities of the paddy fields found at the individual rice cultivation. 5. The number of panicles, number of flowers per panicle and the weight of 1, 000 grains, those contributing components to the rice yield were significantly greater in the cooperative rice cultivation, however, not clear difference in the maturing rate was observed. The variation coefficient of the yield component in the cooperative cultivation showed lower than that or the individual rice cultivation. 6. The average yield of brown rice per 10 are in the cooperative rice cultivation obtained 459.0 kilograms while that of the individual rice cultivation brought 374.8 kilograms. The yield of brown rice in the cooperative rice cultivation increased 84.2 kilogram per 10 are over the individual rice cultivation. With lower variation coefficient of the brown rice yield in the cooperative rice cultivation, it can be said that uniformed higher yield could be obtained through the cooperative rice cultivation. 7. Highly significant positive correlations shown between the seeding date and the number of flowers per panicle, the chemical spray and the number of flowers per panicle, the transplanting date and the number of flowers per panicle, phosphate application and yield, potassium application and maturing rate, the split application of fertilizers and yield. Whilst the significant negative correlation was shown between the transplanting date and the maturing rate 8. The results of investigation from 480 paddy fields obtained through comparative method on the following items are identical in general with those obtained at 18 paddy fields: Application of fertilizers, chemical spray for the control of disease, insects and pests both in the nursery and the paddy field, transplanting date, transplanting density, split application of fertilizers and yield n the paddy fields. a) The number of rice varieties used in the cooperative rice cultivation were 13 varieties while the individual rice cultivation used 47 varieties. b) The cooperative rice cultivation has more successfully adopted improved cultivation techniques such as the practice of seed disinfection, adoption of recommended seeding amount, fall ploughing, application of red soil, introduction of power tillers, the rectangular-type transplanting, midsummer drainage and the periodical irrigation. 9. The following results were also obtained from the same investigation and they are: a) In the cooperative rice cultivation, the greater part of the important practices have been carried out through cooperative operation including seed disinfection, ploughing, application of red soil and compost, the control of disease, insects and pests, harvest, threshing and transportation of the products. b) The labor input to the nursery bed and water control in the cooperative rice cultivation was less than that of the individual rice cultivation while the higher rate of labor input was resulted in the red soil and compost application. 10. From the investigation on the organization and operation scheme of the cooperative rice cultivation, the following results were obtained: a) The size of cooperative rice cultivation farm was varied from. 3 ha to 7 ha and 5 ha farm. occupied 55.9 percent of the total farms. And a single cooperative farm was consisted of 10 to 20 plots of paddies. b) The educational back ground of the staff members involved in the cooperative rice cultivation was superior than that of the individual rice cultivation. c) All of the farmers who participated to the questionaires have responded that the cooperative rice cultivation could promise the increased rice yield mainly through the introduction of the improved method of fertilizer application and the effective control of diseases, insects and pests damages. And the majority of farmers were also in the opinion that preparation of the materials and labor input can be timely carried out and the labor requirement for the rice cultivation possibly be saved through the cooperative rice cultivation. d) The farmers who have expressed their wishes to continue and to make further development of the cooperative rice cultivation was 74.5 percent of total farmers participated to the questionaires. 11. From the analysis of economical feasibility on the two different methods of cultivation, the following results were obtained: a) The value of operation cost for the compost, chemical fertilizers, agricultural chemicals and labor input in the cooperative rice cultivation was respectively higher by 335 won, 199 won, 288 won and 303 won over the individual rice cultivation. However, the other production costs showed no distinct differences between the two cultivation methods. b) Although the total value of expenses for the fertilizers, agricultural chemicals, labor input and etc. in the cooperative rice cultivation were approximately doubled to the amount of the individual rice cultivation, the net income, substracted operation costs from the gross income, was obtained 24, 302 won in the cooperative rice cultivation and 20, 168 won was obtained from the individual rice cultivation. Thereby, it can be said that net income from the cooperative rice cultivation increased 4, 134 won over the individual rice cultivation. It was revealed in this study that the cooperative rice cultivation has not only contributed to increment of the farm income through higher yield but also showed as an effective means to introduce highly improved cultivation techniques to the farmers. It may also be concluded, therefore, the cooperative rice cultivation shall continuously renovate the rice production process of the farmers.