• Journal of Internet Computing and Services
• /
• v.20 no.3
• /
• pp.43-59
• /
• 2019

Today, with the spread of smartphones and the development of social networking services, structured and unstructured big data have stored exponentially. If we analyze them well, we will get useful information to be able to predict data for the future. Large amounts of data need to be collected first in order to analyze big data. The web is repository where these data are most stored. However, because the data size is large, there are also many data that have information that is not needed as much as there are data that have useful information. This has made it important to collect data efficiently, where data with unnecessary information is filtered and only collected data with useful information. Web crawlers cannot download all pages due to some constraints such as network bandwidth, operational time, and data storage. This is why we should avoid visiting many pages that are not relevant to what we want and download only important pages as soon as possible. This paper seeks to help resolve the above issues. First, We introduce basic web-crawling algorithms. For each algorithm, the time-complexity and pros and cons are described, and compared and analyzed. Next, we introduce the state-of-the-art web crawling algorithms that have improved the shortcomings of the basic web crawling algorithms. In addition, recent research trends show that the web crawling algorithms with special purposes such as collecting sentiment words are actively studied. We will one of the introduce Sentiment-aware web crawling techniques that is a proactive web crawling technique as a study of web crawling algorithms with special purpose. The result showed that the larger the data are, the higher the performance is and the more space is saved.

• The Journal of Society for e-Business Studies
• /
• v.26 no.1
• /
• pp.43-65
• /
• 2021

With the advancement of information and communications technology, a means to recruit IT professional has fundamentally changed. Nowadays recruiters search for candidate information from the Web as well as traditional information sources such as résumés or interviews. Particularly, open-source software development (OSSD) platforms have become an opportunity for developers to demonstrate their IT capabilities, making it a way for recruiters to find the right candidates, whom they need. Therefore, this study aims to investigate the impact developers' profiles in an OSSD platform on their finding a job. This study examined four antecedents of developer information that can accelerate their job search: job-seeking status, personal-information posting, learning activities and knowledge contribution activities. For the empirical analysis, we developed a Web crawler and gathered a dataset on 4,005 developers from GitHub, which is a well-known OSSD platform. Proportional hazards regression was used for data analysis because shorter job-seeking period implies more successful result of job change. Our results indicate that developers, who explicitly posted their job-seeking status, had shorter job-seeking periods than those who did not. The other antecedents (i.e., personal-information posting, learning, and knowledge contribution activities) also contributed in reducing the job-seeking period. These findings imply values of OSSD platforms for recruiters to find proper candidates and for developers to successfully find a job.

• Journal of the Korea Society of Computer and Information
• /
• v.27 no.2
• /
• pp.171-177
• /
• 2022

In this paper, we propose a service method that can provide insight into multi-source agricultural data, way to cluster environmental factor which supports data analysis according to time flow, and curate crop environmental factors. The proposed curation service consists of four steps: collection, preprocessing, storage, and analysis. First, in the collection step, the service system collects and organizes multi-source agricultural data by using an OpenAPI-based web crawler. Second, in the preprocessing step, the system performs data smoothing to reduce the data measurement errors. Here, we adopt the smoothing method for each type of facility in consideration of the error rate according to facility characteristics such as greenhouses and open fields. Third, in the storage step, an agricultural data integration schema and Hadoop HDFS-based storage structure are proposed for large-scale agricultural data. Finally, in the analysis step, the service system performs DTW-based time series classification in consideration of the characteristics of agricultural digital data. Through the DTW-based classification, the accuracy of prediction results is improved by reflecting the characteristics of time series data without any loss. As a future work, we plan to implement the proposed service method and apply it to the smart farm greenhouse for testing and verification.

• Journal of Intelligence and Information Systems
• /
• v.20 no.2
• /
• pp.93-107
• /
• 2014

In this paper, we report what we have observed with regard to user-perspective issue clustering based on multi-layered two-mode network analysis. This work is significant in the context of data collection by companies about customer needs. Most companies have failed to uncover such needs for products or services properly in terms of demographic data such as age, income levels, and purchase history. Because of excessive reliance on limited internal data, most recommendation systems do not provide decision makers with appropriate business information for current business circumstances. However, part of the problem is the increasing regulation of personal data gathering and privacy. This makes demographic or transaction data collection more difficult, and is a significant hurdle for traditional recommendation approaches because these systems demand a great deal of personal data or transaction logs. Our motivation for presenting this paper to academia is our strong belief, and evidence, that most customers' requirements for products can be effectively and efficiently analyzed from unstructured textual data such as Internet news text. In order to derive users' requirements from textual data obtained online, the proposed approach in this paper attempts to construct double two-mode networks, such as a user-news network and news-issue network, and to integrate these into one quasi-network as the input for issue clustering. One of the contributions of this research is the development of a methodology utilizing enormous amounts of unstructured textual data for user-oriented issue clustering by leveraging existing text mining and social network analysis. In order to build multi-layered two-mode networks of news logs, we need some tools such as text mining and topic analysis. We used not only SAS Enterprise Miner 12.1, which provides a text miner module and cluster module for textual data analysis, but also NetMiner 4 for network visualization and analysis. Our approach for user-perspective issue clustering is composed of six main phases: crawling, topic analysis, access pattern analysis, network merging, network conversion, and clustering. In the first phase, we collect visit logs for news sites by crawler. After gathering unstructured news article data, the topic analysis phase extracts issues from each news article in order to build an article-news network. For simplicity, 100 topics are extracted from 13,652 articles. In the third phase, a user-article network is constructed with access patterns derived from web transaction logs. The double two-mode networks are then merged into a quasi-network of user-issue. Finally, in the user-oriented issue-clustering phase, we classify issues through structural equivalence, and compare these with the clustering results from statistical tools and network analysis. An experiment with a large dataset was performed to build a multi-layer two-mode network. After that, we compared the results of issue clustering from SAS with that of network analysis. The experimental dataset was from a web site ranking site, and the biggest portal site in Korea. The sample dataset contains 150 million transaction logs and 13,652 news articles of 5,000 panels over one year. User-article and article-issue networks are constructed and merged into a user-issue quasi-network using Netminer. Our issue-clustering results applied the Partitioning Around Medoids (PAM) algorithm and Multidimensional Scaling (MDS), and are consistent with the results from SAS clustering. In spite of extensive efforts to provide user information with recommendation systems, most projects are successful only when companies have sufficient data about users and transactions. Our proposed methodology, user-perspective issue clustering, can provide practical support to decision-making in companies because it enhances user-related data from unstructured textual data. To overcome the problem of insufficient data from traditional approaches, our methodology infers customers' real interests by utilizing web transaction logs. In addition, we suggest topic analysis and issue clustering as a practical means of issue identification.

• Journal of Intelligence and Information Systems
• /
• v.20 no.3
• /
• pp.59-76
• /
• 2014

The revolution of smart media such as smart phone, smart TV and tablets has brought easiness for people to get contents and related information anywhere and anytime. The characteristics of the smart media have changed user behavior for watching the contents from passive attitude into active one. Video is a kind of multimedia resources and widely used to provide information effectively. People not only watch video contents, but also search for related information to specific objects appeared in the contents. However, people have to use extra views or devices to find the information because the existing video contents provide no information through the contents. Therefore, the interaction between user and media is becoming a major concern. The demand for direct interaction and instant information is much increasing. Digital media environment is no longer expected to serve as a one-way information service, which requires user to search manually on the internet finding information they need. To solve the current inconvenience, an interactive service is needed to provide the information exchange function between people and video contents, or between people themselves. Recently, many researchers have recognized the importance of the requirements for interactive services, but only few services provide interactive video within restricted functionality. Only cooking domain is chosen for an interactive cooking video query service in this research. Cooking is receiving lots of people attention continuously. By using smart media devices, user can easily watch a cooking video. One-way information nature of cooking video does not allow to interactively getting more information about the certain contents, although due to the characteristics of videos, cooking videos provide various information such as cooking scenes and explanation for each recipe step. Cooking video indeed attracts academic researches to study and solve several problems related to cooking. However, just few studies focused on interactive services in cooking video and they still not sufficient to provide the interaction with users. In this paper, an interactive cooking video query service system with linked data to provide the interaction functionalities to users. A linked recipe schema is used to handle the linked data. The linked data approach is applied to construct queries in systematic manner when user interacts with cooking videos. We add some classes, data properties, and relations to the linked recipe schema because the current version of the schema is not enough to serve user interaction. A web crawler extracts recipe information from allrecipes.com. All extracted recipe information is transformed into ontology instances by using developed instance generator. To provide a query function, hundreds of questions in cooking video web sites such as BBC food, Foodista, Fine cooking are investigated and analyzed. After the analysis of the investigated questions, we summary the questions into four categories by question generalization. For the question generalization, the questions are clustered in eleven questions. The proposed system provides an environment associating UI (User Interface) and UX (User Experience) that allow user to watch cooking videos while obtaining the necessary additional information using extra information layer. User can use the proposed interactive cooking video system at both PC and mobile environments because responsive web design is applied for the proposed system. In addition, the proposed system enables the interaction between user and video in various smart media devices by employing linked data to provide information matching with the current context. Two methods are used to evaluate the proposed system. First, through a questionnaire-based method, computer system usability is measured by comparing the proposed system with the existing web site. Second, the answer accuracy for user interaction is measured to inspect to-be-offered information. The experimental results show that the proposed system receives a favorable evaluation and provides accurate answers for user interaction.

• Korean journal of applied entomology
• /
• v.60 no.4
• /
• pp.403-415
• /
• 2021

Seasonal development of Parthenolecanium corni was observed from 4 Jun. 2019 (nymph) - 25 Jun. 2020 (1^st Gen. nymph) in blueberry shrub in Sacheon city, Gyeongsangnam-do. To investigate their development, more than 5 twigs sprouted in 2018 were taken from the farm nearly weekly basis. The development of each scale were examined under the stereomicroscope and chemical control was conducted in the blueberry shrub with available three insecticides. The results on the development period and Centigrade Degree-Days accumulation (DDC) obtained are as follows: egg-laying period (peak): 12 -26 May 2020 (24 May)(DDC, 110.0-188.5 (173.6)); egg-hatching period (peak): 9 - 23 June 2020 (19 June)(DDC, 325.2-480.8(435.6); egg period: 26 days; nymph movement from overwintered adult to new leaves 16-25 June 2020 (DDC, 410,5-500.4); nymph movement from leaf to twig (peak) to become adult: 4-18 Feb. 2020 (8 Feb.). Eggs no. /adult (range): 956.8 ± 73.4 (13 - 3497). Size (mm) of egg, 0.29 ± 0.020(L), 0.15 ± 0.013(W); of egg-hatched nymph, 0.35 ± 0.018(L), 0.18 ± 0.007(W), 0.09 ± 0.007(eye distance); and of adult, 4.30 ± 0.893(L), 2.64 ± 0.520(W). The egg-hatched nymphs from the overwintered adult moved to the backside leaf of new shoot in which they found about 95% until leaf is falling by early February in next year. They overwintered as 2^nd instar and occurred univoltine. For the control of the 1^st instar crawler, three insecticides treated on 16 and 30 July at the registered dose for Ceroplastes japonicus. Acetamiprid 8WP showed 96.9% mortality at 21 days after 1^st treatment.

• Explosives and Blasting
• /
• v.9 no.1
• /
• pp.3-13
• /
• 1991

The cautious blasting works had been used with emulsion explosion electric M/S delay caps. Drill depth was from 3m to 6m with Crawler Drill ${\phi}70mm$ on the calcalious sand stone (soft -modelate -semi hard Rock). The total numbers of test blast were 88. Scale distance were induced 15.52-60.32. It was applied to propagation Law in blasting vibration as follows. Propagtion Law in Blasting Vibration $V=K(\frac{D}{W^b})^n$ were V : Peak partical velocity(cm/sec) D : Distance between explosion and recording sites(m) W : Maximum charge per delay-period of eight milliseconds or more (kg) K : Ground transmission constant, empirically determind on the Rocks, Explosive and drilling pattern ets. b : Charge exponents n : Reduced exponents where the quantity $\frac{D}{W^b}$ is known as the scale distance. Above equation is worked by the U.S Bureau of Mines to determine peak particle velocity. The propagation Law can be catagorized in three groups. Cubic root Scaling charge per delay Square root Scaling of charge per delay Site-specific Scaling of charge Per delay Plots of peak particle velocity versus distoance were made on log-log coordinates. The data are grouped by test and P.P.V. The linear grouping of the data permits their representation by an equation of the form ; $V=K(\frac{D}{W^{\frac{1}{3}})^{-n}$ The value of K(41 or 124) and n(1.41 or 1.66) were determined for each set of data by the method of least squores. Statistical tests showed that a common slope, n, could be used for all data of a given components. Charge and reduction exponents carried out by multiple regressional analysis. It's divided into under loom over loom distance because the frequency is verified by the distance from blast site. Empirical equation of cautious blasting vibration is as follows. Over 30m ------- under l00m ${\cdots\cdots\cdots}{\;}41(D/sqrt[2]{W})^{-1.41}{\;}{\cdots\cdots\cdots\cdots\cdots}{\;}A$ Over 100m ${\cdots\cdots\cdots\cdots\cdots}{\;}121(D/sqrt[3]{W})^{-1.66}{\;}{\cdots\cdots\cdots\cdots\cdots}{\;}B$ where ; V is peak particle velocity In cm / sec D is distance in m and W, maximLlm charge weight per day in kg K value on the above equation has to be more specified for further understaring about the effect of explosives, Rock strength. And Drilling pattern on the vibration levels, it is necessary to carry out more tests.

• Journal of Intelligence and Information Systems
• /
• v.19 no.3
• /
• pp.57-71
• /
• 2013

Over a billion people in the world generate new news minute by minute. People forecasts some news but most news are from unexpected events such as natural disasters, accidents, crimes. People spend much time to watch a huge amount of news delivered from many media because they want to understand what is happening now, to predict what might happen in the near future, and to share and discuss on the news. People make better daily decisions through watching and obtaining useful information from news they saw. However, it is difficult that people choose news suitable to them and obtain useful information from the news because there are so many news media such as portal sites, broadcasters, and most news articles consist of gossipy news and breaking news. User interest changes over time and many people have no interest in outdated news. From this fact, applying users' recent interest to personalized news service is also required in news service. It means that personalized news service should dynamically manage user profiles. In this paper, a content-based news recommendation system is proposed to provide the personalized news service. For a personalized service, user's personal information is requisitely required. Social network service is used to extract user information for personalization service. The proposed system constructs dynamic user profile based on recent user information of Facebook, which is one of social network services. User information contains personal information, recent articles, and Facebook Page information. Facebook Pages are used for businesses, organizations and brands to share their contents and connect with people. Facebook users can add Facebook Page to specify their interest in the Page. The proposed system uses this Page information to create user profile, and to match user preferences to news topics. However, some Pages are not directly matched to news topic because Page deals with individual objects and do not provide topic information suitable to news. Freebase, which is a large collaborative database of well-known people, places, things, is used to match Page to news topic by using hierarchy information of its objects. By using recent Page information and articles of Facebook users, the proposed systems can own dynamic user profile. The generated user profile is used to measure user preferences on news. To generate news profile, news category predefined by news media is used and keywords of news articles are extracted after analysis of news contents including title, category, and scripts. TF-IDF technique, which reflects how important a word is to a document in a corpus, is used to identify keywords of each news article. For user profile and news profile, same format is used to efficiently measure similarity between user preferences and news. The proposed system calculates all similarity values between user profiles and news profiles. Existing methods of similarity calculation in vector space model do not cover synonym, hypernym and hyponym because they only handle given words in vector space model. The proposed system applies WordNet to similarity calculation to overcome the limitation. Top-N news articles, which have high similarity value for a target user, are recommended to the user. To evaluate the proposed news recommendation system, user profiles are generated using Facebook account with participants consent, and we implement a Web crawler to extract news information from PBS, which is non-profit public broadcasting television network in the United States, and construct news profiles. We compare the performance of the proposed method with that of benchmark algorithms. One is a traditional method based on TF-IDF. Another is 6Sub-Vectors method that divides the points to get keywords into six parts. Experimental results demonstrate that the proposed system provide useful news to users by applying user's social network information and WordNet functions, in terms of prediction error of recommended news.

• Journal of Intelligence and Information Systems
• /
• v.23 no.3
• /
• pp.155-175
• /
• 2017

As social data become into the spotlight, mainstream web search engines provide data indicate how many people searched specific keyword: Web Search Traffic data. Web search traffic information is collection of each crowd that search for specific keyword. In a various area, web search traffic can be used as one of useful variables that represent the attention of common users on specific interests. A lot of studies uses web search traffic data to nowcast or forecast social phenomenon such as epidemic prediction, consumer pattern analysis, product life cycle, financial invest modeling and so on. Also web search traffic data have begun to be applied to predict tourist inbound. Proper demand prediction is needed because tourism is high value-added industry as increasing employment and foreign exchange. Among those tourists, especially Chinese tourists: Youke is continuously growing nowadays, Youke has been largest tourist inbound of Korea tourism for many years and tourism profits per one Youke as well. It is important that research into proper demand prediction approaches of Youke in both public and private sector. Accurate tourism demands prediction is important to efficient decision making in a limited resource. This study suggests improved model that reflects latest issue of society by presented the attention from group of individual. Trip abroad is generally high-involvement activity so that potential tourists likely deep into searching for information about their own trip. Web search traffic data presents tourists' attention in the process of preparation their journey instantaneous and dynamic way. So that this study attempted select key words that potential Chinese tourists likely searched out internet. Baidu-Chinese biggest web search engine that share over 80%- provides users with accessing to web search traffic data. Qualitative interview with potential tourists helps us to understand the information search behavior before a trip and identify the keywords for this study. Selected key words of web search traffic are categorized by how much directly related to "Korean Tourism" in a three levels. Classifying categories helps to find out which keyword can explain Youke inbound demands from close one to far one as distance of category. Web search traffic data of each key words gathered by web crawler developed to crawling web search data onto Baidu Index. Using automatically gathered variable data, linear model is designed by multiple regression analysis for suitable for operational application of decision and policy making because of easiness to explanation about variables' effective relationship. After regression linear models have composed, comparing with model composed traditional variables and model additional input web search traffic data variables to traditional model has conducted by significance and R squared. after comparing performance of models, final model is composed. Final regression model has improved explanation and advantage of real-time immediacy and convenience than traditional model. Furthermore, this study demonstrates system intuitively visualized to general use -Youke Mining solution has several functions of tourist decision making including embed final regression model. Youke Mining solution has algorithm based on data science and well-designed simple interface. In the end this research suggests three significant meanings on theoretical, practical and political aspects. Theoretically, Youke Mining system and the model in this research are the first step on the Youke inbound prediction using interactive and instant variable: web search traffic information represents tourists' attention while prepare their trip. Baidu web search traffic data has more than 80% of web search engine market. Practically, Baidu data could represent attention of the potential tourists who prepare their own tour as real-time. Finally, in political way, designed Chinese tourist demands prediction model based on web search traffic can be used to tourism decision making for efficient managing of resource and optimizing opportunity for successful policy.