• Title/Summary/Keyword: tweet crawling

Search Result 4, Processing Time 0.02 seconds

Dynamic Seed Selection for Twitter Data Collection (트위터 데이터 수집을 위한 동적 시드 선택)

  • Lee, Hyoenchoel;Byun, Changhyun;Kim, Yanggon;Lee, Sang Ho
    • Journal of KIISE:Databases
    • /
    • v.41 no.4
    • /
    • pp.217-225
    • /
    • 2014
  • Analysis of social media such as Twitter can yield interesting perspectives to understanding human behavior, detecting hot issues, identifying influential people, or discovering a group and community. However, it is difficult to gather the data relevant to specific topics due to the main characteristics of social media data; data is large, noisy, and dynamic. This paper proposes a new algorithm that dynamically selects the seed nodes to efficiently collect tweets relevant to topics. The algorithm utilizes attributes of users to evaluate the user influence, and dynamically selects the seed nodes during the collection process. We evaluate the proposed algorithm with real tweet data, and get satisfactory performance results.

A Study on the Vitalization Strategy Based on Current Status Analysis of National Archives (국내외 국립기록관의 트위터 운용 현황 분석 및 활성화 방안)

  • Gang, JuYeon;Kim, TaeYoung;Choi, JungWon;Oh, Hyo-Jung
    • Journal of the Korean Society for information Management
    • /
    • v.33 no.3
    • /
    • pp.263-285
    • /
    • 2016
  • Nowadays, Social Network Service (SNS), which has been in the spotlight as a way of communication, has become a most effective tool to improve easy of information use and accessibility for users. In this paper, we chose Twitter as the most representative SNS services because of automatic crawling and investigated tweet data gathered from domestic and foreign National Archives - NARA of U.S.A., TNA of U.K.. NAA of Australia, and National Archives of Korea. We also conducted information genres analysis and trend analysis by timeline. Information genres analysis shows how archives satisfied users' information needs as well as trends analysis of tweets helps to understand how users' interestedness was changed. Based on comparison results, we distilled four characteristics of National Archives and suggested vitalization ways for National Archives of Korea.

An Analysis of IT Trends Using Tweet Data (트윗 데이터를 활용한 IT 트렌드 분석)

  • Yi, Jin Baek;Lee, Choong Kwon;Cha, Kyung Jin
    • Journal of Intelligence and Information Systems
    • /
    • v.21 no.1
    • /
    • pp.143-159
    • /
    • 2015
  • Predicting IT trends has been a long and important subject for information systems research. IT trend prediction makes it possible to acknowledge emerging eras of innovation and allocate budgets to prepare against rapidly changing technological trends. Towards the end of each year, various domestic and global organizations predict and announce IT trends for the following year. For example, Gartner Predicts 10 top IT trend during the next year, and these predictions affect IT and industry leaders and organization's basic assumptions about technology and the future of IT, but the accuracy of these reports are difficult to verify. Social media data can be useful tool to verify the accuracy. As social media services have gained in popularity, it is used in a variety of ways, from posting about personal daily life to keeping up to date with news and trends. In the recent years, rates of social media activity in Korea have reached unprecedented levels. Hundreds of millions of users now participate in online social networks and communicate with colleague and friends their opinions and thoughts. In particular, Twitter is currently the major micro blog service, it has an important function named 'tweets' which is to report their current thoughts and actions, comments on news and engage in discussions. For an analysis on IT trends, we chose Tweet data because not only it produces massive unstructured textual data in real time but also it serves as an influential channel for opinion leading on technology. Previous studies found that the tweet data provides useful information and detects the trend of society effectively, these studies also identifies that Twitter can track the issue faster than the other media, newspapers. Therefore, this study investigates how frequently the predicted IT trends for the following year announced by public organizations are mentioned on social network services like Twitter. IT trend predictions for 2013, announced near the end of 2012 from two domestic organizations, the National IT Industry Promotion Agency (NIPA) and the National Information Society Agency (NIA), were used as a basis for this research. The present study analyzes the Twitter data generated from Seoul (Korea) compared with the predictions of the two organizations to analyze the differences. Thus, Twitter data analysis requires various natural language processing techniques, including the removal of stop words, and noun extraction for processing various unrefined forms of unstructured data. To overcome these challenges, we used SAS IRS (Information Retrieval Studio) developed by SAS to capture the trend in real-time processing big stream datasets of Twitter. The system offers a framework for crawling, normalizing, analyzing, indexing and searching tweet data. As a result, we have crawled the entire Twitter sphere in Seoul area and obtained 21,589 tweets in 2013 to review how frequently the IT trend topics announced by the two organizations were mentioned by the people in Seoul. The results shows that most IT trend predicted by NIPA and NIA were all frequently mentioned in Twitter except some topics such as 'new types of security threat', 'green IT', 'next generation semiconductor' since these topics non generalized compound words so they can be mentioned in Twitter with other words. To answer whether the IT trend tweets from Korea is related to the following year's IT trends in real world, we compared Twitter's trending topics with those in Nara Market, Korea's online e-Procurement system which is a nationwide web-based procurement system, dealing with whole procurement process of all public organizations in Korea. The correlation analysis show that Tweet frequencies on IT trending topics predicted by NIPA and NIA are significantly correlated with frequencies on IT topics mentioned in project announcements by Nara market in 2012 and 2013. The main contribution of our research can be found in the following aspects: i) the IT topic predictions announced by NIPA and NIA can provide an effective guideline to IT professionals and researchers in Korea who are looking for verified IT topic trends in the following topic, ii) researchers can use Twitter to get some useful ideas to detect and predict dynamic trends of technological and social issues.

Design of Twitter data collection system for regional sentiment analysis (지역별 감성 분석을 위한 트위터 데이터 수집 시스템 설계)

  • Choi, Kiwon;Kim, Hee-Cheol
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2017.10a
    • /
    • pp.506-509
    • /
    • 2017
  • Opinion mining is a way to analyze the emotions in the text and is used to identify the emotional state of the author and to find out the opinions of the public. As you can analyze individual emotions through opinion mining, if you analyze the text by region, you can find out the emotional state you have in each region. The regional sentiment analysis can obtain information that could not be obtained from personal sentiment analysis, and if a certain area has emotions, it can understand the cause. For regional sentiment analysis, we need text data created by region, so we need to collect data through Twitter crawling. Therefore, this paper designs a Twitter data collection system for regional sentiment analysis. The client requests the tweet data of the specific region and time, and the server collects and transmits the requested tweet data from the client. Through the latitude and longitude values of the region, it collects the tweet data of the area, and it can manage the text by region and time through collected data. We expect efficient data collection and management for emotional analysis through the design of this system.

  • PDF