• Title/Summary/Keyword: Crawler

Search Result 202, Processing Time 0.024 seconds

A Study on Space Consumption Behavior of Contemporary Consumers -Focusing on Analysis of Social Media Big Data- (현대 소비자의 공간소비행동에 관한 연구 -소셜미디어 데이터 분석을 중심으로-)

  • Ahn, Suh Young;Koh, Ae-Ran
    • Journal of the Korean Society of Clothing and Textiles
    • /
    • v.44 no.5
    • /
    • pp.1019-1035
    • /
    • 2020
  • This study examines the millennial generation, who express themselves and share information on social media after experiencing constantly changing 'hot places' (places of interest) in contemporary cities, with the goal of analyzing space consumption behaviors. Data were collected via an Instagram crawler application developed with Python 3.4 administered to 19,262 posts using the term 'hot places' from November 1 and December 15, 2019. Issues were derived from a text mining technique using Textom 2.0; in addition, semantic network analysis using Ucinet6 and the NetDraw program were also conducted. The results are as follows. First, a frequency analysis of keywords for hot places indicated words frequently found in nouns were related to food, local names, SNS and timing. Words related to positive emotions felt in experience, and words related to behavior in hot places appeared in predicate. Based on importance, communication is the most important keyword and influenced all issues. Second, the results of visualization of semantic network analysis revealed four categories in the scope of the definition of "hot place": (1) culinary exploration, (2) atmosphere of cafés, (3) happy daily life of 'me' expressed in images, (4) emotional photos.

The Study for Type of Mask Wearing Dataset for Deep learning and Detection Model (딥러닝을 위한 마스크 착용 유형별 데이터셋 구축 및 검출 모델에 관한 연구)

  • Hwang, Ho Seong;Kim, Dong heon;Kim, Ho Chul
    • Journal of Biomedical Engineering Research
    • /
    • v.43 no.3
    • /
    • pp.131-135
    • /
    • 2022
  • Due to COVID-19, Correct method of wearing mask is important to prevent COVID-19 and the other respiratory tract infections. And the deep learning technology in the image processing has been developed. The purpose of this study is to create the type of mask wearing dataset for deep learning models and select the deep learning model to detect the wearing mask correctly. The Image dataset is the 2,296 images acquired using a web crawler. Deep learning classification models provided by tensorflow are used to validate the dataset. And Object detection deep learning model YOLOs are used to select the detection deep learning model to detect the wearing mask correctly. In this process, this paper proposes to validate the type of mask wearing datasets and YOLOv5 is the effective model to detect the type of mask wearing. The experimental results show that reliable dataset is acquired and the YOLOv5 model effectively recognize type of mask wearing.

Sentiment Analysis on Global Events under Pandemic of COVID-19

  • Junjun, Zhang;Noh, Giseop
    • International Journal of Advanced Culture Technology
    • /
    • v.10 no.3
    • /
    • pp.272-280
    • /
    • 2022
  • During last few years, pandemic of COVID-19 has been a global issue. Under the COVID-19, global events have been restricted or canceled to secure public hygiene and safety. Since one of the largest global events is Olympic Games, we selected recent Olympic Games as our case of analysis. Tokyo Olympic Games (TOG) was held in 2021, but it encountered a millennium disaster, the pandemic of COVID-19. In such a special period, it is of great significance to explore the emotional tendency of global views before and TOG via artificial intelligence. This paper vastly collects the TOG comment data of mainstream websites in South Korea, China, and the United States by implementing crawler program for sentiment analysis (SA). And we use a variety of sentiment analysis models to compare the accuracy of the experimental results, to obtain more reliable SA results. In addition, in the prediction results, to reduce the distortion of opinion by a minority, we introduce an algorithm called "Removing Biased Minority Opinions (RBMO)" and provide how to apply this method to the interpretation domain. Through our method, more authoritative SA results were obtained, which in turn provided a basis for predicting the sentiment tendency of countries around the world in TOG during the COVID-19 epidemic.

Effect of Trust in Creators on Class Preference in Knowledge Marketplaces (지식 마켓플레이스에서 크리에이터에 대한 신뢰가 강의 선호도에 미치는 영향)

  • Kang, Young Ju;Kim, Jin Myeong;Lee, Ui Jun;Oh, Se Hwan
    • The Journal of Information Systems
    • /
    • v.31 no.3
    • /
    • pp.19-45
    • /
    • 2022
  • Purpose Since COVID-19, the demand for online class platforms has increased. However, those platforms have not been clearly defined, and related research is also limited. In the context of the knowledge marketplace (KMs), this study examined the effects of class information and trust in creators on class preferences from the perspective of consumption value theory. Design/methodology/approach By establishing a web crawler through Python, this study collected 1,174 class data in Korea's leading knowledge marketplace, Class 101, focusing on diverse class-related information and the number of Instagram followers for individual class creators. Based on class information, this research analyzed the effects of consumers' utilitarian value, social value, and hedonic value on class preference. In addition, this study examined whether consumers' trust in creators moderates the relationship between class information and class preference. Findings According to analysis results, it was found that the higher the consumers' consumption value for each class on KMs, the more positive their preference for the class. Also, it was confirmed that consumers' trust in creators moderates the relationship between class information and class preference.

Analysis of Social Media Utilization based on Big Data-Focusing on the Chinese Government Weibo

  • Li, Xiang;Guo, Xiaoqin;Kim, Soo Kyun;Lee, Hyukku
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.8
    • /
    • pp.2571-2586
    • /
    • 2022
  • The rapid popularity of government social media has generated huge amounts of text data, and the analysis of these data has gradually become the focus of digital government research. This study uses Python language to analyze the big data of the Chinese provincial government Weibo. First, this study uses a web crawler approach to collect and statistically describe over 360,000 data from 31 provincial government microblogs in China, covering the period from January 2018 to April 2022. Second, a word separation engine is constructed and these text data are analyzed using word cloud word frequencies as well as semantic relationships. Finally, the text data were analyzed for sentiment using natural language processing methods, and the text topics were studied using LDA algorithm. The results of this study show that, first, the number and scale of posts on the Chinese government Weibo have grown rapidly. Second, government Weibo has certain social attributes, and the epidemics, people's livelihood, and services have become the focus of government Weibo. Third, the contents of government Weibo account for more than 30% of negative sentiments. The classified topics show that the epidemics and epidemic prevention and control overshadowed the other topics, which inhibits the diversification of government Weibo.

P-TAF: A Big Data-based Platform for Total Air Traffic Forecast (빅데이터 기반 항공 수요예측 통합 플랫폼 설계 및 실증)

  • Jung, Jooik;Son, Seokhyun;Cha, Hee-June
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2021.01a
    • /
    • pp.281-282
    • /
    • 2021
  • 본 논문에서는 항공 수요예측을 위한 빅데이터 기반 플랫폼의 설계 및 실증 결과를 제시한다. 항공 수요예측 통합 플랫폼은 항공산업 관련 데이터를 Open API, RSS Feed, 웹크롤러(Web Crawler) 등을 이용하여 수집 및 분석하여 자체 개발한 항공 수요예측 알고리즘을 기반으로 결과를 시각화하여 보여주도록 구현되어 있다. 또한, 제안하는 플랫폼의 사용자 인터페이스를 통해 변수 설정을 하여 단위별(Global, National 등), 기간별(단기, 중장기 등), 유형별(여객, 화물 등) 예측 통계 자료를 도출할 수 있다. 플랫폼의 성능 검증을 위해 정형화된 데이터를 비롯하여 소셜네트워크서비스(SNS), 검색엔진 등에서 수집한 비정형 데이터까지 활용하여 특정 키워드의 빈도와 특정 노선에 대한 항공 수요간 상관관계를 분석하였다. 개발한 통합 플랫폼의 지능형 항공 수요예측 알고리즘을 통해 전반적인 공항 운영 및 공항 운영 정책 수립에 기여할 것으로 예상한다.

  • PDF

Load Prediction using Finite Element Analysis and Recurrent Neural Network (유한요소해석과 순환신경망을 활용한 하중 예측)

  • Jung-Ho Kang
    • Journal of the Korean Society of Industry Convergence
    • /
    • v.27 no.1
    • /
    • pp.151-160
    • /
    • 2024
  • Artificial Neural Networks that enabled Artificial Intelligence are being used in many fields. However, the application to mechanical structures has several problems and research is incomplete. One of the problems is that it is difficult to secure a large amount of data necessary for learning Artificial Neural Networks. In particular, it is important to detect and recognize external forces and forces for safety working and accident prevention of mechanical structures. This study examined the possibility by applying the Current Neural Network of Artificial Neural Networks to detect and recognize the load on the machine. Tens of thousands of data are required for general learning of Recurrent Neural Networks, and to secure large amounts of data, this paper derives load data from ANSYS structural analysis results and applies a stacked auto-encoder technique to secure the amount of data that can be learned. The usefulness of Stacked Auto-Encoder data was examined by comparing Stacked Auto-Encoder data and ANSYS data. In addition, in order to improve the accuracy of detection and recognition of load data with a Recurrent Neural Network, the optimal conditions are proposed by investigating the effects of related functions.

Crowd Psychological and Emotional Computing Based on PSMU Algorithm

  • Bei He
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.8
    • /
    • pp.2119-2136
    • /
    • 2024
  • The rapid progress of social media allows more people to express their feelings and opinions online. Many data on social media contains people's emotional information, which can be used for people's psychological analysis and emotional calculation. This research is based on the simplified psychological scale algorithm of multi-theory integration. It aims to accurately analyze people's psychological emotion. According to the comparative analysis of algorithm performance, the results show that the highest recall rate of the algorithm in this study is 95%, while the highest recall rate of the item response theory algorithm and the social network analysis algorithm is 68% and 87%. The acceleration ratio and data volume of the research algorithm are analyzed. The results show that when 400,000 data are calculated in the Hadoop cluster and there are 8 nodes, the maximum acceleration ratio is 40%. When the data volume is 8GB, the maximum scale ratio of 8 nodes is 43%. Finally, we carried out an empirical analysis on the model that compute the population's psychological and emotional conditions. During the analysis, the psychological simplification scale algorithm was adopted and multiple theories were taken into account. Then, we collected negative comments and expressions about Japan's discharge of radioactive water in microblog and compared them with the trend derived by the model. The results were consistent. Therefore, this research model has achieved good results in the emotion classification of microblog comments.

A Study on IoT cloud type middleware platform structure for medical light module control (의료용 Light 모듈 제어를 위한 IoT 클라우드형 미들웨어 플랫폼 구조 연구)

  • Lee, Kack-Hee;Lee, Min-Woo;Cha, Jae-Sang
    • The Journal of the Convergence on Culture Technology
    • /
    • v.3 no.4
    • /
    • pp.177-180
    • /
    • 2017
  • Recently, as aging progresses rapidly around the world, studies using light (UV-LED, etc.) as a method which is harmless to the human body and less burden on the treatment are continuously being carried out. Especially, in some fields, it is not used in a limited way but it is advantageous to be used in various medical fields such as dermatology, dentistry, otorhinolaryngology, etc. In many advanced countries, various studies using it have been carried out. However, Are far behind in developed countries. Therefore, in this paper, we propose a technology that can provide good quality services to human patients by effectively controlling light module that can be used for medical use by using IoT crawler-based middleware platform.

Lumped Track Modeling for Estimating Traction Force of Vecna BEAR Type Robot (Vecna BEAR 형 로봇의 견인력 추정을 위한 Lumped 궤도 모델링)

  • Kim, Tae Yun;Jung, Samuel;Yoo, Wan Suk
    • Transactions of the Korean Society of Mechanical Engineers A
    • /
    • v.39 no.3
    • /
    • pp.275-282
    • /
    • 2015
  • Recently, Vecna BEAR type robot to save injured individuals from inaccessible areas has been developed to minimize the loss of life. Because this robot is driven on rough terrain, there is a risk of rollover and vibration, which could impact the injured. In order to guarantee its stability, an algorithm is required that can estimate the speed limits for various environments in real time. Therefore, a dynamic model for real-time analysis is needed for this algorithm. Because the tracks used as the driving component of Vecna BEAR type robot consist of many parts, it is impossible to analyze the multibody tracks in real time. Thus, a lumped track model that satisfies the requirements of a short computation time and adequate accuracy is required. This study performed lumped track modeling, and the traction force was verified using RecurDyn, which is a dynamic commercial program.