• Title/Summary/Keyword: Beautiful soup

Search Result 10, Processing Time 0.021 seconds

A Scraping Method of In-Frame Web Sources Using Python (파이썬을 이용한 프레임내 웹 페이지 스크래핑 기법)

  • Yun, Sujin;Seung, Li;Woo, Young Woon
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2019.05a
    • /
    • pp.271-274
    • /
    • 2019
  • In this paper, we proposed a detailed address acquisition scheme for automatically collecting data of a web page in a frame that is difficult to access by a general web access method. Using the Python language and the Beautiful Soup library, which can utilize the proposed address resolution technique and the HTML selector, we were able to automatically collect all the bulletin board text data written in several pages. By using the proposed method, we can collect large amount of data automatically by Python web scraping program for web pages of any form of address, and we expect that it can be used for big data analysis.

  • PDF

Django based ChatBot System Using KakaoTalk API (카카오톡 API를 이용한 Django 기반 챗봇 시스템)

  • Ko, Heungchan;Kim, Minsu;Lee, Solbi;Lee, Hyung-Woo
    • Journal of Internet of Things and Convergence
    • /
    • v.4 no.1
    • /
    • pp.31-36
    • /
    • 2018
  • In this paper, we developed a chatbot system using the Django framework using the KakaoTalk API so that college students can easily search for important information in their university. Unlike existing chatbot systems that provide only specific information, the chatbot developed in this research automatically provides search results for various types of user queries such as weather, YouTube, Naver real-time ranking search and language translation as well as important information within their own university. We developed a module using Apache, Python and Django in AWS Ubuntu server and developed a chatbot system that automatically responds to user queries by communicating with KakaoTalk server using KakaoTalk API and BeautifulSoup. The system developed in this study is expected to be applicable to the future university entrance information promotion and election promotion system.

A Study on Analysis of the Trend of Blockchain by Key Words Network Analysis (키워드 네트워크 분석 방법을 활용한 블록체인 트렌드 분석에 관한 연구)

  • Cho, Seong-Hwan
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.11 no.5
    • /
    • pp.550-555
    • /
    • 2018
  • This study aims to identify and compare contents and keywords used in articles related to blockchain applications to various industries. The text mining and Semantic Network Analysis, as methods of keyword network analysis, were used to analyze articles including terms of 'finance' 'energy' and 'logistics', which media and government frequently mentioned as areas that can apply blockchain technologies. For this study, data were collected from 43,093 articles from January, 2017 through July, 2018. Data crawling was carried out by using Python BeautifulSoup and data cleaning was performed in order to eliminate mutual redundancies of the three terms. After that, text mining and semantic network analysis were performed using Textom and UCInet for network analysis between keywords. The results showed that all the three terms were similar in terms of 'technology', but there were differences in the contents of 'government policy' or 'industry' issues. In addition, there were differences in frequencies and centralities of these terms.

Design and Implementation of Typing Practice Application for Learning Using Web Contents (웹 콘텐츠를 활용한 학습용 타자 연습 어플리케이션의 설계와 구현)

  • Kim, Chaewon;Hwang, Soyoung
    • Journal of Korea Multimedia Society
    • /
    • v.24 no.12
    • /
    • pp.1663-1672
    • /
    • 2021
  • There are various typing practice applications. In addition, research cases on learning applications that support typing practice have been reported. These services are usually provided in a way that utilizes their own built-in text. Learners collect various contents through web services and use them a lot for learning. Therefore, this paper proposes a learning application to increase the learning effect by collecting vast amounts of web content and applying it to typing practice. The proposed application is implemented using Tkinter, a GUI module of Python. BeautifulSoup module of Python is used to extract information from the web. In order to process the extracted data, the NLTK module, which is an English data preprocessor, and the KoNLPy module, which is a Korean language processing module, are used. The operation of the proposed function is verified in the implementation and experimental results.

Analysis of YouTube's role as a new platform between media and consumers

  • Hur, Tai-Sung;Im, Jung-ju;Song, Da-hye
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.2
    • /
    • pp.53-60
    • /
    • 2022
  • YouTube realistically shows fake news and biased content based on facts that have not been verified due to low entry barriers and ambiguity in video regulation standards. Therefore, this study aims to analyze the influence of the media and YouTube on individual behavior and their relationship. Data from YouTube and Twitter are randomly imported with selenium, beautiful soup, and Twitter APIs to classify the 31 most frequently mentioned keywords. Based on 31 keywords classified, data were collected from YouTube, Twitter, and Naver News, and positive, negative, and neutral emotions were classified and quantified with NLTK's Natural Language Toolkit (NLTK) Vader model and used as analysis data. As a result of analyzing the correlation of data, it was confirmed that the higher the negative value of news, the more positive content on YouTube, and the positive index of YouTube content is proportional to the positive and negative values on Twitter. As a result of this study, YouTube is not consistent with the emotion index shown in the news due to its secondary processing and affected characteristics. In other words, processed YouTube content intuitively affects Twitter's positive and negative figures, which are channels of communication. The results of this study analyzed that YouTube plays a role in assisting individual discrimination in the current situation where accurate judgment of information has become difficult due to the emergence of yellow media that stimulates people's interests and instincts.

Convergence of Korean Traditional Dance and K-Pop Dance : An Analysis of Comments on 2018 MMA BTS 'IDOL' Videos on YouTube (한국 전통춤과 K-pop 댄스의 융합 : 2018 MMA 방탄소년단 'IDOL' 유튜브 댓글 분석)

  • Yoo, Ji-Young;Kim, Mi-Kyung
    • Journal of Korea Entertainment Industry Association
    • /
    • v.13 no.8
    • /
    • pp.189-198
    • /
    • 2019
  • This study aims to make meaning of the reactions of the Korean people through the text mining of comments on videos of the December 2018 MMA performance of intro on YouTube. For this, comments on 15 YouTube videos were collected over the past 10 months. With the collected data, a total of 5,135 comments were analyzed through crawling using the Python and BeautifulSoup programs, data was refined over a total of 3 sessions, and a final total of 5,080 comments were used as analysis material. A mining technique was used for data analysis and the process of refinement, analysis, and visualization was achieved using the Textom program. Research results showed that keyword analysis showed the keywords of 'performance', 'Korea', 'video', 'top', 'cool', 'dance', 'idol', 'legend', 'love', and 'gratitude' in that order and keywords such as 'patriotism' and 'Olympics' also appeared frequently. N-gram analysis showed that comments with contexts such as 'a top performance that will remain a legend among Korean idol performances', and 'an idol performance that displayed the traditional culture of Korea' were in higher ranks. Based on such keyword analysis results, topic modeling was applied and 5 top keywords were extracted from a total of 5 topics. Analysis results of topic contents and distribution showed that topics in the comments of this performance's videos largely consisted of the 3 reactions of 'high praise regarding the stage performance', 'affection towards the fusion and artistic sublimation of Korean traditional dance', and 'gratitude towards the uploading of cool dance videos'

Sodium Reduction Practice and Importance-Performance Analysis of Sodium Reduction Methods in School Foodservice in Daegu (대구지역 학교급식 나트륨 저감화 실태 및 나트륨 저감화 방법에 대한 중요도-수행도 분석)

  • Kim, Su-Hyeon;Shin, Eun-Kyung;Lee, Yeon-Kyung
    • Korean Journal of Community Nutrition
    • /
    • v.25 no.5
    • /
    • pp.386-395
    • /
    • 2020
  • Objectives: The aim of this study was to investigate sodium reduction practices in school foodservice in Daegu. Methods: The survey included 199 nutrition teachers and dietitians working at elementary, middle and high schools in Daegu. The survey topics included the following: the frequency of salinity measurement, workers in charge of the measurement, average salinity of the soup and stew served, frequency and difficulties of offering low-sodium meals, Importance-Performance Analysis (IPA) of sodium reduction methods in school foodservice and the need for political support in encouraging sodium reduction. Results: The mean salinity of the soup and stew was higher in high school foodservice than in elementary and middle school foodservice. Middle and high schools have difficulties in offering low-sodium meals due to concerns of decreasing satisfaction for the meals. The results of the IPA of programs to reduce sodium in school meals showed that most of the items in the cooking and serving stages were in the 2nd quadrant (Keep up the good work), and all purchasing and menu planning stages occupied the 3rd quadrant (Low priority). To reduce sodium in school meals, government support is required in developing low-sodium recipes for school foodservice, encouraging education on sodium reduction for school foodservice officials and developing low-sodium food for institutional foodservice. Conclusions: To encourage sodium reduction in school meals, the priority is to make low-sodium recipes available. Also, it is necessary to develop a program that calculates the sodium content in menus and processed foods through National Education Information System and to establish standards for sodium levels in school foodservice.

Analysis of Shipping and Logistics News Articles using Topic Modeling (토픽모델링을 활용한 해운물류 뉴스 분석)

  • Hee-Young Yoon;Il-Youp Kwak
    • Korea Trade Review
    • /
    • v.46 no.4
    • /
    • pp.61-76
    • /
    • 2021
  • This study focuses on three logistics-related news (Logistics Newspaper, Korea Shipping Gadget, and Korea Shipping Newspaper) in order to present changes in logistics issues, centering on Corona 19, which has recently had the greatest impact in the world. For data collection, two-year news articles in 2019 and 2020 (title, article, content, date, article classification, article URL) were collected through web crawling (using Python's BeautifulSoup, requests module) on the homepages of three representative logistics-related media companies. As for the data analysis methods, fundamental statistical analysis, Latent Dirichlet Allocation (LDA) for topic modeling, and Scattertext were performed. The analysis results were as follows. First, among the three news media related to logistics, the Korea Shipping Newspaper was carrying out the most active media activities. Second, through topic modeling with LDA, eight logistics-related topics were identified, and keywords and significant issues of each topic were presented. Third, the keywords were visually expressed through Scattertext. This is the first study to present changes in the logistics field, focusing on articles from representative logistics-related media in 2019 and 2020. In particular, 2019 and 2020 can be divided into before and after the outbreak of Corona 19, which has had a great impact not only on the logistics field but also on our lives as a whole. For future work, a multi-faceted approach is required, such as comparative studies of logistics issues between countries or presenting implications based on long-term time-series articles.

A Study of Cookery of Daily Meal (Bankwa Sang: Fruit Table) in Wonheng Ulmyo Jungri Euigwae (1795) (원행을묘정리의궤(園行乙卯整理儀軌) 중(中) 조리면(調理面)에서 본 반과상고(盤果床考))

  • Kim, Sang-Bo;Han, Bok-Jin;Lee, Sung-Woo
    • Journal of the Korean Society of Food Culture
    • /
    • v.5 no.1
    • /
    • pp.1-41
    • /
    • 1990
  • To analyze dialy meal of royal meal, studied Bankwa Sang(Fruit Table) were on record Wonheng Ulmyo Jungri Euigwae (1795). Historic book 'Jungri Euigwae' described the king's visit to his father's royal tomb 'Hyun Neung Won', during the domain of Cheung Jo, the 22nd king of Choson Dynasty. The results obtained from this study areas follows. The fruit table, which similar in concept to desert in the west but quite different in service, was prepared for a guest. The table consisted of two kinds of trays, on which serveral kinds of fruit were stacked to a height of between 4 chon (4寸 : about 12 cm) and 1 chuk(1尺 : about 30.3 cm) according to Korean measurement system. The table was decorated with beautiful artificial flowers made of paper and silk. The number of sets to be arranged on the table were different according to the royal status of the eater: 12 sets-25 sets for king's mother, 7 sets-11 sets for the king. Soy sauce mixed with vinegar and pine-nu meal, mustard were ruled out from kind of sets. Kinds of dishes served with a meal generally were noodles (麵), soup (湯), fried fish (煎油花), fried meats and vegetables (花陽灸), slices of raw fish (魚膾), minced raw meat (肉膾), slices of boiled beef (片肉), stew (蒸), rice cake (餠), sweet rice dish (藥飯), patterned savory cake (茶食), fried cake made of wheat flour, honey and oil (藥果), fried glutinous rice cake (强精), various fruits preserved in honey (正果), sugar candies (各色糖), fruits (果物) honey (淸), soy sauce mixed with vinegar and pine-nut meal (醋醬), mustard (莽子).

  • PDF

Comparison of the Sodium and Sugar Reduction Practices at Samsam Foodservices and General Foodservices in Daegu (대구시 삼삼급식소와 일반급식소의 나트륨 및 당류 저감화 실천도 비교)

  • Kwon, Sung-young;Kim, Kilye;Lee, Yeon-Kyung
    • Korean Journal of Community Nutrition
    • /
    • v.26 no.4
    • /
    • pp.270-279
    • /
    • 2021
  • Objectives: The purpose of this study was to evaluate the effect of the sodium and sugar reduction practices of the Samsam foodservice project of Daegu, in comparison with that of general foodservices in Daegu. Methods: A survey was conducted on 80 Samsam foodservice workers and 80 general foodservice workers from Sep. to Oct. 2020. We compared each worker's taste preferences, stage of behavior change and dietary behavior regarding sodium and sugar, and each foodservice's practices regarding sodium and sugar reduction. Results: There was no significant difference between the salty taste and sweet taste preferences between the workers at the Samsam foodservices and those at the general foodservices. The percentage of foodservice workers in action or maintenance stage of behavior change for eating less salty was higher in the Samsam foodservices than in the general foodservices (P < 0.05). In addition, regarding the degree of saltiness and sweetness of meals, the workers at the general foodservices perceived their meals to be saltier (P < 0.001) and sweeter (P < 0.01) than the workers at Samsam foodservices. The workers at Samsam foodservices had fewer salty dietary behaviors compared to the workers at general foodservices (P < 0.01). The sodium reduction practice was significantly higher in the Samsam foodservices than the general foodservices (P < 0.001), especially in "efforts to make the food as bland as possible overall" (P < 0.001), and "serving less soup and stew" (P < 0.001). The sugar reduction practice too was significantly higher in the Samsam foodservices than the general foodservices (P < 0.001). Conclusions: The Samsam foodservices were shown to be better in the practice of sodium and sugar reduction compared to general foodservices. Therefore, it is necessary to provide continuous and practical support and incentives at the national level to expand the sodium and sugar reduction practices in foodservices.