• Title/Summary/Keyword: Analysis of the data

Search Result 85,993, Processing Time 0.092 seconds

Analysis of Impact Between Data Analysis Performance and Database

  • Kyoungju Min;Jeongyun Cho;Manho Jung;Hyangbae Lee
    • Journal of information and communication convergence engineering
    • /
    • v.21 no.3
    • /
    • pp.244-251
    • /
    • 2023
  • Engineering or humanities data are stored in databases and are often used for search services. While the latest deep-learning technologies, such like BART and BERT, are utilized for data analysis, humanities data still rely on traditional databases. Representative analysis methods include n-gram and lexical statistical extraction. However, when using a database, performance limitation is often imposed on the result calculations. This study presents an experimental process using MariaDB on a PC, which is easily accessible in a laboratory, to analyze the impact of the database on data analysis performance. The findings highlight the fact that the database becomes a bottleneck when analyzing large-scale text data, particularly over hundreds of thousands of records. To address this issue, a method was proposed to provide real-time humanities data analysis web services by leveraging the open source database, with a focus on the Seungjeongwon-Ilgy, one of the largest datasets in the humanities fields.

An Analysis Method of Superlarge Manufacturing Process Data Using Data Cleaning and Graphical Analysis (데이터 정제와 그래프 분석을 이용한 대용량 공정데이터 분석 방법)

  • 박재홍;변재현
    • Journal of Korean Society for Quality Management
    • /
    • v.30 no.2
    • /
    • pp.72-85
    • /
    • 2002
  • Advances in computer and sensor technology have made it possible to obtain superlarge manufacturing process data in real time, letting us extract meaningful information from these superlarge data sets. We propose a systematic data analysis procedure which field engineers can apply easily to manufacture quality products. The procedure consists of data cleaning and data analysis stages. Data cleaning stage is to construct a database suitable for statistical analysis from the original superlarge manufacturing process data. In the data analysis stage, we suggest a graphical easy-to-implement approach to extract practical information from the cleaned database. This study will help manufacturing companies to achieve six sigma quality.

Development of a CAE Middleware and a Visualization System for Supporting Interoperability of Continuous CAE Analysis Data (연속해석 데이터의 상호운용성을 지원하는 CAE 미들웨어와 가시화 시스템의 개발)

  • Song, In-Ho;Yang, Jeong-Sam;Jo, Hyun-Jei;Choi, Sang-Su
    • Korean Journal of Computational Design and Engineering
    • /
    • v.15 no.2
    • /
    • pp.85-93
    • /
    • 2010
  • This paper proposes a CAE data translation and visualization technique that can verify time-varying continuous analysis simulation in a virtual reality (VR) environment. In previous research, the use of CAE analysis data has been problematic because of the lack of any interactive simulation controls for visualizing continuous simulation data. Moreover, the research on post-processing methods for real-time verification of CAE analysis data has not been sufficient. We therefore propose a scene graph based visualization method and a post-processing method for supporting interoperability of continuous CAE analysis data. These methods can continuously visualize static analysis data independently of any timeline; it can also continuously visualize dynamic analysis data that varies in relation to the timeline. The visualization system for continuous simulation data, which includes a CAE middleware that interfaces with various formats of CAE analysis data as well as functions for visualizing continuous simulation data and operational functions, enables users to verify simulation results with more realistic scenes. We also use the system to do a performance evaluation with regard to the visualization of continuous simulation data.

Big Data Analysis Using Principal Component Analysis (주성분 분석을 이용한 빅데이터 분석)

  • Lee, Seung-Joo
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.25 no.6
    • /
    • pp.592-599
    • /
    • 2015
  • In big data environment, we need new approach for big data analysis, because the characteristics of big data, such as volume, variety, and velocity, can analyze entire data for inferring population. But traditional methods of statistics were focused on small data called random sample extracted from population. So, the classical analyses based on statistics are not suitable to big data analysis. To solve this problem, we propose an approach to efficient big data analysis. In this paper, we consider a big data analysis using principal component analysis, which is popular method in multivariate statistics. To verify the performance of our research, we carry out diverse simulation studies.

Network-based Microarray Data Analysis Tool

  • Park, Hee-Chang;Ryu, Ki-Hyun
    • Journal of the Korean Data and Information Science Society
    • /
    • v.17 no.1
    • /
    • pp.53-62
    • /
    • 2006
  • DNA microarray data analysis is a new technology to investigate the expression levels of thousands of genes simultaneously. Since DNA microarray data structures are various and complicative, the data are generally stored in databases for approaching to and controlling the data effectively. But we have some difficulties to analyze and control the data when the data are stored in the several database management systems or that the data are stored to the file format. The existing analysis tools for DNA microarray data have many difficult problems by complicated instructions, and dependency on data types and operating system. In this paper, we design and implement network-based analysis tool for obtaining to useful information from DNA microarray data. When we use this tool, we can analyze effectively DNA microarray data without special knowledge and education for data types and analytical methods.

  • PDF

A Study on the Sentiment Analysis of City Tour Using Big Data

  • Se-won Jeon;Gi-Hwan Ryu
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.15 no.2
    • /
    • pp.112-117
    • /
    • 2023
  • This study aims to find out what tourists' interests and perceptions are like through online big data. Big data for a total of five years from 2018 to 2022 were collected using the Textom program. Sentiment analysis was performed with the collected data. Sentiment analysis expresses the necessity and emotions of city tours in online reviews written by tourists using city tours. The purpose of this study is to extract and analyze keywords representing satisfaction. The sentiment analysis program provided by the big data analysis platform "TEXTOM" was used to study positives and negatives based on sentiment analysis of tourists' online reviews. Sentiment analysis was conducted by collecting reviews related to the city tour. The degree of positive and negative emotions for the city tour was investigated and what emotional words were analyzed for each item. As a result of big data sentiment analysis to examine the emotions and sentiments of tourists about the city tour, 93.8% positive and 6.2% negative, indicating that more than half of the tourists are positively aware. This paper collects tourists' opinions based on the analyzed sentiment analysis, understands the quality characteristics of city tours based on the analysis using the collected data, and sentiment analysis provides important information to the city tour platform for each region.

Automatic Generation of Issue Analysis Report Based on Social Big Data Mining (소셜 빅데이터 마이닝 기반 이슈 분석보고서 자동 생성)

  • Heo, Jeong;Lee, Chung Hee;Oh, Hyo Jung;Yoon, Yeo Chan;Kim, Hyun Ki;Jo, Yo Han;Ock, Cheol Young
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.3 no.12
    • /
    • pp.553-564
    • /
    • 2014
  • In this paper, we propose the system for automatic generation of issue analysis report based on social big data mining, with the purpose of resolving three problems of the previous technologies in a social media analysis and analytic report generation. Three problems are the isolation of analysis, the subjectivity of experts and the closure of information attributable to a high price. The system is comprised of the natural language query analysis, the issue analysis, the social big data analysis, the social big data correlation analysis and the automatic report generation. For the evaluation of report usefulness, we used a Likert scale and made two experts of big data analysis evaluate. The result shows that the quality of report is comparatively useful and reliable. Because of a low price of the report generation, the correlation analysis of social big data and the objectivity of social big data analysis, the proposed system will lead us to the popularization of social big data analysis.

Patterns of Data Analysis\ulcorner

  • Unwin, Antony
    • Journal of the Korean Statistical Society
    • /
    • v.30 no.2
    • /
    • pp.219-230
    • /
    • 2001
  • How do you carry out data analysis\ulcorner There are few texts and little theory. One approach could be to use a pattern language, an idea which has been successful in field as diverse as town planning and software engineering. Patterns for data analysis are defined and discussed, illustrated with examples.

  • PDF

The Development of the Data Acquisition & Analysis System for Multi-Function Radar (다기능레이더 데이터 획득 및 분석 장치 개발)

  • Song, Jun-Ho
    • Journal of the Korea Institute of Military Science and Technology
    • /
    • v.14 no.1
    • /
    • pp.106-113
    • /
    • 2011
  • This paper describes Data Acquisition & Analysis System(DAS) for analysis of the multi-function radar. There are various information - beam probing data, clutter map data, plot data, target tracking data, RT tracking data, radar signal processing data, interface data - this device saves. The most important thing of data analysis is that a researcher gets a view of the whole data. The DAS intergrates with all of the data and provides overall information on the time matters occur. This is very useful advantage for approaching the matter easily. System algorithms of multi-function radar are improved by using this advantage. As a result of, range blank region have fallen about 72% and it is able to keep track in jammer environment.

Analyzing RDF Data in Linked Open Data Cloud using Formal Concept Analysis

  • Hwang, Suk-Hyung;Cho, Dong-Heon
    • Journal of the Korea Society of Computer and Information
    • /
    • v.22 no.6
    • /
    • pp.57-68
    • /
    • 2017
  • The Linked Open Data(LOD) cloud is quickly becoming one of the largest collections of interlinked datasets and the de facto standard for publishing, sharing and connecting pieces of data on the Web. Data publishers from diverse domains publish their data using Resource Description Framework(RDF) data model and provide SPARQL endpoints to enable querying their data, which enables creating a global, distributed and interconnected dataspace on the LOD cloud. Although it is possible to extract structured data as query results by using SPARQL, users have very poor in analysis and visualization of RDF data from SPARQL query results. Therefore, to tackle this issue, based on Formal Concept Analysis, we propose a novel approach for analyzing and visualizing useful information from the LOD cloud. The RDF data analysis and visualization technique proposed in this paper can be utilized in the field of semantic web data mining by extracting and analyzing the information and knowledge inherent in LOD and supporting classification and visualization.