• Title/Summary/Keyword: Data-science

검색결과 56,444건 처리시간 0.063초

A Rapid Locating Protocol of Corrupted Data for Cloud Data Storage

  • Xu, Guangwei;Yang, Yanbin;Yan, Cairong;Gan, Yanglan
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제10권10호
    • /
    • pp.4703-4723
    • /
    • 2016
  • The verification of data integrity is an urgent topic in remote data storage environments with the wide deployment of cloud data storage services. Many traditional verification algorithms focus on the block-oriented verification to resolve the dispute of dynamic data integrity between the data owners and the storage service providers. However, these algorithms scarcely pay attention to the data verification charge and the users' verification experience. The users more concern about the availability of accessed files rather than data blocks. Moreover, the data verification charge limits the number of checked data in each verification. Therefore, we propose a mixed verification protocol to verify the data integrity, which rapidly locates the corrupted files by the file-oriented verification, and then identifies the corrupted blocks in these files by the block-oriented verification. Theoretical analysis and simulation results demonstrate that the protocol reduces the cost of the metadata computation and transmission relative to the traditional block-oriented verification at the expense of little cost of additional file-oriented metadata computation and storage at the data owner. Both the opportunity of data extracted and the scope of suspicious data are optimized to improve the verification efficiency under the same verification cost.

A Visualization Scheme with a Calendar Heat Map for Abnormal Pattern Analysis in the Manufacturing Process

  • Chankhihort, Doung;Lim, Byung-Muk;Lee, Gyu-Jung;Choi, Sungsu;Kwon, Sun-Ock;Lee, Sang-Hyun;Kang, Jeong-Tae;Nasridinov, Aziz;Yoo, Kwan-Hee
    • International Journal of Contents
    • /
    • 제13권2호
    • /
    • pp.21-28
    • /
    • 2017
  • Abnormal data in the manufacturing process makes it difficult to find useful information that can be applied in data management for the manufacturing industry. It causes various problems in the daily process of production. An issue from the abnormal data can be handled by our method that uses big data and visualization. Visualization is a new technology that transforms data representation into a two-dimensional representation. Nowadays, many newly developed technologies provide data analysis, algorithm, optimization, and high efficiency, and they meet user requirements. We propose combined production of the data visualization approach that uses integrative visualization of sources of abnormal pattern analysis results. The perceived idea of the proposed approach can solve the problem as it also works for big data. It can also improve the performance and understanding by using visualization and solving issues that occur in the manufacturing process with a calendar heat map.

Design of Korean Data Center for SDO

  • Choi, Seong-Hwan;Hwang, Eun-Mi;Cho, Kyung-Suk;Kim, Yeon-Han;Park, Young-Deuk;Moon, Yong-Jae
    • 천문학회보
    • /
    • 제36권2호
    • /
    • pp.92.1-92.1
    • /
    • 2011
  • NASA launched Solar Dynamics Observatory (SDO) on February 2011 in order to understand the cause of solar activities and their influences on the Earth and the near-Earth space. KASI is constructing Korean Data Center for SDO based on the letter of agreement between KASI and NASA for space weather research. SDO produces about 1.5 TB a day and its raw data amounts to about 550 TB in a year. Stanford University has been already operating the data center for scientific raw data, but there is a limit to use its data for space weather research and space weather service in real time because of network environment. Korean Data Center for SDO will provide scientific data not only to Korean institutes but also to international space weather societies. KASI has designed the data transfer system by using GLORIAD in order to get higher performance and stability. After the first construction of data transfer system and storage system in this year, we will increase the storage capacity of the data center in phases considering new developments in a storage technology and drop of their prices.

  • PDF

Optical Orbit Determination of a Geosynchronous Earth Orbit Satellite Effected by Baseline Distances between Various Ground-based Tracking Stations II: COMS Case with Analysis of Actual Observation Data

  • Son, Ju Young;Jo, Jung Hyun;Choi, Jin;Kim, Bang-Yeop;Yoon, Joh-Na;Yim, Hong-Suh;Choi, Young-Jun;Park, Sun-Youp;Bae, Young Ho;Roh, Dong-Goo;Park, Jang-Hyun;Kim, Ji-Hye
    • Journal of Astronomy and Space Sciences
    • /
    • 제32권3호
    • /
    • pp.229-235
    • /
    • 2015
  • We estimated the orbit of the Communication, Ocean and Meteorological Satellite (COMS), a Geostationary Earth Orbit (GEO) satellite, through data from actual optical observations using telescopes at the Sobaeksan Optical Astronomy Observatory (SOAO) of the Korea Astronomy and Space Science Institute (KASI), Optical Wide field Patrol (OWL) at KASI, and the Chungbuk National University Observatory (CNUO) from August 1, 2014, to January 13, 2015. The astrometric data of the satellite were extracted from the World Coordinate System (WCS) in the obtained images, and geometrically distorted errors were corrected. To handle the optically observed data, corrections were made for the observation time, light-travel time delay, shutter speed delay, and aberration. For final product, the sequential filter within the Orbit Determination Tool Kit (ODTK) was used for orbit estimation based on the results of optical observation. In addition, a comparative analysis was conducted between the precise orbit from the ephemeris of the COMS maintained by the satellite operator and the results of orbit estimation using optical observation. The orbits estimated in simulation agree with those estimated with actual optical observation data. The error in the results using optical observation data decreased with increasing number of observatories. Our results are useful for optimizing observation data for orbit estimation.

데이터과학 시대에 적합한 컴퓨팅 인프라 구축 (Building a computing infrastructure in the era of data science)

  • 최숙희;한경수;왕철
    • 응용통계연구
    • /
    • 제37권1호
    • /
    • pp.49-59
    • /
    • 2024
  • 2010년을 전후로 미국에서 시작된 데이터과학의 인기는 국내 대학의 여러 통계학과 교육에 큰 영향을 주고 있다. 그러나 국내 학술지에서는 데이터과학을 효율적으로 교육하기 위한 컴퓨팅 환경 구축과 활용을 다루는 연구 결과는 많지 않다. 본 논문은 국내의 통계학과 및 데이터과학 관련 학과의 교육과 연구에 적합한 컴퓨팅 인프라 구축과 활용에 관한 문제를 논의하고 해결책을 제시한다.

Influence of Data Preprocessing

  • Zhu, Changming;Gao, Daqi
    • Journal of Computing Science and Engineering
    • /
    • 제10권2호
    • /
    • pp.51-57
    • /
    • 2016
  • In this paper, we research the influence of data preprocessing. We conclude that using different preprocessing methods leads to different classification performances. Moreover, not all data preprocessing methods are necessary, and a criterion is given to make sure which data preprocessing is necessary and which one is effective. Experiments on some real-world data sets validate that different data preprocessing methods result in different effects. Furthermore, experiments about some algorithms with different preprocessing methods also confirm that preprocessing has a great influence on the performance of a classifier.

Data Science and Deep Learning in Natural Sciences

  • Cha, Meeyoung
    • 천문학회보
    • /
    • 제44권2호
    • /
    • pp.56.1-56.1
    • /
    • 2019
  • We are producing and consuming more data than ever before. Massive data allow us to better understand the world around us, yet they bring a new set of challenges due to their inherent noise and sheer enormity in size. Without smart algorithms and infrastructures, big data problems will remain intractable, and the same is true in natural science research. The mission of data science as a research field is to develop and apply computational methods in support of and in the replacement of costly practices in handling data. In this talk, I will introduce how data science and deep learning has been used for solving various problems in natural sciences. In particular, I will present a case study of analyzing high-resolution satellite images to infer socioeconomic scales of developing countries.

  • PDF

과학교육 연구 자료의 정보 전산화 체제(IV) - 데이터 베이스 프로그램 개발 - (Data base system for the information on science education research and development: (IV) Development of a data base program)

  • 김영수;이원식;박승재
    • 한국과학교육학회지
    • /
    • 제12권3호
    • /
    • pp.35-47
    • /
    • 1992
  • The purpose of this study was to develop a data base system for the information on science education research and development. As a part of this study, a data base program was developed on the Macintosh SE using the 4th Dimension from ACI. The data base consisted of two files, dissertation and journal. The information on the 107 theses including the master's theses and the doctoral dissertations from the Department of Scince Education, Seoul National University and on the 640 papers on science education from the first issues to the 1991 issues of five selected science education journals was input into the data base. The selected five Journals were Journal of the Korean Association for Research in Science Education(published by the Korean Association for Research in Science Education, 148 papers), Teaching Physics(published by Korean Physical Society, 164 papers),Chemical Education(published by The Korean Chemical Society, 98 papers), The Korean Journal of Biological Education(published by The Korean Society of Biological Education, 148 papers), and Journal of Science Education(published by Science Education Center, College of Education, Seoul National University,82 papers).

  • PDF

An Efficient Provable Secure Public Auditing Scheme for Cloud Storage

  • Xu, Chunxiang;Zhang, Yuan;Yu, Yong;Zhang, Xiaojun;Wen, Junwei
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제8권11호
    • /
    • pp.4226-4241
    • /
    • 2014
  • Cloud storage provides an easy, cost-effective and reliable way of data management for users without the burden of local data storage and maintenance. Whereas, this new paradigm poses many challenges on integrity and privacy of users' data, since users losing grip on their data after outsourcing the data to the cloud server. In order to address these problems, recently, Worku et al. have proposed an efficient privacy-preserving public auditing scheme for cloud storage. However, in this paper, we point out the security flaw existing in the scheme. An adversary, who is on-line and active, is capable of modifying the outsourced data arbitrarily and avoiding the detection by exploiting the security flaw. To fix this security flaw, we further propose a secure and efficient privacy-preserving public auditing scheme, which makes up the security flaw of Worku et al.'s scheme while retaining all the features. Finally, we give a formal security proof and the performance analysis, they show the proposed scheme has much more advantages over the Worku et al.'s scheme.

3D Model Compression For Collaborative Design

  • Liu, Jun;Wang, Qifu;Huang, Zhengdong;Chen, Liping;Liu, Yunhua
    • International Journal of CAD/CAM
    • /
    • 제7권1호
    • /
    • pp.1-10
    • /
    • 2007
  • The compression of CAD models is a key technology for realizing Internet-based collaborative product development because big model sizes often prohibit us to achieve a rapid product information transmission. Although there exist some algorithms for compressing discrete CAD models, original precise CAD models are focused on in this paper. Here, the characteristics of hierarchical structures in CAD models and the distribution of their redundant data are exploited for developing a novel data encoding method. In the method, different encoding rules are applied to different types of data. Geometric data is a major concern for reducing model sizes. For geometric data, the control points of B-spline curves and surfaces are compressed with the second-order predictions in a local coordinate system. Based on analysis to the distortion induced by quantization, an efficient method for computation of the distortion is provided. The results indicate that the data size of CAD models can be decreased efficiently after compressed with the proposed method.