• Title/Summary/Keyword: large database

Search Result 1,454, Processing Time 0.03 seconds

Development of Out-of-Core Equation Solver with Virtual Memory Database for Large-Scale Structural Analysis (가상 메모리 데이타베이스를 이용한 대규모 구조해석용 코어 외 방정식 해석기법의 개발)

  • 이성우;송윤환;이동근
    • Computational Structural Engineering
    • /
    • v.4 no.2
    • /
    • pp.103-110
    • /
    • 1991
  • To solve the large problems with limited core memory of computer, a disk management scheme called virtual memory database has been developed. Utilizing this technique along with memory moving scheme, an efficient in-and out-of-core column solver for the sparse symmetric matrix commonly arising in the finite element analysis is developed. Compared with other methods the algorithm is simple, therefore the coding and computational efficiencies are greatly enhanced. Analysis example shows that the proposed method efficiently solve the large structural problem on the small-memory micro-computer.

  • PDF

A Study on Indexing Method using Text Partition (텍스트분할에 의한 색인방법 연구)

  • 강무영;이상구
    • Journal of the Korean Society for information Management
    • /
    • v.16 no.4
    • /
    • pp.75-94
    • /
    • 1999
  • Indexing is a prerequisite function for the information retrieval system in order to retrieve the information of the documents effectively which are saved in database. As a digital data increases in accordance with the development of a computer, the numbers of literatures to be saved in database have also been increased in a large volume. To retrieve such documents of large volume, a lot of system resources and processing time will be required. In this paper, we suggest a advanced indexing method using text partition. This method can retrieve the documents of large volume in short processing time. We applied this suggested indexing method to real information retrieval system, and proved its excellent functions through the demonstration.

  • PDF

A Data Mining Procedure for Unbalanced Binary Classification (불균형 이분 데이터 분류분석을 위한 데이터마이닝 절차)

  • Jung, Han-Na;Lee, Jeong-Hwa;Jun, Chi-Hyuck
    • Journal of Korean Institute of Industrial Engineers
    • /
    • v.36 no.1
    • /
    • pp.13-21
    • /
    • 2010
  • The prediction of contract cancellation of customers is essential in insurance companies but it is a difficult problem because the customer database is large and the target or cancelled customers are a small proportion of the database. This paper proposes a new data mining approach to the binary classification by handling a large-scale unbalanced data. Over-sampling, clustering, regularized logistic regression and boosting are also incorporated in the proposed approach. The proposed approach was applied to a real data set in the area of insurance and the results were compared with some other classification techniques.

NVST DATA ARCHIVING SYSTEM BASED ON FASTBIT NOSQL DATABASE

  • Liu, Ying-Bo;Wang, Feng;Ji, Kai-Fan;Deng, Hui;Dai, Wei;Liang, Bo
    • Journal of The Korean Astronomical Society
    • /
    • v.47 no.3
    • /
    • pp.115-122
    • /
    • 2014
  • The New Vacuum Solar Telescope (NVST) is a 1-meter vacuum solar telescope that aims to observe the fine structures of active regions on the Sun. The main tasks of the NVST are high resolution imaging and spectral observations, including the measurements of the solar magnetic field. The NVST has been collecting more than 20 million FITS files since it began routine observations in 2012 and produces maximum observational records of 120 thousand files in a day. Given the large amount of files, the effective archiving and retrieval of files becomes a critical and urgent problem. In this study, we implement a new data archiving system for the NVST based on the Fastbit Not Only Structured Query Language (NoSQL) database. Comparing to the relational database (i.e., MySQL; My Structured Query Language), the Fastbit database manifests distinctive advantages on indexing and querying performance. In a large scale database of 40 million records, the multi-field combined query response time of Fastbit database is about 15 times faster and fully meets the requirements of the NVST. Our slestudy brings a new idea for massive astronomical data archiving and would contribute to the design of data management systems for other astronomical telescopes.

Representation and Implementation of Graph Algorithms based on Relational Database (관계형 데이타베이스에 기반한 그래프 알고리즘의 표현과 구현)

  • Park, Hyu-Chan
    • Journal of KIISE:Databases
    • /
    • v.29 no.5
    • /
    • pp.347-357
    • /
    • 2002
  • Graphs have provided a powerful methodology to solve a lot of real-world problems, and therefore there have been many proposals on the graph representations and algorithms. But, because most of them considered only memory-based graphs, there are still difficulties to apply them to large-scale problems. To cope with the difficulties, this paper proposes a graph representation and graph algorithms based on the well-developed relational database theory. Graphs are represented in the form of relations which can be visualized as relational tables. Each vertex and edge of a graph is represented as a tuple in the tables. Graph algorithms are also defined in terms of relational algebraic operations such as projection, selection, and join. They can be implemented with the database language such as SQL. We also developed a library of basic graph operations for the management of graphs and the development of graph applications. This database approach provides an efficient methodology to deal with very large- scale graphs, and the graph library supports the development of graph applications. Furthermore, it has many advantages such as the concurrent graph sharing among users by virtue of the capability of database.

Generalized Method for Constructing Cutting Force Coefficients Database in End-milling (엔드밀링 가공에서 절삭력 계수 데이터베이스 구현을 위한 일반화된 방법론)

  • 안성호;고정훈;조동우
    • Journal of the Korean Society for Precision Engineering
    • /
    • v.20 no.8
    • /
    • pp.39-46
    • /
    • 2003
  • Productivity and machining performance can be improved by cutting analysis including cutting force prediction, surface error prediction and machining stability evaluation. In order to perform cutting analysis, cutting force coefficients database have to be constructed. Since cutting force coefficients are dependent on cutting condition in the existing research, a large number of calibration tests are needed to obtain cutting force coefficients, which makes it difficult to build the cutting force coefficients database. This paper proposes a generalized method for constructing the cutting force coefficients database us ins cutting-condition-independent coefficients. The tool geometry and workpiece material were considered as important components for database construction. Cutting force coefficients were calculated and analyzed for various helix and rake angles as well as for several workpiece. Furthermore, the variation of cutting force coefficients according to tool wear was analyzed. Tool wear was found to affect tool geometry, which results in the change of cutting force coefficients.

Search Performance Analysis of SQLite Database on Linux (Linux에서 SQLite 데이터베이스의 검색 성능 분석)

  • Choi, Jin-oh
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2016.05a
    • /
    • pp.390-391
    • /
    • 2016
  • SQLite is a database engine which is suitable for small devices that have large resource restriction and is used a lot in mobile devices based on Linux. SQLite is open source program and has advantages which are lighter weight and faster speed. This paper tests SQLite database performance of select in Linux environment, and analyzes the pros and cons of the results. The analysis is processed by comparison with the results of other database working out at the same environment.

  • PDF

Search Performance Experiments of SQLite Database on Linux Environments (Linux환경에서 SQLite 데이터베이스의 검색 성능 실험)

  • Kim, Soo-hwan;Choi, Jin-oh
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2016.10a
    • /
    • pp.445-447
    • /
    • 2016
  • SQLite, Realm, etc. are database engines which are suitable for small devices that have large resource restriction and are used a lot in mobile devices based on Linux. These engines are open source programs and have advantages which are lighter weight and faster speed than general purpose databases. This paper implements performance test program and performs performance experiment of SQLite database to test search efficiency in Linux environments. The experiment is processed by comparison with the results of a Oracle database working out at the same environment.

  • PDF

The Rule Case Simplification Algorithm to be used in a Rule-Based System (규칙기반 시스템에 사용되는 규칙 간소화 알고리즘)

  • Zheng, Baowei;Yeo, Jeong-Mo
    • The KIPS Transactions:PartD
    • /
    • v.17D no.6
    • /
    • pp.405-414
    • /
    • 2010
  • A rule is defined as a case to determine the target values according to combination of various Business factors. The information system is used to represent enterprise's business, which includes and implements the amount of these rules to Rule-Based System. A Rule-Based System can be constructed by using the rules engine method or Relational Database technology. Because the rules engine method has some disadvantages, the Rule-Based System is mostly developed with Relational Database technology. When business scales become larger and more complex, a large number of various rule cases must be operated in system, and processing these rule cases requires additional time, overhead and storage space, and the speed of execution slows down. To solve these problems, we propose a simplification algorithm that converts a large amount of rule cases to simplification rule cases with same effects. The proposed algorithm is applied to hypothetical business rule data and a large number of simplification experiments and tests are conducted. The final results proved that the number of rows can be reduced to some extent. The proposed algorithm can be used to simplify business rule data for improving performance of the Rule-Based System implemented with the Relational Database.

COMPARISON OF LINEAR AND NON-LINEAR NIR CALIBRATION METHODS USING LARGE FORAGE DATABASES

  • Berzaghi, Paolo;Flinn, Peter C.;Dardenne, Pierre;Lagerholm, Martin;Shenk, John S.;Westerhaus, Mark O.;Cowe, Ian A.
    • Proceedings of the Korean Society of Near Infrared Spectroscopy Conference
    • /
    • 2001.06a
    • /
    • pp.1141-1141
    • /
    • 2001
  • The aim of the study was to evaluate the performance of 3 calibration methods, modified partial least squares (MPLS), local PLS (LOCAL) and artificial neural network (ANN) on the prediction of chemical composition of forages, using a large NIR database. The study used forage samples (n=25,977) from Australia, Europe (Belgium, Germany, Italy and Sweden) and North America (Canada and U.S.A) with information relative to moisture, crude protein and neutral detergent fibre content. The spectra of the samples were collected with 10 different Foss NIR Systems instruments, which were either standardized or not standardized to one master instrument. The spectra were trimmed to a wavelength range between 1100 and 2498 nm. Two data sets, one standardized (IVAL) and the other not standardized (SVAL) were used as independent validation sets, but 10% of both sets were omitted and kept for later expansion of the calibration database. The remaining samples were combined into one database (n=21,696), which was split into 75% calibration (CALBASE) and 25% validation (VALBASE). The chemical components in the 3 validation data sets were predicted with each model derived from CALBASE using the calibration database before and after it was expanded with 10% of the samples from IVAL and SVAL data sets. Calibration performance was evaluated using standard error of prediction corrected for bias (SEP(C)), bias, slope and R2. None of the models appeared to be consistently better across all validation sets. VALBASE was predicted well by all models, with smaller SEP(C) and bias values than for IVAL and SVAL. This was not surprising as VALBASE was selected from the calibration database and it had a sample population similar to CALBASE, whereas IVAL and SVAL were completely independent validation sets. In most cases, Local and ANN models, but not modified PLS, showed considerable improvement in the prediction of IVAL and SVAL after the calibration database had been expanded with the 10% samples of IVAL and SVAL reserved for calibration expansion. The effects of sample processing, instrument standardization and differences in reference procedure were partially confounded in the validation sets, so it was not possible to determine which factors were most important. Further work on the development of large databases must address the problems of standardization of instruments, harmonization and standardization of laboratory procedures and even more importantly, the definition of the database population.

  • PDF