• 제목/요약/키워드: Data Loading Performance

검색결과 333건 처리시간 0.031초

아파치 스쿱을 사용한 하둡의 데이터 적재 성능 영향 요인 분석 (Analysis of the Influence Factors of Data Loading Performance Using Apache Sqoop)

  • ;고정현;여정모
    • 정보처리학회논문지:소프트웨어 및 데이터공학
    • /
    • 제4권2호
    • /
    • pp.77-82
    • /
    • 2015
  • 빅데이터 기술은 데이터 처리 속도가 빠르다는 면에서 주목을 받고 있다. 그리고 관계형 데이터베이스(Relational Database: RDB)에 저장되어있는 대용량 정형 데이터를 더 빠르게 처리하기 위해서 빅데이터 기술을 활용하는 연구도 진행되고 있다. 다양한 분산 처리 도구들을 사용하여 분석 성능을 측정하는 연구는 많지만 분석하기 전 단계인 정형 데이터 적재의 성능에 관한 연구는 미미하다. 때문에 본 연구에서는 RDB 안에 저장되어있는 정형 데이터를 아파치 스쿱(Apache Sqoop)을 사용하여 분산 처리 플랫폼 하둡(Hadoop)으로 적재하는 성능을 측정하였다. 그리고 적재에 영향을 미치는 요인을 분석하기 위해 여러 가지 영향 요소를 변경해가면서 반복적으로 실험을 수행하였고 RDB 기반으로 구성된 서버 간의 적재 성능과 비교하였다. 실험 환경에서 아파치 스쿱의 적재 속도가 낮았지만 실제 운영하고 있는 대규모 하둡 클러스터 환경에서는 더 많은 하드웨어 자원이 확보되기 때문에 훨씬 더 좋은 성능을 기대할 수 있다. 이는 향후 진행할 적재 성능 개선 및 하둡 환경에서 정형 데이터를 분석하는 전체적인 단계의 성능을 향상시킬 수 있는 방법에 대한 연구의 기반이 될 것으로 예상한다.

Adaptive Group Loading and Weighted Loading for MIMO OFDM Systems

  • Shrestha, Robin;Kim, Jae-Moung
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제5권11호
    • /
    • pp.1959-1975
    • /
    • 2011
  • Adaptive Bit Loading (ABL) in Multiple-Input Multiple-Output Orthogonal Frequency-Division Multiplexing (MIMO-OFDM) is often used to achieve the desired Bit Error Rate (BER) performance in wireless systems. In this paper, we discuss some of the bit loading algorithms, compare them in terms of the BER performance, and present an effective and concise Adaptive Grouped Loading (AGL) algorithm. Furthermore, we propose a "weight factor" for loading algorithm to converge rapidly to the final solution for various data rate with variable Signal to Noise Ratio (SNR) gaps. In particular, we consider the bit loading in near optimal Singular Value Decomposition (SVD) based MIMO-OFDM system. While using SVD based system, the system requires perfect Channel State Information (CSI) of channel transfer function at the transmitter. This scenario of SVD based system is taken as an ideal case for the comparison of loading algorithms and to show the actual enhancement achievable by our AGL algorithm. Irrespective of the CSI requirement imposed by the mode of the system itself, ABL demands high level of feedback. Grouped Loading (GL) would reduce the feedback requirement depending upon the group size. However, this also leads to considerable degradation in BER performance. In our AGL algorithm, groups are formed with a number of consecutive sub-channels belonging to the same transmit antenna, with individual gains satisfying predefined criteria. Simulation results show that the proposed "weight factor" leads a loading algorithm to rapid convergence for various data rates with variable SNR gap values and AGL requires much lesser CSI compared to GL for the same BER performance.

A Research on Development of Bills of Material Using Web Grid for Product Lifecycle Management

  • Yoo, Ji-Hyun
    • 한국컴퓨터정보학회논문지
    • /
    • 제22권12호
    • /
    • pp.131-136
    • /
    • 2017
  • PLM(Product Lifecycle Management) is an information management system that can integrate data, processes, business systems and human resources throughout the enterprise. BOM(Bills Of Material) is key data for designing, purchasing materials, manufacturing planning and management, which is basic for product development throughout the product life cycle. In this paper, we propose the efficient system to increase the data loading speed and the processing speed when using such large BOM data. We present the performance and usability of IMDG (In Memory Data Grid) for data processing when loading large amounts of data. In the UI, using the pure web grid of JavaScript instead of the existing data loading method can be improve performance of data managing.

Performance of structural-concrete members under sequential loading and exhibiting points of inflection

  • Jelic, I.;Pavlovic, M.N.;Kotsovos, M.D.
    • Computers and Concrete
    • /
    • 제1권1호
    • /
    • pp.99-113
    • /
    • 2004
  • The article reports data on, and numerical modelling of, beams exhibiting points of inflection and subjected to sequential loading. Both tests and analysis point to inadequacies in current codes of practice. An alternative design methodology, which is strongly associated with the notion that contraflexure points should be designed as "internal supports", is shown to produce superior performance even though it requires significantly less secondary reinforcement than that advocated by codes.

Performance analysis tool for reinforced concrete members

  • Esmaeily, Asad;Peterman, Robert J.
    • Computers and Concrete
    • /
    • 제4권5호
    • /
    • pp.331-346
    • /
    • 2007
  • A computer program was developed to analyze the non-linear, cyclic flexural performance of reinforced concrete structural members under various types of loading paths including non-sequential variations in axial load. This performance is significantly affected by the loading history. Different monotonic material models as well as hysteresis rules for confined and unconfined concrete and steel, some developed and calibrated against test results on material samples, were implemented in a fiber-based moment-curvature and in turn force-deflection analysis. One of the assumptions on curvature distribution along the member was based on a method developed to address the variation of the plastic hinge length as a result of loading pattern. Functionality of the program was verified by reproduction of analytical results obtained by others for several cases, and accuracy of the analytical process and the implemented models were evaluated against the experimental results from large-scale reinforced concrete columns tested under the analyzed loading cases. While the program can be used to predict the response of a member under a certain loading pattern, it can also be used to examine various analytical models and methods or refine a custom material model against test data.

자료포락분석을 활용한 철도화물취급역의 성과평가 (Evaluating the Performance of Freight Railway Stations Using Data Envelopment Analysis)

  • 김성호;오석문
    • 한국철도학회논문집
    • /
    • 제12권2호
    • /
    • pp.315-320
    • /
    • 2009
  • 본 연구에서는 단일기준을 활용한 화물취급역 평가의 대안으로서 다수의 기준을 활용한 평가방법을 제안하고자 한다. 다수의 기준을 활용한 평가에서는 각 성과지표에 대한 가중치가 필요하며 Charnes, Cooper, and Rhodes(1978)가 제안한 자료포락분석은 이러한 가중치를 결정하는 방법으로 사용될 수 있다. 본 연구에서는 353개 화물취급역의 발송규모, 발송수입, 도착규모, 도착수입 등을 성과지표로 사용하고 이들을 자료포락분석을 사용하여 다수의 성과지표를 동시에 모두 반영한 평가결과를 제시한다. 이러한 결과는 철도경영자 또는 철도정책담당자의 경영전략 또는 정책의사결정에 유용하게 사용될 수 있을 것이다.

($XiVE^{(R)}$ 임프란트의 성공률에 관한 후향적 연구;임상성적에 관한 조기평가 (Retrospective Study of Success Rate of the $XiVE^{(R)}$ Implant;Early evaluation of clinical performance)

  • 박진우
    • Journal of Periodontal and Implant Science
    • /
    • 제35권1호
    • /
    • pp.65-76
    • /
    • 2005
  • This retrospective study evaluates the clinical performance of the recently introduced $XiVE^{(R)}$ implant(Dentsply-Friadent) with a new macro-design to improve primary stability. A total of 208 $XiVE^{(R)}$ implants (101 in the maxilla and 107 in the mandible) were placed in 71 patients. The average age of the patients was 49 years. Of the 208 implants, 190 (91.3%) were posterior implants and 82 (39.4%) were placed in compromised sites (grafted sites). Clinical and radiographic evaluation were made at second stage surgery for exposure and after functional loading. 192 implants in 64 patients were evaluated at exposure and 146 implants in 50 patients were loaded (average 170 days-loading) and evaluated after functional loading. Of 192 implants available for evaluation before loading, 3 implants failed (early failure) ; 1 before exposure, 1 at exposure and 1 during prosthetic procedure. 2 implants were in the maxilla and 1 was in the mandible. The success rate before loading was 98.4%. After functional loading, no implant failure was occurred in 146 implants evaluated during this period (100% interval success rate). This preliminary data with a new implant showed excellent success rate although the majority of implants evaluated in this study were placed in the posterior region of the jaw and compromised sites.

Functional Relationships between Fatigue Data

  • Beiss, Paul
    • 한국분말야금학회:학술대회논문집
    • /
    • 한국분말야금학회 2006년도 Extended Abstracts of 2006 POWDER METALLURGY World Congress Part 1
    • /
    • pp.140-141
    • /
    • 2006
  • Most PM components are exposed to cyclic loading over long periods of time, yet, the fatigue performance is often at best characterized by a fully reversed bending strength. The effects of density, deviating loading modes, external notches or mean stresses must usually be estimated. The amount of available data is nowadays sufficient to come to fact-based estimates.

  • PDF

콘크리트 품질ㆍ시공ㆍ유지관리의 조기판정시스템 개발 (Development of Early Evaluation System for Concrete Quality, Construction and Maintenance)

  • 손용우;이증빈;최미라;박봉수
    • 한국전산구조공학회:학술대회논문집
    • /
    • 한국전산구조공학회 2004년도 가을 학술발표회 논문집
    • /
    • pp.517-526
    • /
    • 2004
  • In the resent years, the early evaluation of concrete quality, construction and maintenance has been considered as all is of major concern due to the increase of loading and the degradation of structures related with time. This paper presents evaluation of structural safety performance using measured data of construction, on the basis of a field measurements for the prevention of unreliable concrete works. Measurements analyzed in this paper are early quality condition and performance assessment, serviceability performance by cracks and deflection, rating performance by loading, durability performance by chloride attack and carbonation. Thus, a quantitative assessment model of resistance capacity was developed here to meet the requirement for deteriorated concrete structures. The model focuses on damage mechanical of concrete structures deteriorated by initial damage factors for concrete quality and environment factors such as chloride and carbonation attacks. These results could provide useful information for concrete structures interested in design, construction and maintenance.

  • PDF

Incorporating nonstructural finish effects and construction quality in a performance-based framework for wood shearwall design

  • Kim, Jun Hee;Rosowsky, David V.
    • Structural Engineering and Mechanics
    • /
    • 제21권1호
    • /
    • pp.83-100
    • /
    • 2005
  • This paper presents results from a study to extend a performance-based shearwall selection procedure to take into account the contributions of nonstructural finish materials (such as stucco and gypsum wallboard), construction quality issues, and their effects on the displacement performance of engineered wood shearwalls subject to seismic loading. Shearwall performance is evaluated in terms of peak displacements under seismic loading (characterized by a suite of ordinary ground motion records) considering different combinations of performance levels (drift limits) and seismic hazard. Shearwalls are analyzed using nonlinear dynamic time-history analysis with global assembly hysteretic parameters determined by fitting to actual shearwall test data. Peak displacement distributions, determined from sets of analyses using each of the ground motion records taken to characterize the seismic hazard, are postprocessed into performance curves, design charts, and fragility curves which can be used for risk-based design and assessment applications.