• Title/Summary/Keyword: Large-scale Data

Search Result 2,727, Processing Time 0.031 seconds

RDFS Rule based Parallel Reasoning Scheme for Large-Scale Streaming Sensor Data (대용량 스트리밍 센서데이터 환경에서 RDFS 규칙기반 병렬추론 기법)

  • Kwon, SoonHyun;Park, Youngtack
    • Journal of KIISE
    • /
    • v.41 no.9
    • /
    • pp.686-698
    • /
    • 2014
  • Recently, large-scale streaming sensor data have emerged due to explosive supply of smart phones, diffusion of IoT and Cloud computing technology, and generalization of IoT devices. Also, researches on combination of semantic web technology are being actively pushed forward by increasing of requirements for creating new value of data through data sharing and mash-up in large-scale environments. However, we are faced with big issues due to large-scale and streaming data in the inference field for creating a new knowledge. For this reason, we propose the RDFS rule based parallel reasoning scheme to service by processing large-scale streaming sensor data with the semantic web technology. In the proposed scheme, we run in parallel each job of Rete network algorithm, the existing rule inference algorithm and sharing data using the HBase, a hadoop database, as a public storage. To achieve this, we implement our system and evaluate performance through the AWS data of the weather center as large-scale streaming sensor data.

A Study on Information Strategy Development Using Configuration Management in Large-scale Construction Project (형상관리기법을 활용한 대형 프로젝트 정보화 전략개발)

  • Won, Seo Kyung
    • Proceedings of the Korean Institute of Building Construction Conference
    • /
    • 2018.05a
    • /
    • pp.66-67
    • /
    • 2018
  • Large-scale construction projects require various license and technologies for the manufacturing and handling processes. Also, the whole life cycle business process management determines the success of the project. Then, the efficiency of the business conducted by stakeholders and their possessed technology should be enhanced in order to strengthen their competitive power. For this reason, many experts pointed out to focus on the improvement of the life cycle process and efficient management. Since it is very important to keep up-to-date data and utilize it for work during the long-term project to reflect changes in the large-scale project, the most important part of the project management in project is information change management. Therefore, the objective of this study is applying configuration management(CM) technique in order to managing change data generated for planning in early phase. The result of this research will certainly assist the large-scale project managers in the development of information change management system.

  • PDF

Trends in Compute Express Link(CXL) Technology (CXL 인터커넥트 기술 연구개발 동향)

  • S.Y. Kim;H.Y. Ahn;Y.M. Park;W.J. Han
    • Electronics and Telecommunications Trends
    • /
    • v.38 no.5
    • /
    • pp.23-33
    • /
    • 2023
  • With the widespread demand from data-intensive tasks such as machine learning and large-scale databases, the amount of data processed in modern computing systems is increasing exponentially. Such data-intensive tasks require large amounts of memory to rapidly process and analyze massive data. However, existing computing system architectures face challenges when building large-scale memory owing to various structural issues such as CPU specifications. Moreover, large-scale memory may cause problems including memory overprovisioning. The Compute Express Link (CXL) allows computing nodes to use large amounts of memory while mitigating related problems. Hence, CXL is attracting great attention in industry and academia. We describe the overarching concepts underlying CXL and explore recent research trends in this technology.

Enhancing Network Service Survivability in Large-Scale Failure Scenarios

  • Izaddoost, Alireza;Heydari, Shahram Shah
    • Journal of Communications and Networks
    • /
    • v.16 no.5
    • /
    • pp.534-547
    • /
    • 2014
  • Large-scale failures resulting from natural disasters or intentional attacks are now causing serious concerns for communication network infrastructure, as the impact of large-scale network connection disruptions may cause significant costs for service providers and subscribers. In this paper, we propose a new framework for the analysis and prevention of network service disruptions in large-scale failure scenarios. We build dynamic deterministic and probabilistic models to capture the impact of regional failures as they evolve with time. A probabilistic failure model is proposed based on wave energy behaviour. Then, we develop a novel approach for preventive protection of the network in such probabilistic large-scale failure scenarios. We show that our method significantly improves uninterrupted delivery of data in the network and reduces service disruption times in large-scale regional failure scenarios.

A Study on Competition Analysis in Retail Distribution Industry Using GIS in Seoul

  • YOO, Byong-Kook;KIM, Soon-Hong
    • Journal of Distribution Science
    • /
    • v.19 no.3
    • /
    • pp.49-60
    • /
    • 2021
  • Purpose: This study aims to utilize geographic data to analyze how various retail formats of large-scale stores around the traditional market affect the performance of the traditional market in Seoul, Korea. Research design, data, and methodology: The two types of catchment areas were demarcated (circle of 1km radius and Thiessen polygon) for each traditional market, and the large-scale stores located within each catchment area were identified for 153 traditional markets in Seoul, Korea. Additionally, multiple regression analysis was utilized. Results: The results revealed that the influence on the performance of the traditional markets were different depending on the retail format of the large-scale stores. Large discount stores were found to have a negative effect on the sales and the visitors of traditional markets, whereas complex shopping malls and department stores had a positive effect on the traditional markets. Conclusions: As a result of the differences in the retail format such as product categories and leisure functions, the impact of some large-scale stores on the traditional market may have a greater agglomeration effect than the consumer churn effect. Therefore, it is suggested that in the regulation of these large-scale stores, the differences in retail format should be considered for the future.

Implementation of the Large-scale Data Signature System Using Hash Tree Replication Approach (해시 트리 기반의 대규모 데이터 서명 시스템 구현)

  • Park, Seung Kyu
    • Convergence Security Journal
    • /
    • v.18 no.1
    • /
    • pp.19-31
    • /
    • 2018
  • As the ICT technologies advance, the unprecedently large amount of digital data is created, transferred, stored, and utilized in every industry. With the data scale extension and the applying technologies advancement, the new services emerging from the use of large scale data make our living more convenient and useful. But the cybercrimes such as data forgery and/or change of data generation time are also increasing. For the data security against the cybercrimes, the technology for data integrity and the time verification are necessary. Today, public key based signature technology is the most commonly used. But a lot of costly system resources and the additional infra to manage the certificates and keys for using it make it impractical to use in the large-scale data environment. In this research, a new and far less system resources consuming signature technology for large scale data, based on the Hash Function and Merkle tree, is introduced. An improved method for processing the distributed hash trees is also suggested to mitigate the disruptions by server failures. The prototype system was implemented, and its performance was evaluated. The results show that the technology can be effectively used in a variety of areas like cloud computing, IoT, big data, fin-tech, etc., which produce a large-scale data.

  • PDF

Large-scale 3D fast Fourier transform computation on a GPU

  • Jaehong Lee;Duksu Kim
    • ETRI Journal
    • /
    • v.45 no.6
    • /
    • pp.1035-1045
    • /
    • 2023
  • We propose a novel graphics processing unit (GPU) algorithm that can handle a large-scale 3D fast Fourier transform (i.e., 3D-FFT) problem whose data size is larger than the GPU's memory. A 1D FFT-based 3D-FFT computational approach is used to solve the limited device memory issue. Moreover, to reduce the communication overhead between the CPU and GPU, we propose a 3D data-transposition method that converts the target 1D vector into a contiguous memory layout and improves data transfer efficiency. The transposed data are communicated between the host and device memories efficiently through the pinned buffer and multiple streams. We apply our method to various large-scale benchmarks and compare its performance with the state-of-the-art multicore CPU FFT library (i.e., fastest Fourier transform in the West [FFTW]) and a prior GPU-based 3D-FFT algorithm. Our method achieves a higher performance (up to 2.89 times) than FFTW; it yields more performance gaps as the data size increases. The performance of the prior GPU algorithm decreases considerably in massive-scale problems, whereas our method's performance is stable.

A study on high dimensional large-scale data visualization (고차원 대용량 자료의 시각화에 대한 고찰)

  • Lee, Eun-Kyung;Hwang, Nayoung;Lee, Yoondong
    • The Korean Journal of Applied Statistics
    • /
    • v.29 no.6
    • /
    • pp.1061-1075
    • /
    • 2016
  • In this paper, we discuss various methods to visualize high dimensional large-scale data and review some issues associated with visualizing this type of data. High-dimensional data can be presented in a 2-dimensional space with a few selected important variables. We can visualize more variables with various aesthetic attributes in graphics or use the projection pursuit method to find an interesting low-dimensional view. For large-scale data, we discuss jittering and alpha blending methods that solve any problem with overlapping points. We also review the R package tabplot, scagnostics, and other R packages for interactive web application with visualization.

A case study of large-scale slope failure in Granite - Andesite contact area (화강암-안산암 접촉부 대규모 사면의 붕괴 사례 연구)

  • 이수곤;양홍석;황의성
    • Proceedings of the Korean Geotechical Society Conference
    • /
    • 2003.03a
    • /
    • pp.503-508
    • /
    • 2003
  • In this study, we peformed ahead a field geological investigation, boring investigation for slope stability analysis in large scale slope failure area. But the geological stratum was not clearly grasped, because ground was very disturbed by large scale Granite intrusion. Furthermore, the existing test data was not pertinent to the large scale Granite intrusion site like here. Therefore, various kind of field test were performed to grasp clearly for geological stratum. And the results of back analysis, various kind tests used to slope stability analysis.

  • PDF

Design of Distributed Cloud System for Managing large-scale Genomic Data

  • Seine Jang;Seok-Jae Moon
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.16 no.2
    • /
    • pp.119-126
    • /
    • 2024
  • The volume of genomic data is constantly increasing in various modern industries and research fields. This growth presents new challenges and opportunities in terms of the quantity and diversity of genetic data. In this paper, we propose a distributed cloud system for integrating and managing large-scale gene databases. By introducing a distributed data storage and processing system based on the Hadoop Distributed File System (HDFS), various formats and sizes of genomic data can be efficiently integrated. Furthermore, by leveraging Spark on YARN, efficient management of distributed cloud computing tasks and optimal resource allocation are achieved. This establishes a foundation for the rapid processing and analysis of large-scale genomic data. Additionally, by utilizing BigQuery ML, machine learning models are developed to support genetic search and prediction, enabling researchers to more effectively utilize data. It is expected that this will contribute to driving innovative advancements in genetic research and applications.