• 제목/요약/키워드: real time analytics

검색결과 58건 처리시간 0.023초

A Computational Intelligence Based Online Data Imputation Method: An Application For Banking

  • Nishanth, Kancherla Jonah;Ravi, Vadlamani
    • Journal of Information Processing Systems
    • /
    • 제9권4호
    • /
    • pp.633-650
    • /
    • 2013
  • All the imputation techniques proposed so far in literature for data imputation are offline techniques as they require a number of iterations to learn the characteristics of data during training and they also consume a lot of computational time. Hence, these techniques are not suitable for applications that require the imputation to be performed on demand and near real-time. The paper proposes a computational intelligence based architecture for online data imputation and extended versions of an existing offline data imputation method as well. The proposed online imputation technique has 2 stages. In stage 1, Evolving Clustering Method (ECM) is used to replace the missing values with cluster centers, as part of the local learning strategy. Stage 2 refines the resultant approximate values using a General Regression Neural Network (GRNN) as part of the global approximation strategy. We also propose extended versions of an existing offline imputation technique. The offline imputation techniques employ K-Means or K-Medoids and Multi Layer Perceptron (MLP)or GRNN in Stage-1and Stage-2respectively. Several experiments were conducted on 8benchmark datasets and 4 bank related datasets to assess the effectiveness of the proposed online and offline imputation techniques. In terms of Mean Absolute Percentage Error (MAPE), the results indicate that the difference between the proposed best offline imputation method viz., K-Medoids+GRNN and the proposed online imputation method viz., ECM+GRNN is statistically insignificant at a 1% level of significance. Consequently, the proposed online technique, being less expensive and faster, can be employed for imputation instead of the existing and proposed offline imputation techniques. This is the significant outcome of the study. Furthermore, GRNN in stage-2 uniformly reduced MAPE values in both offline and online imputation methods on all datasets.

BIG DATA ANALYSIS ROLE IN ADVANCING THE VARIOUS ACTIVITIES OF DIGITAL LIBRARIES: TAIBAH UNIVERSITY CASE STUDY- SAUDI ARABIA

  • Alotaibi, Saqar Moisan F
    • International Journal of Computer Science & Network Security
    • /
    • 제21권8호
    • /
    • pp.297-307
    • /
    • 2021
  • In the vibrant environment, documentation and managing systems are maintained autonomously through education foundations, book materials and libraries at the same time as information are not voluntarily accessible in a centralized location. At the moment Libraries are providing online resources and services for education activities. Moreover, libraries are applying outlets of social media such as Facebook as well as Instagrams to preview their services and procedures. Librarians with the assistance of promising tools and technology like analytics software are capable to accumulate more online information, analyse them for incorporating worth to their services. Thus Libraries can employ big data to construct enhanced decisions concerning collection developments, updating public spaces and tracking the purpose of library book materials. Big data is being produced due to library digitations and this has forced restrictions to academicians, researchers and policy creator's efforts in enhancing the quality and effectiveness. Accordingly, helping the library clients with research articles and book materials that are in line with the users interest is a big challenge and dispute based on Taibah university in Saudi Arabia. The issues of this domain brings the numerous sources of data from various institutions and sources into single place in real time which can be time consuming. The most important aim is to reduce the time that lapses among the authentic book reading and searching the specific study material.

Improving Elasticsearch for Chinese, Japanese, and Korean Text Search through Language Detector

  • Kim, Ki-Ju;Cho, Young-Bok
    • Journal of information and communication convergence engineering
    • /
    • 제18권1호
    • /
    • pp.33-38
    • /
    • 2020
  • Elasticsearch is an open source search and analytics engine that can search petabytes of data in near real time. It is designed as a distributed system horizontally scalable and highly available. It provides RESTful APIs, thereby making it programming-language agnostic. Full text search of multilingual text requires language-specific analyzers and field mappings appropriate for indexing and searching multilingual text. Additionally, a language detector can be used in conjunction with the analyzers to improve the multilingual text search. Elasticsearch provides more than 40 language analysis plugins that can process text and extract language-specific tokens and language detector plugins that can determine the language of the given text. This study investigates three different approaches to index and search Chinese, Japanese, and Korean (CJK) text (single analyzer, multi-fields, and language detector-based), and identifies the advantages of the language detector-based approach compared to the other two.

심화 학습 기반 이동통신기술 연구 동향 (Research Trends of Deep Learning-based Mobile Communication Technology)

  • 권동승
    • 전자통신동향분석
    • /
    • 제34권6호
    • /
    • pp.71-86
    • /
    • 2019
  • The unprecedented demands of mobile communication networks by the rapid rising popularity of mobile applications and services require future networks to support the exploding mobile traffic volumes, the real time extraction of fine-rained analytics, and the agile management of network resources, so as to maximize user experience. To fulfill these needs, research on the use of emerging deep learning techniques in future mobile systems has recently emerged; as such, this study deals with deep learning based mobile communication research activities. A thorough survey of the literature, conference, and workshops on deep learning for mobile communication networks is conducted. Finally, concluding remarks describe the major future research directions in this field.

BigCrawler: 엣지 서버 컴퓨팅·스토리지 모듈의 동적 구성을 통한 효율적인 빅데이터 처리 시스템 구현 및 성능 분석 (Implementation and Performance Aanalysis of Efficient Big Data Processing System Through Dynamic Configuration of Edge Server Computing and Storage Modules)

  • 김용연;전재호;강성주
    • 대한임베디드공학회논문지
    • /
    • 제16권6호
    • /
    • pp.259-266
    • /
    • 2021
  • Edge Computing enables real-time big data processing by performing computing close to the physical location of the user or data source. However, in an edge computing environment, various situations that affect big data processing performance may occur depending on temporary service requirements or changes of physical resources in the field. In this paper, we proposed a BigCrawler system that dynamically configures the computing module and storage module according to the big data collection status and computing resource usage status in the edge computing environment. And the feature of big data processing workload according to the arrangement of computing module and storage module were analyzed.

공간 데이터 분석 기반의 비즈니스의 혁신: 해외 사례 분석을 중심으로 (Business Innovation Through Spatial Data Analysis: A Multi-Case Analysis)

  • 함유근
    • 한국빅데이터학회지
    • /
    • 제4권1호
    • /
    • pp.83-97
    • /
    • 2019
  • 서 및 통신 기술 발전으로 기업경영과 관련된 공간 데이터가 급증하고 있다. 공간 데이터는 이제 2차원적인 지리 데이터를 벗어나 3차원 이상의 공간에 관한 비정형 데이터로 진화하고 있다. 가상공간과 현실공간을 연결해야 하는 제4차산업혁명과 함께 기업들이 이를 활용할 기회도 크게 확대되고 있다. 최근의 해외 사례들의 분석 결과 특히 공간 속에 위치한 고객과 사물의 상황을 파악하여 맞춤화된 서비스를 제공하고, 위험관리를 하며, 더 나아가 업무 프로세스의 혁신도 공간 데이터 분석으로 가능해지고 있다. 향후 공간 속 사람과 사물 들 간의 관계 및 상황을 다양한 소스로부터의 공간 데이터를 결합하여 실시간으로 분석하는 비즈니스 혁신이 모든 분야에서 확대될 전망된다.

  • PDF

타겟 샘플링 검사를 통한 출하품질 향상에 관한 사례 연구 (A Case Study on the Target Sampling Inspection for Improving Outgoing Quality)

  • 김준세;이창기;김경남;김창우;송혜미;안성수;오재원;조현상;한상섭
    • 품질경영학회지
    • /
    • 제49권3호
    • /
    • pp.421-431
    • /
    • 2021
  • Purpose: For improving outgoing quality, this study presents a novel sampling framework based on predictive analytics. Methods: The proposed framework is composed of three steps. The first step is the variable selection. The knowledge-based and data-driven approaches are employed to select important variables. The second step is the model learning. In this step, we consider the supervised classification methods, the anomaly detection methods, and the rule-based methods. The applying model is the third step. This step includes the all processes to be enabled on real-time prediction. Each prediction model classifies a product as a target sample or random sample. Thereafter intensive quality inspections are executed on the specified target samples. Results: The inspection data of three Samsung products (mobile, TV, refrigerator) are used to check functional defects in the product by utilizing the proposed method. The results demonstrate that using target sampling is more effective and efficient than random sampling. Conclusion: The results of this paper show that the proposed method can efficiently detect products that have the possibilities of user's defect in the lot. Additionally our study can guide practitioners on how to easily detect defective products using stratified sampling

A COMPARISON STUDY OF EXPLICIT AND IMPLICIT NUMERICAL METHODS FOR THE EQUITY-LINKED SECURITIES

  • YOO, MINHYUN;JEONG, DARAE;SEO, SEUNGSUK;KIM, JUNSEOK
    • 호남수학학술지
    • /
    • 제37권4호
    • /
    • pp.441-455
    • /
    • 2015
  • In this paper, we perform a comparison study of explicit and implicit numerical methods for the equity-linked securities (ELS). The option prices of the two-asset ELS are typically computed using an implicit finite diffrence method because an explicit finite diffrence scheme has a restriction for time steps. Nowadays, the three-asset ELS is getting popularity in the real world financial market. In practical applications of the finite diffrence methods in computational finance, we typically use relatively large space steps and small time steps. Therefore, we can use an accurate and effient explicit finite diffrence method because the implementation is simple and the computation is fast. The computational results demonstrate that if we use a large space step, then the explicit scheme is better than the implicit one. On the other hand, if the space step size is small, then the implicit scheme is more effient than the explicit one.

Pump availability prediction using response surface method in nuclear plant

  • Parasuraman Suganya;Ganapathiraman Swaminathan;Bhargavan Anoop
    • Nuclear Engineering and Technology
    • /
    • 제56권1호
    • /
    • pp.48-55
    • /
    • 2024
  • The safety-related raw water system's strong operational condition supports the radiation defense and biological shield of nuclear plant containment structures. Gaps and failures in maintaining proper working condition of main equipment like pump were among the most common causes of unavailability of safety related raw water systems. We integrated the advanced data analytics tools to evaluate the maintenance records of water systems and gave special consideration to deficiencies related to pump. We utilized maintenance data over a three-and-a-half-year period to produce metrics like MTBF, MTTF, MTTR, and failure rate. The visual analytic platform using tableau identified the efficacy of maintenance & deficiency in the safety raw water systems. When the number of water quality violation was compared to the other O&M deficiencies, it was discovered that water quality violations account for roughly 15% of the system's deficiencies. The pumps were substantial contributors to the deficit. Pump availability was predicted and optimized with real time data using response surface method. The prediction model was significant with r-squared value of 0.98. This prediction model can be used to predict forth coming pump failures in nuclear plant.

Hadoop을 이용한 스마트 자동차 서비스용 빅 데이터 솔루션 개발 (Addressing Big Data solution enabled Connected Vehicle services using Hadoop)

  • 라이오넬;장종욱
    • 한국정보통신학회논문지
    • /
    • 제19권3호
    • /
    • pp.607-612
    • /
    • 2015
  • 자동차 진단 데이터의 양이 증가함에 따라 자동차 에코시스템의 액터는 스마트 자동차에서 수집된 데이터에 따라 새로운 서비스를 시뮬레이션 하거나 설계하기 위하여 실시간으로 분석을 해야 하는 어려움에 직면하게 된다. 본 논문에서는 자동차에서 생성된 막대한 양의 자동차 내장 진단 데이터를 처리하고 분석하는데 필수적이고 심오한 해석학을 제시하는 빅 데이터 솔루션에 관한 연구를 하였다. Hadoop 및 그 에코시스템은 자동차 소유자에 대한 새로운 서비스 제공을 위해 자동차 에코시스템의 액터에 의해 사용될 수 있는 막대한 데이터 및 전달된 유용한 결과를 처리하기 위해 개발된 것이다. 지능형 교통시스템이 안전성 보장, 속도로 인한 사고로 입는 상해 및 충돌의 비율 감소 등에 관여함에 따라, 자동차 진단 데이터 기반의 빅 데이터 솔루션 개발을 통해 향후 실시간 결과 감시, 여러 스마트 자동차에서의 데이터 수집, 수집된 데이터에 대한 신뢰성 있는 처리 및 용이한 저장을 실현화하게 된다.