• Title/Summary/Keyword: data scalability

Search Result 574, Processing Time 0.026 seconds

User-based Collaborative Filtering Recommender Technique using MapReduce (맵리듀스를 이용한 사용자 기반 협업 필터링 추천 기법)

  • Yun, So-young;Youn, Sung-dae
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2015.10a
    • /
    • pp.331-333
    • /
    • 2015
  • Data is increasing explosively with the spread of networks and mobile devices and there are problems in effectively processing the rapidly increasing data using existing recommendation techniques. Therefore, researches are being conducted on how to solve the scalability problem of the collaborative filtering technique. In this paper applies MapReduce, which is a distributed parallel process framework, to the collaborative filtering technique to reduce the scalability problem and heighten accuracy. The proposed technique applies MapReduce and the index technique to a user-based collaborative filtering technique and as a method which improves neighbor numbers which are used in similarity calculations and neighbor suitability, scalability and accuracy improvement effects can be expected.

  • PDF

Methods to Enhance Service Scalability Using Service Replication and Migration (서비스 복제 및 이주를 이용한 서비스 확장성 향상 기법)

  • Kim, Ji-Won;Lee, Jae-Yoo;Kim, Soo-Dong
    • Journal of KIISE:Software and Applications
    • /
    • v.37 no.7
    • /
    • pp.503-517
    • /
    • 2010
  • Service-oriented computing, the effective paradigm for developing service applications by using reusable services, becomes popular. In service-oriented computing, service consumer has no responsibility for managing services, just invokes services what service providers are producing. On the other hand, service providers should manage any resources and data for service consumers can use the service anytime and anywhere. However, it is hard service providers manage the quality of the services because an unspecified number of service consumers. Therefore, service scalability for providing services with higher quality of services specified in a service level agreement becomes a potential problem in service-oriented computing. There have been many researches for scalability in network, database, and distributed computing area. But a research about a definition of service scalability and metrics of measuring service scalability is still not mature in service engineering area. In this paper, we construct a service network which connects multiple service nodes, and integrate all the resources to manage it. And we also present a service scalability framework for managing service scalability by using a mechanism of service migration or replication. In section 3, we, firstly, present the structure of the scalability management framework and basic functionalities. In section 4, we propose scalability enhancement mechanism which is needed to release functionality of the framework. In section 5, we design and implement the framework by using proposed mechanism. In section 6, we demonstrate the result of our case study which dynamically manages services in multi-nodes environment by applying our framework. Through the case study, we show the applicability of our scalability management framework and mechanism.

A Study for Transaction Processing Supporting Scalability in the Cloud (클라우드 환경에서 확장성을 지원하는 트랜잭션 처리 방법)

  • Kim, Chi-Yeon
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.7 no.4
    • /
    • pp.873-879
    • /
    • 2012
  • Recently the cloud computing paradigm has been widely accepting in various applications. Data management system of cloud computing requires ability to manage tremendous data and supporting scalability. The former can be achieved by weaken consistency and limitation of transactions, and the latter needs expand or shrink of components. In this paper, we propose a transaction processing model and a scalable module management algorithm when transaction is executed in the cloud. Transaction processing model consists of a transaction management module and a data management module. Scalable module management algorithm has no redistribution of components and may alleviates loads of existed modules. With simulation results, we can see the improvement of response time and decrease abort ratio of transactions.

Non-Equal Region Split for Efficient Scalability of Multi-dimensional Sensor Data Storage (다차원 센서 데이타 저장의 효율적인 Scalability를 위한 비균등 영역 분할)

  • Jeon Sang-Hun;Kang Hong-Koo;Hong Dong-Suk;Park Chun-Geol;Han Ki-Joon
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2006.06d
    • /
    • pp.34-36
    • /
    • 2006
  • 데이타 중심 저장 방식의 센서 네트워크는 비슷한 값의 데이타를 같은 센서 노드에 저장한다. 따라서 센서 네트워크가 확장되고 비슷한 값의 데이타가 빈번히 발생 시 하나의 센서 노드에 저장이 집중되는 문제가 있다. 기존의 데이타 중심 저장 방식에서 센서 데이타 저장 기법들은 저장 데이타의 효율적인 관리에만 치우쳐 센서 네트워크의 확장 시 하나의 센서 노드에 저장이 집중되는 문제점을 고려하지 않았다. 본 논문은 센서 네트워크의 확장 시 다차원 센서 데이타 저장의 효율적인 scalability를 지원하는 비균등 분할 기법을 제안한다. 제안한 기법은 센서 네트워크를 센서 노드의 분포에 따라 같은 센서 노드 개수를 갖는 영역으로 분할하고 분할된 각 영역 내에서 측정된 센서 데이타를 해당 영역에서 저장 및 관리함으로써 센서 네트워크의 확장에 따른 저장 비용을 줄였다. 그리고, 생성한 영역 개수를 센서 네트워크의 크기와 센서 노드의 개수, 발생하는 데이타의 양에 따라 증가시킴으로서 전체 센서 노드의 에너지 소모가 분산되어 기존의 방식과 비교하였을 때 센서 네트워크의 수명과 scalability가 향상되었다.

  • PDF

Design of the new parallel processing architecture for commercial applications (상용 응용을 위한 병렬처리 구조 설계)

  • 한우종;윤석한;임기욱
    • Journal of the Korean Institute of Telematics and Electronics B
    • /
    • v.33B no.5
    • /
    • pp.41-51
    • /
    • 1996
  • In this paper, anew parallel processing system based on a cluster architecture which provides scalability of a parallel processing system while maintains shared memory multiprocessor characteristics is proposed. In recent days low cost, high performnce microprocessors have led to construction of large scale parallel processing systems. Such parallel processing systems provides large scalability but are mainly used for scientific applications which have large data parallelism. A shared memory multiprocessor system like TICOM is currently used as aserver for the commercial application, however, the shared memory multiprocessor system is known to have very limited scalability. The proposed architecture can support scalability and performance of the parallel processing system while it provides adaptability for the commerical application, hence it can overcome the limitation of the shared memory multiprocessor. The architecture and characteristics of the proposed system shall be described. A proprietary hierarchical crsossbar network is designed for this system, of which the protocol, routing and switching technique and the signal transfer technique are optimized for the proposed architecture. The design trade-offs for the network are described in this paper and with simulation usihng the SES/workbench, it is explored that the network fits to the proposed architecture.

  • PDF

Performance Improvement of a Collaborative Recommendation System using Feature Selection (속성추출을 이용한 협동적 추천시스템의 성능 향상)

  • Yoo, Sang-Jong;Kwon, Young- S.
    • IE interfaces
    • /
    • v.19 no.1
    • /
    • pp.70-77
    • /
    • 2006
  • One of the problems in developing a collaborative recommendation system is the scalability. To alleviate the scalability problem efficiently, enhancing the performance of the recommendation system, we propose a new recommendation system using feature selection. In our experiments, the proposed system using about a third of all features shows the comparable performances when compared with using all features in light of precision, recall and number of computations, as the number of users and products increases.

Scalable and Robust Data Dissemination Scheme for Large-Scale Wireless Sensor Networks (대규모 무선 센서 네트워크를 위한 확장성과 강건성이 있는 데이터 전송 방안)

  • Park, Soo-Chang;Lee, Eui-Sin;Park, Ho-Sung;Lee, Jeong-Cheol;Oh, Seung-Min;Jung, Ju-Hyun;Kim, Sang-Ha
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.34 no.12B
    • /
    • pp.1359-1370
    • /
    • 2009
  • In wireless sensor networks, data dissemination is based on data-centric routing that well matches the publish/subscribe communication paradigm. The publish/subscribe paradigm requires decoupling properties: space, time, and synchronization decoupling. For large-scale applications, the three decoupling properties provide scalability and robust communication. However, existing data dissemination schemes for wireless sensor networks do not achieve full decoupling. Therefore, we propose a novel data dissemination scheme that fully accomplishes the three decoupling, called ARBIETER. ARBITER constructs an independent network structure as a logical software bus. Information interworking between publishers and subscribers is indirectly and asynchronously performed via the network structure. ARBITER also manages storage and mapping of queries and data on the structure because of supporting different time connection of publishers and subscribers. Our simulation proves ARBITER show better performance in terms of scalability, network robustness, data responsibility, mobility support, and energy efficiency.

Healthcare System using Pegged Blockchain considering Scalability and Data Privacy

  • Azizan, Akmal;Pham, Quoc-Viet;Han, Suk Young;Kim, Jung Eon;Kim, Hoon;Park, Junseok;Hwang, Won-Joo
    • Journal of Korea Multimedia Society
    • /
    • v.22 no.5
    • /
    • pp.613-625
    • /
    • 2019
  • The rise of the Internet of Things (IoT) devices have greatly influenced many industries and one of them is healthcare where wearable devices started to track all your daily activities for better health monitoring accuracy and even down to tracking daily food intake in some cases. With the amounts of data that are being tracked and shared between from these devices, questions were raised on how to uphold user's data privacy when data is shared between these IoT devices and third party. With the blockchain platforms started to mature since its inception, the technology can be implemented according to a variety of use case scenarios. In this paper, we present a system architecture based on the healthcare system and IoT network by leveraging on multiple blockchain networks as the medium in between that should enable users to have direct authority on data accessibility of their shared data. We provide proof of concept implementation and highlight the results from our testing to show how the efficiency and scalability of the healthcare system improved without having a significant impact on the performance of the Electronic Medical Record (EMR) that mostly affected by the previous solution since these solutions directly connected to a public blockchain network and which resulted in significant delays and high cost of operation when a large amount of data or complicated functions are involved.

A Study on Improvement of Blockchain Scalability (블록체인 확장성 개선 연구)

  • Lee, Daesung
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2018.05a
    • /
    • pp.86-87
    • /
    • 2018
  • As blockchain technology has the potential to revolutionize trust models and business processes across industries, applications are expected to be endless. However, this technology is still in the early stage, and the scalability caused by the accumulation of transaction data due to the increase of blocks is emerging as a serious problem. In this paper, we propose various alternatives to solve the scalability problem.

  • PDF

Monte-Carlo Methods for Social Network Analysis (사회네트워크분석에서 몬테칼로 방법의 활용)

  • Huh, Myung-Hoe;Lee, Yong-Goo
    • The Korean Journal of Applied Statistics
    • /
    • v.24 no.2
    • /
    • pp.401-409
    • /
    • 2011
  • From a social network of n nodes connected by l lines, one may produce centrality measures such as closeness, betweenness and so on. In the past, the magnitude of n was around 1,000 or 10,000 at most. Nowadays, some networks have 10,000, 100,000 or even more than that. Thus, the scalability issue needs the attention of researchers. In this short paper, we explore random networks of the size around n = 100,000 by Monte-Carlo method and propose Monte-Carlo algorithms of computing closeness and betweenness centrality measures to study the small world properties of social networks.