• Title/Summary/Keyword: Distributed Processing Platform

Search Result 130, Processing Time 0.023 seconds

A Design and Implementation of Distributed Object Group Platform for Supporting Real-Time Application in CORBA Environments (CORBA 환경에서 실시간 응용을 자원을 위한 분산 객체그룹 플랫폼의 설계 및 구현)

  • Kim, Myeong-Hui;Lee, Jae-Wan;Ju, Su-Jong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.7 no.4
    • /
    • pp.1062-1072
    • /
    • 2000
  • The applications developing in distributed object computing enviroments are faced with the difficulties for managing various lots of distributed objects. Also, because the most multimedia service, like video, audio, and so forth, must be satisfied itself with real-time constraints, the users also are feeling with necessary to apply real-time mechanisms to distributed multimedia services. The goal of this paper is to solve the problems for managing distributed objects, and to be easy to develop complex applications that can provide real-time services. To do this, we designed and implemented a real-time object group platform that can be placed between applications and CORBA. This platform is extended the existing object group model[13,14] added to the scheduler and timer object components for supporting real-time concept. We designed the components for platform by using James Rumbaugh object modeling technology that consists of object, function, and dynamic model. And then we described the detailed interfaces of the components by IDL, and implemented our real-time object group's platform using OrbixMT 22 which is the IONA Technologies' ORB product. Finally, we showed the execution procedures of the schduler object of each components in a real-time object group platform.

  • PDF

A Study on a Distributed Data Fabric-based Platform in a Multi-Cloud Environment

  • Moon, Seok-Jae;Kang, Seong-Beom;Park, Byung-Joon
    • International Journal of Advanced Culture Technology
    • /
    • v.9 no.3
    • /
    • pp.321-326
    • /
    • 2021
  • In a multi-cloud environment, it is necessary to minimize physical movement for efficient interoperability of distributed source data without building a data warehouse or data lake. And there is a need for a data platform that can easily access data anywhere in a multi-cloud environment. In this paper, we propose a new platform based on data fabric centered on a distributed platform suitable for cloud environments that overcomes the limitations of legacy systems. This platform applies the knowledge graph database technique to the physical linkage of source data for interoperability of distributed data. And by integrating all data into one scalable platform in a multi-cloud environment, it uses the holochain technique so that companies can easily access and move data with security and authority guaranteed regardless of where the data is stored. The knowledge graph database mitigates the problem of heterogeneous conflicts of data interoperability in a decentralized environment, and Holochain accelerates the memory and security processing process on traditional blockchains. In this way, data access and sharing of more distributed data interoperability becomes flexible, and metadata matching flexibility is effectively handled.

Development of Big-data Management Platform Considering Docker Based Real Time Data Connecting and Processing Environments (도커 기반의 실시간 데이터 연계 및 처리 환경을 고려한 빅데이터 관리 플랫폼 개발)

  • Kim, Dong Gil;Park, Yong-Soon;Chung, Tae-Yun
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.16 no.4
    • /
    • pp.153-161
    • /
    • 2021
  • Real-time access is required to handle continuous and unstructured data and should be flexible in management under dynamic state. Platform can be built to allow data collection, storage, and processing from local-server or multi-server. Although the former centralize method is easy to control, it creates an overload problem because it proceeds all the processing in one unit, and the latter distributed method performs parallel processing, so it is fast to respond and can easily scale system capacity, but the design is complex. This paper provides data collection and processing on one platform to derive significant insights from various data held by an enterprise or agency in the latter manner, which is intuitively available on dashboards and utilizes Spark to improve distributed processing performance. All service utilize dockers to distribute and management. The data used in this study was 100% collected from Kafka, showing that when the file size is 4.4 gigabytes, the data processing speed in spark cluster mode is 2 minute 15 seconds, about 3 minutes 19 seconds faster than the local mode.

Design and Implementation of An Educational VDB System in Distributed Environments Based on Object Groups (객체그룹 기반의 분산환경에서 교육용 VDB 시스템의 설계 및 구현)

  • Yu, Gyeong-Taek;Lee, Hyeon-Cheol;Ju, Su-Jong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.11
    • /
    • pp.3034-3045
    • /
    • 1999
  • For efficiently providing multimedia services, distributed computing environments are specified the requirements of various services and distributed object platforms applied an object-oriented technology by TINA Consortium and OMG CORBA. Because multimedia service applications are becoming large and distributing, their servicing managing interfaces among objects are being complicated. In order to solve these defects, it is necessary to suggest a new object grouping model and specify object service/management requirements can be introduced under the object groups. We have been developed the distributed object group platform that can group all individual objects by the relating services and can supply trading functions for interconnecting between distributed objects or object groups. In this paper, we designed and implemented the Virtual Drawing Board for remote equational services on the distributed object group platform we mentioned above. As results, we designed a basic structure and service interfaces, and showed execution procedures of VDB system consisted of distributed objects and objects groups for educational services. For supporting distributed services of VDB system, we used three kinds of tools as follows; IONA orbix 2.2 of CORBA compliance as an object middleware, OrbixTrader 1.0 for interconnection of distributed objects, and the OGTG we developed for interconnection of distributed object groups and checking access rights of objects included in an arbitrary object group.

  • PDF

A Package Design for Multimedia Live Streaming in Distributed Environment (분산 환경에서 멀티미디어 실시간 스트리밍을 위한 패키지 설계)

  • Seo Bong-Kun;Kim Yun-Ho
    • Journal of Korea Multimedia Society
    • /
    • v.9 no.4
    • /
    • pp.490-504
    • /
    • 2006
  • It needs to control each objects on various platform and transmit multimedia data to multiple receivers which are for developing a multimedia service with multimedia live streaming in a distributed environment. In this paper, we present a DLS (Distributed Live Streaming) package which support l:N multimedia live streaming in a distributed environment. Also, it has extended RMI which is a distributed object technology and JMF using multimedia transmission/processing. A java-based DLS package has been designed to separate a transmission and a control for more efficient distributed processing. It is possible to apply in development of multimedia service supported 1:N transmission and runned independently to any platform.

  • PDF

Research on the Sharing Strategy of Electronic Book Resources in Universities in the Internet Era

  • Guiya Gao
    • Journal of Information Processing Systems
    • /
    • v.19 no.5
    • /
    • pp.590-601
    • /
    • 2023
  • University books are an important information resource. University book resources can be shared not only in the traditional paper form, but also electronic form under the background of the Internet. In order to better manage the sharing of electronic book resources in universities, this study put forward three resource sharing strategies: centralized sharing strategy, distributed sharing strategy, and centralized-distributed sharing strategy by analyzing the combined development of books and the Internet as well as the significance and development of book resource sharing. The centralized sharing strategy, however simple, was difficult to handle large traffic; while the resource nodes were independent and self-consistent, the distributed sharing strategy was not easy to find and had a high repetition rate. Combining the advantages of both strategies, the centralized-distributed sharing strategy was more suitable for the heterogeneous form of university book sharing. Finally, a teaching resources sharing platform for university libraries was designed based on the strategy of centralized and distributed sharing, and three interfaces including platform login, resource search, and resource release were displayed. The results of the simulated comparison experiment showed that centralized and distributed sharing strategies had limitations in resource searching and had low efficiencies; the efficiency of the centralized strategy reduced with an increase in search subjects; however, the centralized-distributed sharing strategy was able to search more resources efficiently and main stability.

Design and Implementation of a TMN Agent Platform based on a Multi-thread Parallel Processing Architecture (멀티쓰레드 기반 병렬처리 구조를 이용한 TMN 에이젼트 플랫폼 설계 및 구현)

  • Kim, Seong-U;Kim, Yeong-Tak
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.5 no.6
    • /
    • pp.793-800
    • /
    • 1999
  • TMN Agent Platform은 망 요소의 운영상태와 자원들을 GDMO에 따라 관리객체(Managed Object : MO)로 모델링 하고, 자원들의 현재 상태를 유지하며, 관리자(Manager)로부터의 망 관리 기능 요구에 따라 조작된다. 그러므로, 에이전트의 성능향상은 전체적인 통신망 관리의 성능향상에 직접적인 영향을 미친다.본 논문에서는 TMN 에이전트의 기능요구 사항을 분석하고, 이를 토대로 성능향상을 위해 멀티스레드 기법을 사용하는 병렬 처리 구조의 TMN Agent Platform의 기능구조를 제시한다. 또한 에이전트와 다양한 자원들간의 효율적인 메시지전달을 위한 체계를 제시하며, 구현된 TMN Agent Platform의 성능을 분석한다.Abstract TMN Agent manages the operational status and real-resources of network elements, such as switching nodes and transmission systems. It performs the requested management functions from manager and maintains consistent status data of real-resource. The performance of agent system affects directly the performance of network management operation. If the agent is implemented by sequential processing scheme with single process, the agent processing can be delayed or blocked according to the status of real-resources. This problem can be solved by parallel and distributed processing scheme.To improve the processing performance of TMN Agent, we propose a TMN Agent Platform's functional architecture that is based on parallel processing with multi-tread and effective message transferring scheme between agent and various real-resource. We analyze the performance of the implemented TMN Agent Platform.

Processing Method of Mass Small File Using Hadoop Platform (하둡 플랫폼을 이용한 대량의 스몰파일 처리방법)

  • Kim, Chang-Bok;Chung, Jae-Pil
    • Journal of Advanced Navigation Technology
    • /
    • v.18 no.4
    • /
    • pp.401-408
    • /
    • 2014
  • Hadoop is composed with MapReduce programming model for distributed processing and HDFS distributed file system. Hadoop is suitable framework for big data processing, but processing of mass small files have many problems. The processing of mass small file in hadoop have problems to created one mapper per one file, and it have problems to needed many memory for store of meta information of file. This paper have comparison evaluation processing method of mass small file with various method in hadoop platform. The processing of general compression format is inadequate because of processing by one mapper regardless of data size. The processing of sequence and hadoop archive file is removed memory problem of namenode by compress and combine of small file. Hadoop archive file is faster then sequence file about combine time of small file. The processing using CombineFileInputFormat class is needed not combine of small file, and it have similar speed big data processing method.

Optimization and Performance Analysis of Cloud Computing Platform for Distributed Processing of Big Data (대용량 데이터의 분산 처리를 위한 클라우드 컴퓨팅 환경 최적화 및 성능평가)

  • Hong, Seung-Tae;Shin, Young-Sung;Chang, Jae-Woo
    • Spatial Information Research
    • /
    • v.19 no.4
    • /
    • pp.55-71
    • /
    • 2011
  • Recently, interest in cloud computing which provides IT resources as service form in IT field is increasing. As a result, much research has been done on the distributed data processing that store and manage a large amount of data in many servers. Meanwhile, in order to effectively utilize the spatial data which is rapidly increasing day by day with the growth of GIS technology, distributed processing of spatial data using cloud computing is essential. Therefore, in this paper, we review the representative distributed data processing techniques and we analyze the optimization requirements for performance improvement of the distributed processing techniques for a large amount of data. In addition, we uses the Hadoop and we evaluate the performance of the distributed data processing techniques for their optimization requirements.

Design of Spark SQL Based Framework for Advanced Analytics (Spark SQL 기반 고도 분석 지원 프레임워크 설계)

  • Chung, Jaehwa
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.10
    • /
    • pp.477-482
    • /
    • 2016
  • As being the advanced analytics indispensable on big data for agile decision-making and tactical planning in enterprises, distributed processing platforms, such as Hadoop and Spark which distribute and handle the large volume of data on multiple nodes, receive great attention in the field. In Spark platform stack, Spark SQL unveiled recently to make Spark able to support distributed processing framework based on SQL. However, Spark SQL cannot effectively handle advanced analytics that involves machine learning and graph processing in terms of iterative tasks and task allocations. Motivated by these issues, this paper proposes the design of SQL-based big data optimal processing engine and processing framework to support advanced analytics in Spark environments. Big data optimal processing engines copes with complex SQL queries that involves multiple parameters and join, aggregation and sorting operations in distributed/parallel manner and the proposing framework optimizes machine learning process in terms of relational operations.