• Title/Summary/Keyword: 데이터그리드

Search Result 512, Processing Time 0.03 seconds

Implementation of High Speed Big Data Processing System using In Memory Data Grid in Semiconductor Process (반도체 공정에서 인 메모리 데이터 그리드를 이용한 고속의 빅데이터 처리 시스템 구현)

  • Park, Jong-Beom;Lee, Alex;Kim, Tony
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.15 no.5
    • /
    • pp.125-133
    • /
    • 2016
  • Data processing capacity and speed are rapidly increasing due to the development of hardware and software in recent time. As a result, data usage is geometrically increasing and the amount of data which computers have to process has already exceeded five-thousand transaction per second. That is, the importance of Big Data is due to its 'real-time' and this makes it possible to analyze all the data in order to obtain accurate data at right time under any circumstances. Moreover, there are many researches about this as construction of smart factory with the application of Big Data is expected to have reduction in development, production, and quality management cost. In this paper, system using In-Memory Data Grid for high speed processing is implemented in semiconductor process which numerous data occur and improved performance is proven with experiments. Implemented system is expected to be possible to apply on not only the semiconductor but also any fields using Big Data and further researches will be made for possible application on other fields.

A Dynamic Reorganization Method using the Hierarchical Structure in the Grid Database (그리드 데이터베이스에서 계층 구조를 이용한 동적 재조직 기법)

  • Cheon, Jong-Hyeon;Jang, Yong-Il;Cho, Sook-Kyoung;Lee, Soon-Jo;Bae, Hae-Young
    • Journal of Korea Spatial Information System Society
    • /
    • v.8 no.1 s.16
    • /
    • pp.93-106
    • /
    • 2006
  • A Grid Database is a database management system to process effectively and use the distributed data in a grid computing environment. Spatial data is more important than other general data according to the local characteristics and requires a large storage. The grid database can be used as the optimal system for the management of the spatial data. However, contrary to the conventional distributed database systems, the Grid Database which guarantees the local autonomy has a possibility not to provide an effective system, or it is impossible to use a centralized management environment. In order to allow flexible responses to a dynamically changing environment, it is required to use effectively reorganized method. In this paper, hierarchical reorganization method is presented for dynamic reorganization in a grid database. When the reorganization is conducted, an organizer is created to collect the information of databases. In addition, the organizer which is constructed by the hierarchical structure supports information communication and reorganization, and then it allows the support of regional reorganization operation and effective balance control. The performance assessment of the proposed method shows that the processing capacity is increased after the reorganization.

  • PDF

Interoperability Framework between GRID and PACS based on Web Services (웹서비스 기반의 Grid-PACS 상호운용성 프레임워크)

  • Lee, Bong-Hwan;Cho, Hyun-Sug
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.14 no.8
    • /
    • pp.1799-1808
    • /
    • 2010
  • Recently the increasing collaborative research requires the remote medical and clinical data sharing and access of external institutions. In this paper, an interoperability framework between Grid and PACS using Web services is proposed and implemented in order to provide flexible and efficient medical data management. The Digital Imaging and Communications in Medicine(DICOM) standard defines medical image data exchange and transfer between PACSs and image databases. However, medical data exchange between hospitals is limited within the trusted and static environments. Moreover, DICOM does not provide medical data management and the Grid middleware does not include standard toolkit to access DICOM data. To address this issue, a Web services-based Grid Service Mediator (WGSM) which provides PACS integration and medical image data management is developed. The WGSM consists of several service mediators such as compress mediator, GridFTP mediator, RFT mediator, MyProxy mediator, MDS mediator, and RLS mediator and others. The proposed Web services-based framework provides user authentication and secure data access between PACSs in collaborative environments. In particular, the WGSM allows ordinary users to access remote PACS data in a simple and efficient manner without any the knowledge about underlying Grid middleware.

High Voltage Electron Microscopy DataBase System Using Data Grid (데이터 그리드를 이용한 초고전압 투과전자현미경 데이터베이스 시스템)

  • Ahn Young-Heon;Kweon Hee-Seok;Kim Youn-Joong
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2005.07a
    • /
    • pp.583-585
    • /
    • 2005
  • 바이오 및 재료 분야 등 NT(Nano Technology), BT(Bio Technology)에 관련된 연구를 보다 더 수월하게 하기 위해 본원에 설치되어 있는 초고전압 투과전자현미경(High Voltage Electron Microscopy - 이하 HVEM)을 이용한다. HVEM을 통해 획득한 이미지의 정보는 매우 방대하여 하나의 시료를 관찰하는데 있어서 수백 메가 이상의 용량을 차지하고, 연구를 수행하는데 있어서 데이터를 여러 형태로 관찰 분석하기 때문에 수월한 지원을 위해 체계적으로 관리할 수 있는 데이터베이스 시스템이 필요하다. 그러나 일반적인 범용 데이터베이스로는 이러한 대규모의 데이터를 저장할 수 없다. 따라서 본 논문에서는 이러한 용량 데이터를 체계적으로 관리할 수 있도록, 데이터 그리드와 연구 데이터의 정보를 갖는 metadata 테이블을 통해 서로 먼 거리에 있는 연구원들이 데이터를 접근하고 대규모 저장 공간을 갖는 데이터베이스 시스템을 제안한다.

  • PDF

Definition of Network Resource and Network Service Interface for Grid Network Resource Manager (그리드 네트워크 자원 관리기를 위한 네트워크 자원과 네트워크 서비스 인터페이스의 정의)

  • Kim, Hae-Hyun;Cha, Young-Wook;Han, Jang-Soo;Kim, Choon-Hee;Kong, Jong-Uk;Seok, Woo-Jin
    • The KIPS Transactions:PartC
    • /
    • v.16C no.4
    • /
    • pp.511-520
    • /
    • 2009
  • Grid technology enables high performance computation and mass data processing by connecting geographically distributed computing resources to high-speed network. To carry on grid tasks requiring guaranteed quality of service, we need to build converged infrastructure which controls and manages network resources as well as computing resources. In this paper, we defined grid network service interface and information model to support grid network path, resource creation and reservation. In addition to resource creation and reservation messages of the existing GLIF, grid network path and notification messages were defined in the grid network service interface. We also defined related statuses and messages for two-phase resource reservation mechanism. We designed and implemented network resource management system based on the grid network information model and service interface. We experimented in resource reservation, allocation and release of network resources in grid environment with GMPLS control and transport network.

A Dynamically Segmented DCT Technique for Grid Artifact Suppression in X-ray Images (X-ray 영상에서 그리드 아티팩트 개선을 위한 동적 분할 기반 DCT 기법)

  • Kim, Hyunggue;Jung, Joongeun;Lee, Jihyun;Park, Joonhyuk;Seo, Jisu;Kim, Hojoon
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.8 no.4
    • /
    • pp.171-178
    • /
    • 2019
  • The use of anti-scatter grids in radiographic imaging has the advantage of preventing the image distortion caused by scattered radiation. However, it carries the side effect of leaving artifacts in the X-ray image. In this paper, we propose a grid line suppression technique using discrete cosine transform(DCT). In X-ray images, the grid lines have different characteristics depending on the shape of the object and the area of the image. To solve this problem, we adopt the DCT transform based on a dynamic segmentation, and propose a filter transfer function for each individual segment. An algorithm for detecting the band of grid lines in frequency domain and a band stop filter(BSF) with a filter transfer function of a combination of Kaiser window and Butterworth filter have been proposed. To solve the blocking effects, we present a method to determine the pixel values using multiple structured images. The validity of the proposed theory has been evaluated from the experimental results using 140 X-ray images.

A Load Balancing Method Using Ring Network Structure in the Grid Database (그리드 데이터베이스에서 링 기반 연결 구조를 이용한 부하 분산 기법)

  • Jang Yong-Il;Shin Soong-Sun;Park Soon-Young;Bae Hae-Young
    • Journal of Korea Multimedia Society
    • /
    • v.9 no.9
    • /
    • pp.1107-1117
    • /
    • 2006
  • In this paper, a load balancing method using ring network structure in the Grid database is proposed. In the Grid database, generally, data is replicated for performance and availability. And, user's request is transferred to node and processed in that node which has target data. But, in such environment, a decline of performance can be occurred because unbalanced workload. A traditional research is proposed to solve unbalanced load problem. However, the Grid database has a number of systems and user's request always changes dynamically. Therefore, a traditional research can not be applied. The proposed method connects each node which has a same replicated data through ing network structure. If workload is overflowed in some node, user's request is transferred to a linked node which has a target data. And, this node stops another request processing until workload is decreased. Then, it changes the link structure through sending a message to a previous node, to stop request forwarding from a previous node. This paper shows a proposed method increases performance than existing research through performance evaluation and is more suitable for a complex and dynamic environment.

  • PDF

SBR-k(Sized-base replacement-k) : File Replacement in Data Grid Environments (SBR-k(Sized-based replacement-k) : 데이터 그리드 환경에서 파일 교체)

  • Park, Hong-Jin
    • The Journal of the Korea Contents Association
    • /
    • v.8 no.11
    • /
    • pp.57-64
    • /
    • 2008
  • The data grid computing provides geographically distributed storage resources to solve computational problems with large-scale data. Unlike cache replacement policies in virtual memory or web-caching replacement, an optimal file replacement policy for data grids is the one of the important problems by the fact that file size is very large. The traditional file replacement policies such as LRU(Least Recently Used), LCB-K(Least Cost Beneficial based on K), EBR(Economic-based cache replacement), LVCT(Least Value-based on Caching Time) have the problem that they have to predict requests or need additional resources to file replacement. To solve theses problems, this paper propose SBR-k(Sized-based replacement-k) that replaces files based on file size. The proposed policy considers file size to reduce the number of files corresponding to a requested file rather than forecasting the uncertain future for replacement. The results of the simulation show that hit ratio was similar when the cache size was small, but the proposed policy was superior to traditional policies when the cache size was large.

Symmetric key based user authentication between Grid Service and Portal (그리드서비스와 포털간의 대칭키 기반 사용자 단일인증에 관한 연구)

  • Hwang, Dae-Bok;Heo, Dae-Young;Hwang, Sun-Tae
    • Journal of the Korea Society of Computer and Information
    • /
    • v.12 no.3
    • /
    • pp.19-26
    • /
    • 2007
  • In recent rears. web portal system has received much attention as a user interface for the grid environment. Grid system uses symmetric key for authenticating user identity while the traditional portal system does a password-based authentication. Regarding this, many researches are progressing to integrate portal accounts with symmetric key. Specially. researches such as GAMA and PURSE are active and those focus on easy usability for users who familiar with password-based authentication. However the protection of data and resources is a critical issue in Grid environment, because those are shared through a wide-area network. In this paper, we suggest a new authentication mechanism which unify authentication mechanisms between portal system and grid service by using symmetric key. It will improve a security level in UI layer as much as in grid service.

  • PDF

A Study on Disk Cache Replacement for Data Grids Computing Environment (데이타 그리드 컴퓨팅 환경에서 디스크 캐쉬 교체 연구)

  • Park Hong-Jin
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2006.05a
    • /
    • pp.1167-1170
    • /
    • 2006
  • 웹 인프라와 P2P 기술 개념을 기반으로 연구되고 있는 데이터 그리드 컴퓨팅 환경에서 중요한 문제 중 하나는 거대한 데이터 자료 접근에 관한 이슈이다. 요청된 자원 공간을 확보하기위해 저장되어 있는 자원을 선택하는 '캐쉬 교체 정책' 연구는 많이 연구되어 왔으나, 상항을 미리 예측하거나 스택과 같은 추가적인 자원이 필요하다. 본 논문은 기존 연구의 문제점을 해결하고 요청된 파일에 대응되는 파일의 개수가 되도록 적게 하기위해 파일 크기를 고려한 캐쉬 교체 정책을 제안하고 있다.

  • PDF