• Title/Summary/Keyword: HDFs

Search Result 151, Processing Time 0.027 seconds

Visual Semantic Based 3D Video Retrieval System Using HDFS

  • Ranjith Kumar, C.;Suguna, S.
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.8
    • /
    • pp.3806-3825
    • /
    • 2016
  • This paper brings out a neoteric frame of reference for visual semantic based 3d video search and retrieval applications. Newfangled 3D retrieval application spotlight on shape analysis like object matching, classification and retrieval not only sticking up entirely with video retrieval. In this ambit, we delve into 3D-CBVR (Content Based Video Retrieval) concept for the first time. For this purpose we intent to hitch on BOVW and Mapreduce in 3D framework. Here, we tried to coalesce shape, color and texture for feature extraction. For this purpose, we have used combination of geometric & topological features for shape and 3D co-occurrence matrix for color and texture. After thriving extraction of local descriptors, TB-PCT (Threshold Based- Predictive Clustering Tree) algorithm is used to generate visual codebook. Further, matching is performed using soft weighting scheme with L2 distance function. As a final step, retrieved results are ranked according to the Index value and produce results .In order to handle prodigious amount of data and Efficacious retrieval, we have incorporated HDFS in our Intellection. Using 3D video dataset, we fiture the performance of our proposed system which can pan out that the proposed work gives meticulous result and also reduce the time intricacy.

RDP: A storage-tier-aware Robust Data Placement strategy for Hadoop in a Cloud-based Heterogeneous Environment

  • Muhammad Faseeh Qureshi, Nawab;Shin, Dong Ryeol
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.9
    • /
    • pp.4063-4086
    • /
    • 2016
  • Cloud computing is a robust technology, which facilitate to resolve many parallel distributed computing issues in the modern Big Data environment. Hadoop is an ecosystem, which process large data-sets in distributed computing environment. The HDFS is a filesystem of Hadoop, which process data blocks to the cluster nodes. The data block placement has become a bottleneck to overall performance in a Hadoop cluster. The current placement policy assumes that, all Datanodes have equal computing capacity to process data blocks. This computing capacity includes availability of same storage media and same processing performances of a node. As a result, Hadoop cluster performance gets effected with unbalanced workloads, inefficient storage-tier, network traffic congestion and HDFS integrity issues. This paper proposes a storage-tier-aware Robust Data Placement (RDP) scheme, which systematically resolves unbalanced workloads, reduces network congestion to an optimal state, utilizes storage-tier in a useful manner and minimizes the HDFS integrity issues. The experimental results show that the proposed approach reduced unbalanced workload issue to 72%. Moreover, the presented approach resolve storage-tier compatibility problem to 81% by predicting storage for block jobs and improved overall data block placement by 78% through pre-calculated computing capacity allocations and execution of map files over respective Namenode and Datanodes.

The Effect of a Long-Term Cyclic Strain on Human Dermal Fibroblasts Cultured in a Bioreactor on Chitosan-Based Scaffolds for the Development of Tissue Engineered Artificial Dermis

  • Lim, Sae-Hwan;Son, Young-Sook;Kim, Chun-Ho;Shin, Heung-Soo;Kim, Jong-Il
    • Macromolecular Research
    • /
    • v.15 no.4
    • /
    • pp.370-378
    • /
    • 2007
  • Mechanical stimulation is known to activate several cellular signal transduction pathways, leading to the induction of signaling molecules and extracellular matrix (ECM) proteins, thereby modulating cellular activities, such as proliferation and survival. In this study, primary human dermal fibroblasts (HDFs) were seeded onto chitosan-based scaffolds, and then cultured for 3 weeks in a bioreactor under a cyclic strain of 1 Hz frequency. Compared to control samples cultured under static conditions, the application of a cyclic strain stimulated the proliferation of HDFs in I week, and by week 3 the thickness of the cell/scaffold composites increased 1.56 fold. Moreover, immunohistochemical staining of the culture media obtained from the cell/scaffold samples subjected to the cyclic strain, revealed increases in the expression and secretion of ECM proteins, such as fibronectin and collagen. These results suggest that the preconditioning of cell/scaffold composites with a cyclic strain may enhance the proliferation of HDFs, and even facilitate integration of the engineered artificial dermal tissue into the host graft site.

A Study on Phon Call Big Data Analytics (전화통화 빅데이터 분석에 관한 연구)

  • Kim, Jeongrae;Jeong, Chanki
    • Journal of Information Technology and Architecture
    • /
    • v.10 no.3
    • /
    • pp.387-397
    • /
    • 2013
  • This paper proposes an approach to big data analytics for phon call data. The analytical models for phon call data is composed of the PVPF (Parallel Variable-length Phrase Finding) algorithm for identifying verbal phrases of natural language and the word count algorithm for measuring the usage frequency of keywords. In the proposed model, we identify words using the PVPF algorithm, and measure the usage frequency of the identified words using word count algorithm in MapReduce. The results can be interpreted from various viewpoints. We design and implement the model based HDFS (Hadoop Distributed File System), verify the proposed approach through a case study of phon call data. So we extract useful results through analysis of keyword correlation and usage frequency.

Access efficiency of small sized files in Big Data using various Techniques on Hadoop Distributed File System platform

  • Alange, Neeta;Mathur, Anjali
    • International Journal of Computer Science & Network Security
    • /
    • v.21 no.7
    • /
    • pp.359-364
    • /
    • 2021
  • In recent years Hadoop usage has been increasing day by day. The need of development of the technology and its specified outcomes are eagerly waiting across globe to adopt speedy access of data. Need of computers and its dependency is increasing day by day. Big data is exponentially growing as the entire world is working in online mode. Large amount of data has been produced which is very difficult to handle and process within a short time. In present situation industries are widely using the Hadoop framework to store, process and produce at the specified time with huge amount of data that has been put on the server. Processing of this huge amount of data having small files & its storage optimization is a big problem. HDFS, Sequence files, HAR, NHAR various techniques have been already proposed. In this paper we have discussed about various existing techniques which are developed for accessing and storing small files efficiently. Out of the various techniques we have specifically tried to implement the HDFS- HAR, NHAR techniques.

Antioxidative Activity of Extracts of Acanthopanax divaricatus var. albeofructus Leaves in Human Dermal Fibroblast Irradiated by UVA (자외선이 조사된 사람피부 섬유아세포에서 흰털오가피 잎추출물의 항산화작용)

  • Shin, Ai-Hyang;Lyu, Su-Yun;Noh, Bin-Na;Kim, Ja-In;Kim, Ok-Kyoung;Park, Won-Bong
    • YAKHAK HOEJI
    • /
    • v.51 no.4
    • /
    • pp.229-234
    • /
    • 2007
  • We investigated antioxidative activity of the water and ethanol extracts of leaves of Acanthopanax divaricatus var. albeofructus in human dermal fibroblast (HDFs) irradiated by UVA. The irradiation of UVA did not affect the cell viability of HDFs. The antioxidative activity of the extract was investigated by xylenol orange, TBARS (thiobarbituric acid reactive substances) and antioxidant enzyme assay. Both extracts showed H202 scavenging activity and inhibited lipid peroxidation in HDF cells irradiated by UVA. The extracts also recovered enzyme activity in the same cells.

Distributed Cache for High-Performance in real time cloud (실시간 클라우드 환경에서 HDFS의 고 성능을 위한 분산캐시)

  • Choi, Ji Hyeon;Youn, Hee Yong
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2014.07a
    • /
    • pp.351-352
    • /
    • 2014
  • 분산 파일시스템은 서로 분산된 여러 서버들을 가지고 파일 시스템을 구성함으로써 높은 확장성과 고가용성을 지원한다. HDFS는 대용량 데이터 저장장치로 처리되고 있지만 실시간 파일 접근에 관한 고려는 부족하다. 파일을 읽을 때 네임노드와 데이터 노드는 상호 작용을 하지만 엄청난 대용량의 데이터 그리고 동시작업량이 많을 때 접근수행속가 급격하게 감소하게 된다. 따라서 실시간 클라우드 서비스 환경에서 HDFS 파일 접근 수행속도를 향상시키기 위한 연구가 이슈이다. 본 논문에서는 HDFS의 위에 분산 캐시를 둔 새로운 캐시시스템을 제안한다.

  • PDF

A Design of the Small File Grouping System Based on Naive Bayesian Classifier Model (나이브 베이지안 분류기 모델 기반의 소용량 파일 그룹화 시스템 설계)

  • Kim, Min-Jae;Kim, Kyung-Tae;Youn, Hee-Young
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2014.07a
    • /
    • pp.221-222
    • /
    • 2014
  • 빠른 웹의 성장으로 대용량 데이터를 효과적으로 처리할 수 있는 플랫폼 기술에 대한 관심이 높아지고 있다. 특히, HDFS는 이상적인 분산 파일 시스템으로 각광받고 있으며 대용량 파일의 처리를 목적으로 개발되었다. 하지만, 실제 파일들의 집합에서 소용량 파일이 차지하는 비중은 높은 편이다. 많은 수의 소용량 파일은 HDFS 성능 감소에 치명적인 원인이 된다. 많은 수의 소용량 파일들이 HDFS에 저장된다면 NameNode의 메모리 소비량이 증가하게 되며 많은 수의 소용량 파일은 많은 수의 DataNode와 NameNode를 요구하므로 상대적으로 처리시간이 많이 소모된다. 따라서 본 논문에서는 HDFS에서 소용량 파일의 저장과 액세스 효율성을 향상시키기 위하여 나이브 베이지안 분류기 알고리즘을 적용한 파일 그룹화 시스템을 설계하였다.

  • PDF

Design of RDBMS-based HDFS ACLs (RDBMS 기반 HDFS ACL 설계)

  • Son, Siwoon;Gil, Myeong-Seon;Moon, Yang-Sae;Nguyen, Minh Chau;Won, Hee-Sun
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2015.04a
    • /
    • pp.697-699
    • /
    • 2015
  • 본 논문에서는 하둡의 인메모리 기반 ACL(access control list)을 RDBMS 기반으로 관리하도록 기존 하둡을 재설계하였다. 기존 하둡은 ACL을 인메모리에서 관리하기 때문에 대용량 ACL 정보를 관리함에 있어 메모리 오버헤드, ACL 정보 관리의 비효율성 등 몇 가지 문제가 발생할 수 있다. 본 논문에서는 ACL 관리에 RDBMS를 사용함으로써 메모리 크기에 종속되지 않으며, 외부 응용 프로그램에서도 쉽고 일관성있게 ACL 정보를 관리할 수 있다. 이 같은 결과에 따라, 본 논문은 빅데이터를 하둡에서 안정하게 관리할 수 있는 우수한 연구 설계 결과라 생각된다.

Efficient Multimedia Data File Management and Retrieval Strategy on Big Data Processing System

  • Lee, Jae-Kyung;Shin, Su-Mi;Kim, Kyung-Chang
    • Journal of the Korea Society of Computer and Information
    • /
    • v.20 no.8
    • /
    • pp.77-83
    • /
    • 2015
  • The storage and retrieval of multimedia data is becoming increasingly important in many application areas including record management, video(CCTV) management and Internet of Things (IoT). In these applications, the files containing multimedia that need to be stored and managed is tremendous and constantly scaling. In this paper, we propose a technique to retrieve a very large number of files, in multimedia format, using the Hadoop Framework. Our strategy is based on the management of metadata that describes the characteristic of files that are stored in Hadoop Distributed File System (HDFS). The metadata schema is represented in Hbase and looked up using SQL On Hadoop (Hive, Tajo). Both the Hbase, Hive and Tajo are part of the Hadoop Ecosystem. Preliminary experiment on multimedia data files stored in HDFS shows the viability of the proposed strategy.