• Title/Summary/Keyword: HDFs

Search Result 151, Processing Time 0.033 seconds

A File Merging Scheme for Efficient Handling of Small Files in Hadoop Distributed File System (Hadoop Distribute file system에서 Small file을 효과적으로 처리하기 위한 파일 병합 기법 연구)

  • Park, Jong-Chang;Youn, Hee-Yong
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2013.11a
    • /
    • pp.15-17
    • /
    • 2013
  • HDFS(Hadoop Distribute File System)는 대용량 파일 처리를 목적으로 설계 되었으며 현재 이상적인 분산 파일 시스템으로 각광 받고 있다. 이러한 HDFS는 기존 분산파일 시스템과 많은 유사성을 가지고 있으나, Fault Tolerance를 제공하고, 데이터 엑세스 패턴을 스트리밍 방식으로 지원하여 대용량 파일을 효율적으로 저장할 수 있다는 차별성을 가지고 있다. 하지만 실제 HDFS 데이터 집합에는 Small file이 차지하는 비중이 상당히 높으며, 이러한 다수의 Small file 은 데이터 처리에 있어 높은 비용을 초래할 뿐 아니라 Master Node 의 파일 처리 및 메모리 성능에 악영향을 미친다. 따라서 본 논문에서는 HDFS에서 Small file 이 미치는 영향을 분석하고 이러한 문제점을 해결 할 수 있는 로컬 인덱스 파일기반의 파일 병합 기법을 제안한다.

Design of Distributed Cloud System for Managing large-scale Genomic Data

  • Seine Jang;Seok-Jae Moon
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.16 no.2
    • /
    • pp.119-126
    • /
    • 2024
  • The volume of genomic data is constantly increasing in various modern industries and research fields. This growth presents new challenges and opportunities in terms of the quantity and diversity of genetic data. In this paper, we propose a distributed cloud system for integrating and managing large-scale gene databases. By introducing a distributed data storage and processing system based on the Hadoop Distributed File System (HDFS), various formats and sizes of genomic data can be efficiently integrated. Furthermore, by leveraging Spark on YARN, efficient management of distributed cloud computing tasks and optimal resource allocation are achieved. This establishes a foundation for the rapid processing and analysis of large-scale genomic data. Additionally, by utilizing BigQuery ML, machine learning models are developed to support genetic search and prediction, enabling researchers to more effectively utilize data. It is expected that this will contribute to driving innovative advancements in genetic research and applications.

Anti-fouling Property of Hydrophobic Surfaces in Sea Water (소수성 표면의 해수 방오성능)

  • Cho, S.H.;Ryu, S.N.;Hwang, W.B.;Yoon, B.S.
    • Journal of the Korean Society for Marine Environment & Energy
    • /
    • v.16 no.2
    • /
    • pp.82-87
    • /
    • 2013
  • Effects of material surface property, hydrophobic or hydrophilic, on the bio-fouling occurred on the bodies submerged in the sea water are investigated experimentally. 4 test models are used in the experiment, which includes aluminum foil in common use, AAO applied hydrophobic surface, HDFS coated hydrophobic surface and hydrophilic surface. Hydrophobic surfaces with numerous micro & nano-scale pillars on it seems to play very important role of preventing them from fouling in initial stage while the effects disappear in long term sense of fouling process. It is concluded that the surface hydrophobicity retards the initial fouling until the fouling thickness is smaller than the heights of the pillars on it but the effects diminish with the fouling proceeds so that the thickness grows bigger than the pillar heights.

The Isolation and Antioxidative Effects of Vitexin from Acer palmatum

  • Kim Jin Hwa;Lee Bum Chun;Kim Jin Hui;Sim Gwan Sub;Lee Dong Hwan;Lee Kyung Eun;Yun Yeo Pyo;Pyo Hyeong Bae
    • Archives of Pharmacal Research
    • /
    • v.28 no.2
    • /
    • pp.195-202
    • /
    • 2005
  • Free radicals and reactive oxygen species (ROS) caused by UV exposure or other environmental factors are critical players in cellular damage and aging. In order to develop a new antiphotoaging agent, this work focused on the antioxidant effects of the extract of tinged autumnal leaves of Acer palmatum. One compound was isolated from an ethyl acetate soluble fraction of the A. palmatum extract using silica gel column chromatography. The chemical structure was identified as apigenin-8-C-beta-D-glucopyranoside, more commonly known as vitexin, by spectral analysis including LC-MS, FT-IR, UV, $^{1}H-$, and $^{13}C-NMR$. The biological activities of vitexin were investigated for the potential application of its anti-aging effects in the cosmetic field. Vitexin inhibited superoxide radicals by about $70\%$ at a concentration of $100\;{\mu}g/mL$ and 1,1-diphenyl-2-picrylhydrazyl (DPPH) radicals by about $60\%$ at a concentration of $100\;{\mu}g/mL$. Intracellular ROS scavenging activity was indicated by increases in dichlorofluorescein (DCF) fluorescence upon exposure to UVB $20\;mJ/cm^2$ in cultured human dermal fibroblasts (HDFs) after the treatment of vitexin. The results show that oxidation of 5-(6-)chloromethyl-2',7'-dichlo-rodihydrofluorescein diacetate ($CM-H_{2}DCFDA$) is inhibited by vitexin effectively and that vitexin has a potent free radical scavenging activity in UVB-irradiated HDFs. In ROS imaging using a confocal microscope we visualized DCF fluorescence in HDFs directly. In conclusion, our findings suggest that vitexin can be effectively used for the prevention of UV-induced adverse skin reactions such as free radical production and skin cell damage.

Service Delivery Time Improvement using HDFS in Desktop Virtualization (데스크탑 가상화에서 HDFS를 이용한 서비스 제공시간 개선 연구)

  • Lee, Wan-Hee;Lee, Bong-Hwan
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.16 no.5
    • /
    • pp.913-921
    • /
    • 2012
  • The current PC-based desktop environment is being converted into server-based virtual desktop environment due to security, mobility, and low upgrade cost. In this paper, a desktop virtualization system is implemented using an open source-based cloud computing platform and hypervisor. The implemented system is applied to the virtualziation of computer in university. In order to reduce the image transfer time, we propose a solution using HDFS. In addition, an image management structure needed for desktop virtualization is designed and implemented, and applied to a real computer lab which accommodates 30 PCs. The performance of the proposed system is evaluated in various aspects including implementation cost, power saving rate, reduction rate of license cost, and management cost. The experimental results showed that the proposed system considerably reduced the image transfer time for desktop service.

Glycation Inhibitory and Antioxidative Activities of Ergothioneine (에르고티오네인의 당화 억제 및 항산화 활성에 관한 연구)

  • Bae, Jun-Tae;Lee, Chung-Hee;Lee, Geun-Soo;Kim, Jin-Hwa;Hong, Jin-Tae
    • Journal of the Society of Cosmetic Scientists of Korea
    • /
    • v.45 no.2
    • /
    • pp.151-159
    • /
    • 2019
  • Ergothioneine has been known as an excellent antioxidant and a cellular protector against oxidative damage in vivo. In the present study, ergothioneine was demonstrated to possess antioxidant and anti-glycation activities. The radical scavenging activity of ergothioneine enhanced the viability of human dermal fibroblasts (HDFs) exposed to ultraviolet (UV) light. The UVA irradiation increased the proportion of senescence-associated ${\beta}$-galactosidase (SA-${\beta}$-gal) positive cells in comparison with the normal control group. The treatment of UVA-irradiated HDFs with ergothioneine decreased the level of SA-b-gal (by approximately 45% at an ergothioneine concentration of $400{\mu}M$) compared with the UVA-irradiated HDFs. We also found that ergothioneine inhibited production of glyceraldehyde-derived advanced glycation endproducts (AGEs) in a concentration-dependent manner. The ergothioneine educed carboxymethyl-lysine (CML) expression in comparison to the glyoxal treatment. In addition, in the Western blot analysis, treatment of glyoxal-stimulated HDFs with ergothioneine resulted in a dose-dependent decrease in the expression level of the receptor for AGE (RAGE). These results suggest that ergothioneine may have potent anti-aging effects and could be used as a cosmetic material against cellular accumulation of AGEs.

The Design of Method for Efficient Processing of Small Files in the Distributed System based on Hadoop Framework (하둡 프레임워크 기반 분산시스템 내의 작은 파일들을 효율적으로 처리하기 위한 방법의 설계)

  • Kim, Seung-Hyun;Kim, Young-Geun;Kim, Won-Jung
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.10 no.10
    • /
    • pp.1115-1122
    • /
    • 2015
  • Hadoop framework was designed to be suitable for processing very large files. On the other hand, when processing the Small Files, it waste the resource of a distributed system, and occur performance degradation. It is shown noticeable the more the Small Files. This problem is caused by the Small Files, it can be solved through the merging of associated Small Files. But a way of merging of Small Files has some limited point. in this paper, examines existing limit of merging method, design merging method Small Files for effective process.

Development of Smart Healthcare Wear System for Acquiring Vital Signs and Monitoring Personal Health (생체신호 습득과 건강 모니터링을 위한 스마트 헬스케어 의복 개발)

  • Joo, Moon-Il;Ko, Dong-Hee;Kim, Hee-Cheol
    • Journal of Korea Multimedia Society
    • /
    • v.19 no.5
    • /
    • pp.808-817
    • /
    • 2016
  • Recently, the wearable computing technology with bio-sensors has been rapidly developed and utilized in various areas such as personal health, care-giving for senior citizens who live alone, and sports activities. In particular, the wearable computing equipment to measure vital signs by means of digital yarns and bio sensors is noticeable. The wearable computing devices help users monitor and manage their health in their daily lives through the customized healthcare service. In this paper, we suggest a system for monitoring and analyzing vital signs utilizing smart healthcare clothing with bio-sensors. Vital signs that can be continuously acquired from the clothing is well-known as unstructured data. The amount of data is huge, and they are perceived as the big data. Vital sings are stored by Hadoop Distributed File System(HDFS), and one can build data warehouse for analyzing them in HDFS. We provide health monitoring system based on vital sings that are acquired by biosensors in smart healthcare clothing. We implemented a big data platform which provides health monitoring service to visualize and monitor clinical information and physical activities performed by the users.

Downregulation of matrix metalloproteinases in hyperplastic dental follicles results in abnormal tooth eruption

  • Kim, Seong-Gon;Kim, Myung-Hee;Chae, Chang-Hoon;Jung, Youn-Kwan;Choi, Je-Yong
    • BMB Reports
    • /
    • v.41 no.4
    • /
    • pp.322-327
    • /
    • 2008
  • In this study, we compared the gene expression profiles of non-syndromic hyperplastic dental follicle (HDF) fibroblasts and normal dental follicle (NDF) fibroblasts using cDNA micro-arrays, quantitative PCR, and immunohistochemical staining. Microarray analysis showed that several collagens genes were upregulated in the HDF's, including collagen types I, IV, VIII, and XI and TIMP-1, -3, and -4 (fold ratio > 2.0). In contrast, the expression of MMP-1, -3, -10, and -16 together with IL-8 was more than two fold downregulated. The differential expression of the genes encoding alkaline phosphatase, MMP-1, -3, -8, and IL-8 was confirmed by quantitative RT-PCR, while that of 24 HDFs and 18 NDFs was confirmed by immunohistochemical analysis. However, HDFs showed stronger expression of MMP-3 than NDFs (P < 0.001). Collectively, these results indicate that defective regulation of MMPs mediating connective tissue remodeling may be responsible for abnormal tooth eruption.