• Title/Summary/Keyword: HDFs

Search Result 151, Processing Time 0.054 seconds

Design of a Sentiment Analysis System to Prevent School Violence and Student's Suicide (학교폭력과 자살사고를 예방하기 위한 감성분석 시스템의 설계)

  • Kim, YoungTaek
    • The Journal of Korean Association of Computer Education
    • /
    • v.17 no.6
    • /
    • pp.115-122
    • /
    • 2014
  • One of the problems with current youth generations is increasing rate of violence and suicide in their school lives, and this study aims at the design of a sentiment analysis system to prevent suicide by uising big data process. The main issues of the design are economical implementation, easy and fast processing for the users, so, the open source Hadoop system with MapReduce algorithm is used on the HDFS(Hadoop Distributed File System) for the experimentation. This study uses word count method to do the sentiment analysis with informal data on some sns communications concerning a kinds of violent words, in terms of text mining to avoid some expensive and complex statistical analysis methods.

  • PDF

MRSPAKE : A Web-Scale Spatial Knowledge Extractor Using Hadoop MapReduce (MRSPAKE : Hadoop MapReduce를 이용한 웹 규모의 공간 지식 추출기)

  • Lee, Seok-Jun;Kim, In-Cheol
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.11
    • /
    • pp.569-584
    • /
    • 2016
  • In this paper, we present a spatial knowledge extractor implemented in Hadoop MapReduce parallel, distributed computing environment. From a large spatial dataset, this knowledge extractor automatically derives a qualitative spatial knowledge base, which consists of both topological and directional relations on pairs of two spatial objects. By using R-tree index and range queries over a distributed spatial data file on HDFS, the MapReduce-enabled spatial knowledge extractor, MRSPAKE, can produce a web-scale spatial knowledge base in highly efficient way. In experiments with the well-known open spatial dataset, Open Street Map (OSM), the proposed web-scale spatial knowledge extractor, MRSPAKE, showed high performance and scalability.

Mechanism to Select the Data Source of HDFS with SSD Cache Based on Storage I / O Cost (SSD 캐시를 적용한 HDFS의 I/O 비용 기반 데이터 선택 기법)

  • Kim, Minkyung;Shin, Mincheol;Park, Sanghyun
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2015.04a
    • /
    • pp.676-679
    • /
    • 2015
  • 빅데이터 분석을 위한 Hadoop 환경에서 고성능 저장장치인 SSD에 대한 중요성이 증가하면서 일반적으로 사용되는 저장장치인 HDD와 혼합하여 사용하는 연구들이 주목 받고 있다. 특히 SSD를 HDD의 캐시로 사용했을 때 저장장치에 대한 I/O 성능을 향상할 수 있다는 연구 결과들이 있다. 본 연구는 이를 바탕으로 SSD를 HDD의 캐시로 사용한다. HDFS는 저장장치에 접근하여 I/O를 수행하는데 기존에는 로컬 서버에서 캐시 미스가 발생한 경우 로컬 HDD로 접근한다. 이러한 방식은 접근하는 데이터에 따라 SSD의 높은 Bandwidth를 활용하지 못하게 되는 경우를 발생시키고 그 결과 특정 서버의 I/O 지연으로 전체 분산 처리의 성능을 저하시킬 수 있다. 이를 해결하기 위해 본 연구는 HDFS 레벨에서 로컬 서버의 HDD와 데이터 복제본들이 저장된 원격 서버의 SSD에서 I/O를 수행하는 경우에 대해 수식을 통해 비용을 비교한다. 그 결과 항상 기대 성능이 높은 저장 장치를 선택하여 데이터를 읽어오게 함으로써 기존 방식보다 성능이 개선될 수 있음을 입증한다.

Search for a user-centered system design and implementation (사용자 중심 검색 시스템 설계 및 구현)

  • Kim, A-Yong;Park, Man-Seub;Kim, Jong-Moon;Jeong, Dae-Jin;Jung, Hoe-kyung
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2014.05a
    • /
    • pp.619-621
    • /
    • 2014
  • addition to the advances in information technology and the latest IT technology for their issue. To enable users who are using the Web to find need the information your search data they're sifting through about how many are struggling. In this paper, we propose a user-centered search system. Lucene search system to offer Hadoop's MapReduce with the Apache project Nutch, Solr, HDFS, utilizing design and implementation. This is the Web search users who wish to use depending on the intentions of the data that you want to collect and index information will be utilized in the search field.

  • PDF

Sim-Hadoop : Leveraging Hadoop Distributed File System and Parallel I/O for Reliable and Efficient N-body Simulations (Sim-Hadoop : 신뢰성 있고 효율적인 N-body 시뮬레이션을 위한 Hadoop 분산 파일 시스템과 병렬 I / O)

  • Awan, Ammar Ahmad;Lee, Sungyoung;Chung, Tae Choong
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2013.05a
    • /
    • pp.476-477
    • /
    • 2013
  • Gadget-2 is a scientific simulation code has been used for many different types of simulations like, Colliding Galaxies, Cluster Formation and the popular Millennium Simulation. The code is parallelized with Message Passing Interface (MPI) and is written in C language. There is also a Java adaptation of the original code written using MPJ Express called Java Gadget. Java Gadget writes a lot of checkpoint data which may or may not use the HDF-5 file format. Since, HDF-5 is MPI-IO compliant, we can use our MPJ-IO library to perform parallel reading and writing of the checkpoint files and improve I/O performance. Additionally, to add reliability to the code execution, we propose the usage of Hadoop Distributed File System (HDFS) for writing the intermediate (checkpoint files) and final data (output files). The current code writes and reads the input, output and checkpoint files sequentially which can easily become bottleneck for large scale simulations. In this paper, we propose Sim-Hadoop, a framework to leverage HDFS and MPJ-IO for improving the I/O performance of Java Gadget code.

Effects of human collagen α-1 type I-derived proteins on collagen synthesis and elastin production in human dermal fibroblasts

  • Hwang, Su Jin;Kim, Su Hwan;Seo, Woo-Young;Jeong, Yelin;Shin, Min Cheol;Ryu, Dongryeol;Lee, Sang Bae;Choi, Young Jin;Kim, KyeongJin
    • BMB Reports
    • /
    • v.54 no.6
    • /
    • pp.329-334
    • /
    • 2021
  • Collagen type I is the most abundant form of collagen in human tissues, and is composed of two identical α-1 type I chains and an α-2 type I chain organized in a triple helical structure. A previous study has shown that human collagen α-2 type I (hCOL1A2) promotes collagen synthesis, wound healing, and elastin production in normal human dermal fibroblasts (HDFs). However, the biological effects of human collagen α-1 type I (hCOL1A1) on various skin properties have not been investigated. Here, we isolate and identify the hCOL1A1-collagen effective domain (CED) which promotes collagen type I synthesis. Recombinant hCOL1A1-CED effectively induces cell proliferation and collagen biosynthesis in HDFs, as well as increased cell migration and elastin production. Based on these results, hCOL1A1-CED may be explored further for its potential use as a preventative agent against skin aging.

Analysis of identification of Spectrum for HDFSS (HDFSS 주파수 분배 동향 분석)

  • Oh, D.S.;Ahn, D.S.
    • Electronics and Telecommunications Trends
    • /
    • v.17 no.5 s.77
    • /
    • pp.149-156
    • /
    • 2002
  • 2000년에 개최된 세계전파통신회의에서는 차기 회의까지 글로벌 환경에서의 고밀도 고정위성업무를 위한 주파수 분배에 대한 연구를 의제로 결정하였다. 이후 ITU-R 회의에서는 17.3GHz 대역 이상의 주파수 대역에서 HDFSS에 적합한 주파수 대역을 연구하고 있는 중이다. 본 고에서에서는 국내 주파수 분배를 고려하여 적절한 HDFSS 주파수 대역을 고찰하고, 외국의 주파수 분배 현황에 대해 비교 검토하였다.

Adaptive Cache Management Scheme in HDFS (HDFS에서 적응형 캐시 관리 기법)

  • Choi, Hyoung-Rak;Yoo, Jae-Soo
    • Proceedings of the Korea Contents Association Conference
    • /
    • 2019.05a
    • /
    • pp.461-462
    • /
    • 2019
  • 스마트팩토리는 정보통신기술(ICT)를 이용한 공정의 모든 데이터를 수집, 분석하여 제어하고 있다. 기존보다 방대한 양의 데이터를 처리하기 위해 기업들은 하둡을 이용한다. 다양한 크기의 데이터가 나타나는 환경에서 HDFS을 효율적으로 관리하기 위한 적응형 캐시 관리 기법을 제안한다. 제안하는 기법은 데이터 노드의 로컬 디스크의 공간 이용 효율성을 높이고 평균 데이터 크기를 분석하여 데이터 노드 확장시 적합한 블록 크기를 적용할 수 있게 관리한다. 성능 평가를 통해 제안하는 기법의 데이터 노드에서 로컬 디스크 효율 향상과 읽기와 쓰기 속도의 속도에 효과를 보인다.

  • PDF

Ferment Red Ginseng Suppresses the Expression of Matrix Metalloproteinases in UVA-irradiated Human Dermal Fibroblast Cells (발효홍삼의 인간진피섬유모세포에서 UVA로 유도한 염증 및 기질단백분해효소 발현 억제 효능)

  • Lee, Keun-Hyeun;Jeong, Seung-Il;Lee, Chang-Hyun;Shin, Sang Woo;Jeong, Han-Sol
    • Journal of Physiology & Pathology in Korean Medicine
    • /
    • v.31 no.2
    • /
    • pp.105-110
    • /
    • 2017
  • Prolonged exposure to solar ultraviolet A (UVA) radiation has been known to cause premature skin aging (photo-aging). UVA radiation generates ROS thereby induce degenerative changes of skin such as degradation of dermal collagen, elastic fibers. Matrix metalloproteinases (MMPs), the proteolytic enzymes have been implicated as a major player in the development of UVA-induced photo-aging. Many studies have been conducted to block the harmful effects of UV radiation on the skin. Recently, we are interested in the availability of fermented red ginseng (FRG) as natural matrix metalloproteinases inhibitors (MMPIs). The efficacy difference between red ginseng and FRG has been compared. Both RG and FRG have no cytotoxic effects below the concentration of $300{\mu}g/ml$. Human dermal fibroblasts (HDFs) were pretreated with FRG or RG for 24h, followed by irradiation of UVA. Then, we measured the intracellular ROS production and the expression of MMP, $IL-1{\beta}$ at the mRNA level. We also examined the intracellular localization of $NF-{\kappa}B$ and MMP-9 on the FRG or RG treated and UVA-irradiated HDFs. FRG decreased the intracellular ROS production elicited by UVA. In addition, FRG decreased the mRNA expression of MMP-3, MMP-9, and $IL-1{\beta}$ more efficiently than RG. Furthermore, FRG suppressed the nuclear localization of $NF-{\kappa}B$, and the expression of MMP-9. Taken together, our results suggest that FRG is promising agents to prevent UVA-induced photo-aging by suppressing MMP expression and inflammation.

A Security Log Analysis System using Logstash based on Apache Elasticsearch (아파치 엘라스틱서치 기반 로그스태시를 이용한 보안로그 분석시스템)

  • Lee, Bong-Hwan;Yang, Dong-Min
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.22 no.2
    • /
    • pp.382-389
    • /
    • 2018
  • Recently cyber attacks can cause serious damage on various information systems. Log data analysis would be able to resolve this problem. Security log analysis system allows to cope with security risk properly by collecting, storing, and analyzing log data information. In this paper, a security log analysis system is designed and implemented in order to analyze security log data using the Logstash in the Elasticsearch, a distributed search engine which enables to collect and process various types of log data. The Kibana, an open source data visualization plugin for Elasticsearch, is used to generate log statistics and search report, and visualize the results. The performance of Elasticsearch-based security log analysis system is compared to the existing log analysis system which uses the Flume log collector, Flume HDFS sink and HBase. The experimental results show that the proposed system tremendously reduces both database query processing time and log data analysis time compared to the existing Hadoop-based log analysis system.