• Title/Summary/Keyword: Map reduce

Search Result 852, Processing Time 0.031 seconds

The Brazilian Housing program "Minha Casa Minha Vida" - A Systematic Literature Review - (브라질의 사회주택 프로그램에 관한 문헌분석연구 - "민하 카사 민하 비다" 사례를 중심으로-)

  • Kowaltowski, Doris;Granja, Ariovaldo Denis;Moreira, Daniel de Carvalho;Pina, Silvia Mikami;Oliva, Carolina Asensio;Castro, Mariana Rios
    • Journal of the Korean housing association
    • /
    • v.26 no.6
    • /
    • pp.35-42
    • /
    • 2015
  • The Brazilian Federal Housing Program "Minha Casa Minha Vida" -PMCMV (My House My Life) is part of a large housing policy to eradicate the housing deficit, estimated at around five million homes in the country. The Ministry for Cities created a network of independent research organizations to evaluate the program, and the PMCMV has attracted many studies. Despite the inherent worth of these individual studies, there is a need for a holistic evaluation. A Systematic Literature Review (SLR) is seen as an important contribution to focus research efforts on the problems identified, to avoid the repetition of studies and to provide novel insights. This article carried out a SLR to map and quantify the types of research and represent a graphic panorama of published studies. Gap spotting was part of the study. In the five-year period from 2010-2014, the SLR identified 1489 bibliographic items. The panorama is categorised according to types of publications, and the analysis of items shows that most studies are concentrated on housing policies, financial questions, urban impacts and Brazilian social development. The results of this SLR raise some new research questions and show opportunities to reduce errors in the housing projects themselves.

Reverse k-Nearest Neighbor Query Processing Method for Continuous Query Processing in Bigdata Environments (빅데이터 환경에서 연속 질의 처리를 위한 리버스 k-최근접 질의 처리 기법)

  • Lim, Jongtae;Park, Sunyong;Seo, Kiwon;Lee, Minho;Bok, Kyoungsoo;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.14 no.10
    • /
    • pp.454-462
    • /
    • 2014
  • With the development of location aware technologies and mobile devices, location-based services have been studied. To provide location-based services, many researchers proposed methods for processing various query types with Mapreduce(MR). One of the proposed methods, is a Reverse k-nearest neighbor(RkNN) query processing method with MR. However, the existing methods spend too much cost to process the continuous RkNN query. In this paper, we propose an efficient continuous RkNN query processing method with MR to resolve the problems of the existing methods. The proposed method uses the 60-degree-pruning method. The proposed method does not need to reprocess the query for continuous query processing because the proposed method draws and monitors the monitoring area including the candidate objects of a RkNN query. In order to show the superiority of the proposed method, we compare it with the query processing performance of the existing method.

Development of a Collision Risk Assessment System for Optimum Safe Route (최적안전항로를 위한 충돌위험도 평가시스템의 개발)

  • Jeon, Ho-Kun;Jung, Yun-Chul
    • Journal of the Korean Society of Marine Environment & Safety
    • /
    • v.24 no.6
    • /
    • pp.670-678
    • /
    • 2018
  • In coastal waters where the traffic volume of the ship is high, there is a high possibility of a collision accident because complicated encounter situations frequently occurs between ships. To reduce the collision accidents at sea, a quantitative collision risk assessment is required in addition to the navigator's compliance with COLREG. In this study, a new collision risk assessment system was developed to evaluate the collision risk on ship's planned sailing routes. The appropriate collision risk assessment method was proposed on the basis of reviewing existing collision risk assessment models. The system was developed using MATLAB and it consists of three parts: Map, Bumper and Assessment. The developed system was applied to the test sea area with simple computational conditions for testing and to actual sea areas with real computational conditions for validation. The results show the length of own ship, ship's sailing time and sailing routes affect collision risks. The developed system is expected to be helpful for navigators to choose the optimum safe route before sailing.

LDBAS: Location-aware Data Block Allocation Strategy for HDFS-based Applications in the Cloud

  • Xu, Hua;Liu, Weiqing;Shu, Guansheng;Li, Jing
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.12 no.1
    • /
    • pp.204-226
    • /
    • 2018
  • Big data processing applications have been migrated into cloud gradually, due to the advantages of cloud computing. Hadoop Distributed File System (HDFS) is one of the fundamental support systems for big data processing on MapReduce-like frameworks, such as Hadoop and Spark. Since HDFS is not aware of the co-location of virtual machines in the cloud, the default scheme of block allocation in HDFS does not fit well in the cloud environments behaving in two aspects: data reliability loss and performance degradation. In this paper, we present a novel location-aware data block allocation strategy (LDBAS). LDBAS jointly optimizes data reliability and performance for upper-layer applications by allocating data blocks according to the locations and different processing capacities of virtual nodes in the cloud. We apply LDBAS to two stages of data allocation of HDFS in the cloud (the initial data allocation and data recovery), and design the corresponding algorithms. Finally, we implement LDBAS into an actual Hadoop cluster and evaluate the performance with the benchmark suite BigDataBench. The experimental results show that LDBAS can guarantee the designed data reliability while reducing the job execution time of the I/O-intensive applications in Hadoop by 8.9% on average and up to 11.2% compared with the original Hadoop in the cloud.

Processing large-scale data with Apache Spark (Apache Spark를 활용한 대용량 데이터의 처리)

  • Ko, Seyoon;Won, Joong-Ho
    • The Korean Journal of Applied Statistics
    • /
    • v.29 no.6
    • /
    • pp.1077-1094
    • /
    • 2016
  • Apache Spark is a fast and general-purpose cluster computing package. It provides a new abstraction named resilient distributed dataset, which is capable of support for fault tolerance while keeping data in memory. This type of abstraction results in a significant speedup compared to legacy large-scale data framework, MapReduce. In particular, Spark framework is suitable for iterative machine learning applications such as logistic regression and K-means clustering, and interactive data querying. Spark also supports high level libraries for various applications such as machine learning, streaming data processing, database querying and graph data mining thanks to its versatility. In this work, we introduce the concept and programming model of Spark as well as show some implementations of simple statistical computing applications. We also review the machine learning package MLlib, and the R language interface SparkR.

Route Optimization Scheme in Nested NEMO Environment based on Prefix Delegation (프리픽스 할당에 기반한 중첩된 NEMO 환경에서의 경로최적화 기법)

  • Rho, Kyung-Taeg;Kang, Jeong-Jin
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.8 no.5
    • /
    • pp.95-103
    • /
    • 2008
  • The Network Mobility (NEMO) basic support protocol extends the operation of Mobile IPv6 to provide uninterrupted Internet connectivity to the communicating nodes of mobile networks. The protocol is not efficient to offer delays in data delivery and higher overheads in the case of nested mobile networks because it uses fairly sub-optimal routing and multiple encapsulation of data packets. In this paper, our scheme combining Hierarchical Mobile IPv6 (HMIPv6) functionality and Hierarchical Prefix Delegation (HPD) protocol for IPv6, which provide more effective route optimization and reduce packet header overhead and the burden of location registration for handoff. The scheme also uses hierarchical mobile network prefix (HMNP) assignment and tree-based routing mechanism to allocate the location address of mobile network nodes (MNNs) and support micro-mobility and intra-domain data communication. The performance is evaluated using NS-2.

  • PDF

A Design of Parallel Turbo Decoder based on Double Flow Method Using Even-Odd Cross Mapping (짝·홀 교차 사상을 이용한 Double Flow 기법 기반 병렬 터보 복호기 설계)

  • Jwa, Yu-Cheol;Rim, Chong-Suck
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.54 no.7
    • /
    • pp.36-46
    • /
    • 2017
  • The turbo code, an error correction code, needs a long decoding time since the same decoding process must be repeated several times in order to obtain a good BER performance. Thus, parallel processing may be used to reduce the decoding time, in which case there may be a memory contention that requires additional buffers. The QPP interleaving has been proposed to avoid such case, but there is still a possibility of memory contention when a decoder is constructed using the so-called double flow technique. In this paper, we propose an even-odd cross mapping technique to avoid memory conflicts even in decoding using the double-flow technique. This method uses the address generation characteristic of the QPP interleaving and can be used to implement the interleaving circuit between the decoding blocks and the LLR memory blocks. When the decoder implemented by applying the double flow and the proposed methods is compared with the decoder by the conventional MDF techniques, the decoding time is reduced by up to 32% with the total area increase by 8%.

Yet Another BGP Archive Forensic Analysis Tool Using Hadoop and Hive (하둡과 하이브를 이용한 BGP 아카이브 데이터의 포렌직 분석 툴)

  • Lee, Yeonhee;Lee, YoungSeok
    • Journal of KIISE
    • /
    • v.42 no.4
    • /
    • pp.541-549
    • /
    • 2015
  • A large volume of continuously growing BGP data files can raise two technical challenges regarding scalability and manageability. Due to the recent development of the open-source distributed computing infrastructure, Hadoop, it becomes feasible to handle a large amount of data in a scalable manner. In this paper, we present a new Hadoop-based BGP tool (BGPdoop) that provides the scale-out performance as well as the extensible and agile analysis capability. In particular, BGPdoop realizes a query-based BGP record exploration function using Hive on the partitioned BGP data structure, which enables flexible and versatile analytics of BGP archive files. From the experiments for the scalability with a Hadoop cluster of 20 nodes, we demonstrate that BGPdoop achieves 5 times higher performance and the user-defined analysis capability by expressing diverse BGP routing analytics in Hive queries.

GIS Landslide Hazard Mapping Using Root Strength Reinforcement Model (GIS 기반의 뿌리보강모델을 활용한 산사태 위험도 작성방법 연구)

  • Kim, Min-Gu;Yoon, Young-Jin;Chang, Hoon
    • Journal of Korean Society for Geospatial Information Science
    • /
    • v.13 no.1 s.31
    • /
    • pp.45-53
    • /
    • 2005
  • Korea is formed a mountainous district at 70% of country. Damages are occurred by landslide. It is increased annually by industrial development and extension of urban area. Serious damages occurred in Kangreng area of Gangwondo by typoon Rusa in August, 2002. Landslide hazard mapping needs to forecast and to manage landslide hazard area. GIS fits landslide hazard mapping, to reduce of time, manpower and costs, to investigate a board area, to do a spatial forecast model in study site. Variety of methods is for landslide hazard mapping. A study use root strength reinforcement model. This model use factors such as topography of slope, soil depth, soil properties, and vegetation of slope. Landslide hazard map is made by this model.

  • PDF

On the Height Growth of Several Species growing in the Middle Korea (국내주요수종의 수고생장에 대하여)

  • Ma, Sang-Kyu
    • Journal of Korean Society of Forest Science
    • /
    • v.21 no.1
    • /
    • pp.39-45
    • /
    • 1974
  • The objects of this study are to induce the experimental equation suitable to estimate the height growing process, and to get some information that could be to reduce the error percent on determining the site index. The height growing process per sample plot were made up from the height age curve getted through the stem-analysis of the dominant tree. The 11 equations were calculated in the functional formulas between the age and the dominant height of Alnus hirsuta, Larix leptolepsis, L. Gmelini, P. koraiensis, Abies holophylla, P. rigida, Q. acutissima and Q. aliena. The suitable equation for estimating the guide curve were selected with the highest correlation and the low standard error with comparision on the age-height distributed map as Fig. 2. The suitable equation is different between the species as their calculated results as follows; The growing process of tree height are occasionally found to be polymorphous in the growth pattern as Fig. 3. By this result, the site index are practically found to be always not constant but shows variation to age in same plot as example 5. So, that the site index are estimated from the height-growth curve to age drawing by the stem-analysis or the polymorphic-curve, are concluded to be suitable on the study of site evaluation.

  • PDF