Browse > Article
http://dx.doi.org/10.5391/JKIIS.2013.23.5.385

The Efficient Method of Parallel Genetic Algorithm using MapReduce of Big Data  

Hong, Sung-Sam (Department of Computer Engineering, Gachon University)
Han, Myung-Mook (Department of Computer Engineering, Gachon University)
Publication Information
Journal of the Korean Institute of Intelligent Systems / v.23, no.5, 2013 , pp. 385-391 More about this Journal
Abstract
Big Data is data of big size which is not processed, collected, stored, searched, analyzed by the existing database management system. The parallel genetic algorithm using the Hadoop for BigData technology is easily realized by implementing GA(Genetic Algorithm) using MapReduce in the Hadoop Distribution System. The previous study that the genetic algorithm using MapReduce is proposed suitable transforming for the GA by MapReduce. However, they did not show good performance because of frequently occurring data input and output. In this paper, we proposed the MRPGA(MapReduce Parallel Genetic Algorithm) using improvement Map and Reduce process and the parallel processing characteristic of MapReduce. The optimal solution can be found by using the topology, migration of parallel genetic algorithm and local search algorithm. The convergence speed of the proposal method is 1.5 times faster than that of the existing MapReduce SGA, and is the optimal solution can be found quickly by the number of sub-generation iteration. In addition, the MRPGA is able to improve the processing and analysis performance of Big Data technology.
Keywords
Big Data; Hadoop Framework; MapReduce; Parallel Genectic Algorithm; Optimization;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Dongho Song, Yougil Lee, Tae-Hyoung Kim, "A Study on Distributed Particle Swarm Optimization Algorithm with Quantum-infusion Mechanism," INTERNATIONAL JOURNAL of FUZZY LOGIC and INTELLIGENT, Vol.22, No.4, pp 527-531, 2012
2 Young Jun Kim1, Kyung Soon Hwang and Keon Myung Lee, "A MapReduce-based Algorithm for Semantic Hashing with Extended Boundaries," Proceedings of KIIS Fall Conference, Vol. 22, No. 2, 2012
3 Chong-Ho Yi and Dong W. Kim, "Comparisons of Robot-Moving Strategies with Evolutionary Algorithm and Neuro-Fuzzy Method", Journal of KIIT, Vol. 10, No. 2, pp. 227-232, 2012..
4 E. Cantu-Paz, "A survey of parallel genetic algorithms," Calculateurs Paralleles, Reseaux et Systemes Repartis, Vol. 10, No. 2, pp. 141-171, 1998.
5 Jae Hoon Cho , Dae-Jong Lee , Jin-Il Park , Myung-Geun Chun, "Hybrid Feature Selection Using Genetic Algorithm and Information Theory," INTERNATIONAL JOURNAL of FUZZY LOGIC and INTELLIGENT SYSTEMS, Vol.13 No.1, pp 69-77, 2013
6 Myung-Mook Han, "Parallel Genetic Algorithm based on a Multiprocessor System FIN and Its Application to a Classifier Machine," INTERNATIONAL JOURNAL of FUZZY LOGIC and INTELLIGENT SYSTEMS, Vol.8, No5, pp 61-71, 1998
7 Seung-Hyung Jung, Jeoung-Nae Chi, Sung-Kwun Oh, Hyun-ki Kim, "Design of Optimized Fuzzy PD Cascade Controller Based on Parallel Genetic Algorithms," INTERNATIONAL JOURNAL of FUZZY LOGIC and INTELLIGENT, Vol.19, No.3, pp 329-336, 2009
8 Dean, J. and Ghemawat, S., "MapReduce: Simplified Data Processing on Large Clusters," Communication of the ACM, Vol. 51, No. 1, pp.107-113, 2008.
9 Yun-Hee Kang amd Myoung-Woo Hong, "Sensory Data Processing by Using Hadoop Framework," Journal of KIIT, Vol. 11, No.2, pp 169-174 , 2013
10 Feng Wang, Jie Qiu, Jie Yang, Bo Dong, Xinhui Li, and Ying Li, "Hadoop high availabilitythrough metadata replication", Proceeding of the first International Workshop on Cloud datamanagement, pp. 37-44, 2009.
11 Suan Lee and Jinho Kim, "Sort-Based Distributed Parallel Data Cube Computation Algorithm using MapReduce," Journal of The Institute of Electronics Engineers of Korea, Vol. 49, NO. 9, 2012.
12 Ghemawat, S., Gobioff, H., and Leung, S. T.,"The Google File System," In Proc. 19th on Operating Systems Principles, pp. 29-43, 2003.
13 D. E. Goldberg. Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading, MA, 1989.
14 HDFS, http://hadoop.apache.org/hdfs/.
15 Hadoop, http://hadoop.apache.org/.
16 Abhishek Verma, Xavier Llor'a, David E. Goldberg, and Roy H. Campbell., "Scaling Genetic Algorithms Using MapReduce," In Proceedings of the Ninth International Conference on Intelligent Systems Design andApplications (ISDA) , 2009.
17 D. E. Goldberg. The Design of Innovation: Lessons fromand for Competent Genetic Algorithms. Kluwer Academic Publishers, Norwell, MA, 2002.
18 G. Sywerda., "Uniform crossover in genetic algorithms," Proceedings of the third international conference on Genetic algorithms, pages 2-9, San 1989.
19 J. Schaffer and L. Eshelman., "On Crossover as an Evolutionary Viable Strategy," Proceedings of the 4th International Conference on Genetic Algorithms, pp 61-68, 1991.