Browse > Article
http://dx.doi.org/10.9708/jksci.2012.17.3.011

Inference of Context-Free Grammars using Binary Third-order Recurrent Neural Networks with Genetic Algorithm  

Jung, Soon-Ho (Dept. of Computer Engineering, Pukyong National University)
Publication Information
Journal of the Korea Society of Computer and Information / v.17, no.3, 2012 , pp. 11-25 More about this Journal
Abstract
We present the method to infer Context-Free Grammars by applying genetic algorithm to the Binary Third-order Recurrent Neural Networks(BTRNN). BTRNN is a multiple-layered architecture of recurrent neural networks, each of which is corresponding to an input symbol, and is combined with external stack. All parameters of BTRNN are represented as binary numbers and each state transition is performed with any stack operation simultaneously. We apply Genetic Algorithm to BTRNN chromosomes and obtain the optimal BTRNN inferring context-free grammar of positive and negative input patterns. This proposed method infers BTRNN, which includes the number of its states equal to or less than those of existing methods of Discrete Recurrent Neural Networks, with less examples and less learning trials. Also BTRNN is superior to the recent method of chromosomes representing grammars at recognition time complexity because of performing deterministic state transitions and stack operations at parsing process. If the number of non-terminals is p, the number of terminals q, the length of an input string k, and the max number of BTRNN states m, the parallel processing time is O(k) and the sequential processing time is O(km).
Keywords
Binary Third-order Recurrent Neural Networks; Context-Free Grammars; Genetic Algorithm;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 N. S. Choubey, and M. V. Kharat, "PDA Simulator for CFG Induction Using Genetic Algorithm," 12th Int'l Conference on Computer Modeling and Simulation, pp.92-97, March 2010.
2 H. M. Pandey, "Context Free Grammar Induction Library Using Genetic Algorithms," Int'l Conference on Computer & Communication Technology(ICCCT), pp.752-758, Sept. 2010.
3 S. Lawrence, L. Giles, and S. Fong, "Natural Language Grammatical Inference with RNN," IEEE Trans. Knowledge and Data Engineering, vo.12, No.1, JAN/FEB, 2000.
4 E. Rodrigues and H. S. Lopes, "Genetic Programming for Induciton of Context-free Grammars," 7th International Conference on Intelligent Systems Design and Applications, pp.297-302, Oct. 2007.
5 J. E. Hopcroft and J. D. Ullman, "Introduction Automata Theory, Languages, and Computation," Addison-Wesley, pp.217-232, 1979.
6 U. Harigopal, and H. C. Chen, "Grammatical Inference using Higher Order Recurrent Neural Networks," Proceedings SSST '93., 25th Southeastern Symposium on System Theory, pp.338-342, March 1993.
7 S, Jung, "Generalized Binary Second-order Recurrent Neural Networks Equivalent to Regular Grammars," Journal Of Korea Intelligent Information System Society(KIISS), vol. 12, no. 1, pp.107-123, MAR, 2006.   과학기술학회마을
8 N. S. Chouney and M. U. Kharat, "Stochastic Mutation Approach for Grammar Induction Using Genetic Algorithm," International Conference on Electronic Computer Technology (ICECT), pp.143-146, May 2010.
9 K. Fu and T. L. Booth, "Grammatical inference: Introduction and survey," IEEE Trans. Pattern Anal. & Machine Intell., vol.8, no.3, pp.343-375, May 1986.   DOI   ScienceOn
10 A. Cleeremans, D. Servan-Schreiber, and J. L. McClelland, "Finite state automata and simple recurrent networks," Neural Comp., vol.1, no.3, pp.327-381, Mar. 1989.
11 C. L. Giles, C. B. Miller, D. Chen, H. H. Chen, G. Z. Sun, and Y. C. Lee, "Second-order recurrent neural networks," Neural Comp., vol.4, no.3, pp.393-405, Mar. 1972.
12 C. L. Giles, and C. B. Miller, "The effect of higher order in recurrent neural networks: Experiments," Artificial Neural Networks for Speech and Vision, Chapman & Hall, pp.54-78, 1993.
13 C. L. Giles, C. B. Miller, D. Chen, H. H. Chen, G. Z. Sun, and Y. C. Lee, "Learning and Extracting finite state automata with second-order recurrent neural networks," Comm. ACM, vol.4, no.3, pp.393-405, March 1992.
14 Z. Zeng, R. M. Goodman, and P. Smyth, "Discrete recurrent neural networks for grammatical inference," IEEE Trans. Neural Networks, vol.5, no.2, pp.320-330, June 1994.   DOI   ScienceOn
15 M. W. Goudreau, C. L. Giles, S. T. Chakradhar, and D. Chen, "First-order versus second-order single-layer recurrent neural networks," IEEE Trans. Neural Networks, vol.5, no.3, pp.511-513, May 1994.   DOI   ScienceOn
16 S. Jung and H. Yoon, "Binary Second-Order Recurrent Neural Networks for Inferring Regular Grammars," IEICE Trans. on Info. and Sys., vol.E83-D, no.11, pp.1996-2007, Nov. 2000.
17 S. Porat and J. A. Feldman, "Learning automata from ordered examples," Machine Learning, vol.7, no. 2-3, pp.109-138, Sep. 1991.   DOI   ScienceOn
18 I. Tsoulos, D. Gavrilis, and E. Glavas, "Neural network construction and training using grammatical evolution," Neurocomputing, pp.1-9, Dec. 2008.
19 P. Wyard, "Context Free Grammar Induction Using Genetic Algorithm," 1994, Proceedings, Sixth Int'l Conference on Tools with Artificial Intelligence, pp.828-831, April 1994.
20 J. L. Elman, "Finding structure in time," Cognitive Science, vol.14, pp.179-211, Apr./Jun. 1990   DOI   ScienceOn