International Journal of Fuzzy Logic and Intelligent Systems

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지
DOI QR코드

DOI QR Code

On Neural Fuzzy Systems

  • Su, Shun-Feng (Department of Electrical Engineering, National Taiwan University of Science and Technology) ;
  • Yeh, Jen-Wei (Department of Electrical Engineering, National Taiwan University of Science and Technology)
  • Received : 2014.11.05
  • Accepted : 2014.12.07
  • Published : 2014.12.25
Download PDF
⟨ Previous Next ⟩

Abstract

Neural fuzzy system (NFS) is basically a fuzzy system that has been equipped with learning capability adapted from the learning idea used in neural networks. Due to their outstanding system modeling capability, NFS have been widely employed in various applications. In this article, we intend to discuss several ideas regarding the learning of NFS for modeling systems. The first issue discussed here is about structure learning techniques. Various ideas used in the literature are introduced and discussed. The second issue is about the use of recurrent networks in NFS to model dynamic systems. The discussion about the performance of such systems will be given. It can be found that such a delay feedback can only bring one order to the system not all possible order as claimed in the literature. Finally, the mechanisms and relative learning performance of with the use of the recursive least squares (RLS) algorithm are reported and discussed. The analyses will be on the effects of interactions among rules. Two kinds of systems are considered. They are the strict rules and generalized rules and have difference variances for membership functions. With those observations in our study, several suggestions regarding the use of the RLS algorithm in NFS are presented.

Keywords

References

  1. B. Kosko, Neural Networks and Fuzzy Systems: A Dynamical Systems Approach to Machine Intelligence, Englewood Ciffs, NJ: Prentice Hall, 1992.
  2. C. T. Lin and C. S. G. Lee, Neural Fuzzy Systems: A Neuro-Fuzzy Synergism to Intelligent Systems, Upper Saddle River, NJ: Prentice Hall PTR, 1996.
  3. K. Hornik, M. Stinchcombe, and H. White, "Multilayer feedforward networks are universal approximators," Neural Networks, vol. 2, no. 5, pp. 359-366, 1989. http://dx.doi.org/10.1016/0893-6080(89)90020-8
  4. L. A. Zadeh, "Outline of a new approach to the analysis of complex systems and decision processes," IEEE Transactions on Systems, Man and Cybernetics, vol. SMC-3, no. 1, pp. 28-44, Jan. 1973. http://dx.doi.org/10.1109/TSMC.1973.5408575
  5. W. Pedrycz, "Structured fuzzy models," Cybernetics and Systems, vol. 16, no. 1, pp. 103-117, Jan. 1985. http://dx. doi.org/10.1080/01969728508927757
  6. L. X. Wang and J. M. Mendel, "Generating fuzzy rules by learning from examples," IEEE Transactions on Systems, Man and Cybernetics, vol. 22, no. 6, pp. 1414-1427, Nov. 1992. http://dx.doi.org/10.1109/21.199466
  7. R. R. Yager and D. P. Filev, Essentials of Fuzzy Modeling and Control, New York, NY: Wiley, 1994.
  8. S. Horikawa, T. Furuhashi, and Y. Uchikawa, "On fuzzy modeling using fuzzy neural networks with the backpropagation algorithm," IEEE Transactions on Neural Networks, vol. 3, no. 5, pp. 801-806, Sep. 1992. http://dx.doi.org/10.1109/72.159069
  9. K. Tanaka, M. Sano, and H. Watanabe, "Modeling and control of carbon monoxide concentration using a neurofuzzy technique," IEEE Transactions on Fuzzy Systems, vol. 3, no. 3, pp. 271-279, Aug. 1995. http://dx.doi.org/10.1109/91.413233
  10. Y. Lin and G. A. Cunningham, III, "A new approach to fuzzy-neural system modeling," IEEE Transactions on Fuzzy Systems, vol. 3, no. 2, pp. 190-198, May 1995. http://dx.doi.org/10.1109/91.388173
  11. J. S. R. Jang, C. T. Sun, and E. Mizutani, Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence, Upper Saddle River, NJ: Prentice Hall, 1997.
  12. S. F. Su and K. Y. Chen, "Conceptual discussions and benchmark comparison for neural networks and fuzzy systems," Differential Equations and Dynamical Systems, vol. 13, no. 1, pp. 35-61, 2005.
  13. C. F. Juang and C. T. Lin, "An online self-constructing neural fuzzy inference network and its applications," IEEE Transactions on Fuzzy Systems, vol. 6, no. 1, pp. 12-32, Feb. 1998. http://dx.doi.org/10.1109/91.660805
  14. S. F. Su and F. Y. P. Yang, "On the dynamical modeling with neural fuzzy networks," IEEE Transactions on Neural Networks, vol. 13, no. 6, pp. 1548-1553, Nov. 2002. http://dx.doi.org/10.1109/TNN.2002.804313
  15. J. S. R. Jang, "ANFIS: adaptive-network-based fuzzy inference system," IEEE Transactions on Systems, Man and Cybernetics, vol. 23, no. 3, pp. 665-685, May 1993. http://dx.doi.org/10.1109/21.256541
  16. T. Takagi and M. Sugeno, "Fuzzy identification of systems and its applications to modeling and control," IEEE Transactions on Systems, Man and Cybernetics, vol. SMC-15, no. 1, pp. 116-132, Jan. 1985. http://dx.doi.org/10.1109/TSMC.1985.6313399
  17. C. T. Lin and C. S. G. Lee, "Neural-network-based fuzzy logic control and decision system," IEEE Transactions on Computers, vol. 40, no. 12, pp. 1320-1336, Dec. 1991. http://dx.doi.org/10.1109/12.106218
  18. J. W. Yeh, S. F. Su, J. T. Jeng, and B. S. Chen, "On learning analysis of neural fuzzy systems," in Proceedings of the IEEE International Conference on Fuzzy Systems (FUZZ), Barcelona, Italy, July 18-23, 2010, pp. 1-6. http://dx.doi.org/10.1109/FUZZY.2010.5584389
  19. J. W. Yeh, S. F. Su, and I. Rudas, "Analysis of using RLS in neural fuzzy systems," in Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Anchorage, AK, October 9-12, 2011, pp. 1831-1836. http://dx.doi.org/10.1109/ICSMC.2011.6083937
  20. J. W. Yeh and S. F. Su, "Learning analysis for correlation of fuzzy rules in applying RLS for neural fuzzy systems," in Proceedings of the IEEE International Conference on Granular Computing, Hangzhou, China, August 11-13, 2012, pp. 609-613. http://dx.doi.org/10.1109/GrC.2012.6468690
  21. Y. Zhang and X. R. Li, "A fast U-D factorization-based learning algorithm with applications to nonlinear system modeling and identification," IEEE Transactions on Neural Networks, vol. 10, no. 4, pp. 930-938, Jul. 1999. http://dx.doi.org/10.1109/72.774266
  22. Y. S. Cho, S. B. Kim, and E. J. Powers, "Time-varying spectral estimation using AR models with variable forgetting factors," IEEE Transactions on Signal Processing, vol. 39, no. 6, pp. 1422-1426, Jun. 1991. http://dx.doi.org/10.1109/78.136549
  23. S. R. Huang, "Analysis of model-free estimators: applications to stock market with the use of technical indices," M.S. thesis, National Taiwan University of Science and Technology, Taipei, Taiwan, 1999.
  24. T. Kohonen, Self-organization and Associative Memory, 3rd ed., New York, NY: Springer-Verlag, 1989.
  25. J. A. Dickerson and B. Kosko, "Fuzzy function approximation with ellipsoidal rules," IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 26, no. 4, pp. 542-560, Aug. 1996. http://dx.doi.org/10.1109/3477.517030
  26. A. Kroll, "Identification of functional fuzzy models using multidimensional reference fuzzy sets," Fuzzy Sets and Systems, vol. 80, no. 2, pp. 149-158, Jun. 1996. http://dx. doi.org/10.1016/0165-0114(95)00140-9
  27. F. Klawonn and R. Kruse, "Constructing a fuzzy controller from data," Fuzzy Sets and Systems, vol. 85, no. 2, pp. 177-193, Jan. 1997. http://dx.doi.org/10.1016/0165-0114(95)00350-9
  28. E. Kim, M. Park, S. Ji, and M. Park, "A new approach to fuzzy modeling," IEEE Transactions on Fuzzy Systems, vol. 5, no. 3, pp. 328-337, Aug. 1997. http://dx.doi.org/10.1109/91.618271
  29. C. C. Chuang, S. F. Su, and S. S. Chen, "Robust TSK fuzzy modeling for function approximation with outliers," IEEE Transactions on Fuzzy Systems, vol. 9, no. 6, pp. 810-821, Dec. 2001. http://dx.doi.org/10.1109/91.971730
  30. H. Frigui and R. Krishnapuram, "A robust competitive clustering algorithm with applications in computer vision," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 5, pp. 450-465, May 1999. http://dx.doi.org/10.1109/34.765656
  31. D. M. Hawkins, Identification of Outliers, New York, NY: Chapman and Hall, 1980.
  32. M. Smith, Neural Networks for Statistical Modeling, New York, NY: Van Nostrand Reinhold, 1993.
  33. W. S. Sarle, "Stopped training and other remedies for overfitting," in Proceedings of the 27th Symposium on the Interface of Computing Science and Statistics, Pittsburgh, PA, June 21-24, 1995, pp. 352-360.
  34. P. L. Bartlett, "For valid generalization, the size of the weights is more important than the size of the network," Advances in Neural Information Processing Systems, vol. 9, pp. 134-140, 1996.
  35. J. C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms, New York, NY: Plenum Press, 1981.
  36. A. K. Jain and R. C. Dubes, Algorithms for Clustering Data, Englewood Cliffs, NJ: Prentice Hall, 1988.
  37. R. N. Dave and R. Krishnapuram, "Robust clustering methods: a unified view," IEEE Transactions on Fuzzy Systems, vol. 5, no. 2, pp. 270-293, May 1997. http://dx.doi.org/10.1109/91.580801
  38. A. Cichocki and R. Unbehauen, Neural Networks for Optimization and Signal Processing, New York, NY: J. Wiley, 1993.
  39. P. J. Rousseeuw and A. M. Leroy, Robust Regression and Outlier Detection, New York, NY: Wiley, 1987.
  40. D. S. Chen and R. C. Jain, "A robust backpropagation learning algorithm for function approximation," IEEE Transactions on Neural Networks, vol. 5, no. 3, pp. 467-479, May 1994. http://dx.doi.org/10.1109/72.286917
  41. J. T. Connor, R. D. Martin, and L. E. Atlas, "Recurrent neural networks and robust time series prediction," IEEE Transactions on Neural Networks, vol. 5, no. 2, pp. 240-254, Mar. 1994. http://dx.doi.org/10.1109/72.279188
  42. K. Liano, "Robust error measure for supervised neural network learning with outliers," IEEE Transactions on Neural Networks, vol. 7, no. 1, pp. 246-250, Jan. 1996. http://dx.doi.org/10.1109/72.478411
  43. V. D. S'anchez A, "Robustization of a learning method for RBF networks," Neurocomputing, vol. 9, no. 1, pp. 85-94, Sep. 1995. http://dx.doi.org/10.1016/0925-2312(95)00000-V
  44. L. Huang, B.-L. Zhang, and Q. Huang, "Robust interval regression analysis using neural networks," Fuzzy Sets and Systems, vol. 97, no. 3, pp. 337-347, Aug. 1998. http://dx.doi.org/10.1016/S0165-0114(96)00325-9
  45. C. C. Chuang, S. F. Su, and C. C. Hsiao, "The annealing robust backpropagation (ARBP) learning algorithm," IEEE Transactions on Neural Networks, vol. 11, no. 5, pp. 1067-1077, Sep. 2000. http://dx.doi.org/10.1109/72.870040
  46. S. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed., Upper Saddle River, NJ: Prentice Hall, 1999.
  47. C. F. Juang and C. T. Lin, "A recurrent self-organizing neural fuzzy inference network," IEEE Transactions on Neural Networks, vol. 10, no. 4, pp. 828-845, Jul. 1999. http://dx.doi.org/10.1109/72.774232
  48. C. H. Lee and C. C. Teng, "Identification and control of dynamic systems using recurrent fuzzy neural networks," IEEE Transactions on Fuzzy Systems, vol. 8, no. 4, pp. 349-366, Aug. 2000. http://dx.doi.org/10.1109/91.868943
  49. Y. Y. Lin, J. Y. Chang, and C. T. Lin, "Identification and prediction of dynamic systems using an interactively recurrent self-evolving fuzzy neural network," IEEE Transactions on Neural Networks and Learning Systems, vol. 24, no. 2, pp. 310-321, Dec. 2013. http://dx.doi.org/10.1109/TNNLS.2012.2231436
  50. K. S. Narendra and K. Parthasarathy, "Identification and control of dynamical systems using neural networks," IEEE Transactions on Neural Networks, vol. 1, no. 1, pp. 4-27, Mar. 1990. http://dx.doi.org/10.1109/72.80202
  51. J. Espinosa and J. Vandewalle, "Constructing fuzzy models with linguistic integrity from numerical data-AFRELI algorithm," IEEE Transactions on Fuzzy Systems, vol. 8, no. 5, pp. 591-600, Oct. 2000. http://dx.doi.org/10.1109/91.873582
  52. P. S. Sastry, G. Santharam, and K. P. Unnikrishnan, "Memory neuron networks for identification and control of dynamical systems," IEEE Transactions on Neural Networks, vol. 5, no. 2, pp. 306-319, Mar. 1994. http://dx.doi.org/10.1109/72.279193
  53. S. S. Haykin, Adaptive Filter Theory, 2nd ed., Englewood Cliffs, NJ: Prentice Hall, 1991.
  54. L. A. Zadeh, "The concept of a linguistic variable and its application to approximate reasoning-I," Information Sciences, vol. 8, no. 3, pp. 199-249, 1975. http://dx.doi.org/10.1016/0020-0255(75)90036-5
  55. L. A. Zadeh, "The concept of a linguistic variable and its application to approximate reasoning-II," Information Sciences, vol. 8, no. 4, pp. 301-357, 1975. http://dx.doi.org/10.1016/0020-0255(75)90046-8
  56. L. A. Zadeh, "The concept of a linguistic variable and its application to approximate reasoning-III," Information Sciences, vol. 9, no. 1, pp. 43-80, 1975. http://dx.doi.org/10.1016/0020-0255(75)90017-1
  57. M. Grabisch, "The representation of importance and interaction of features by fuzzy measures," Pattern Recognition Letters, vol. 17, no. 6, pp. 567-575, May 1996. http://dx.doi.org/10.1016/0167-8655(96)00020-7
  58. R. D. Jones, Y. C. Lee, C. W. Barnes, G. W. Flake, K. Lee, P. S. Lewis, and S. Qian, "Function approximation and time series prediction with neural networks," in Proceedings of the IJCNN International Joint Conference on Neural Networks, San Diego, CA, June 17-21, 1990, pp. 649-665. http://dx.doi.org/10.1109/IJCNN.1990.137644

Cited by

  1. Efficient Approach for RLS Type Learning in TSK Neural Fuzzy Systems vol.47, pp.9, 2017, https://doi.org/10.1109/TCYB.2016.2638861