KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

A biologically inspired model based on a multi-scale spatial representation for goal-directed navigation

  • Li, Weilong (Information and Navigation College) ;
  • Wu, Dewei (Information and Navigation College) ;
  • Du, Jia (Information and Navigation College) ;
  • Zhou, Yang (Information and Navigation College)
  • Received : 2016.09.09
  • Accepted : 2017.01.11
  • Published : 2017.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

Inspired by the multi-scale nature of hippocampal place cells, a biologically inspired model based on a multi-scale spatial representation for goal-directed navigation is proposed in order to achieve robotic spatial cognition and autonomous navigation. First, a map of the place cells is constructed in different scales, which is used for encoding the spatial environment. Then, the firing rate of the place cells in each layer is calculated by the Gaussian function as the input of the Q-learning process. The robot decides on its next direction for movement through several candidate actions according to the rules of action selection. After several training trials, the robot can accumulate experiential knowledge and thus learn an appropriate navigation policy to find its goal. The results in simulation show that, in contrast to the other two methods(G-Q, S-Q), the multi-scale model presented in this paper is not only in line with the multi-scale nature of place cells, but also has a faster learning potential to find the optimized path to the goal. Additionally, this method also has a good ability to complete the goal-directed navigation task in large space and in the environments with obstacles.

Keywords

References

  1. A. Barrera, G. Tejera, M. Llofriu and A. Weitzenfeld, "Learning Spatial Localization: From Rat Studies to Computational Models of the Hippocampus," Spatial Cognition & Computation: An Interdisciplinary Journal, vol. 15, no. 1, pp. 27-59, 2015. https://doi.org/10.1080/13875868.2014.961602
  2. D. Derdikman and E.I. Moser, "A manifold of spatial maps in the brain," Trends in Cognitive Sciences, vol. 14, no. 12, pp. 561-569, 2010. https://doi.org/10.1016/j.tics.2010.09.004
  3. F. Chersi and N. Burgess, "The Cognitive Architecture of Spatial Navigation: Hippocampal and Striatal Contributions," Neuron, vol. 88, no. 1, pp. 64-77, 2015. https://doi.org/10.1016/j.neuron.2015.09.021
  4. P.K. Raymond and T.R. Edmund, "A computational theory of hippocampal function, and tests of the theory: New developments," Neuroscience and Biobehavioral Reviews, vol. 48, pp. 92-147, 2015. https://doi.org/10.1016/j.neubiorev.2014.11.009
  5. B.E. Pfeiffer and D.J. Foster, "Hippocampal place-cell sequences depict future paths to remembered goals," Nature, vol. 49, no. 7447, pp. 74-79, 2013.
  6. E.C. Tolman, "Cognitive maps in rats and men," Psychological Review, vol. 55, no. 554, pp. 189-208, 1948. https://doi.org/10.1037/h0061626
  7. J. O'Keefe and D.H. Conway, "Hippocampal place units in the freely moving rat: Why they fire where they fire," Experimental Brain Research, vol. 31, no. 4, pp. 573-590, 1978.
  8. J. O'Keefe and J. DosLrovskv, "The Hippocampus as a spatial Map," Brain Research, vol. 34, no. 1, pp. 171-175, 1971. https://doi.org/10.1016/0006-8993(71)90358-1
  9. J. O'Keefe and N. Burgess, "Geometric determinants of the place fields of hippocampal neurons," Nature, vol. 381, no. 6581, pp. 425-428, 1996. https://doi.org/10.1038/381425a0
  10. J.S. Taube, R.U. Muller and J.B. Ranck, "Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis," Journal of Neuroscience, vol. 10, no. 2, pp. 420-435, 1990. https://doi.org/10.1523/JNEUROSCI.10-02-00420.1990
  11. L.M. Giocomo, T. Stensola, T. Bonnevie, T.V. Cauter, M.B. Moser and E.I. Moser, "Topography of head direction cells in medial entorhinal cortex," Current Biology, vol. 24, no. 3, pp. 252-262, 2014. https://doi.org/10.1016/j.cub.2013.12.002
  12. T. Hafting, M. Fyhn, S. Molden, M.B. Moser and E.I. Moser, "Microstructure of a spatial map in the entorhinal cortex," Nature, vol. 436, no. 7052, pp. 801-806, 2005. https://doi.org/10.1038/nature03721
  13. E.I. Moser, Y. Roudi, M.P. Witter, C. Kentros, T. Bonhoeffer and M.B. Moser, "Grid cells and cortical representation," Nature Reviews Neuroscience, vol. 15, no. 7, pp. 466-481, 2014. https://doi.org/10.1038/nrn3766
  14. T. Solstad, C.N. Boccara, E. Kropff, M.B. Moser and E.I. Moser, "Representation of geometric borders in the entorhinal cortex," Science, vol. 322, no. 5909, pp. 1865-1868, 2008. https://doi.org/10.1126/science.1166466
  15. E. Kropff, J.E. Carmichael, M.B. Moser and E.I. Moser, "Speed cells in the medial entorhinal cortex," Nature, vol. 523, no. 7561, pp. 419-424, 2015. https://doi.org/10.1038/nature14622
  16. T. Wolbers, "Spatial Navigation," International Encyclopedia of the Social & Behavioral Sciences, vol. 23, pp. 161-171, 2015.
  17. S. Grossberg, "From brain synapses to systems for learning and memory: Object recognition, spatial navigation, timed conditioning, and movement control," Brain Research, vol. 6121, pp. 270-293, 2015.
  18. R.M. Tavares, A. Mendelsohn, Y. Grossman, C.H. Williams, M. Shapiro, Y. Trope and D. Schiller, "A Map for Social Navigation in the Human Brain," Neuron, vol. 87, no. 1, pp. 231-243, 2015. https://doi.org/10.1016/j.neuron.2015.06.011
  19. I. Kostavelis, K. Charalampous, A. Gasteratos and J.K. Tsotsos, "Robot navigation via spatial and temporal coherent semantic maps," Engineering Applications of Artificial Intelligence, vol. 48, pp. 173-187, 2016. https://doi.org/10.1016/j.engappai.2015.11.004
  20. Q. Zhu, R.B. Wang and Z.Y. Wang, "A cognitive map model based on spatial and goal-oriented mental exploration in rodents," Behavioural Brain Research, vol. 256, no. 11, pp. 128-139, 2013. https://doi.org/10.1016/j.bbr.2013.05.050
  21. D. Bush, C. Barry, D. Manson and N. Burgess, "Using Grid Cells for Navigation," Neuron, vol. 87, no. 3, pp. 507-520, 2015. https://doi.org/10.1016/j.neuron.2015.07.006
  22. H.J. Spiers and C. Barry, "Neural systems supporting navigation," Current Opinion in Behavioral Sciences, vol. 1, no. 1, pp. 47-55, 2015. https://doi.org/10.1016/j.cobeha.2014.08.005
  23. U.M. Erdem, M.E. Hasselmo, "A goal-directed spatial navigation model using forward trajectory planning based on grid cells," European Journal of Neuroscience, vol. 35, no. 6, pp. 916-931, 2012. https://doi.org/10.1111/j.1460-9568.2012.08015.x
  24. M.G. Sagiv, L. Las, Y. Yovel and N. Ulanovsky, "Spatial cognition in bats and rats: from sensory acquisition to multiscale maps and navigation," Nature Reviews Neuroscience, vol. 16, no. 2, pp. 94-108, 2015. https://doi.org/10.1038/nrn3888
  25. A.T. Keinath, M.E. Wang, E.G. Wann, R.K. Yuan, J.T. Dudman and I. A. Muzzio, "Precise spatial coding is preserved along the longitudinal hippocampal axis," Hippocampus, vol. 24, no. 12, pp. 1533-1548, 2014. https://doi.org/10.1002/hipo.22333
  26. U.M. Erdem, M.J. Milford and M.E. Hasselmo, "A hierarchical model of goal directed navigation selects trajectories in a visual environment," Neurobiology of Learning and Memory, vol. 117, pp. 109-121, 2015. https://doi.org/10.1016/j.nlm.2014.07.003
  27. Z. Chen, S. Lowry, A. Jacobson, M.E. Hasselmo and M.J. Milford, "Bio-inspired homogeneous multi-scale place recognition," Neural Networks, vol. 72, pp. 48-61, 2015. https://doi.org/10.1016/j.neunet.2015.10.002
  28. L.L. Long, J.G. Bunce and J.J. Chrobak, "Theta variation and spatiotemporal scaling along the septotemporal axis of the hippocampus," Frontiers in Systems Neuroscience, vol. 9, no. 37, pp. 1-14, 2015.
  29. R.J. Robitsek, J.A. White and H. Eichenbaum, "Place cell activation predicts subsequent memory," Behavioural Brain Research, vol. 254, no. 4, pp.65-72, 2013. https://doi.org/10.1016/j.bbr.2012.12.034
  30. A. Konar, I.G. Chakraborty, J.S. Sapam, C.J. Lakhmi and K.N. Atulya, "A Deterministic Improved Q-Learning for Path Planning of a Mobile Robot," IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 43, no. 5, pp. 1141-1153, 2015. https://doi.org/10.1109/TSMCA.2012.2227719
  31. N. Cuperlier, M. Quoy and P. Gaussier, "Navigation and Planning in an Unknown Environment Using Vision and a Cognitive Map," European Robotics Symposium, vol. 22, pp. 129-142, 2006.
  32. P. Giovanni, A.A. Matthijs, C.S. Lansink and M.A. C. Pennartz, "Internally generated sequences in learning and executing goal-directed behavior," Trends in Cognitive Sciences, vol. 18, no. 12, pp. 647-657, 2014. https://doi.org/10.1016/j.tics.2014.06.011