Journal of the Korean Institute of Landscape Architecture (한국조경학회지)

The Korean Institute of Landscape Architecture (한국조경학회)
권호 표지
DOI QR코드

DOI QR Code

Development of Image Classification Model for Urban Park User Activity Using Deep Learning of Social Media Photo Posts

소셜미디어 사진 게시물의 딥러닝을 활용한 도시공원 이용자 활동 이미지 분류모델 개발

  • Lee, Ju-Kyung (Interdisciplinary Program in Landscape Architecture, Seoul National University) ;
  • Son, Yong-Hoon (Graduate School of Environment Studies, Seoul National University)
  • 이주경 (서울대학교 협동과정 조경학) ;
  • 손용훈 (서울대학교 환경대학원 환경조경학과 )
  • Received : 2022.10.13
  • Accepted : 2022.11.16
  • Published : 2022.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

This study aims to create a basic model for classifying the activity photos that urban park users shared on social media using Deep Learning through Artificial Intelligence. Regarding the social media data, photos related to urban parks were collected through a Naver search, were collected, and used for the classification model. Based on the indicators of Naturalness, Potential Attraction, and Activity, which can be used to evaluate the characteristics of urban parks, 21 classification categories were created. Urban park photos shared on Naver were collected by category, and annotated datasets were created. A custom CNN model and a transfer learning model utilizing a CNN pre-trained on the collected photo datasets were designed and subsequently analyzed. As a result of the study, the Xception transfer learning model, which demonstrated the best performance, was selected as the urban park user activity image classification model and evaluated through several evaluation indicators. This study is meaningful in that it has built AI as an index that can evaluate the characteristics of urban parks by using user-shared photos on social media. The classification model using Deep Learning mitigates the limitations of manual classification, and it can efficiently classify large amounts of urban park photos. So, it can be said to be a useful method that can be used for the monitoring and management of city parks in the future.

본 연구의 목적은 인공지능의 딥러닝을 활용하여 소셜미디어에서 공유되는 도시공원 이용자 활동사진을 분류하는 기초 모델을 만드는 것이다. 소셜미디어 데이터는 네이버 검색을 통해 수집된 도시공원 관련 사진들을 수집하여 분류모델에 활용하였다. 도시공원 특성 평가에 활용할 수 있는 지표인 자연성(naturalness), 잠재적 매력성(potential attraction), 활동(activity)을 기반으로 최종 21개의 분류 항목체계를 만들고, 항목별로 네이버에서 공유되는 실제 도시공원 사진을 수집하여 주석이 달린 데이터 세트를 구축했다. 수집한 사진 데이터 세트에 대해 커스텀(cuntom) CNN 모델과 사전 훈련된 CNN의 전이학습 모델을 설계하고 분석하였다. 연구결과, 가장 우수한 성능을 보였던 Xception 전이학습 모델이 최종적으로 도시공원 이용자 활동 이미지 분류모델로 선정되었으며, 그 외 다양한 평가 지표를 통해 모델을 평가했다. 본 연구는 소셜미디어에 공유되는 이용자 사진을 활용하여 도시공원 특성을 평가할 수 있는 지표로서 AI를 구축한 것에 의의가 있다. 딥러닝을 활용한 분류모델은 수동분류에 대한 한계를 보완하고, 대량의 도시공원 사진을 효율적으로 분류할 수 있어서 향후 도시공원의 모니터링 및 관리에 활용할 수 있는 유용한 방법이라고 할 수 있다.

Keywords

Acknowledgement

이 논문은 서울대 환경계획연구소의 지원을 받았습니다.

References

  1. Aghdam, H. H. and E. J. Heravi(2017) Guide to Convolutional Neural Networks. New York, NY: Springer, 10(978-973), 51. 
  2. Alizadehtazi, B., K. Tangtrakul, S. Woerdeman, A. Gussenhoven, N. Mostafavi and F. A. Montalto(2020). Urban park usage during the COVID-19 pandemic. Journal of Extreme Events 7(4): 2150008. 
  3. Angradi, T. R., J. J. Launspach and R. Debbout(2018) Determining preferences for ecosystem benefits in Great Lakes areas of concern from photographs posted to social media. Journal of Great Lakes Research 44(2): 340-351.  https://doi.org/10.1016/j.jglr.2017.12.007
  4. Cetin, M.(2015) Using GIS analysis to assess urban green space in terms of accessibility: Case study in Kutahya. International Journal of Sustainable Development & World Ecology 22(5): 420-424.  https://doi.org/10.1080/13504509.2015.1081651
  5. Chae, I. Y.(2017) A Method for Analysis of Preferences of Places Based on Sentimental Analysis Using SNS Data: Case Study on Theme Parks in Seoul, South Korea. Seoul University, 1-92. 
  6. Chen, M., D. Arribas-Bel and A. Singleton(2020) Quantifying the characteristics of the local urban environment through geotagged flickr photographs and image recognition. ISPRS International Journal of Geo-Information 9(4): 264. 
  7. Chen, Y., X. Liu, W. Gao, R. Y. Wang, Y. Li and W. Tu(2018) Emerging social media data on measuring urban park use. Urban Forestry & Urban Greening 31: 130-141.  https://doi.org/10.1016/j.ufug.2018.02.005
  8. Chiang, Y. C. and D. Li(2019) Metric or topological proximity? The associations among proximity to parks, the frequency of residents' visits to parks, and perceived stress. Urban forestry & Urban Greening, 38, 205-214.  https://doi.org/10.1016/j.ufug.2018.12.011
  9. Chiesura, A.(2004) The role of urban parks for the sustainable city. Landscape and Urban Planning, 68(1), 129-138.  https://doi.org/10.1016/j.landurbplan.2003.08.003
  10. Chollet, F.(2017) Xception: Deep Learning with Depthwise Separable Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1251-1258). 
  11. Clemente, P., M. Calvache, P. Antunes, R. Santos, J. O. Cerdeira and M. J. Martins(2019) Combining social media photographs and species distribution models to map cultural ecosystem services: The case of a Natural Park in Portugal. Ecological Indicators 96: 59-68.  https://doi.org/10.1016/j.ecolind.2018.08.043
  12. Dunkel, A.(2015) Visualizing the perceived environment using crowdsourced photo geodata. Landscape and Urban Planning 142: 173-186.  https://doi.org/10.1016/j.landurbplan.2015.02.022
  13. Egorova, E.(2021) Using textual volunteered geographic information to model nature-based activities: A case study from Aotearoa New Zealand. Journal of Spatial Information Science (23): 25-63. 
  14. Garcia-Palomares, J. C., J. Gutierrez and C. Minguez(2015) Identification of tourist hot spots based on social networks: A comparative analysis of European metropolises using photo-sharing services and GIS. Applied Geography 63: 408-417.  https://doi.org/10.1016/j.apgeog.2015.08.002
  15. Geng, D. C., J. Innes, W. Wu and G. Wang(2021) Impacts of COVID-19 pandemic on urban park visitation: A global analysis. Journal of Forestry Research 32(2): 553-567.  https://doi.org/10.1007/s11676-020-01249-w
  16. Goodchild, M. F.(2007) Citizens as sensors: The world of volunteered geography. GeoJournal 69(4): 211-221.  https://doi.org/10.1007/s10708-007-9111-y
  17. Gosal, A. S., I. R. Geijzendorffer, T. Vaclavik, B. Poulin and G. Ziv(2019) Using social media, machine learning and natural language processing to map multiple recreational beneficiaries. Ecosystem Services 38: 100958. 
  18. Gosal, A. S. and G. Ziv(2020) Landscape aesthetics: Spatial modelling and mapping using social media images and machine learning. Ecological Indicators 117: 106638. 
  19. Huang, G., Z. Liu, L. Van Der Maaten, and Weinberger, K. Q. (2017). Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4700-4708). 
  20. Huang, M. H. and R. T. Rust(2018) Artificial intelligence in service. Journal of Service Research 21(2): 155-172.  https://doi.org/10.1177/1094670517752459
  21. Jais, I. K. M., A. R. Ismail and S. Q. Nisa(2019). Adam optimization algorithm for wide and deep neural network. Knowledge Engineering and Data Science 2(1): 41-46.  https://doi.org/10.17977/um018v2i12019p41-46
  22. Jang, Y. and D. Chung(2019) Technology trend for image analysis based on deep learning. Current Industrial and Technological Trends in Aerospace 17(1): 113-122. 
  23. Jiang, T., J. L. Gradus and A. J. Rosellini(2020) Supervised machine learning: A brief primer. Behavior Therapy 51(5): 675-687.  https://doi.org/10.1016/j.beth.2020.05.002
  24. Jeong, K. R.(2021) An Extended Theory of Planned Behavior Approach to Analyzing the Impact of COVID19 on Urban Park Users' Behavioral Decision-making. The Graduate School Seoul National University. Master's thesis. 
  25. Kang, Y., N. Cho, S. Park and J. Kim(2021) Exploring Tourism Activities of Tourists and Residents through Convolutional Neural Network-based SNS Photo Classification. 
  26. Kim, D., Y. Kang, Y. Park, N. Kim and J. Lee(2020) Understanding tourists' urban images with geotagged photos using convolutional neural networks. Spatial Information Research 28(2): 241-255.  https://doi.org/10.1007/s41324-019-00285-x
  27. Ilic, L., M. Sawada and A. Zarzelli(2019) Deep mapping gentrification in a large Canadian city using deep learning and Google Street View. PloS one 14(3): e0212814. 
  28. Kaczynski, A. T. and K. A. Henderson(2007) Environmental correlates of physical activity: A review of evidence about parks and recreation. Leisure Sciences 29(4): 315-354.  https://doi.org/10.1080/01490400701394865
  29. Kim, I, H. and J. H. Kim(2021) Multivariate time series clustering of electricity consumption data. Journal of the Korean Data And Information Science Society 32(3): 569-84.  https://doi.org/10.7465/jkdi.2021.32.3.569
  30. Kim, J. Y. and Y. H. Son(2021) Assessing and mapping the aesthetic value of Bukhansan national park using geotagged images. Journal of the Korean Institute of Landscape Architecture 49(4): 64-73. 
  31. Larson, L. R., Z. Zhang, J. I. Oh, W. Beam, S. S. Ogletree, J. N. Bocarro, K. J. Lee, J. M. Casper, J. A. Hipp, L. E. Mullenbach, M. Carusona and M. Wells(2021) Urban park use during the COVID-19 pandemic: Are socially vulnerable communities disproportionately impacted? Frontiers in Sustainable Cities, 3. 
  32. Lee, J. K. and Y. H. Son(2021) Perception and appraisal of urban park users using text mining of google maps review-cases of Seoul Forest, Boramae Park, Olympic Park. Journal of the Korean Institute of Landscape Architecture 49(4): 15-29.  https://doi.org/10.9715/KILA.2021.49.4.015
  33. LeCun, Y., Y. Bengio and G. Hinton(2015) Deep learning. Nature 521(7553): 436-444.  https://doi.org/10.1038/nature14539
  34. Lyu, F. and L. Zhang(2019) Using multi-source big data to understand the factors affecting urban park use in Wuhan. Urban Forestry & Urban Greening 43: 126367. 
  35. MA.(2005). Millennium Ecosystem Assessment. Washington, DC: New Island. 
  36. Martinez Pastur, G., P. L. Peri, M. V. Lencinas, M. Garcia-Llorente and B. Martin-Lopez(2016) Spatial patterns of cultural ecosystem services provision in Southern Patagonia. Landscape Ecology 31(2): 383-399.  https://doi.org/10.1007/s10980-015-0254-9
  37. Medical Device Information and Technology Assistance Center, MDITAC(2017) New Product Development Trend Newsletter: Artificial Intelligence(AI) Healthcare. 
  38. Millward, A. A. and S. Sabir(2011) Benefits of a forested urban park: What is the value of Allan Gardens to the city of Toronto, Canada? Landscape and Urban Planning 100(3): 177-188.  https://doi.org/10.1016/j.landurbplan.2010.11.013
  39. Munoz, L., V. H. Hausner, C. Runge, G. Brown and R. Daigle(2020) Using crowdsourced spatial data from Flickr vs. PPGIS for understanding nature's contribution to people in Southern Norway. People and Nature 2(2): 437-449.  https://doi.org/10.1002/pan3.10083
  40. Nanni, L., G. Maguolo and A. Lumini(2021) Exploiting Adam-like Optimization Algorithms to Improve the Performance of Convolutional Neural Networks. arXiv preprint arXiv:2103.14689. 
  41. Oteros-Rozas, E., B. Martin-Lopez, N. Fagerholm, C. Bieling and T. Plieninger(2018) Using social media photos to explore the relation between cultural ecosystem services and landscape features across five European sites. Ecological Indicators 94: 74-86. 
  42. Praveenkumar, G. D. and D. Muthusamy(2022) Machine Transfer Learning Deep Softmax Regression Neural Network for Image Classification. In Inventive Communication and Computational Technologies (pp. 695-708). Springer, Singapore. 
  43. Rajoub, B.(2020) Supervised and unsupervised learning. In Biomedical Signal Processing and Artificial Intelligence in Healthcare (pp. 51-89). Academic Press. 
  44. Retka, J., P. Jepson, R. J. Ladle, A. C. Malhado, F. A. Vieira, I. C. Normande, C. N. Souza, C. Bragagnolo, and R. A. Correia(2019) Assessing cultural ecosystem services of a large marine protected area through social media photographs. Ocean & Coastal Management 176: 40-48.  https://doi.org/10.1016/j.ocecoaman.2019.04.018
  45. Richards, D. R., and B. Tuncer(2018) Using image recognition to automate assessment of cultural ecosystem services from social media photographs. Ecosystem Services 31: 318-325.  https://doi.org/10.1016/j.ecoser.2017.09.004
  46. Schipperijn, J., P. Bentsen, J. Troelsen, M. Toftager and U. K. Stigsdotter(2013) Associations between physical activity and characteristics of urban green space. Urban Forestry & Urban Greening 12(1): 109-116.  https://doi.org/10.1016/j.ufug.2012.12.002
  47. Schnell, I., N. Harel and D. Mishori(2019) The benefits of discrete visits in urban parks. Urban Forestry & Urban Greening 41: 179-184.  https://doi.org/10.1016/j.ufug.2019.03.019
  48. Sinclair, M., A. Ghermandi and A. M. Sheela(2018) A crowdsourced valuation of recreational ecosystem services using social media data: An application to a tropical wetland in India. Science of the Total Environment 642: 356-365.  https://doi.org/10.1016/j.scitotenv.2018.06.056
  49. Song, M. K. and H. Chang(2010) Charaterization of cities in Seoul metropolitan area by cluster analysis. Journal of Korean Society for Geospatial Information Science 18(1): 83-88. 
  50. Stedman, R. C., B. L. Amsden, T. M. Beckley and K. G. Tidball(2014) Photo-based methods for understanding place meanings as foundations of attachment. In L. Manzo & P. Devine-Wright (Eds.), Place Attachment: Advances in Theory, Methods and Applications. Abingdon: Routledge. 
  51. Szegedy, C., V. Vanhoucke, S. Ioffe, J. Shlens and Z. Wojna(2016) Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2818-2826). 
  52. Tenerelli, P., C. Puffel and S. Luque(2017) Spatial assessment of aesthetic services in a complex mountain region: Combining visual landscape properties with crowdsourced geographic information. Landscape Ecology 32(5): 1097-1115.  https://doi.org/10.1007/s10980-017-0498-7
  53. Tveit, M. S.(2009) Indicators of visual scale as predictors of landscape preference: A comparison between groups. Journal of Environmental Management 90(9): 2882-2888.  https://doi.org/10.1016/j.jenvman.2007.12.021
  54. Van Zanten, B. T., D. B. Van Berkel, R. K. Meentemeyer, J. W. Smith, K. F. Tieskens and P. H. Verburg(2016) Continental-scale quantification of landscape values using social media data. Proceedings of the National Academy of Sciences 113(46): 12974-12979.  https://doi.org/10.1073/pnas.1614158113
  55. Vesal, S., N. Ravikumar, A. Davari, S. Ellmann and A. Maier(2018) Classification of breast cancer histology images using transfer learning. In International Conference Image Analysis and Recognition (pp. 812-819). Springer, Cham. 
  56. Wang, R., J. Luo and S. S. Huang(2020) Developing an artificial intelligence framework for online destination image photos identification. Journal of Destination Marketing & Management 18: 100512. 
  57. Wang, S., F. Liu and B. Liu(2021) Escaping the gradient vanishing: Periodic alternatives of softmax in attention mechanism. IEEE Access 9: 168749-168759.  https://doi.org/10.1109/ACCESS.2021.3138201
  58. Wang, Y., X. Shi, K. Cheng, J. Zhang and Q. Chang(2022) How do urban park features affect cultural ecosystem services: Quantified evidence for design practices. Urban Forestry & Urban Greening 76: 127713. 
  59. Wood, S. A., A. D. Guerry, J. M. Silver and M. Lacayo(2013) Using social media to quantify nature-based tourism and recreation. Scientific Reports 3(1): 1-7. 
  60. Yi, H. M.(2010) Interpreting the Spatial Politics of the Changing Process of Urban Park: Focus on Seoul Childrens Grand Park. The Graduate School Seoul National University. Master's thesis. 
  61. Yoon, J. Y. and Y. O Kang(2021) Tourism scene analysis through CNN-based multi-label transfer learning. Journal of Korean Society for Geospatial Information Science 29(4): 15-26  https://doi.org/10.7319/kogsis.2021.29.4.015
  62. Yoshimura, N. and T. Hiura(2017) Demand and supply of cultural ecosystem services: Use of geotagged photos to map the aesthetic value of landscapes in Hokkaido. Ecosystem Services 24: 68-78.  https://doi.org/10.1016/j.ecoser.2017.02.009
  63. Zhang, K., Y. Chen and C. Li(2019) Discovering the tourists' behaviors and perceptions in a tourism destination by analyzing photos' visual content with a computer deep learning model: The case of Beijing. Tourism Management 75: 595-608.  https://doi.org/10.1016/j.tourman.2019.07.002
  64. Zhang, K., X. Tang, Y. Zhao, B. Huang, L. Huang, M. Liu, E. Luo, Y. Li, T. Jiang, L. Zhang, Y. Wang and J. Wan(2022) Differing perceptions of the youth and the elderly regarding cultural ecosystem services in urban parks: An exploration of the tour experience. Science of The Total Environment 821: 153388. 
  65. Google Cloud. https://cloud.google.com/natural-language/automl/docs/beginners-guide?hl=ko 
  66. Naver(2021) 2021 Naver Blog Report. https://campaign.naver.com/2021blog/blogreport/
  67. Opensurvey(2022) Social Media.Portal Site Trend Report 2022. https://blog.opensurvey.co.kr/trend report/socialmedia-2022/