The Journal of the Convergence on Culture Technology (문화기술의 융합)

The International Promotion Agency of Culture Technology (국제문화기술진흥원)
권호 표지
DOI QR코드

DOI QR Code

Graph Convolutional - Network Architecture Search : Network architecture search Using Graph Convolution Neural Networks

그래프 합성곱-신경망 구조 탐색 : 그래프 합성곱 신경망을 이용한 신경망 구조 탐색

  • 최수연 (숭실대학교 베어드학부 ) ;
  • 박종열 (배화여자대학교 스마트IT학과)
  • Received : 2022.12.26
  • Accepted : 2023.01.09
  • Published : 2023.01.31
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes the design of a neural network structure search model using graph convolutional neural networks. Deep learning has a problem of not being able to verify whether the designed model has a structure with optimized performance due to the nature of learning as a black box. The neural network structure search model is composed of a recurrent neural network that creates a model and a convolutional neural network that is the generated network. Conventional neural network structure search models use recurrent neural networks, but in this paper, we propose GC-NAS, which uses graph convolutional neural networks instead of recurrent neural networks to create convolutional neural network models. The proposed GC-NAS uses the Layer Extraction Block to explore depth, and the Hyper Parameter Prediction Block to explore spatial and temporal information (hyper parameters) based on depth information in parallel. Therefore, since the depth information is reflected, the search area is wider, and the purpose of the search area of the model is clear by conducting a parallel search with depth information, so it is judged to be superior in theoretical structure compared to GC-NAS. GC-NAS is expected to solve the problem of the high-dimensional time axis and the range of spatial search of recurrent neural networks in the existing neural network structure search model through the graph convolutional neural network block and graph generation algorithm. In addition, we hope that the GC-NAS proposed in this paper will serve as an opportunity for active research on the application of graph convolutional neural networks to neural network structure search.

본 논문은 그래프 합성곱 신경망을 이용한 신경망 구조 탐색 모델 설계를 제안한다. 딥 러닝은 블랙박스로 학습이 진행되는 특성으로 인해 설계한 모델이 최적화된 성능을 가지는 구조인지 검증하지 못하는 문제점이 존재한다. 신경망 구조 탐색 모델은 모델을 생성하는 순환 신경망과 생성된 네트워크인 합성곱 신경망으로 구성되어있다. 통상의 신경망 구조 탐색 모델은 순환신경망 계열을 사용하지만 우리는 본 논문에서 순환신경망 대신 그래프 합성곱 신경망을 사용하여 합성곱 신경망 모델을 생성하는 GC-NAS를 제안한다. 제안하는 GC-NAS는 Layer Extraction Block을 이용하여 Depth를 탐색하며 Hyper Parameter Prediction Block을 이용하여 Depth 정보를 기반으로 한 spatial, temporal 정보(hyper parameter)를 병렬적으로 탐색합니다. 따라서 Depth 정보를 반영하기 때문에 탐색 영역이 더 넓으며 Depth 정보와 병렬적 탐색을 진행함으로 모델의 탐색 영역의 목적성이 분명하기 때문에 GC-NAS대비 이론적 구조에 있어서 우위에 있다고 판단된다. GC-NAS는 그래프 합성곱 신경망 블록 및 그래프 생성 알고리즘을 통하여 기존 신경망 구조 탐색 모델에서 순환 신경망이 가지는 고차원 시간 축의 문제와 공간적 탐색의 범위 문제를 해결할 것으로 기대한다. 또한 우리는 본 논문이 제안하는 GC-NAS를 통하여 신경망 구조 탐색에 그래프 합성곱 신경망을 적용하는 연구가 활발히 이루어질 수 있는 계기가 될 수 있기를 기대한다.

Keywords

References

  1. Barret Zoph, and Quoc V. Le. "Neural architecture search with reinforcement learning." arXiv preprint arXiv:1611.01578, 2016. DOI: https://doi.org/10.48550/arXiv.1611.01578 
  2. J. J. Hopfield. "Neural networks and physical systems with emergent collective computational abilities." Proceedings of the national academy of sciences, 79.8: 2554-2558, 1982. DOI: https://doi.org/10.1073/pnas.79.8.2554 
  3. Kunihiko Fukushima and Sei Miyake, "Neocogni tron: A self-organizing neural network model for a mechanism of visual pattern recognition," Competition and cooperation in neural nets. Springer, Berlin, Heidelberg, p. 267-285, 1982. DOI: https://doi.org/10.1007/978-3-642-46466-9_18 
  4. Se-Jin Choi, and Jun-Mo Jung. "A Method for accelerating training of Convolutional Neural Networks," The Journal of the Convergence on Cultural Technology, Vol. 3, No. 4, pp. 171-175, 2017. DOI: 10.17801/JCCT.2017.3.4.171 
  5. KIPF, Thomas N. and WELLING. Max. "Semi-supervised classification with graph convolutional networks." arXiv preprint arXiv:1609.02907, 2016. DOI: https://doi.org/10.48550/arXiv.1609.02907 
  6. Rana, Amrita. and Kyung-Ki KIM. "Robust architecture search using network adaptation." Journal of Sensor Science and Technology, 30.5: 290-294, 2021. DOI: https://doi.org/10.46670/JSST.2021.30.5.290 
  7. BOZINOVSKI, and Stevo. "Reminder of the first paper on transfer learning in neural networks," Informatica, Vol 44. No 3, 2020. DOI: https://doi.org/10.31449/inf.v44i3.2828 
  8. DENG. Jia. Dong Wei, and Socher Richard "Imagenet: A large-scale hierarchical image database," 2009 IEEE conference on computer vision and pattern recognition. Ieee, p. 248-255, 2009. DOI: 10.1109/CVPR.2009.5206848 
  9. G. Hinton, O. Vinyals, and J. Dean. "Distilling the knowledge in a neural network." arXiv preprint arXiv:1503.02531, Feb 2015. 
  10. Mingxing Tan, and Quoc Le, "Efficientnet: Rethinking model scaling for convolutional neural networks," International conference on machine learning. PMLR, p. 6105-6114, 2019. 
  11. A. Mirhoseini, A. Goldie, "A graph placement methodology for fast chip design." Nature, 594.7 862: 207-212, 2021.  https://doi.org/10.1038/s41586-021-03544-w
  12. S. Yan, Y. Xiong, and D. Lin, "Spatial temporal graph convolutional networks for skeleton-based action recognition," Thirty-second AAAI conference on artificial intelligence, 2018. 
  13. W. Peng, X. Hong, and H. Chen, "Learning graph convolutional network for skeleton-based human action recognition by neural searching," Proceedings of the AAAI Conference on Artificial Intelligence, p. 2669-2676, 2020. DOI: https://doi.org/10.1609/aaai.v34i03.5652