Browse > Article
http://dx.doi.org/10.5909/JBE.2021.26.5.489

Analysis and Design of Arts and Culture Content Creation Tool powered by Artificial Intelligence  

Shin, Choonsung (Program of Media Content and Culture Technology, Graduate School of Culture, Chonnam National University)
Jeong, Hieyong (Department of Artificial Intelligence Convergence, Chonnam National University)
Publication Information
Journal of Broadcast Engineering / v.26, no.5, 2021 , pp. 489-499 More about this Journal
Abstract
This paper proposes an arts and culture content creation tool powered by artificial intelligence. With the recent advances in technologies including artificial intelligence, there are active research activities on creating art and culture contents. However, it is still difficult and cumbersome for those who are not familiar with programming and artificial intelligence. In order to deal with the content creation with new technologies, we analyze related creation tools, services and technologies that process with raw visual and audio data, generate new media contents and visualize intermediate results. We then extract key requirements for a future creation tool for creators who are not familiar with programming and artificial intelligence. We finally introduce an intuitive and integrated content creation tool for end-users. We hope that this tool will allow creators to intuitively and creatively generate new media arts and culture contents based on not only understanding given data but also adopting new technologies.
Keywords
art and culture content; artificial intelligence; content creation tool; deep learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 L. A. Gatys, A. S. Ecker, M. Bethge, "Image Style Transfer Using Convolutional Neural Networks," In Proc of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2414-2423, 2016.
2 P. Isola, J. Zhu, T. Zhou and A. A. Efros, "Image-to-Image Translation with Conditional Adversarial Networks," In Proc of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967-5976, 2017. doi: 10.1109/CVPR.2017.632.   DOI
3 P.Isola, J. Zhu, T. Zhou, A. Efros, "Image-to-Image Translation with Conditional Adversarial Networks," In Proc of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967-5976, 2017. doi: 10.1109/CVPR.2017.632.   DOI
4 A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, M. Chen, I. Sutskever, "Zero-Shot Text-to-Image Generation," ArXiv, abs/2102.12092. 2020.
5 C. Sievert, K. Shirley, "LDAvis: A method for visualizing and interpreting topics," In Proceedings of the workshop on interactive language learning, visualization, and interfaces, pp. 63-70, 2014.
6 S. Hershey, S. Chaudhuri, D. P. W. Ellis, J. F. Gemmeke, A. Jansen, C. Moore, M. Plakal, D. Platt, R. A. Saurous, B. Seybold, M. Slaney, R. Weiss, K. Wilson, "CNN architectures for large-scale audio classification," In Proc of 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017, pp. 131-135, doi: 10.1109/ICASSP.2017.7952132.   DOI
7 Autodraw, https://www.autodraw.com/ (accessed June. 27, 2021)
8 C. Weng, B. Curless, I. Kemelmacher, "Photo Wake-Up: 3D Character Animation From a Single Photo," pp. 5901-5910, 2019. doi: 10.1109/CVPR.2019.00606.   DOI
9 L. Maaten and G.E. Hinton. Visualizing High-Dimensional Data Using t-SNE, Journal of Machine Learning Research, Vol 9, pp. 2579-2605, 2008.
10 Music visualization, https://www.ocf.berkeley.edu/~acowen/music.html# (accessed June. 27, 2021)
11 Z. Cao, G. Hidalgo, T. Simon, S. -E. Wei and Y. Sheikh, "OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields," In Proc of IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 1, pp. 172-186, 2021. doi: 10.1109/TPAMI.2019.2929257.   DOI
12 J.P. Briot, G. Hadjeres, F. Pachet, "Deep Learning Techniques for Music Generation - A Survey," Computational Synthesis and Creative Systems, Springer, 2017.
13 J. Redmon and A. Farhadi, "YOLO9000: Better, Faster, Stronger," In Proc of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6517-6525, 2017. doi: 10.1109/CVPR.2017.690.   DOI
14 A.S. Cowen, X. Fang, D. Sauter, D. Keltner, "What music makes us feel: At least 13 dimensions organize subjective experiences associated with music across different cultures," In Proc of the National Academy of Sciences, Vol. 117(4), pp. 1924-1934, 2020. doi: 10.1073/pnas.1910704117.   DOI
15 H. Dong, W. Hsiao, Li Yang, and Y. Yang, "MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment," In Proc of the 32nd AAAI Conference on Artificial Intelligence (AAAI), 2018.
16 M. Wattenberg, F. Viegas, I. Johnson, "How to Use t-SNE Effectively," Distill, 2016. http://doi.org/10.23915/distill.00002 (accessed June. 27, 2021)   DOI
17 T-SNE visualization, https://lvdmaaten.github.io/tsne/ (accessed June. 27, 2021)
18 The Next Rembrandt, https://thenextrembrandt.pr.co/(accessed June. 27, 2021)
19 Magenta, https://magenta.tensorflow.org/(accessed June. 27, 2021)
20 Teachable Machine, https://teachablemachine.withgoogle.com/(accessed June. 27, 2021)
21 Google Arts & Culture, https://artsandculture.google.com/ (accessed June. 27, 2021)
22 Deep dream generator, https://deepdreamgenerator.com/(accessed June. 27, 2021)
23 D. M. Blei, A. Y. Ng, M. I. Jordan, "Latent Dirichlet Allocation," Journal of Machine Learning Research, Vol. 3, pp. 993-1022, 2003.
24 tSNE Java Script demo, https://cs.stanford.edu/people/karpathy/tsnejs/ (accessed June. 27, 2021)
25 A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," In Proc of the 25th International Conference on Neural Information Processing Systems(NIPS'12), Vol. 1,Curran Associates Inc., Red Hook, NY, USA, 1097-1105.
26 K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," In Proc of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016. doi: 10.1109/CVPR.2016.90.   DOI
27 J. Redmon, A. Farhadi, "YOLOv3: An Incremental Improvement," ArXiv abs/1804.02767, 2018.
28 T. Ishibashi, Y. Nakao, and Y. Sugano, "Investigating audio data visualization for interactive sound recognition," In Proc of the 25th International Conference on Intelligent User Interfaces (IUI), Association for Computing Machinery, New York, NY, USA, 67-77. doi:https://doi.org/10.1145/3377325.3377483   DOI
29 Z. Wolkowicz, S. Brooks, V. Keselj, "Midivis: Visualizing Music Structure via Similarity Matrices," In Proc of International computer music conference(ICMC), pp. 53-56, 2009.
30 Z. Kastrati, L. Ahmedi, A. Kurti, F. Kadriu, D. Murtezaj, F. Gashi, "A Deep Learning Sentiment Analyser for Social Media Comments in Low-Resource Languages," MDPI Electronics, Vol. 10, No 1133, 2021. https://doi.org/10.3390/electronics10101133   DOI
31 S. Minaee, N. Kalchbrenner, E. Cambria, N. Nikzad, M. Chenaghlu, and J. Gao, "Deep Learning-based Text Classification: A Comprehensive Review. ACM Computing Survey," Vol. 54, No. 3, Article 62, 2021. doiI:https://doi.org/10.1145/3439726   DOI