Acknowledgement
This work was supported by an Institute of Information & Communications Technology Planning & Evaluation (IITP) grant (2022-0-00995, automated reliable source code generation from natural language descriptions, 95%) and a National Research Council of Science & Technology (NST) grant (Global-23-001, SeCode: Collaborative intelligent model for secure program code generator, 5%) funded by the Korea government (MSIT).
References
- M. M. Abdollah Pour and S. Momtazi, Comparative study of text representation and learning for persian named entity recognition, ETRI J. 44 (2022), no. 5, 794-804. DOI 10.4218/etrij.2021-0269
- C. Park, J. Lim, J. Ryu, H. Kim, and C. Lee, Simple and effective neural coreference resolution for korean language, ETRI J. 43 (2021), no. 6, 1038-1048. DOI 10.4218/etrij.2020-0282
- A. Prakash, N. K. Singh, and S. K. Saha, Automatic extraction of similar poetry for study of literary texts: An experiment on hindi poetry, ETRI J. 44 (2022), no. 3, 413-425. DOI 10.4218/etrij.2019-0396
- M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba, Evaluating large language models trained on code, arXiv preprint, 2021. DOI 10.48550/arXiv.2107.03374
- Y. Li, D. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. D. Lago, T. Hubert, P. Choy, C. de Masson d'Autume, I. Babuschkin, X. Chen, P.-S. Huang, J. Welbl, S. Gowal, A. Cherepanov, J. Molloy, D. J. Mankowitz, E. S. Robson, P. Kohli, N. de Freitas, K. Kavukcuoglu, and O. Vinyals, Competition-level code generation with alphacode, Sci. 378 (2022), no. 6624, 1092-1097. https://doi.org/10.1126/science.abq1158
- E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y. Zhou, S. Savarese, and C. Xiong, Codegen: An open large language model for code with multi-turn program synthesis, (International Conference on Learning Representations), 2022. https://api.semanticscholar.org/CorpusID:252668917
- K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, BLEU: A method for automatic evaluation of machine translation, (Association for Computational Linguistics, Philadelphia, PA, USA), 2002, pp. 311-318.
- S. Ren, D. Guo, S. Lu, L. Zhou, S. Liu, D. Tang, N. Sundaresan, M. Zhou, A. Blanco, and S. Ma, Codebleu: a method for automatic evaluation of code synthesis, arXiv preprint, 2020, DOI 10.48550/arXiv.2009.10297
- S. Kulal, P. Pasupat, K. Chandra, M. Lee, O. Padon, A. Aiken, and P. S. Liang, SPoC: Search-based Pseudocode to Code, Proceedings of the 33rd International Conference on Neural Information Processing Systems Edited by H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alche-Buc, E. Fox, and R. Garnett, Vol. 32, Curran Associates, Inc., 2019.
- Y. Feng, S. Vanam, M. Cherukupally, W. Zheng, M. Qiu, and H. Chen, Investigating code generation performance of chatgpt with crowdsourcing social data, (IEEE 47th Annual Computers, Software, and Applications conference (COMPSAC), Torino, Italy), 2023, pp. 876-885.
- N. Nguyen and S. Nadi, An empirical evaluation of github copilot's code suggestions, (IEEE/ACM 19th International Conference on Mining Software Repositories (MSR), Pittsburgh, PA, USA), 2022, pp. 1-5.
- B. Yetistiren, I. Ozsoy, and E. Tuzun, Assessing the quality of github copilot's code generation, (Proc. of the 18th International Conference on Predictive Models and Data Analytics in Software Engineering, Singapore), 2022, pp. 62-71.
- T. Kim, Y. Jang, C. Lee, H. Koo, and H. Kim, Smartmark: Software watermarking scheme for smart contracts, (IEEE/ACM 45th International Conference on Software Engineering (ICSE), Melbourne, Australia), 2023, pp. 283-294.
- J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, and C. Sutton, Program synthesis with large language models, arXiv preprint, 2021. DOI 10.48550/arXiv.2108.07732
- Q. Zheng, X. Xia, X. Zou, Y. Dong, S. Wang, Y. Xue, Z. Wang, L. Shen, A. Wang, Y. Li, T. Su, Z. Yang, and J. Tang, Code-GeeX: A pre-trained model for code generation with multilingual evaluations on humaneval-XX, (Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Long Beach, CA, USA), 2023, pp. 5673-5684.
- B. Athiwaratkun, S. K. Gouda, Z. Wang, X. Li, Y. Tian, M. Tan, W. U. Ahmad, S. Wang, Q. Sun, and M. Shang, Multilingual evaluation of code generation models, (The Eleventh International Conference on Learning Representations), 2023.
- B. Roziere, 2023. Code llama: Open foundation models for code.
- D. Hendrycks, S. Basart, S. Kadavath, M. Mazeika, A. Arora, E. Guo, C. Burns, S. Puranik, H. He, D. Song, and J. Steinhardt, Measuring coding challenge competence with apps, (Proc. of the Neural Information Processing Systems Track on Datasets and Benchmarks), 2021.
- X.-Y. Li, J.-T. Xue, Z. Xie, and M. Li, Think outside the code: Brainstorming boosts large language models in code generation, arXiv preprint, 2023, DOI 10.48550/arXiv.2305.10679
- OpenAI, Gpt-4 technical report, 2023.