Journal of Korean Society of Industrial and Systems Engineering (산업경영시스템학회지)

Society of Korea Industrial and System Engineering (한국산업경영시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Instruction Fine-tuning and LoRA Combined Approach for Optimizing Large Language Models

대규모 언어 모델의 최적화를 위한 지시형 미세 조정과 LoRA 결합 접근법

  • Sang-Gook Kim (Korea Institute of Science and Technology Information) ;
  • Kyungran Noh (Korea Institute of Science and Technology Information) ;
  • Hyuk Hahn (Korea Institute of Science and Technology Information) ;
  • Boong Kee Choi (Korea Institute of Science and Technology Information)
  • 김상국 (한국과학기술정보연구원) ;
  • 노경란 (한국과학기술정보연구원) ;
  • 한혁 (한국과학기술정보연구원) ;
  • 최붕기 (한국과학기술정보연구원)
  • Received : 2024.06.10
  • Accepted : 2024.06.20
  • Published : 2024.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

This study introduces and experimentally validates a novel approach that combines Instruction fine-tuning and Low-Rank Adaptation (LoRA) fine-tuning to optimize the performance of Large Language Models (LLMs). These models have become revolutionary tools in natural language processing, showing remarkable performance across diverse application areas. However, optimizing their performance for specific domains necessitates fine-tuning of the base models (FMs), which is often limited by challenges such as data complexity and resource costs. The proposed approach aims to overcome these limitations by enhancing the performance of LLMs, particularly in the analysis precision and efficiency of national Research and Development (R&D) data. The study provides theoretical foundations and technical implementations of Instruction fine-tuning and LoRA fine-tuning. Through rigorous experimental validation, it is demonstrated that the proposed method significantly improves the precision and efficiency of data analysis, outperforming traditional fine-tuning methods. This enhancement is not only beneficial for national R&D data but also suggests potential applicability in various other data-centric domains, such as medical data analysis, financial forecasting, and educational assessments. The findings highlight the method's broad utility and significant contribution to advancing data analysis techniques in specialized knowledge domains, offering new possibilities for leveraging LLMs in complex and resource-intensive tasks. This research underscores the transformative potential of combining Instruction fine-tuning with LoRA fine-tuning to achieve superior performance in diverse applications, paving the way for more efficient and effective utilization of LLMs in both academic and industrial settings.

Keywords

Acknowledgement

This research was supported by Korea Institute of Science and Technology Information(KISTI) ((KISTI) K-24-L3-M2-C4-02).

References

  1. Bilgram, V., Laarmann, F., Accelerating Innovation With Generative AI: AI-Augmented Digital Prototyping and Innovation Methods, IEEE Engineering Management Review, 2023, Vol. 51, No. 2, pp. 18-25.
  2. Crothers, E., Japkowicz, N., Viktor, H., Machine-Generated Text: A Comprehensive Survey of Threat Models and Detection Methods, IEEE Access, 2023, Vol. 11, pp. 70977-71002.
  3. Camara, J., Troya, J., Burgueno, L., Vallecillo, A., On the assessment of generative AI in modeling tasks: an experience report with ChatGPT and UML, Software and Systems Modeling, 2023, Vol. 22, No. 3, pp. 781-793.
  4. Hassija, V., Chakrabarti, A., Singh, A., Chamola, V., Sikdar, B., Unleashing the Potential of Conversational AI: Amplifying Chat-GPT's Capabilities and Tackling Technical Hurdles, IEEE Access, 2023, Vol.11, pp. 143657-143682.
  5. Hommel, B., Expanding the methodological toolbox: Machine-based item desirability ratings as an alternative to human-based ratings, Personality and Individual Differences, 2023, Vol. 213, 112307.
  6. Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., LoRA: Low-Rank Adaptation of Large Language Models, ICLR 2022 Conference Poster, 2022, pp. 1-26.
  7. Kamnis, S., Generative pre-trained transformers(GPT) for surface engineering, Surface and Coatings Technology, 2023, Vol. 466, 129680.
  8. Karkera, N., Acharya, S., Palaniappan, S., Leveraging pre-trained language models for mining microbiome-disease relationships, BMC Bioinformatics, 2023, Vol. 24, No. 1, Article 290.
  9. Kheddar, H., Himeur, Y., Al Maadeed, S., Amira, A., Bensaali, F., Deep transfer learning for automatic speech recognition: Towards better generalization, Knowledge-Based Systems, 2023, Vol.277, pp. 1-34.
  10. Kim, J., Yoon, S., Choi, T., Sull, S., Unsupervised Video Anomaly Detection Based on Similarity with Predefined Text Descriptions, Sensors, 2023, Vol.23, No. 14, 6256.
  11. Kolides, A., Nawaz, A., Rathor, A., Beeman, D., Hashmi, M., Fatima, S., Berdik, D., Al Ayyoub, M., Jararweh, Y., Artificial intelligence foundation and pre-trained models: Fundamentals, applications, opportunities, and social impacts, Simulation Modelling Practice and Theory, 2023, Vol.126, 102754.
  12. Lankford, S., Afli, H., Way, A., adaptMLLM: Fine-Tuning Multilingual Language Models on Low-Resource Languages with Integrated LLM Playgrounds, Information(Switzerland), 2023, Vol. 14, No. 12, pp. 1-24.
  13. Lin, K., Agia, C., Migimatsu, T., Pavone, M., Bohg, J., Text2Motion: from natural language instructions to feasible plans, Autonomous Robots, 2023, Vol.47, No. 8, pp. 1345-1365.
  14. Mazumdar, H., Chakraborty, C., Sathvik, M., Mukhopadhyay, S., Panigrahi, P., GPTFX: A Novel GPT-3 Based Framework for Mental Health Detection and Explanations, IEEE Journal of Biomedical and Health Informatics, 2023, PMID:37903039.
  15. Megahed, F., Chen, Y., Ferris, J., Knoth, S., Jones Farmer, L., How generative AI models such as ChatGPT can be(mis)used in SPC practice, education, and research? An exploratory study, Quality Engineering, 2023, pp. 278-315.
  16. Nicula, B., Dascalu, M., Arner, T., Balyan, R., McNamara, D., Automated Assessment of Comprehension Strategies from Self-Explanations Using LLMs, Information (Switzerland), 2023, Vol. 14, No. 10, 567.
  17. Pan, W., Jiang, P., Li, Y., Wang, Z., Huang, J., Research on automatic pilot repetition generation method based on deep reinforcement learning, Frontiers in Neurorobotics, 2023, Vol. 17.
  18. Porsdam Mann, S., Earp, B., Moller, N., Vynn, S., Savulescu, J., AUTOGEN: A Personalized Large Language Model for Academic Enhancement- Ethics and Proof of Principle, American Journal of Bioethics, 2023, Vol. 23, No. 10, pp. 28-41.
  19. Singh, I., Blukis, V., Mousavian, A., Goyal, A., Xu, D., Tremblay, J., Fox, D., Thomason, J., Garg, A., ProgPrompt: Program generation for situated robot task planning using large language models, Autonomous Robots, 2023, Vol. 47, No. 8, pp. 999-1012.
  20. Saetra, H., Generative AI: Here to stay, but for good?, Technology in Society, 2023, Vol.75, 102372.
  21. Yin, C., Du, K., Nong, Q., Zhang, H., Yang, L., Yan, B., Huang, X., Wang, X., Zhang, X., PowerPulse: Power energy chat model with LLaMA model fine-tuned on Chinese and power sector domain knowledge, Expert Systems, 2023.
  22. Zhao, C., Yuan, S., Jiang, C., Cai, J., Yu, H., Wang, M., Chen, Q., ERRA: An Embodied Representation and Reasoning Architecture for Long-Horizon Language-Conditioned Manipulation Tasks, IEEE Robotics and Automation Letters, 2023, Vol. 8, No. 6, pp. 3230-3237.
  23. Zhu, Q., Zhang, X., Luo, J., Biologically Inspired Design Concept Generation Using Generative Pre-Trained Transformers, Journal of Mechanical Designs, 2023, Vol.145, No. 4, pp. 1-23.