Online Submission and Review System for Open Science: A Case of AccessON Peer Review Management System Plus (ACOMS+)

  • Jaemin Chung (Open Access Center, Korea Institute of Science and Technology Information) ;
  • Eunkyung Nam (Open Access Center, Korea Institute of Science and Technology Information) ;
  • Sung-Nam Cho (Open Access Center, Korea Institute of Science and Technology Information) ;
  • Jeong-Mee Lee (Department of Library & Information Science, Seoul Women's University) ;
  • Hyunjung Kim (Department of Library & Information Science, Seoul Women's University) ;
  • Hye-Sun Kim (Open Access Center, Korea Institute of Science and Technology Information) ;
  • Wan Jong Kim (Open Access Center, Korea Institute of Science and Technology Information)
  • Received : 2023.09.27
  • Accepted : 2023.11.28
  • Published : 2024.03.30


As the academic publishing environment evolves rapidly and the open science paradigm emerges, the demand for efficient and transparent peer review is growing. This study outlines efforts to actively introduce advanced concepts in scholarly communication into the submission and review system. AccessON Peer Review Management System Plus (ACOMS+), developed and operated by the Korea Institute of Science and Technology Information, is an online submission and peer review system that aims for open science. This study provides an overview of ACOMS+ and presents its four main features: open peer review, open access publishing and self-archiving, online quantitative/qualitative evaluation, and peer reviewer invitation. The directions for further developing ACOMS+ to fully support open science are also discussed. ACOMS+ is the first system in Korea to introduce the open peer review process and is distinguished as a system that supports open access publishing and digital transformation of academic journals. Furthermore, ACOMS+ is expected to contribute to the advancement of the academic publishing environment through the increasing shift toward open access publishing, transparent peer review, and open science.



Journals are an important tool and medium in scholarly communication. Researchers seek to disseminate and share research results by publishing in academic journals, whose editorial boards organize peer reviews to evaluate the academic value of scholars’ papers. Research results published in peer-reviewed academic journals are the most recognized form of scholarly communication. Peer review is an important process contributing to academic advancement as one’s research is reviewed and recognized by fellow researchers for the validity, originality, and reliability of its process and results (Daniel, 1993).

Peer review was first adopted by the British journal Philosophical Transactions of the Royal Society in the early seventeenth century. In 1973, the peer review process was made mandatory by Nature, and is used by most journals today as a tool to improve the quality of articles and to decide whether to publish an article (Csiszar, 2016). In general, peer review is conducted in the following order: submission, assignment of reviewers, review, decision, revision, editing, and publication. To ensure that research results are disseminated speedily, the process of peer review must be made more efficient, while securing transparency and fairness (Wicherts, 2016). To this end, it is necessary to establish a system to manage data and procedures associated with peer review in a transparent manner.

Before submission management systems were developed and distributed in Korea, most journals used to submit articles and conduct reviews offline. This made it difficult to ensure fairness and transparency in the submission and review process, posing an obstacle to gaining competitiveness to advance as an international journal. Some Korean academic organizations used submission and review systems developed overseas, including ScholarOne of Clarivate or the Editorial Manager of the Aries System; however, these had limitations, such as an increase in annual costs and dependence on overseas systems. In 1999, the Korea Institute of Science and Technology Information (KISTI) developed Article Contribution Management System (ACOMS) and distributed it to academic organizations in Korea, making it the first online submission and peer review system developed in Korea.

As peer review has become a common procedure for scholarly communication, traditional blind peer review is being criticized for being unfair or biased (Benos et al., 2007). It has also been criticized for adding to the prestige of commercial publishing journals, leading to attempts at new ways of producing scholarly information (Tennant et al., 2017). The emergence of Open Journal Systems (OJS) in 2001, which allowed for a variety of peer review models to be tried, inspired several attempts to return to peer reviews led by the scientific community (Edgar & Willinsky, 2010), and in 2013, F1000Research attempted open peer review (Kirkham & Moher, 2018).

Excessive commercialization of the academic publishing market has pushed up subscription fees continuously, leading to the demand for new models of academic publishing distribution, which has resulted in the birth of new concepts such as open science and open access. Open science is the practice of making the results of research available in any process of the research, and includes publication and access to research results such as open access, disclosure of research data, and creation of a collaborative research environment (Heise & Pearce, 2020; Watson, 2015).

In other countries, submission and review systems such as Editorial Manager, ScholarOne, and OJS support open peer review and data sharing with related systems. Currently in Korea, only the traditional reviewing process is supported, which inherently limits the implementation of new trends in academic publishing, such as open access and the sharing of data with a variety of platforms and tools that are being developed to respond to the demand for new scholarly publishing distribution models.

Against this backdrop, it is necessary to develop and distribute a new submission and review system that reflects these changes in scholarly communication, namely open access publishing and the transition to open science, secures the global competitiveness of Korean journals, and integrates technologies such as artificial intelligence and big data. This study aims to outline efforts to develop a new submission and review management system, AccessON Peer Review Management System Plus (ACOMS+), and describe ACOMS+ and its main features. Cases of international and Korean submission and review systems are described. This study also presents the design of a new system that introduces advanced concepts in scholarly communication and reflects the requirements of researchers in Korea. In addition, the major features provided by ACOMS+ and to be developed in the near future to support a complete shift to open science are proposed.

The remainder of this paper is organized as follows: Section 2 describes the trends and characteristics of Korean and international submission and review systems; Section 3 explains the architecture, flowchart, and main features of ACOMS+ in detail; Section 4 discusses the directions for further development of the system to complete the open science features followed by the contributions of this work; and Section 5 presents the conclusions of this study.


The submission and review systems available worldwide are diverse, with many tailored to each country’s context. In this section, characteristics of major submission and review systems in the world and Korea are presented. As examples of international systems, Editorial Manager, ScholarOne, and OJS—the three most popular systems used by the top 20 publishers listed in the Directory of Open Access Journals (DOAJ)—are introduced. As examples of Korean systems, the publicly provided Journal & Article Management System (JAMS) and the commercially provided DBpiaONE are introduced. The history of ACOMS, the predecessor of ACOMS+, which is the target system of this study is also discussed. In this study, a case study was conducted by referring to the homepage and user guides of each system and the previous study (Kim et al., 2018).

Editorial Manager by Aries Systems is used by more than 8,500 journals and publishers around the world, including major publishers such as Elsevier, Wiley, and Springer. Editorial Manager systematically enhances the ease of submission and review (Salem et al., 2016). For example, it extracts manuscript metadata at the time of submission, reducing authors’ time and effort, and analyzes information about reviewers and papers to recommend reviewers. It provides a system that can configure the review workflow for each journal, and in particular, both blind and open peer review functions can be selected. It also provides a service for connecting other scholarly communication systems, and complies with metadata standards such as ORCID, Ringgold, Open Funder Registry, and CRediT.

ScholarOne by Clarivate is used by over 7,000 journals and publishers including Wiley, SAGE Publishing, and Taylor & Francis Group (Gahutu, 2017). The main advantage of ScholarOne is its ability to search and recommend reviewers. It utilizes the Web of Science journal indexing database operated by its parent company Clarivate, which matches SCIE, SSCI, and A&HCI listed journals and references with the metadata of the paper to recommend reviewers. ScholarOne organizes review workflows for each journal, connects to external scholarly communication services, and complies with metadata standards. It also supports both blind and open peer review process.

OJS is an open-source software application for the management and publication of academic journals, developed and distributed by Public Knowledge Project (Willinsky, 2005). OJS is the most widely used application among existing open science journal publishing platforms, with over 25,000 users globally. Since it is an open-source software, it offers a hosting service with multiple pricing plans and is maintained on GitHub for feedback and ongoing maintenance. Moreover, OJS can be further developed and used by users and can disclose reviewer identification information, allowing for both blind and open peer review.

JAMS was developed by the National Research Foundation of Korea (NRF) and distributed free of charge to academic organizations in Korea (Kim et al., 2018). More than 1,600 journals are currently using the system. JAMS supports websites of academic organizations, conference organizers, and electronic publishing. Also, articles published in JAMS are linked to the Korea Citation Index (KCI) and Korean Researcher Information, operated by NRF. However, as there is no setting to disclose reviewer information, only blind peer review can be selected.

DBpiaONE is provided by Korea’s largest commercial academic database provider, Nurimedia, and used by more than 120 journals (Lee et al., 2016). DBpiaONE employs technologies focused on user convenience. Features such as accepting reviews without logging into the system and consolidating author submissions into a single PDF file for reviewers are available. However, it lacks the function of setting a Creative Commons License (CCL) to support open access journals and only supports blind peer review.

ACOMS is an online submission and review system provided for free by KISTI (Kim et al., 2022). Currently, the system is used by 118 journals, mainly in the field of science and technology in Korea. KISTI developed ACOMS 1.0 in 2000 and began distributing it to academic journals for free, and in 2010, the English version, ACOMS 4.0, was developed to give journals a global reach. The most recent version, ACOMS 4.5, was developed in 2020 and is still operational. ACOMS 4.5 offers features such as submission, review, and online journal publication to enhance the efficiency and convenience of the process. ACOMS, however, lacked metadata standards, open peer review processes, review history management functions, and functions to support open access. In addition, the system underwent a long period of development of more than 20 years, which made the operational and development environment less efficient than desired. To address these problems, KISTI developed a next-generation submission and review system, ACOMS+. ACOMS+ is designed to overcome the main limitations of ACOMS, thus it can support the publication of international-level journals, reflect the latest open science trends, and link with KISTI’s internal open science support systems. ACOMS+ was officially launched in March 2023 after the planning, designing, and building of the system was undertaken in 2021 and 2022. As part of the upgrade, journals that had used ACOMS were automatically transferred to ACOMS+. KISTI plans to ensure the continuity of submission and reviewing work for journals currently using ACOMS.


3.1. Overview of ACOMS+

3.1.1. System Architecture

ACOMS+ is a system that facilitates a flexible and efficient peer review process and journal management. The objectives of ACOMS+ can be summarized as follows: First, ACOMS+ aims to streamline the submission process for academic organizations by providing high-quality peer review features; second, ACOMS+ aims to promote and advance open access and open science by offering features reflecting the latest academic publishing trends; and third, ACOMS+ can be used free of charge to reduce the financial burden of academic organizations. The system architecture has been designed to achieve these goals, as illustrated in Fig. 1. The system architecture, focusing on the user roles and related systems, is now described.

E1JSCH_2024_v12n1_87_f0001.png 이미지

Fig. 1. System architecture of ACOMS+. ACOMS+, AccessON Peer Review Management System Plus.

All ACOMS+ users become AccessON integrated members. AccessON provides a service to help Korean researchers access open access papers more easily and academic organizations to disclose and disseminate research results with open access. ACOMS+ users can use all services provided by AccessON, including Collab, Self-Archiving, and SAFE. Notably, they can access the submission and review systems of all journals using ACOMS+ with a single integrated account provided by AccessON.

Authors represent the most basic role in ACOMS+. Once registered to a journal’s submission and review system, a member is automatically granted permissions as an author. Authors can submit their papers to the journal of their choice by entering details regarding the paper and uploading the manuscript. Upon completion of peer review, they can further improve the manuscript based on the review comments and resubmit it. When the manuscript is finally accepted for publication, the author can print out the certificate of acceptance and proof the edited manuscript before publication.

The role of reviewers is crucial in the peer review process. The members who are registered in the system are granted rights as a reviewer and can act as reviewers from the moment they are invited to do so. After accepting an invitation to review an article, they should review the article according to the journal’s requirements. ACOMS+ stores the history of review completed, prints out the review certificate, and automatically enters the history of peer review activities in association with ORCID.

Editors or chief editors are in charge of submission management. When a manuscript is submitted, an editorial board member and a reviewer are assigned. Once the reviewer has completed the review, the chief editor will make a decision based on the review comments and notify the author of the result. The chief editor may manage the publication of papers and assign the accepted papers for publication in each issue. The authority of chief editor may be granted by the academic organization, and the authority of the editorial board may be granted by the chief editor or the academic organization to journal members registered in the system.

Publishing managers are responsible for the publication of articles. They assign articles accepted for publication to volumes/issues, set the issue type, communicate with authors, and proof the edited article iteratively. Once the manuscript has been proofed by the author, it is ready for publication as a journal article.

Academic societies handle the setup and administration of the system to be used. They enter basic information and policies of journals, establish the conditions for submission and review, and manage the issues of published papers and the rights of journal members. ACOMS+ provides features that allow journal administrators to view the history of submission and review as well as the statistics and data to be used to register in Korean and international indexing databases.

The system administrator is in charge of managing the system in general. They perform tasks such as managing system-wide announcements, approving applications to utilize the system, and creating journal systems. They also improve the system to reflect the needs of academic organizations and correct system errors. They can also view the overall statistics of journals using the system.

ACOMS+ provides linkage with external and internal systems. External systems include ORCID and KCI. ACOMS+ also provides features including notifications of the review process through mobile messengers and payment of article processing charges (APC). Internally, it is linked to various systems provided by KISTI, primarily AccessON, which is explained in more detail in Section 3.1.2 and Section 3.2.2. Also, it supports linkages with the Korea DOI Center. If a journal is issued a DOI via this DOI Center, the DOI can be automatically issued by entering the metadata and full-text of manuscripts. Finally, all actions occurring in the ACOMS+ system are stored in a database systematically.

3.1.2. Flowchart of the Peer Review Process

Due to its organic interaction with users in various roles and a diversity of internal and external systems, the submission and review system involves a complicated process. Fig. 2 shows the flowchart of the ACOMS+ process. A series of procedures in the ACOMS+ system—from application and setup to submission and review, publication, dissemination, and distribution—is now described.

E1JSCH_2024_v12n1_87_f0002.png 이미지

Fig. 2. Process flowchart of ACOMS+. ACOMS+, AccessON Peer Review Management System Plus; KISTI, Korea Institute of Science and Technology Information; APC, article processing charges; DOAJ, Directory of Open Access Journals; DOI, Digital Object Identifier; JATS XML, Journal Article Tag Suite Extensible Markup Language.

First, the academic organization fills in and submits the application form online. For the application process, the academic organization is required to enter basic information, including the name of the journal, ISSN, date of foundation, and publication cycle. Based on the application submitted, the system administrator checks the information to ensure that the information is accurate and that the journal is not predatory. If all conditions are satisfied, the system administrator generates a system and provides the URL of the submission and review system and journal administrator page to the academic organization.

The next step is the setup of the journal information for the operation of the submission and review system. The journal administrator of the academic organization selects the journal’s open access, copyright, self-archiving, APC, and peer review policies, and creates the relevant policy guidance pages. The conditions for submission review should also be set. To provide the optimal environment for each journal, ACOMS+ provides several options such as journal language, abstract length, number of keywords, file types, and journal discipline. Lastly, templates of alert mails sent by the system during the submission and review process can be customized and saved for each journal preference to complete the setup.

When the setup is completed, users are ready to submit manuscripts and review articles. ACOMS+ provides a basic submission and review process. An author logs in and submits an article. The author checks the basic policies of the journal and enters the article type, title, abstract, keywords, author information, and field(s), and uploads the files required by the journal (e.g., manuscript, cover letter, supplementary materials, figures, tables) for submission. Then, the chief editor receives the submission and follows an established protocol to check whether the manuscript is submitted in accordance with the peer review policy and whether there is plagiarism. If a submission is not appropriate for the journal, the chief editor may reject it without review. If the manuscript requires simple corrections or enhancements, the chief editor may return the submission to the author without rejecting it. In this case, the author modifies and resubmits the manuscript based on the chief editor’s comments.

If it is judged that the manuscript is ready for review, the chief editor checks in the manuscript and assigns an appropriate editor to the article. The editor then decides whether to accept the assignment. For example, they may decline an assignment if there is a conflict of interest with the author(s) of the assigned manuscript, if the manuscript does not fit their research area, or if the manuscript is of poor quality and needs to be rejected or enhanced. If the assignment is accepted, the editor invites reviewers. Specifically, the editors determine the number of reviews for editor recommendation, find the right experts, and send them emails asking them to review the manuscript. Next, the reviewers perform the review. Reviewers can download full-text files only if they accept the invitation. For double-blind peer review journals, the system restricts information disclosure so that reviewers do not know who the authors are. Once all reviewers have completed their review, the editor integrates the reviewers’ comments to make an editor recommendation, and the chief editor will send the decision (e.g., accept, major revision, minor revision, reject) to the author. Finally, the author submits a revised version of the manuscript along with a response letter to the reviewers, or submits the final version for publication.

The manuscript accepted after peer review is published and disseminated in linkage with AccessON Journal Repository. Journal Repository is an online service platform for open access journals supporting journal publishing, distribution, and archiving. Journal publishing managers can generate new volumes/issues of the journal and assign manuscripts accepted for publication in ACOMS+ to volumes/issues. After the process of editing, they communicate with the author to review and proofread the published version. Once verified, the published version is made available immediately on Journal Repository. The published articles are indexed in AccessON and ScienceON as well as on the homepage of each journal, the Journal Repository, to facilitate dissemination via academic search engines such as Google Scholar.

3.2. Main Features

3.2.1. Open Peer Review Process

ACOMS+ is Korea’s first submission and peer review system to support the open peer review process. Submission and peer review systems or publishers have different open peer review process (Tennant et al., 2017). Fig. 3 shows the open peer review process provided by ACOMS+. ACOMS+ provides the basic and hybrid options based on the transparent peer review model. In the basic option, the journal administrator can set the open peer review option to disclose review information when the paper is published after peer review, which may be single-blinded or double-blinded, so that authors do not know who the reviewers are. The information disclosed with the publication may include reviewer information, review reports, author response, manuscript version, and editor information, and the journal can select the information to be disclosed as they want.

E1JSCH_2024_v12n1_87_f0003.png 이미지

Fig. 3. Open peer review process of ACOMS+. ACOMS+, AccessON Peer Review Management System Plus.

The hybrid option is based on the agreement of four major parties in the peer review process—journal, author, reviewer, and chief editor. First, the journal administrator decides whether to allow hybrid open peer review when setting the journal’s peer review policy. If open peer review is allowed, the next choice is passed on to the author. When submitting a new manuscript to a hybrid open peer review journal, the author can choose whether to accept open peer review or not. If the author agrees, the reviewers choose whether to allow open peer review when they accept the invitation to review. The reviewers are notified that their details and the review report will be made public with the publication. If both the author and reviewers accept the use of hybrid open peer review, the chief editor selects whether to allow open peer review when deciding the acceptance of the paper for publication. This is the final step in determining whether it is appropriate to disclose the reviewer comments and author responses that were exchanged during the peer review process. The entire peer review process is conducted as a single- or double-blind review, where the authors are blinded to the reviewers’ identity.

The settings and four-way agreements for open peer review made during the peer review process are stored and managed in a database. When open peer review is selected, journals can make the review information public with the publication of the paper while the readers are allowed to view the review process transparently.

3.2.2. Open Access Publishing and Self-Archiving

ACOMS+ supports both gold open access and green open access—two representative open access models. To this end, ACOMS+ is closely linked to AccessON. This subsection describes in detail the linkage functions to support each model.

The gold open access features are provided in conjunction with Journal Repository. ACOMS+ and Journal Repository are connected as a single system to support the entire process of open access academic publishing from submission and review to publication and dissemination. An open access journal can integrate ACOMS+ and Journal Repository and use the integrated administrator page to manage both systems at once. This integration saves journal administrators the effort of managing the same information into two different systems because the journal’s basic information and journal policies are linked. Additionally, as described in Section 3.1.2, submission and review data of ACOMS+ can be linked to the publication process. For papers that have undergone open peer review, the reviewers’ information and review reports can be disclosed with the publication without the need to enter any information or upload any files separately in Journal Repository.

ACOMS+ and Journal Repository provide open access declaration and policy templates. Fig. 4 presents the concept of auto-populating policy templates based on journal settings. For example, if a journal declares open access and selects CC BY-NC as its license policy, the settings are stored in the database. The system will retrieve the license policy wording corresponding to the set value from the database and automatically populate it on the user page. Journal administrators can modify the wording as needed and disclose it. ACOMS+ and Journal Repository offer policy templates for CCL, copyright, self-archiving, APC, and journal archiving, which can be used to evaluate the publishing environment of an open access journal (Kim et al., 2023). In terms of journal operation, this feature is expected to eliminate barriers for journals concerned about the difficulty of setting policies with regard to transition to open access publishing, and to help open access journals to be listed in prestigious journal indexes such as Web of Science or DOAJ.

E1JSCH_2024_v12n1_87_f0004.png 이미지

Fig. 4. Concept of rule-based open access policy generation.

The features for green open access are provided through a connection with AccessON’s Self-Archiving. The journal administrator page provides a feature to set author self-archiving conditions. Authors can use this linkage if the journal allows pre-print or post-print self-archiving of accepted manuscripts to any website. In this case, when submitting the final article after being accepted for publication, authors are asked if they wish to self-archive their manuscripts to AccessON. If the author agrees, self-archiving happens automatically without the need to review the journal’s policies or upload files. If an article is under embargo, the system will calculate the embargo period and automatically release the pre-print and post-print after the embargo ends. To sum up, ACOMS+ has built an optimal environment for green open access by minimizing the author’s work for self-archiving.

3.2.3. Online Quantitative/Qualitative Evaluation

ACOMS+ ensures objective and fair review and improves the quality of peer review by providing a structured set of quantitative and qualitative metrics. A case study, survey, and focus group interview that were conducted to build the metrics and the results reflected in the system are outlined in this subsection.

A case study of journals, publishers, and systems in Korea and abroad to comprehensively review items widely used to evaluate academic articles was firstly conducted. For Korea, 177 items from 20 journals, including the Journal of Korean Academic Society of Nursing Education and journals in the field of library and records management were collected. For the international context, 149 items were collected from 12 systems, publishers, and journals, including article submission and review systems such as Editorial Manager and ScholarOne, and large publishers such as Elsevier and Emerald Publishing, and journals such as F1000Research and PLOS ONE. Based on the collected items, items and contents to initially classify the major themes and detailed evaluation items were analyzed. Then, a survey was conducted to evaluate the appropriateness and consistency of the initial evaluation items among journals using ACOMS. The questionnaire comprised questions on whether the division of major themes is appropriate, whether the detailed evaluation items are appropriate for each major theme, and what detailed evaluation items should be included in major themes. It was conducted online between June 3 and June 18, 2022, and 102 of the 113 journals responded. During the survey period, a focus group interview was conducted simultaneously with the editorial board members of five journals to get opinions on the organization of the major themes, detailed evaluation items, and evaluation methods. Finally, the results of the survey and interview were analyzed to integrate similar themes and move detailed evaluation items under appropriate themes. Consequently, five major themes and 32 detailed evaluation items were finalized (Appendix A).

Appendix A. Major themes and evaluation items for quantitative/qualitative evaluation

E1JSCH_2024_v12n1_87_t0002.png 이미지

ACOMS+ supports all major themes and evaluation items by default and provides a feature to customize evaluation methods to suit the needs of each journal. Each item can be modified or removed depending on the subject area or evaluation perspective of journals. For quantitative evaluation, journals can choose between a 5-point Likert scale, a 7-point scale, and a scoring system. They can also choose whether to use quantitative or qualitative evaluation. Reviewers perform quantitative and/or qualitative evaluations based on the settings selected by the journal administrator (Fig. 5); the editors, chief editor, and authors can view the evaluation results in detail. Also, ACOMS+ provides a feature for authors to submit a revised manuscript with responses to each of the reviewers’ qualitative evaluations.

E1JSCH_2024_v12n1_87_f0005.png 이미지

Fig. 5. Snippets from the qualitative/quantitative evaluation table.

3.2.4. Peer Reviewer Invitation

One of the key challenges in the operation of a journal’s peer review system is to secure a pool of reviewers and allocate appropriate reviewers to an article. This subsection briefly describes the features for editors to use to invite reviewers.

First, ACOMS+ provides a function of searching reviewers through diverse reviewer pools. Total five reviewer search pools are provided: existing reviewers, corresponding authors, ACOMS+ members, ORCID referrals, and journals’ own reviewers. Existing reviewer pools, author pools, and ACOMS+ member pools can be searched against users of other journals or ACOMS+ as a whole as well as against users within a journal. It is planned to introduce a data-driven algorithm that recommends appropriate reviewers based on each user’s publication and review history. Second, ACOMS+ enables editors to invite reviewers simply by entering the name and e-mail address. When an expert is considered a good fit even though not in a pool of reviewers, this feature can be used to invite the expert as a reviewer. Third, ACOMS+ enables non-members to review an article. Reviewers are allowed access to reviewing without the need to register as ACOMS+ users. If a reviewer signs up for membership after completing a review, the review history accumulated as a non-member is consolidated, making it easier to manage review history. Finally, ACOMS+ also offers a consolidated view of the review history for a reviewer (Fig. 6). Editors can learn about a reviewer’s reviewing behavior through their history and use it to make decisions about reviewer invitations. It is also planned to enhance this feature by providing a list of published papers and keywords of the research conducted by the reviewer.

E1JSCH_2024_v12n1_87_f0006.png 이미지

Fig. 6. Sample of review history page.


4.1. Future Directions for Open Science

Despite the architecture, work flow, and features introduced above, there is still room for improvement in terms of open science support. This section describes future directions for ACOMS+.

A new feature to support open data is underway. Open data, which means opening up and sharing the data used in research, is one of the main components of open science. Academic papers are one of the final outcomes of research and development, and data are essential to complete them. ACOMS+ will provide a feature for authors to submit the research data used to write the paper. In particular, large data will be deposited in DataON, a data repository provided by KISTI, to make them publicly available. DataON systematically manages and preserves research data and provides an environment where each dataset can be shared externally by issuing DOIs. The process has been designed to have the data automatically registered to DataON if the author uploads the manuscript and research data together and the manuscript is accepted for publication in ACOMS+. After the registration, research data is associated with the article and made available to the public via the article’s Data Availability Statement. The publicly available research data can be used to verify the results and reproducibility of articles, enhancing the quality and transparency of journals. This is also expected to help promote an open research ecosystem where research data can be reused by other researchers in follow-up studies.

In addition, ACOMS+ plans to provide Journal Article Tag Suite Extensible Markup Language (JATS XML) publishing. With the open access movement, JATS XML has gained importance as a way of distributing digital files. JATS XML is an XML standard for storing and publishing academic papers and is emerging as an alternative to the traditional PDF-centric academic publishing environment. ACOMS+ supports JATS XML publishing in conjunction with Collab, an online co-authoring tool provided by AccessON. One of Collab’s key features, Collab PEN, provides an environment for co-authors to work together on papers via a Word plug-in, which will be linked to ACOMS+ for submission. When authors have written and submitted an article in Collab PEN according to the journal’s template and it has been accepted for publication in ACOMS+, the journal’s publishing manager can convert the manuscript to JATS XML once it has been proofread and edited. Converted JATS XML can be provided in a publishable format, albeit not in a clean state, with minimal processing, and can be published in Journal Repository immediately. Accordingly, this is expected to be of great help to small and financially constrained academic organizations in their academic publishing activities. It will also streamline the open access publishing process to contribute to the promotion of open access in Korea.

In addition, the feature of summarizing journal information for SHERPA/RoMEO and DOAJ registration is under development. SHERPA/RoMEO is a database that provides self-archiving policies of publishers and journals, and DOAJ is an open access journal indexing database. For green open access of scholarly articles, it is crucial for journals to set and disclose their authors’ self-archiving policies. In addition, gold open access journals are also required to publish high-quality articles to meet the global open access standards. ACOMS+ is developing a feature that enables academic organizations to easily apply for registration and listing with SHERPA/RoMEO and DOAJ by simply referring to the application report provided by ACOMS+. The application report is provided by consolidating information entered in the journal administrator page, which is expected to minimize the efforts of revising or additionally entering information. This will contribute to the growth of Korean journals according to global standards and the promotion of open access in Korea.

4.2. Academic Contributions and Practical Implications

ACOMS+ is a submission and review management system that reflects the latest academic publishing trends. Table 1 shows the results of comparing existing systems focusing on the main features of ACOMS+. ACOMS+ has introduced many differentiated features compared to the previous ACOMS system, and has been developed to a level where it can compete with international systems. In this section, the major contributions of ACOMS+ are described in detail.

Table 1. Comparison of main features with current submission and review management systems

E1JSCH_2024_v12n1_87_t0001.png 이미지

EM, Editorial Manager; SO, ScholarOne; OJS, Open Journal Systems; DO, DBpiaONE; JAMS, Journal and Article Management System; ACOMS, Article Contribution Management System; ACOMS+, AccessON Peer Review Management System Plus.

a)△: Features to be developed in the near future.

First of all, ACOMS+ is the first Korean submission and review system that supports open peer review. Importantly, it has established the open peer review process to suit the academic research environment of Korea. Korean researchers often have a cautious stance toward disclosing reviewer information from the peer review stage because the domestic academic field is relatively small (Jeong et al., 2022). Against this backdrop, ACOMS+ has developed the two open peer review options where firstly blind peer reviews are conducted, and review information is disclosed when the paper is published. In specific, the hybrid option is based on a four-way agreement between the journal, author, reviewers, and chief editor. Since this open peer review feature is developed to reflect Korean academia, it is expected to contribute to enhancing open access maturity and the transparency of Korean journals.

Second, ACOMS+ has introduced advanced concepts such as self-archiving, open peer review, and data sharing, which are somewhat unfamiliar to Korea’s academic community. Thus, it will familiarize Korean researchers with terms and concepts related to scholarly communication. Furthermore, many features of ACOMS+ are linked to Journal Repository. Journals can be encouraged to further utilize Journal Repository by promoting the benefits of integrating the two systems.

Third, with a variety of linkage features, ACOMS+ has contributed to the development of a life-cycle open access support system centered on AccessON. AccessON is designed to aid article discovery, co-authoring, self-archiving, monitoring, and publication/dissemination. ACOMS+ completes the submission and review phase, an essential activity in academic publishing; and it enables AccessON to support the entire open access publishing experience, from production to dissemination. Articles written using Collab in AccessON can be peer-reviewed and copy edited in ACOMS+, and published directly through Journal Repository. With ACOMS+, a full-cycle support system of AccessON is finally completed to provide support for sustainable open access publishing in Korea’s academic community.

Fourth, from the global point of view, ACOMS+ can greatly help academic organizations to continue journal publishing by reducing the financial burden. Commercial submission and review systems such as Editorial Manager and ScholarOne charge significant system fees. In contrast, ACOMS+ is free for academic publishing organizations. If organizations can save the cost of using a submission and review system, then they can spend more on improving the quality of journal articles. They will be able to increase their number of journal citations and even be listed in global indexing databases.

Finally, in terms of global academic perspectives, ACOMS+ can help the advancement and digital transformation of academic publishing. For example, the quantitative and qualitative evaluation feature helps to better assess the quality of manuscripts on a more objective basis. It also ensures that the peer review process controls against fraudulent behavior by editors and chief editors who might be tempted to arbitrarily modify the content of the reviewer’s comment. The open peer review function, which discloses all the results of the review process, can create a transparent environment where the public can view and judge whether an article has undergone proper peer review. In added, ACOMS+ can play a core role to share Korean research outputs by linking review history globally thorough ORCID and supporting indexing in global indexing databases.


This study designed a system framework and step-by-step features for the development of a next-generation article submission and review system that supports open access publishing, open peer review, and open science. To achieve this goal, the technical and environmental limitations of ACOMS, which has been supporting Korean academic organizations as the primary article management system in Korea, were analyzed, and the features to be developed for a new submission and review system were derived by analyzing the requirements of stakeholders in the Korean academic community.

While new academic publishing models are being tried to improve the transparency, efficiency, and speed of the academic publishing process, submission and review platforms that enable the verification and disclosure of peer review history are also being promoted. This study aimed to develop a new submission and review system to gain a global competitive edge in the Korean research environment and to support an open science that encompasses open access publishing, open peer review, and research data sharing. Furthermore, it was linked to AccessON and DataON, KISTI’s two open science platforms, to enable data to be shared with various publishing support platforms.

Further studies are required after the development of the system features. First, future researchers will benefit from evaluating the usability of ACOMS+ and the convenience of its functions to check whether the needs of researchers and academic organizations have been adequately reflected and to derive future development directions. It will also be possible to conduct a survey of ACOMS+ journals and an evaluation by experts involved in scholarly communication and academic publishing. By doing so, qualitative verification of the excellence of ACOMS+ and receiving feedback on its shortcomings to improve the system will be available.

Second, this study described the content and results of research conducted by KISTI to develop ACOMS+. A comparative analysis of ACOMS+ and other major systems could be conducted in the future. A previous study has investigated and compared eight submission and review systems and their functions (Kim et al., 2018). However, over the years, each system will have improved its capabilities, and ACOMS, one of the systems examined in this study, has been upgraded to ACOMS+. Therefore, it would be significant to conduct another comparative analysis when ACOMS+ is truly mature to provide insight into the latest trends in peer review.

Third, as it targets journals in the field of science and technology, this study does not reflect the characteristics and requirements of academic fields such as humanities and social sciences. This will be addressed by follow-up studies that reflect the characteristics of various academic fields. In further research, features to fully support open peer review that reflect the environment of Korean academic organizations and the specificities of the Korean scholarly publishing distribution market could be studied.

Finally, from the perspective of the system, the transparency of peer review must be further strengthened. In the future, decentralized technologies such as blockchain will be able to be incorporated into the system (Tenorio-Fornés et al., 2021). Through a decentralized infrastructure, the entire peer review process will be opened to the public, and furthermore, it will be possible to develop a system that can prove the originality of manuscripts, review reports, tables, and figures.


This research was supported by Korea Institute of Science and Technology Information (KISTI) (No. K-23-L01-C02, 1711198421).


No potential conflict of interest relevant to this article was reported.



  1. Benos, D. J., Bashari, E., Chaves, J. M., Gaggar, A., Kapoor, N., LaFrance, M., Mans, R., Mayhew, D., McGowan, S., Polter, A., Qadri, Y., Sarfare, S., Schultz, K., Splittgerber, R., Stephenson, J., Tower, C., Walton, R. G., & Zotov, A. (2007). The ups and downs of peer review. Advances in Physiology Education, 31(2), 145-152. advan.00104.2006
  2. Csiszar, A. (2016). Peer review: Troubled from the start. Nature, 532(7599), 306-308.
  3. Daniel, H. D. (1993). Guardians of science: Fairness and reliability of peer review. VCH Verlagsgesellschaft.
  4. Edgar, B. D., & Willinsky, J. (2010). A survey of the scholarly journals using open journal systems. OJS Pa Dansk, 1(1), 1-40.
  5. Gahutu, J. B. (2017). Editorial: Publishing with ScholarOne manuscripts. Rwanda Journal, 4(1).
  6. Heise, C., & Pearce, J. M. (2020). From open access to open science: The path from scientific reality to open scientific communication. SAGE Open, 10(2).
  7. Jeong, Y., Ro, J. Y., Cho, S., & Ahn, S. (2022). A study on science and technology scholarly societies' understanding on open peer review. Journal of the Korea Contents Association, 22(10), 59-73.
  8. Kim, H., Kim, W., Kim, J., Kim, J., Kim, H., Nam, E., Ro, J., Park, S., Seo, T., Yoo, S., Ahn, B., Ahn, S., Lee, S., Jeong, S., Jung, Y., Jeong, Y., Chung, J., Chung, J., Cho, S., ... & Kim, S. (2022). Establishment of open access support system of science and technology information through digital transformation. Ministry of Science and ICT.
  9. Kim, N., Yoon, J. W., Lee, J. Y., Joung, K. H., Hwang, H., Bai, S. Y., & Chung, E. (2023). Development of a diagnostic framework and its application to open access journal publishing in Korea. Learned Publishing, 36(3), 379-393.
  10. Kim, S., Choi, H., Kim, N., Chung, E. K., & Lee, J. Y. (2018). Comparative analysis of manuscript management systems for scholarly publishing. Science Editing, 5(2), 124-134.
  11. Kirkham, J., & Moher, D. (2018). Who and why do researchers opt to publish in post-publication peer review platforms? - Findings from a review and survey of F1000 Research. F1000Research, 7, 920.
  12. Lee, Y. C., Yeom, J., Song, K., Ha, J., Lee, K., Yeo, J., & Kim, S. W. (2016, October 9-12). Recommendation of research papers in DBpia: A hybrid approach exploiting content and collaborative data. Proceedings of the 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (pp. 002966-002971). IEEE.
  13. Salem, R. M., Culbertson, N. M., & O'Connell, A. (2016). Process for selecting and implementing a manuscript management system: Experiences of a new peer-reviewed journal. Learned Publishing, 29(1), 55-59.
  14. Tennant, J. P., Dugan, J. M., Graziotin, D., Jacques, D. C., Waldner, F., Mietchen, D., Elkhatib, Y., B Collister, L., Pikas, C. K., Crick, T., Masuzzo, P., Caravaggi, A., Berg, D. R., Niemeyer, K. E., Ross-Hellauer, T., Mannheimer, S., Rigling, L., Katz, D. S., Greshake Tzovaras, B., … Colomb, J. (2017). A multidisciplinary perspective on emergent and future innovations in peer review. F1000Research, 6, 1151.
  15. Tenorio-Fornes, A., Tirador, E. P., Sanchez-Ruiz, A. A., & Hassan, S. (2021). Decentralizing science: Towards an interoperable open peer review ecosystem using blockchain. Information Processing & Management, 58(6), 102724.
  16. Watson, M. (2015). When will 'open science' become simply 'science'? Genome Biology, 16(1), 101.
  17. Wicherts, J. M. (2016). Peer review quality and transparency of the peer-review process in open access and subscription journals. PLoS One, 11(1), e0147913.
  18. Willinsky, J. (2005). Open journal systems: An example of open source software for journal management and publishing. Library Hi Tech, 23(4), 504-519.