Return-on-Investment Measurement and Assessment of Research Fund: A Case Study in Malaysia |
SANUSI, Nur Azura
(Department of Economics, Faculty of Business, Economics and Social Development, University Malaysia Terengganu)
SHAFIEE, Noor Hayati Akma (Department of Economics, Faculty of Business, Economics and Social Development, University Malaysia Terengganu) HUSSAIN, Nor Ermawati (Department of Economics, Faculty of Business, Economics and Social Development, University Malaysia Terengganu) ABU HASAN, Zuha Rosufila (Department of Management and Marketing, Faculty of Business, Economics and Social Development, University Malaysia Terengganu) ABDULLAH, Mohd Lazim (Department of Mathematics, Faculty of Ocean Engineering, Technology and Informatics, University Malaysia Terengganu) SA'AT, Nor Hayati (Department of Social Ecology, Faculty of Business, Economics and Social Development, University Malaysia Terengganu) |
1 | Audretsch, D. B., & Keilbach, M. (2004). Entrepreneurship capital and economic performance. Regional Studies, 38(8), 949-959. https://doi.org/10.1080/0034340042000280956 DOI |
2 | Svider, P. F., Husain, Q., Folbe, A. J., Couldwell, W. T. Liu, J. K., & Eloy, J. A. (2014). Assessing national institutes of health funding and scholarly impact in neurological surgery: Clinical article. Journal of Neurosurgery, 120, 1-295. https://doi.org/10.3171/2013.8.JNS13938 DOI |
3 | European Commission. (2010). Assessing Europe's universitybased research. https://ec.europa.eu/research/sciencesociety/document_library/pdf_06/assessing-europeuniversity-based-research_en.pdf |
4 | Frank, C., & Nason, E. (2009). Health research: Measuring the social, health, and economic impacts. Canadian Medical Association Journal, 180(5), 528-534. https://doi.org/10.1503/cmaj.090016 DOI |
5 | Friedman, J., & Silberman, J. (2003). University technology transfer: Do incentives, management, and location matter? The Journal of Technology Transfer, 28(1), 17-30. https://doi.org/10.1023/A:1021674618658 DOI |
6 | Botchkarev, A., & Andru, P. (2011). A return on investment as a metric for evaluating information systems: Taxonomy and application. Interdisciplinary Journal of Information, Knowledge, and Management, 6(6), 245-269. https://doi.org/10.28945/1535 DOI |
7 | Australian Research Council. (2017). Engagement and impact assessment pilot 2017. Commonwealth of Australia. http://www.arc.gov.au/sites/default/files/filedepot/Public/EI/Engagement_and_Impact_Assessment_Pilot_2017_Report.pdf. |
8 | Azoulay, P., Zivin, J. S. G., Li, D., & Sampat, B. N. (2019). Public R&D investments and private-sector patenting: Evidence from NIH funding rules. Review of Economic Studies, 86(1), 117-152. https://doi.org/10.1093/restud/rdy034 DOI |
9 | Belenzon, S., & Schankerman, M. (2013). Spreading the word: Geography, policy, and knowledge spillovers. Review of Economics and Statistics, 95(3), 884-903. https://doi.org/10.1162/REST_a_00334 DOI |
10 | Bunn, F., Trivedi, D., Alderson, P., Hamilton, L., Martin, A., Pinkney, E., & Iliffe, S. (2015). The impact of Cochrane Reviews: A mixed-methods evaluation of outputs from Cochrane Review Groups supported by the National Institute for Health Research. Health Technology Assessment, 19(28), 1-99. https://doi.org/10.3310/hta19280 DOI |
11 | Anderson, D. L., Smart, W., & Tressler, J. (2013). Evaluating research: Peer review team assessment and journal based bibliographic measures: New Zealand PBRF research output scores in 2006. New Zealand Economic Papers, 47(2), 140-157. https://doi.org/10.1080/00779954.2013.772879 DOI |
12 | Wooding, S., Hanney, S., Buxton, M., & Grant, J. (2005). Payback arising from research funding: Evaluation of the arthritis research campaign. Rheumatology, 44(9), 1145-1156. https://doi.org/10.1093/rheumatology/keh708 DOI |
13 | Fursov, K., Roschina, Y., & Balmush, O. (2016). Determinants of research productivity: An individual-level lens. Foresight and STI Governance, 10(2), 44-56. https://doi.org/10.17323/1995-459X.2016.2.44.56 DOI |
14 | Gellings, C. (2007). Outsourcing relationships: The contract as IT governance tool. 40th Annual Hawaii International Conference on System Sciences (HICSS '07), 3-6 Jan 2007, Big Island, Hawaii, USA (pp.236c-236c). US, Europe: The IT Governance Institute. https://doi.org/10.1109/HICSS.2007.421 DOI |
15 | Whitehead, A. N. (1929). The aims of education and other essays: Universities and their function. New York: Macmillan Company. |
16 | Rhoten, D., & Calhoun, C. (2011). Knowledge matters: The public mission of the research university. New York: Columbia University Press. |
17 | Oliveira, M. C. L. A., Martelli, D. R., Quirino, I. G., Colosimo, E. A., Silva, A. C. S., Junior, H. M., & De Oliveira, E. A. (2014). Profile and scientific production of the Brazilian Council for scientific and technological development (CNPq) researchers in the field of hematology/oncology. Revista da Associacao Medica Brasileira, 60(6), 542-547. https://doi.org/10.1590/1806-9282.60.06.012 DOI |
18 | Ozanne, J. L., Davis, B., Murray, J. B., Grier, S., Benmecheddal, A., Downey, H., Ekpo, A. E., Garnier, M., Hietanen, J., Le Gall-Ely, M., Seregina, A., Thomas, K. D., & Veer, E. (2016). Assessing the societal impact of research: The relational engagement approach. Journal of Public Policy and Marketing, 36(1), 1-14. https://doi.org/10.1509/jppm.14.121 DOI |
19 | Reddy, K. S., Xie, E., & Tang, Q. (2016). Higher education, highimpact research, and world university rankings: A Case of India and comparison with China. Pacific Science Review B: Humanities and Social Sciences, 2(1), 1-21. https://doi.org/10.1016/j.psrb.2016.09.004 DOI |
20 | Research Excellence Framework. (2014). Results. http://results.ref.ac.uk/ |
21 | Sanusi, N. A., Johari, M. S., & Padli, J. (2019). Science social scientific research. Kuala Nerus: UMT Publisher. |
22 | Heyeres, M., Tsey, K., Yang, Y., Yan, L., & Jiang, H. (2019). The characteristics and reporting quality of research impact case studies: A systematic review. Evaluation and Program Planning, 73, 10-23. https://doi.org/10.1016/j.evalprogplan.2018.11.002 DOI |
23 | Xing, Z., Yu, F., Du, J., Walker, J. S., Paulson, C. B., Mani, N. S., & Song, L. (2019). Conversational interfaces for health: Bibliometric analysis of grants, publications, and patents. Journal of Medical Internet Research, 21(11), e14672. https://doi.org/10.2196/14672 DOI |
24 | Yeo, B. (2018). The societal impact of university innovation. Management Research Review, 41(11), 1309-1335. https://doi.org/10.1108/MRR-12-2017-0430 DOI |
25 | Oakleaf, M. (2010). Value of academic libraries: A comprehensive research review and report. Chicago, IL: Association of College and Research Libraries. |
26 | Sekaran, U. (2002). Research methods for business (4th ed.). Hoboken, NJ: Wiley. |
27 | Seppala, N., & Smith, C. (2020). Teaching awards in higher education: A qualitative study of motivation and outcomes. Studies in Higher Education, 45(7), 1398-1412. https://doi.org/10.1080/03075079.2019.1593349 DOI |
28 | Silver, J. K., Bank, A. M., Slocum, C. S., Blauwet, C. A., Bhatnagar, S., Poorman, J. A., Goldstein, R., Reilly, J. M., & Zafonte, R. D. (2018). Women physicians were underrepresented in the American Academy of Neurology recognition awards. American Academy of Neurology, 91(7), e603-e614. https://doi.org/10.1212/WNL.0000000000006004 DOI |
29 | Hammond, G. W., Le, M., Novotny, T., Caligiuri, S. P. B., Pierce, G. N., & Wade, J. (2017). An output evaluation of a health research foundation's enhanced grant review process for new investigators. Health Research Policy and Systems, 15, 57. https://doi.org/10.1186/s12961-017-0220-x DOI |
30 | Hausman, N. (2012). University innovation, local economic growth, and entrepreneurship (Working Paper No. CESWP-12-10). U.S. Census Bureau Center for Economic Studies Paper. Maryland, US: U.S. Census Bureau. https://ssrn.com/abstract=2097842 |
31 | Hicks, D. M. (2004). Bibliometric evaluation of federally funded research in the United States. Research Evaluation, 13(2), 76-86. https://doi.org/10.3152/147154404781776446 DOI |
32 | Higher Education Funding Council. (2011). REF Assessment framework and guidance on submissions. Bristol, UK: HEFCE. |
33 | Audretsch, D. B. (2014). From the entrepreneurial university to the university for the entrepreneurial society. The Journal of Technology Transfer, 39(3), 313-321. https://doi.org/10.1007/s10961-012-9288-1 DOI |
34 | Mansfield, E. (1991). Academic research and industrial innovation. Research Policy, 20(1), 1-12. https://doi.org/10.1016/0048-7333(91)90080-A DOI |
35 | Sim, S. Y., Watts, E., Constenla, D., Brenzel, L., & Patenaude, B. N. (2020). Return on investment from immunization against 10 pathogens in 94 low and middle-income countries, 2011-30. Health Affairs, 39(8), 1343-1353. https://doi.org/10.1377/hlthaff.2020.00103 DOI |
36 | Strouse, R. (2003). Demonstrating value and return on investment: The ongoing imperative. Information Outlook, 7(3), 45-59. https://www.semanticscholar.org/paper/Demonstratingvalue-and-return-on-investment%3A-The-Strouse |
37 | Sanusi, N. A., Moosin, A. F., & Kusairi, S. (2020). Neural network analysis in forecasting the Malaysian GDP. Journal of Asian Finance, Economics, and Business, 7(12), 109-114. https://doi.org/10.13106/jafeb.2020.vol7.no12.109 DOI |
38 | Hinrichs-Krapels, S., & Grant, J. (2016). Exploring the effectiveness, efficiency, and equity (3e's) of research and research impact assessment. Palgrave Communications, 2, 16090. https://doi.org/10.1057/palcomms.2016.90 DOI |
39 | Ho, C. S. M., Lu, J. F., & Bryant, D. A. (2020). The impact of teacher entrepreneurial behavior: A timely investigation of an emerging phenomenon. Journal of Educational Administration, 58(6), 697-712. https://doi.org/10.1108/JEA-08-2019-0140 DOI |
40 | Guthrie, S., Bienkowska-Gibbs, T., Manville, C., Pollitt, A., Kirtley, A., & Wooding, S. (2015). The impact of the national institute for health research health technology assessment program, 2003-13: A multimethod evaluation. Health Technology Assessment, 19(67), 1-291. https://doi.org/10.3310/hta19670 DOI |
41 | Donovan, C., Butler, L., Butt, A. J., Jones, T. H., & Hanney, S. R. (2014). Evaluation of the impact of national breast cancer foundation-funded research. The Medical Journal of Australia, 200(4), 214-218. https://doi.org/10.5694/mja13.10798 DOI |
42 | Guerrero, M., Cunningham, J. A., & Urbano, D. (2015). The economic impact of entrepreneurial universities' activities: An exploratory study of the United Kingdom. Research Policy, 44(3), 748-764. https://doi.org/10.1016/j.respol.2014.10.008 DOI |
43 | Jones, L., Schedler, K., & Mussari, R. (2004). Strategies for public management reform. Elsevier. |
44 | Yunus, M. A. S., & Pang, V. (2015). Academic promotion in Malaysia: Meeting academics' expectations and institutional needs. RIHE International Seminar Reports, 23, 61-81. https://files.eric.ed.gov/fulltext/ED574241.pdf |
45 | El-Boghdadly, K., Docherty, A. B., & Klein, A. A. (2018). Analysis of the distribution and scholarly output from National Institute of Academic Anaesthesia (NIAA) research grants. Anaesthesia, 73(6), 679-691. https://doi.org/10.1111/anae.14277 DOI |
46 | Elsevier. (2019). Malaysia's R&D investment paying off with higher research productivity and improved university ranking. https://www.elsevier.com/about/press-releases/corporate/malaysias-r-and-d-investment-paying-off-withhigher-research-productivity-and-improved-universityranking |
47 | Tenopir, C., & King, D. W. (2007). Perceptions of value and value beyond perceptions: Measuring the quality and value of journal article readings. Serials, 20(3), 199-207. https://doi.org/10.1629/20199 DOI |
48 | Valero, A., & Van Reenen, J. (2016). The economic impact of universities: Evidence from across the globe. Economic of Education Review, 68, 53-67. https://doi.org/10.1016/j.econedurev.2018.09.001 DOI |
49 | Van Der Steen, M., & Enders, J. (2008). Universities in evolutionary systems of innovation. Creativity and Innovation Management, 17(4), 281-292. https://doi.org/10.1111/j.1467-8691.2008.00496.x DOI |
50 | Johnson, I. M., Williams, D. A., Wavell, C., & Baxter, G. (2004). Impact evaluation, professional practice, and policymaking. New Library World, 105(1/2), 33-46. https://doi.org/10.1108/03074800410515255 DOI |
51 | Joosse, G. (2009). Revolutionary changes in Alberta postsecondary funding: Comparison of two college's responses [Unpublished doctoral dissertation]. ProQuest database. |
52 | Joshi, M. A. (2014). Bibliometric indicators for evaluating the quality of scientific publications. The Journal Contemporary Dental Practice, 15(2), 258-262. https://doi.org/10.5005/jpjournals-10024-1525 DOI |
53 | Kalutkiewicz, M. J., & Ehman, R. L. (2014). Patents as proxies: NIH hubs of innovation. Nature Biotechnology, 32(6), 536-537. https://doi.org/10.1038/nbt.2917 DOI |
54 | Kantor, S., & Whalley, A. (2014). Knowledge spillovers from research universities: Evidence from endowment value shocks. Review of Economics and Statistics, 96(1), 171-188. https://doi.org/10.1162/REST_a_00357 DOI |
55 | Kingma, B., & McClure, K. (2015). Lib-value: Values, outcomes and return on investment of academic libraries, phase III: ROI of the Syracuse University Library. College & Research Libraries, 76(1), 63-80. https://doi.org/10.5860/crl.76.1.63 DOI |
56 | Wahab, E., Shamsuddin, A., Abdullah, N. H., & Abdul Hamid, N. (2016). Users' satisfaction and return on investment (ROI) for online database library databases: A Malaysian technical university perspective. Procedia-Social and Behavioral Sciences, 219, 777-783. https://doi.org/10.1016/j.sbspro.2016.05.075 DOI |
57 | White, L. N. (2007). An old tool with potential new uses: Return on investment. The Bottom Line, 20(1), 5-9. https://doi.org/10.1108/08880450710747407 DOI |
58 | Tenopir, C. (2012). Beyond usage: Measuring library outcomes and value. Library Management, 233(1/2), 5-13. https://doi.org/10.1108/01435121211203275 DOI |
59 | Kelly, U., & McNicoll, I. (2011). Through a glass, darkly: Measuring the social value of universities. National Co-ordinating Centre for Public Engagement. https://www.publicengagement.ac.uk/sites/default/files/80096%20NCCPE%20Social%20Value%20Report.pdf. |
60 | Hulten, C. (2001). Total factor productivity: A short biography. In C. R. Hulten, E. R. Dean, & M. J. Harper (Eds.), New developments in productivity analysis (pp. 1-54). Chicago, IL: University of Chicago Press. |
61 | Rau, H., Goggins, G., & Fahy, F. (2018). From invisibility to impact: Recognising the scientific and societal relevance of interdisciplinary sustainability research. Research Policy, 47(1), 266-276. https://doi.org/10.1016/j.respol.2017.11.005 DOI |
62 | Toivanen, O., & Vaananen, L. (2016). Education and invention. Review of Economics and Statistics, 98(2), 382-396. https://doi.org/10.1162/REST_a_00520 DOI |
63 | Peik, S., Schimmel, E., & Hejazi, S. (2019). Projected return on investment of a corporate global health program. BMC Public Health, 19, 1476. https://doi.org/10.1186/s12889-019-7857-z DOI |
64 | Penfield, T., Baker, M. J., Scoble, R., & Wykes, M. C. (2014). Assessment, evaluations, and definitions of research impact: A review. Research Evaluation, 23(1), 21-32. https://doi.org/10.1093/reseval/rvt021 DOI |
65 | Preuss, M. (2016). Return on investment and grants: A review of present understandings and recommendations for change. Research Management Review, 21(1), 1-26. https://files.eric.ed.gov/fulltext/EJ1134098.pdf |
66 | Ranasinghe, P., Jayawardena, R., & Katulanda, P. (2012). Sri Lanka in global medical research: A scientific analysis of the Sri Lankan research output during 2000-2009. BMC Research Notes, 5, 121. http://www.biomedcentral.com/1756-0500/5/121 DOI |
67 | Costello, L. (2020). A survey of Canadian academic librarians outlines the integration of traditional and emerging services. Evidence-Based Library and Information Practice, 15(3), 184-186. https://doi.org/10.18438/eblip29789 DOI |
68 | Corbyn, Z. (2009). Bang for your bucks. The Times Higher Education Supplement, 32, 43-56. |
69 | Hsu, D. W., Shen, Y.-C., Yuan, B. J., & Chou, C. J. (2015). Toward successful commercialization of university technology: Performance drivers of university technology transfer in Taiwan. Technological Forecasting and Social Change, 92, 25-39. https://doi.org/10.1016/j.techfore.2014.11.002 DOI |
70 | Lin, Y. Y. C., & Chen, Y. C. M. (2007). Does innovation lead to performance? An empirical study of SMEs in Taiwan. Management Research News, 30(2), 115-132. https://doi.org/10.1108/01409170710722955 DOI |
71 | Duryea, M., Hochman, M., & Parfitt, A. (2007). Measuring the impact of research. http://www.atn.edu.au/docs/Research%20Global%20-%20.Measuring%20the%20impact%20of%20research.pdf. |
72 | Elsayed, A. M. & Saleh, E. I. (2015). Measuring the return on investment of academic libraries in Arab countries: A proposed model. Information Development, 31(3), 219-228. https://doi.org/10.1177/0266666913512146 DOI |
73 | Carpenter, C. R., Cone, D. C., & Sarli, C. C. (2014). Using publication metrics to highlight academic productivity and research impact. Academic Emergency Medicine: Official Journal of the Society for Academic Emergency Medicine, 21(10), 1160-1172. https://doi.org/10.1111/acem.12482 DOI |
74 | Altbach, P. G. (2009). Peripheries and centers: Research universities in developing countries. Asia Pacific Education Review, 10, 15-27. https://doi.org/10.1007/s12564-009-9000-9 DOI |
75 | Ahmed, K. K. M., & Gupta, B. M. (2018). Nano-enabled drug delivery research: A scientometric assessment of Indian publications during 1995-2018. International Journal of Pharmaceutical Investigation, 8(4), 182-191. https://doi.org/10.4103/jphi.JPHI_11_19 DOI |
76 | Ahmed, Z., Shakoor, Z., Khan, M. A., & Ullah, W. (2021). The role of financial risk management in predicting financial performance: A case study of commercial banks in Pakistan. Journal of Asian Finance, Economics, and Business, 8(5), 0639-0648. https://doi.org/10.13106/jafeb.2021.vol8.no5.0639 DOI |
77 | Aldrich, H. E. (2012). The emergence of entrepreneurship as an academic field: A personal essay on institutional entrepreneurship. Research Policy, 41(7), 1240-1248. https://doi.org/10.1016/j.respol.2012.03.013 DOI |
78 | League of European Research Universities. (2013). Research universities and research assessment. Leuven: LERU. |
79 | Luther, J. (2008). University investment in the library: What's the return? A case study at the University of Illinois at UrbanaChampaign. Amsterdam, The Netherlands: Elsevier. |
80 | Macilwain, C. (2010). Science economics: What science is really worth. Nature, 465, 682-684. https://doi.org/10.1038/465682a DOI |
81 | Morton, S. (2015). Progressing research impact assessment: A 'contributions' approach. Research Evaluation, 24(4), 405-419. https://doi.org/10.1093/reseval/rvv016 DOI |
82 | Moriarty, P. (2011). False economies. The Times Higher Education Supplement 2010. Retrieved January 20, 2021, from: https://www.timeshighereducation.com/news/falseeconomies/417033.article |
83 | Mahmoud, A. S., Sanni-Anibire, M. O., Hassanain, M. A., & Ahmed, W. (2019). Key performance indicators for the evaluation of academic and research laboratory facilities. International Journal of Building Pathology and Adaptation, 37(2), 208-230. https://doi.org/10.1108/IJBPA-08-2018-0066 DOI |
84 | Mirani, M. A., & Yusof, M. (2016). Entrepreneurial engagements of academics in engineering Universities of Pakistan. Procedia Economics and Finance, 35, 411-417. https://doi.org/10.1016/S2212-5671(16)00051-4 DOI |
85 | Moaniba, I. M., Lee, P. C., & Su, H. N. (2020). How does external knowledge sourcing enhance product development? Evidence from drug commercialization. Technology in Society, 63, 101414. https://doi.org/10.1016/j.techsoc.2020.101414 DOI |
86 | Morgan, N. (2005). Characteristics associated with the effectiveness of resource development programs at Florida community colleges. Electronic Theses and Dissertations, 362, 55-129. https://stars.library.ucf.edu/etd/362 |
87 | Le, O. T. T., & Bui, N. T. (2020). Responsibility accounting in public universities: A case in Vietnam. Journal of Asian Finance, Economics, and Business, 7(7), 169-178. https://doi.org/10.13106/jafeb.2020.vol7.no7.169 DOI |
88 | Sweileh, W. M., Huijer, H. A., Al-Jabi, S. W., Zyoud, S. H., & Sawalha, A. F. (2019). Nursing and midwifery: Research activity in Arab countries from 1950 to 2017. BMC Health Services Research, 19, 340. https://doi.org/10.1186/s12913-019-4178-y DOI |
89 | Sweileh, W. M. (2018). Global research output on HIV/AIDS: Related medication adherence from 1980 to 2017. BMC Health Services Research, 18, 765. https://doi.org/10.1186/s12913-018-3568-x DOI |
90 | Sweileh, W. M., Al-Jabi, S. W., Zyoud, S. H., & Sawalha, A. F. (2018). Outdoor air pollution and respiratory health: A bibliometric analysis of publications in peer-reviewed journals (1900-2017). Multidisciplinary Respiratory Medicine, 13, 15. https://doi.org/10.1186/s40248-018-0128-5 DOI |
91 | Zikmund, W. G. (2003). Business research method (7th ed.). Nashville, United States: South-Western. |
92 | Kelly, U., McNicoll, I., & White, J. (2014). The impact of universities on the UK economy. London, UK: University UK. |
93 | Tenopir, C., Love, A., Park, J., Wu, L., Baer, A., & Mays, R. (2010). University investment in the library, phase II: An international study of the library's value to the grants process. Library Connect White Paper, Elsevier Library Connect. |
![]() |