Asian Journal of Innovation and Policy

Asian Society for Innovation and Policy (아시아기술혁신학회)
권호 표지
DOI QR코드

DOI QR Code

Ex-ante Evaluation Process for Public R&D: Korean Case and its Implications for Indonesian R&D System

  • Lee, Elly Hyanghee (Division of Global Innovation Strategy, Science and Technology Policy Institute) ;
  • Jang, Young-Sun (Korea Research Institute of Chemical Technology) ;
  • Ariyani, Luthfina (Research Center for Science, Technology, and Innovation Policy and Management, Indonesian Institute of Sciences) ;
  • Sari, Karlina (Research Center for Science, Technology, and Innovation Policy and Management, Indonesian Institute of Sciences) ;
  • Hardiyati, Ria (Research Center for Science, Technology, and Innovation Policy and Management, Indonesian Institute of Sciences)
  • Received : 2020.12.06
  • Accepted : 2020.12.26
  • Published : 2020.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

A variety of approaches are being applied to improve the existing ex-ante evaluation by expert panels in publicly funded R&D. While the objective evaluation criteria are constantly being improved to screen and select the superior projects, alternative approaches such as random prioritization and logical modeling are also underway to overcome the conservative bias of reviewers and to secure disruptive innovation. This study intends to find critical implications for ex-ante evaluation of public R&D system from the comparison of Indonesia and Korea. For the comparative analysis, literature review and expert in-depth interviews are conducted on the national R&D system and the selection evaluation process. In Korea, the selection criteria of projects are legally promulgated for establishing an objective evaluation system, and at the program level, the major considerations in the planning process are specified by Presidential Decree. On the other hand, while Indonesia conducts R&D in 47 strategic fields largely by public research institutes (PRI) based on the non-competitive government contributions. This study draws out implications of institutionalizing the planning process at the level of program, and of increasing the ratio of contract-based competitive funding at the level of project in the national R&D portfolio.

Keywords

References

  1. Arrow, K. (1962) Economic welfare and allocation of resources for invention. In R. Nelson (eds.). The rate and direction of inventive activity. Princeton: Princeton University Press, 609-625.
  2. Brezis, E.S. (2007) Focal randomisation: an optimal mechanism for the evaluation of R&D projects, Science and Public Policy, 34(10), 691-698. https://doi.org/10.3152/030234207X265394
  3. Bulathsinhala, N.A. (2015) Ex-ante evaluation of publicly funded R&D projects: Searching for exploration, Science and Public Policy, 42, 162-175. https://doi.org/10.1093/scipol/scu035
  4. Duch-Brown, N., Garcia-Quevedo, J., and Montolio, D. (2008) Assessing the assignation of public subsidies: Do the experts choose the most efficient R&D projects? World Review of Science Technology and Sustainable Development, 9(2).
  5. Heinze, T. (2008) How to sponsor ground-breaking research: a comparison of funding schemes, Science and Public Policy, 35(5), 302-318. https://doi.org/10.3152/030234208X317151
  6. Horrobin, D.F. (1996) Peer review of grant applications: A harbinger for mediocrity in clinical research? The Lancet, 348, 1293-1295. https://doi.org/10.1016/S0140-6736(96)08029-4
  7. Hwang, H.S., and Yu. J.C. (1998) R&D project evaluation model based on fuzzy set priority, Computers & Industrial Engineering, 35(3-4), 567-570. https://doi.org/10.1016/S0360-8352(98)00160-0
  8. Johnson, R.A. (1995) Evaluating the viability of on-going product oriented internal research and development projects: Fact or fiction? Computers & Industrial Engineering, 29(1-4), 573-577. https://doi.org/10.1016/0360-8352(95)00136-O
  9. Klenner, P., Husig, S., and Dowling, M. (2013) Ex-ante evaluation of disruptive susceptibility in established value networks - When are markets ready for disruptive innovations? Research Policy, 43, 914-927. https://doi.org/10.1016/j.respol.2013.09.006
  10. Klette, T.J., Moen, J., and Griliches, Z. (2000) Do subsidies to commercial R&D reduce market failures? Microeconometric evaluation studies, Research Policy, 29, 471-295. https://doi.org/10.1016/S0048-7333(99)00086-4
  11. Linton, J.D., Walsh, S., and Morabito, J. (2002) Analysis, ranking and selection of R&D projects in a portfolio, R&D Management, 32, 139-148. https://doi.org/10.1111/1467-9310.00246
  12. Martin, B. (1997) Peer review as scholarly conformity. In Suppression Stories, B. Martin, 69-83. Wollongong: Fund for Intellectual Dissent.
  13. Ministry of Research, Technology and Higher Education (MORTHE), Republic of Indonesia (2017) National research master plan (Rencana Induk Riset Nasional) (2017-2045) (in Indonesian).
  14. Ministry of Research, Technology and Higher Education (MORTHE), Republic of Indonesia (2018) MORTHE Decree 36/2018 on procedures for developing national research priorities and monitoring and evaluation mechanisms for the implementation of national research master plan (in Indonesian).
  15. Ministry of Research and Technology (2019) Prioritas Riset Nasional 2020-2024 Kebijakan untuk Mendorong Pengembangan dan Pemanfaatan Produksi Dalam Negeri [PowerPoint slides]. (in Indonesian). https://balitbanghub.dephub.go.id/file/138.
  16. Ministry of Research, Technology and Higher Education, Republic of Indonesia (2019a) National research priority (Prioritas Riset Nasional) (2020-2024) (in Indonesian).
  17. Ministry of Research, Technology and Higher Education, Republic of Indonesia (2019b) MORTHE Decree no. 38/2019 on National research priority 2020-2024 (in Indonesian).
  18. Ministry of Science and ICT, Republic of Korea (2017) Regulations on the Management, Etc. of National Research and Development Projects. Presidential Decree No. 28210, July 26, 2017.
  19. Ministry of Science and ICT, Republic of Korea (2018a) Framework Act on Science and Technology. Act No. 15556, April 17, 2018.
  20. Ministry of Science and ICT, Republic of Korea (2018b) Enforcement Decree of the Framework Act on Science and Technology. Act No. 28800, April 17, 2018.
  21. Ministry of Science and ICT, Republic of Korea (2020a) Manual on management of national R&D programs. Sejong: MSIT (in Korean).
  22. Ministry of Science and ICT, Republic of Korea (2020b) Notification no. 2020 - 0324. Sejong: MSIT (in Korean).
  23. Ministry of Trade, Industry and Energy, Republic of Korea (2020) Notification no. 2020 - 537. Sejong: MOTIE (in Korean).
  24. Oey-Gardiner, M. (2011) In search of an identity for the DRN. Final Report. Jakarta: Insan Hitawasana Sejahtera: 1-28
  25. Park, J.M., Lim. S.I., and Seol, S.S. (2017) Measurement of public research outcomes: A technology valuation method, Asian Journal of Innovation and Policy, 6(2), 206-224. https://doi.org/10.7545/ajip.2017.6.2.206
  26. Park, S.Y., Son, J.K., Seo, J.H., and Seo, J. (2013) Performance evaluation index of TRM: A Korean case for SMEs, Asian Journal of Innovation and Policy, 2(1), 063-096. https://doi.org/10.7545/ajip.2013.2.1.063
  27. Republic of Indonesia (2002) Law no. 18/2002 on the National systems of research, development and application of science and technology (in Indonesian).
  28. Republic of Indonesia (2019) Law no. 19/2019 on National system of science and technology (in Indonesian).
  29. Rigby, J. (2002) Expert panels and peer review. In European Commission, RTD Evaluation Toolbox - Assessing the socio-economic impact of RTD policies, IPTS Technical Report series EUR 20382 EN, Institute for Prospective Technological Studies y Joanneum Research.
  30. Roper, S., Hewitt-Dundas, N., and Love, J.H. (2004) An ex ante evaluation framework for the regional benefits of publicly supported R&D projects, Research Policy, 33, 487-509. https://doi.org/10.1016/j.respol.2003.10.002
  31. Roy, R. (1985) The real defects of peer review and an alternative to it, 10(3), 73-81. https://doi.org/10.1177/016224398501000309
  32. Sampurno-Kuffal, F. H. (2011). The Collapse of Indonesia's Strategic Industries. Jakarta: Khazanah Bahari (in Indonesian).
  33. Setiadarma, E.G. (2018) Understanding the Evidence-Based Policy Making (EBPM) Discourse in the Making of the Master Plan of National Research (RIRN) Indonesia 2017-2045. STI Policy Review, 9(1), 30-54. https://doi.org/10.22675/STIPR.2018. 9.1.030.
  34. Thee, K. W. (2012). Indonesia's Economy since Independence. Singapore: ISEAS.
  35. Yim, D.S., Kim, S.S., Ko, Y.J., Kwon, K.S., Lee, E.H.H., Hidayat, D., Indraprahasta, G., Asmara, A. Y., Triyono, B., Akbar, M., Pradana, A., Dinaseviani, A., Purwaningsih, I., Sari, K., Hardiyati, R., Ariyani, L. (forthcoming). Policy Consultation on the Arrangement of Science, Technology and Innovation (STI) Governance: Coordination Mechanism, Institutional Framework & Financial System and Investment in Indonesia. Sejong: Science and Technology Policy Institute.