An examination of the Indonesian journal ranking system
DOI:
https://doi.org/10.47989/ir31154705Keywords:
Academic publishing, Bibliometric Analysis, journal evaluation, Science and Technology Index, SINTAAbstract
Introduction. The study presents a thorough statistical analysis on the Science and Technology Index (SINTA), a journal ranking system established by the Indonesian Ministry of Higher Education. SINTA ranks Indonesian journals into six tiers (S1 to S6) based on scientific content and administrative compliance. The system differs from widely accepted quartile-based ranking system which ranks journal purely from citation perspective. This study assesses whether SINTA reflects journal quality when contrasted to the globally recognized citation-based indicators.
Method. The H5-index and Impact Factor distributions data of 10,478 Indonesian journals spanning over ten subject areas as of November 2024 were retrieved. Statistical analysis was conducted using Kruskal-Wallis and Dunn’s pairwise tests to detect and identify statistically distinguishable ranking clusters.
Analysis. The study analysed whether the six SINTA rankings (S1 – S6) corresponds to six distinct clusters when mapped out based on the H5-index and Impact Factor distributions. All analysis was performed on SPSS.
Results. The analysis showed that there are at most four statistically distinguishable groups, based on the H5-index and Impact distributions, instead of six independent groups as indicated by the six SINTA rankings. A substantial portion of non-adjacent SINTA rankings exhibited indistinguishable citation distributions. In some subject areas, all six SINTA rankings showed statistically indistinguishable citation-based distributions, indicating that SINTA fails to reflect citation-based quality.
Conclusions. Findings indicate that SINTA is not a reliable measure of journal quality and suggest adopting a more citation-based framework aligned with global practice.
References
Akoglu, H. (2018). User's guide to correlation coefficients. Turkish Journal of Emergency Medicine, 18, 91–93. https://doi.org/10.1016/j.tjem.2018.08.001
Akreditasi Jurnal Ilmiah. (2018). Permen Ristekdikti no 9 Tahun 2018. https://peraturan.bpk.go.id/Details/140402/permen-ristekdikti-no-9-tahun-2018 (Archived at https://web.archive.org/web/20250909083959/https://peraturan.bpk.go.id/Details/140402/permen-ristekdikti-no-9-tahun-2018)
Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2009). h-Index: A review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics, 3(4), 273–289. https://doi.org/10.1016/j.joi.2009.04.001
Antara News. (2017). Menristek luncurkan SINTA, portal kinerja peneliti. https://www.antaranews.com/berita/609670/menristek-luncurkan-sinta-portal-kinerja-peneliti (Archived at https://web.archive.org/web/20250614060223/https://www.antaranews.com/berita/609670/menristek-luncurkan-sinta-portal-kinerja-peneliti)
Aprile, K. T., Ellem, P., & Lole, L. (2021). Publish, perish, or pursue? Early career academics’ perspectives on demands for research productivity in regional universities. Higher Education Research & Development, 40(6), 1131–1145. https://doi.org/10.1080/07294360.2020.1804334
Asan, A., & Aslan, A. (2020). Quartile scores of scientific journals: Meaning, importance and usage. Acta Medica Alanya, 4(1), 102–108. https://doi.org/10.30565/medalanya.653661
Badan Strategi Kebijakan Dalam Negeri. (2017). Kemenristek Luncurkan Aplikasi SINTA 2.0. https://bskdn.kemendagri.go.id/website/kemenristek-luncurkan-aplikasi-sinta-2-0/ (Archived at https://web.archive.org/web/20250910235240/https://bskdn.kemendagri.go.id/website/kemenristek-luncurkan-aplikasi-sinta-2-0/)
Cagan, R. (2013). The San Francisco declaration on research assessment. DMM Disease Models and Mechanisms, 6(4), 869–870. https://doi.org/10.1242/dmm.012955
CiteScore, S. (n.d.). Elsevier. Retrieved February 9, 2025, from https://www.elsevier.com/products/scopus/metrics/citescore (Archived at https://web.archive.org/web/20250828045418/https://www.elsevier.com/products/scopus/metrics/citescore)
Di Bitetti, M. S., & Ferreras, J. A. (2017). Publish (in English) or perish: The effect on citation rate of using languages other than English in scientific publications. Ambio, 46(1), 121–127. https://doi.org/10.1007/s13280-016-0820-7
Fiala, D. (2022). Indonesia’s place in the research landscape of Southeast Asia. In Unisia (pp. 45–66). https://doi.org/10.20885/unisia.vol40.iss1.art3
Fry, C. V., Lynham, J., & Tran, S. (2023). Ranking researchers: Evidence from Indonesia. Research Policy, 52(5), 104753. https://doi.org/10.1016/j.respol.2023.104753
Galbraith, Q., Carlie Butterfield, A., & Cardon, C. (2023). Judging journals: How Impact Factor and other metrics differ across disciplines. College & Research Libraries, 84(6). https://doi.org/10.5860/crl.84.6.888
Garfield, E. (1999). Journal impact factor: A brief review. CMAJ. Canadian Medical Association Journal, 161(8), 979–980. https://pmc.ncbi.nlm.nih.gov/articles/instance/1230709/pdf/cmaj_161_8_979.pdf (Archived at https://web.archive.org/web/20250204171302/https://pmc.ncbi.nlm.nih.gov/articles/instance/1230709/pdf/cmaj_161_8_979.pdf)
Garfield, E. (2006). The history and meaning of the journal Impact Factor. JAMA, 295(1), 90–93. https://doi.org/10.1001/jama.295.1.90
González-Pereira, B., Guerrero-Bote, V. P., & Moya-Anegón, F. (2010). A new approach to the metric of journals scientific prestige: The SJR indicator. Journal of Informetrics, 4(3), 379–391. https://doi.org/10.1016/j.joi.2010.03.002
Hanson, M. A., Barreiro, P. G., Crosetto, P., & Brockington, D. (2024). The strain on scientific publishing. Quantitative Science Studies, 5(4), 823–843. https://doi.org/10.1162/qss_a_00327
Heffernan, T. (2021). Academic networks and career trajectory: ‘There’s no career in academia without networks.’ Higher Education Research & Development, 40(5), 981–994. https://doi.org/10.1080/07294360.2020.1799948
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. https://doi.org/10.1038/520429a
Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572. https://doi.org/10.1073/pnas.0507655102
Jaffé, R. (2020). Qualis: The journal ranking system undermining the impact of Brazilian science. Anais Da Academia Brasileira de Ciencias, 92(3), 1–13. https://doi.org/10.1590/0001-3765202020201116
Jin, B., & Wang, S. (1999). Division of SCI journal grading regions and distributions of Chinese papers. Scientific Research Management, 02, 2–8. https://doi.org/10.19571/j.cnki.1000-2995.1999.02.001
Lukman, L., Dimyati, M., Rianto, Y., Subroto, I. M. I., Sutikno, T., Hidayat, D. S., Nadhiroh, I. M., Stiawan, D., Haviana, S. F. C., Heryanto, A., & Yuliansyah, H. (2018). Proposal of the S-score for measuring the performance of researchers, institutions, and journals in Indonesia. Science Editing, 5(2), 135–141. https://doi.org/10.6087/KCSE.138
Moed, H. F. (2010). Measuring contextual citation impact of scientific journals. Journal of Informetrics, 4(3), 265–277. https://doi.org/10.1016/j.joi.2010.01.002
Moskaleva, O., Pislyakov, V., Sterligov, I., Akoev, M., & Shabanova, S. (2018). Russian Index of Science Citation: Overview and review. Scientometrics, 116(1), 449–462. https://doi.org/10.1007/s11192-018-2758-y
Pedoman Akreditasi Jurnal Ilmiah. (2021). Surat Keputusan Dirjen Dikti No 134/E/KPT/2021. https://arjuna.kemdikbud.go.id/ (Archived at https://web.archive.org/web/20250909104116/https://arjuna.kemdikbud.go.id/)
Robinson, P. G. (2024). How to get your work published. Community Dental Health, 41(3), 154–157. https://www.cdhjournal.org/issues/41-3-september-2024/1373-editorial-how-to-get-your-work-published (Archived at https://web.archive.org/web/20250417224207/https://www.cdhjournal.org/issues/41-3-september-2024/1373-editorial-how-to-get-your-work-published)
Sánchez-García, E., Martínez-Falcó, J., Seva-Larrosa, P., & Marco-Lajara, B. (2024). Delving into the analysis of scientific production and communication in academic literature. Journal of Librarianship and Information Science, 57(2), 433-449. https://doi.org/10.1177/09610006231223168
Schober, P., Boer, C., & Schwarte, L. A. (2018). Correlation coefficients: Appropriate use and interpretation. Anesthesia & Analgesia, 126(5), 1763–1768. https://doi.org/10.1213/ANE.0000000000002864
Szomszor, M., Adams, J., Fry, R., Gebert, C., Pendlebury, D. A., Potter, R. W. K., & Rogers, G. (2020). Interpreting bibliometric data. Frontiers in Research Metrics and Analytics, 5, 1–20. https://doi.org/10.3389/frma.2020.628703
Teixeira da Silva, J. A. (2020). CiteScore: Advances, evolution, applications, and limitations. Publishing Research Quarterly, 36(3), 459–468. https://doi.org/10.1007/s12109-020-09736-y
Walters, W. H. (2017). Citation-based journal rankings: Key questions, metrics, and data sources. IEEE Access, 5, 22036–22053. https://doi.org/10.1109/ACCESS.2017.2761400
Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365–391. https://doi.org/10.1016/j.joi.2016.02.007
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Eddy Yusuf
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
