Scopus weighted CiteScore: A better alternative to plain CiteScore

Authors

DOI:

https://doi.org/10.47909/ijsmc.170

Keywords:

plain CiteScore, weighted CiteScore, Scopus journals, citation impact, research evaluation, journals ranking, education journals

Abstract

Objective. CiteScore has become a widely used tool for assessing journal performance in recent years. This paper aims to show the limitations of the plain CiteScore and propose a better alternative: the weighted CiteScore, which captures the percentage of publications cited and improves journal ranking.

Design/Methodology/Approach. Using an exploratory research methodology, we show the limitation of a plain CiteScore, calculated by dividing the past four years' citations by publications in the past four years. We demonstrate how a plain CiteScore can convey misleading results about the overall quality of a journal based on one or a few high-performing publications. Using the Wilcoxon signed-rank test, we prove that the journal Citescore ranking changed significantly using a weighted CiteScore method.

Results/Discussion. The analysis shows how a single open-access paper’s citation can significantly swing the overall ranks and percentiles in a specific domain due to plain CiteScore. Weighted CiteScore better represents the journal's performance, considering the number of publications cited.

Conclusions. Results of ranking journals based on plain CiteScore can be misleading. Our hypothetical and empirical analysis shows the need for a weighted CiteScore methodology. There has to be a level playing field by factoring in the percentage of publication citations.

Originality/value. The paper makes a novel contribution by suggesting an accurate and fair performance metric. It will be of significant value to libraries and researchers when assessing the quality of a publication.

Downloads

Download data is not yet available.

References

Ahn, B. S. (2011). Compatible weighting method with rank order centroid: Maximum entropy ordered weighted averaging approach. European journal of operational research, 212(3), 552-559. https://doi.org/10.1016/j.ejor.2011.02.017 DOI: https://doi.org/10.1016/j.ejor.2011.02.017

Al-Hoorie, AH, Vitta, JP. (2019). The seven sins of L2 research: A review of 30 journals’ statistical quality and their CiteScore, SJR, SNIP, JCR Impact Factors. Language Teaching Research, 23(6), 727-44. https://doi.org/10.1177/1362168818767191 DOI: https://doi.org/10.1177/1362168818767191

Ali, M.F. (2021). Evaluating the correlation between different impact indicators for library and information science journals: Comparing the journal citation reports and scopus. Learned Publishing, 34(3), 315-30. https://doi.org/10.1002/leap.1353 DOI: https://doi.org/10.1002/leap.1353

Archambault, É., & Larivière, V. (2009). History of the journal impact factor: Contingencies and consequences. Scientometrics, 79(3), 635–649. https://doi.org/10.1007/s11192-007-2036-x DOI: https://doi.org/10.1007/s11192-007-2036-x

Atayero, A.A., Popoola, S.I., Egeonu, J., & Oludayo, O. (2018). Citation analytics: Data exploration and comparative analyses of CiteScores of Open Access and Subscription-Based publications indexed in Scopus (2014–2016). Data in brief, 19, 198-213. https://doi.org/10.1016/j.dib.2018.05.005 DOI: https://doi.org/10.1016/j.dib.2018.05.005

Brown, T., & Gutman, S.A. (2018). Impact factor, eigenfactor, paper influence, scopus SNIP, and SCImage journal rank of occupational therapy journals. Scandinavian Journal of Occupational Therapy, 26(7), 475-483. https://doi.org/10.1080/11038128.2018.1473489 DOI: https://doi.org/10.1080/11038128.2018.1473489

Chatterjee, A., Ghosh, A., & Chakrabarti, B. K. (2016). Universality of citation distributions for academic institutions and journals. PloS one, 11(1), e0146762. https://doi.org/10.1371/journal.pone.0146762 DOI: https://doi.org/10.1371/journal.pone.0146762

Colledge, L., James, C., Azoulay, N., Meester, W., Plume, A. (2017). CiteScore metrics are suitable to address different situations–A case study. Euro. Sci. Edit., 43(2), 27-31. DOI: 10.20316/ESE.2017.43.003. DOI: https://doi.org/10.20316/ESE.2017.43.003

Colson, A. R., & Cooke, R. M. (2017). Cross validation for the classical model of structured expert judgment. Reliability Engineering & System Safety, 163, 109-120. https://doi.org/10.1016/j.ress.2017.02.003 DOI: https://doi.org/10.1016/j.ress.2017.02.003

Croft, W.L., & Sack, J.R. (2022). Predicting the citation count and CiteScore of journals one year in advance. Journal of Informetrics, 16(4), 101349. https://doi.org/10.1016/j.joi.2022.101349 DOI: https://doi.org/10.1016/j.joi.2022.101349

Dana, J., & Dawes, R. M. (2004). The superiority of simple alternatives to regression for social science predictions. Journal of Educational and Behavioral Statistics, 29(3), 317-331. https://doi.org/10.3102/10769986029003317 DOI: https://doi.org/10.3102/10769986029003317

Dias, N.W. (2021). The growing international relevance of Ambiente & Água according to Scopus CiteScore results. Revista Ambiente & Água, 16. https://doi.org/10.4136/ambi-agua.2670 DOI: https://doi.org/10.4136/ambi-agua.2670

Erfanmanesh, M. (2017). Status and quality of open access journals in Scopus. Collection building, 36(4), 155-162. https://doi.org/10.1108/CB-02-2017-0007 DOI: https://doi.org/10.1108/CB-02-2017-0007

Fang, H. (2021). Analysis of the new Scopus CiteScore. Scientometrics, 126(6), 5321-31. https://doi.org/10.1007/s11192-021-03964-5 DOI: https://doi.org/10.1007/s11192-021-03964-5

Fernandez-Llimos, F. (2018). Differences and similarities between journal impact factor and citescore. Pharmacy Practice (Granada), 16(2). https://dx.doi.org/10.18549/pharmpract.2018.02.1282 DOI: https://doi.org/10.18549/PharmPract.2018.02.1282

Ghosh, A., & Chakrabarti, B. K. (2021). Limiting value of the Kolkata index for social inequality and a possible social constant. Physica A: Statistical Mechanics and its Applications, 573, 125944. https://doi.org/10.1016/j.physa.2021.125944 DOI: https://doi.org/10.1016/j.physa.2021.125944

Ghosh, A., Chattopadhyay, N., & Chakrabarti, B. K. (2014). Inequality in societies, academic institutions and science journals: Gini and k-indices. Physica A: Statistical Mechanics and its Applications, 410, 30-34. https://doi.org/10.1016/j.physa.2014.05.026 DOI: https://doi.org/10.1016/j.physa.2014.05.026

Graham, M. A., Chakraborti, S., & Human, S. W. (2011). A nonparametric exponentially weighted moving average signed-rank chart for monitoring location. Computational Statistics & Data Analysis, 55(8), 2490-2503. https://doi.org/10.1016/j.csda.2011.02.013 DOI: https://doi.org/10.1016/j.csda.2011.02.013

Henao-Rodríguez, C., Lis-Gutiérrez, J.P., Bouza, C., Gaitán-Angulo, M., Viloria, A. (2019). Citescore of publications indexed in Scopus: an implementation of panel data. In International Conference on Data Mining and Big Data (pp. 53-60). Springer, Singapore. DOI: 10.1007/978-981-32-9563-6_6 DOI: https://doi.org/10.1007/978-981-32-9563-6_6

Hidouri, M., & Rebai, A. (2019). A multi-attribute ranking approach based on net inferiority and superiority indexes, two weight vectors, and generalized Heronian means. Decision Science Letters, 8(4), 471-482. http://dx.doi.org/10.5267/j.dsl.2019.4.005 DOI: https://doi.org/10.5267/j.dsl.2019.4.005

Higher Education for the Future, (2024). Accessed from https://journals.sagepub.com/home/HEF

James, C., Colledge, L., Meester, W., Azoulay, N., & Plume, A. (2018). CiteScore metrics: Creating journal metrics from the Scopus citation index. arXiv preprint. https://doi.org/10.48550/arXiv.1812.06871 DOI: https://doi.org/10.1002/leap.1246

Kapilan, N., Vidhya, P., & Gao, X. Z. (2021). Virtual laboratory: A boon to the mechanical engineering education during covid-19 pandemic. Higher Education for the Future, 8(1), 31-46. https://doi.org/10.1177/2347631120970757 DOI: https://doi.org/10.1177/2347631120970757

Krauskopf E. (2020). Sources without a CiteScore value: more clarity is required. Scientometrics, 122(3), 1801-12. https://doi.org/10.1007/s11192-020-03350-7 DOI: https://doi.org/10.1007/s11192-020-03350-7

Kumar, A., Paliwal, J. M., Brar, V., Singh, M., Patil, P. R. T., & Raibagkar, S. S. (2023). Previous Year’s Cite Score Strongly Predicts the Next Year’s Score: Ten Years of Evidence for the Top 400 Scopus-indexed Journals of 2021. Journal of Scientometric Research, 12(2), 254-263. https://jscires.org/full-text/6557/ DOI: https://doi.org/10.5530/jscires.12.2.020

Li, Y., Wu, C., Yan, E., Li, K. (2018). Will open access increase journal CiteScores? An empirical investigation over multiple disciplines. PloS one, 13(8), e0201885. https://doi.org/10.1371/journal.pone.0201885 DOI: https://doi.org/10.1371/journal.pone.0201885

Liu, Z. (2021). A bibliometric study of family studies journals using journal impact factors, CiteScore and H-index. International Journal of Librarianship, 6(1), 1-2. https://doi.org/10.23974/ijol.2021.vol6.1.174 DOI: https://doi.org/10.23974/ijol.2021.vol6.1.174

Martin, B. R. (2016). Editors’ JIF-boosting stratagems – Which are appropriate and which not? Research Policy, 45(1), 1–7. https://doi.org/10.1016/j.respol.2015.09.001 DOI: https://doi.org/10.1016/j.respol.2015.09.001

Matthews, D. (2015). Journal impact factors ‘no longer credible.’ Available at https://www.timeshighereducation.com/news/journal-impact-factors-no-longer-credible.

Meho, L.I. (2019). Using Scopus’s CiteScore for assessing the quality of computer science conferences. Journal of Informetrics, 13(1), 419-33. https://doi.org/10.1016/j.joi.2019.02.006 DOI: https://doi.org/10.1016/j.joi.2019.02.006

Okagbue, H.I., Akhmetshin, E.M., Teixeira da Silva, J.A. (2021). Distinct clusters of CiteScore and percentiles in top 1000 journals in Scopus. COLLNET Journal of Scientometrics and Information Management, 15(1), 133-43. https://doi.org/10.1080/09737766.2021.1934604 DOI: https://doi.org/10.1080/09737766.2021.1934604

Okagbue, H.I., Atayero, A.A., Adamu, M.O., Bishop, S.A., Oguntunde, P.E., Opanuga, A.A. (2018). Exploration of editorial board composition, Citescore and percentiles of Hindawi journals indexed in Scopus. Data in Brief, 19, 743-52. https://doi.org/10.1016/j.dib.2018.05.066 DOI: https://doi.org/10.1016/j.dib.2018.05.066

Okagbue, H.I., Bishop, S.A., Adamu, P.I., Opanuga, A.A., & Obasi E.C. (2020). Analysis of percentiles of computer science, theory and methods journals: CiteScore versus impact factor. DESIDOC Journal of Library & Information Technology, 40(1), 359-65. DOI: 10.14429/djlit.40.1.14866 DOI: https://doi.org/10.14429/djlit.40.01.14866

Okagbue H.I., Bishop, S.A., Oguntunde, P.E., Adamu, P.I., Opanuga, A.A., Akhmetshin, E.M. (2019). Modified CiteScore metric for reducing the effect of self-citations. Telkomnika (Telecommunication Computing Electronics and Control), 17(6), 3044-9. http://doi.org/10.12928/telkomnika.v17i6.12292 DOI: https://doi.org/10.12928/telkomnika.v17i6.12292

Okagbue, H.I., & Teixeira da Silva, J.A. (2020). Correlation between the CiteScore and Journal Impact Factor of top-ranked library and information science journals. Scientometrics, 124(1), 797-801. https://doi.org/10.1007/s11192-020-03457-x DOI: https://doi.org/10.1007/s11192-020-03457-x

Pokhrel, S., & Chhetri, R. (2021). A literature review on impact of COVID-19 pandemic on teaching and learning. Higher education for the future, 8(1), 133-141. https://doi.org/10.1177/2347631120983481 DOI: https://doi.org/10.1177/2347631120983481

Rajkumar, K.V., Adimulam, Y., Subrahmanyam, K. (2018). A critical study and analysis of journal metric ‘CiteScore’cluster and regression analysis. International Journal of Engineering & Technology, 7(2.7), 28-32. DOI: https://doi.org/10.14419/ijet.v7i2.7.10251

Ramentol, E., Vluymans, S., Verbiest, N., Caballero, Y., Bello, R., Cornelis, C., & Herrera, F. (2014). IFROWANN: imbalanced fuzzy-rough ordered weighted average nearest neighbor classification. IEEE Transactions on Fuzzy Systems, 23(5), 1622-1637. https://doi.org/10.1109/TFUZZ.2014.2371472 DOI: https://doi.org/10.1109/TFUZZ.2014.2371472

Sage. (2020). SAGE Waives Paper Processing Charges for Research Related to COVID-19. Accessed from https://us.sagepub.com/en-us/nam/press/sage-waives-paper-processing-charges-for-research-related-to-covid-19

Salisbury, L. (2020). Scopus CiteScore and Clarivate Journal Citation Reports. The Charleston Advisor. 21(4), 5-15. https://doi.org/10.5260/chara.21.4.5 DOI: https://doi.org/10.5260/chara.21.4.5

Scopus. (2024). Sources. Available from https://www.scopus.com/sources.uri?zone=TopNavBar&origin=

Stanley, T. D., & Doucouliagos, H. (2015). Neither fixed nor random: weighted least squares meta‐analysis. Statistics in medicine, 34(13), 2116-2127. https://doi.org/10.1002/sim.6481 DOI: https://doi.org/10.1002/sim.6481

Tarvainen, A., & Valpola, H. (2017). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Advances in neural information processing systems, 30. Available from https://proceedings.neurips.cc/paper/2017/hash/68053af2923e00204c3ca7c6a3150cf7-Abstract.html

Teixeira da Silva, J.A., & Memon, A.R. (2017). CiteScore: A cite for sore eyes, or a valuable, transparent metric?. Scientometrics, 111(1), 553-6. https://doi.org/10.1007/s11192-017-2250-0 DOI: https://doi.org/10.1007/s11192-017-2250-0

Teixeira da Silva, J.A. (2020). CiteScore: Advances, evolution, applications, and limitations. Publishing Research Quarterly, 36(3), 459-68. https://doi.org/10.1007/s12109-020-09736-y DOI: https://doi.org/10.1007/s12109-020-09736-y

Teixeira da Silva, J.A. (2021). CiteScore: risk of copy-cat, fake and misleading metrics. Scientometrics. 126(2):1859-62. https://doi.org/10.1007/s11192-020-03791-0 DOI: https://doi.org/10.1007/s11192-020-03791-0

Torres, J. (2022). An Innovative Approach to Bridging Open Access, Collection Development, and Faculty: Altmetric and CiteScore Analyses at a Large Public University. Qualitative and Quantitative Methods in Libraries, 11(2), 385-411. Available from https://www.qqml-journal.net/index.php/qqml/paper/view/769

Trapp, J.V. (2020). The new Scopus CiteScore formula and the Journal Impact Factor: a look at top ranking journals and middle ranking journals in the Scopus categories of General Physics and Astronomy, Materials Science, General Medicine and Social Sciences. Physical and Engineering Sciences in Medicine, 43(3), 739-48. https://doi.org/10.1007/s13246-020-00903-1 DOI: https://doi.org/10.1007/s13246-020-00903-1

Van Noorden, R. (2016). Impact factor gets heavyweight rival. Journal Citation Reports, 30(20). Available from https://www.nature.com/articles/nature.2016.21131.pdf

Vanclay, J. K. (2012). Impact factor: Outdated artefact or stepping-stone to journal certification? Scientometrics, 92(2). https://doi.org/10.1007/s11192-011-0561-0 DOI: https://doi.org/10.1007/s11192-011-0561-0

Villaseñor-Almaraz, M., Islas-Serrano, J., Murata, C., Roldan-Valadez, E. (2019). Impact factor correlations with Scimago journal rank, source normalized impact per paper, Eigenfactor score, and the CiteScore in radiology, nuclear medicine & medical imaging journals. La radiologia medica, 124(6), 495-504. https://doi.org/10.1007/s11547-019-00996-z DOI: https://doi.org/10.1007/s11547-019-00996-z

Wahakit, S., Boonsom, N., Kusakunniran, W., Thongkanchorn, K. (2021). Construction of CiteScore based metric for Conferences on a subject area of Computer Science in Scopus. In 2021 25th International Computer Science and Engineering Conference (ICSEC). (pp. 289-294). IEEE. https://doi.org/10.1109/ICSEC53205.2021.9684581 DOI: https://doi.org/10.1109/ICSEC53205.2021.9684581

Xi, M., Sun, J., & Xu, W. (2008). An improved quantum-behaved ppaper swarm optimization algorithm with weighted mean best position. Applied Mathematics and Computation, 205(2), 751-759. https://doi.org/10.1016/j.amc.2008.05.135 DOI: https://doi.org/10.1016/j.amc.2008.05.135

Yadlowsky, S., Fleming, S., Shah, N., Brunskill, E., & Wager, S. (2024). Evaluating treatment prioritization rules via rank-weighted average treatment effects. Journal of the American Statistical Association, 1-25. https://doi.org/10.1080/01621459.2024.2393466 DOI: https://doi.org/10.1080/01621459.2024.2393466

Zolfaghari, Z., Shokrpour, N., Ghahramani, L., & Sarveravan, P. (2022). CiteScores of cardiology and cardiovascular journals indexed in Scopus in 2019: A bibliometric analysis. European Science Editing, 48, e73949. https://doi.org/10.3897/ese.2022.e73949 DOI: https://doi.org/10.3897/ese.2022.e73949

Iberoamerican Journal of Science Measurement and Communication

Downloads

Published

2025-01-23

How to Cite

Kumar, A., Gawande, A., Kale, S., Agarwal, A., Brar, V., & Raibagkar, S. (2025). Scopus weighted CiteScore: A better alternative to plain CiteScore. Iberoamerican Journal of Science Measurement and Communication, 5(1), 1–15. https://doi.org/10.47909/ijsmc.170

Issue

Vol. 5 No. 1 (2025): Iberoamerican Journal of Science Measurement and Communication

Section

Original article

License

Copyright (c) 2025 Atul Kumar, Amol Gawande, Shailendrakumar Kale, Akash Agarwal, Vinaydeep Brar, Shirish Raibagkar

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

This is an open access article distributed under the terms of the CC BY-NC 4.0, which permits copying and redistributing the material in any medium or format, adapting, transforming, and building upon the material as long as the license terms are followed.