Beyond the loop: a research agenda towards a framework for critical AI literacy in the AI-assisted literature review
DOI:
https://doi.org/10.47989/ir31iConf64166Keywords:
Archival intelligence, Literature review, Automate literature review, Academic writingAbstract
Introduction. AI tools hold significant promise for enhancing the efficiency and transparency of academic literature reviews by streamlining literature search and data synthesis. However, their uncritical use poses risks to methodological rigor and academic integrity due to algorithmic bias, fabricated information (‘hallucinations’), and opaque AI processes. Current ‘human-in-the-loop’ guidelines often lack clear oversight protocols, potentially fostering intellectual passivity.
Method. Through a critical literature review, this paper identifies three paradoxes: the Verification Paradox, where efficiency compromises factual integrity; the Homogenization Paradox, where bias reinforces dominant perspectives, stifling innovation; and the Automation Paradox, where opaque algorithms undermine critical thinking and rigor.
Analysis/Results. To address these challenges, we propose a ‘framework for critical AI literacy in IS’, grounded in epistemological agility, methodological transparency, and ethical responsibility. This framework must offer actionable guidance for information systems scholars to responsibly integrate AI into literature reviews, ensuring human expertise and intellectual integrity remain central.
Conclusion. By fostering critical engagement with AI tools, this study sets a research agenda for a robust, transparent, and ethically sound research practices in an AI-Assisted research landscape.
References
Attard-Frost, B., De los Ríos, A., & Walters, D. R. (2023). The ethics of AI business practices: A review of 47 AI ethics guidelines. AI and Ethics, 3(2), 389–406. https://doi.org/10.1007/s43681-022-00156-6
Bawack, R. E., Fosso Wamba, S., & Carillo, K. D. A. (2021). A framework for understanding artificial intelligence research: Insights from practice. Journal of Enterprise Information Management, 34(2), 645–678. https://doi.org/10.1108/JEIM-07-2020-0284
Boell, S. K., & Cecez-Kecmanovic, D. (2015). On being ‘Systematic’ in Literature Reviews in IS. Journal of Information Technology, 30(2), 161–173. https://doi.org/10.1057/jit.2014.26
Bolaños, F., Salatino, A., Osborne, F., & Motta, E. (2024). Artificial intelligence for literature reviews: Opportunities and challenges. Artificial Intelligence Review, 57(10), 259. https://doi.org/10.1007/s10462-024-10902-3
Casal-Otero, L., Catala, A., Fernández-Morante, C., Taboada, M., Cebreiro, B., & Barro, S. (2023). AI literacy in K-12: A systematic literature review. International Journal of STEM Education, 10(1), 29. https://doi.org/10.1186/s40594-023-00418-7
Chiu, T. K. F., Ahmad, Z., Ismailov, M., & Sanusi, I. T. (2024). What are artificial intelligence literacy and competency? A comprehensive framework to support them. Computers and Education Open, 6, 100171. https://doi.org/10.1016/j.caeo.2024.100171
Cram, W., Templier, M., & Paré, G. (2020). (Re)considering the Concept of Literature Review Reproducibility. Journal of the AIS. https://doi.org/10.17705/1JAIS.00630
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., … Wright, R. (2023). Opinion Paper: ‘So what if ChatGPT wrote it?’ Multidisciplinary perspectives on opportunities, challenges, and implications of generative conversational AI for research, practice, and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642
Else, H. (2023). Abstracts written by ChatGPT fool scientists. Nature, 613(7944), 423–423. https://doi.org/10.1038/d41586-023-00056-7
Haefner, N., Parida, V., Gassmann, O., & Wincent, J. (2023). Implementing and scaling artificial intelligence: A review, framework, and research agenda. Technological Forecasting and Social Change, 197, 122878. https://doi.org/10.1016/j.techfore.2023.122878
Huettemann, S., Mueller, R., Larsen, K., & Dinter, B. (2024). A Framework for Ontology-Based Knowledge Synthesis from Research Articles.
Kacena, M. A., Plotkin, L. I., & Fehrenbacher, J. C. (2024). The Use of Artificial Intelligence in Writing Scientific Review Articles. Current Osteoporosis Reports, 22(1), 115–121. https://doi.org/10.1007/s11914-023-00852-0
Kittler, H. (2024). Will AI revolutionize literature reviews? Journal of the European Academy of Dermatology and Venereology, 38(12), 2213–2214. https://doi.org/10.1111/jdv.20354
Lu, J., & Mayhook, Z. A. (2024). AI Tools for Literature Reviews.
Luomala, M., Naarmala, J., & Tuomi, V. (2025). Technology-Assisted Literature Reviews with Technology of Artificial Intelligence: Ethical and Credibility Challenges. Procedia Computer Science, 256, 378–387. https://doi.org/10.1016/j.procs.2025.02.133
Messeri, L., & Crockett, M. J. (2024). Artificial intelligence and illusions of understanding in scientific research. Nature, 627(8002), 49–58. https://doi.org/10.1038/s41586-024-07146-0
Miao, J., Thongprayoon, C., Suppadungsuk, S., Garcia Valencia, O. A., Qureshi, F., & Cheungpasitporn, W. (2024). Ethical Dilemmas in Using AI for Academic Writing and an Example Framework for Peer Review in Nephrology Academia: A Narrative Review. Clinics and Practice, 14(1), 89–105. https://doi.org/10.3390/clinpract14010008
Michalak, R., & Ellixson, D. (2024). AI-Driven Discovery: How Litmaps Shapes Research and Teaching & Learning. The Serials Librarian, 85(5–6), 117–129. https://doi.org/10.1080/0361526X.2024.2433255
Mirbabaie, M., Brendel, A., & Hofeditz, L. (2022). Ethics and AI in Information Systems Research. Communications of the Association for Information Systems, 50(1). https://doi.org/10.17705/1CAIS.05034
Morandini, S., Fraboni, F., Balatti, E., Hackmann, A., Brendel, H., Puzzo, G., Volpi, L., Giusino, D., Angelis, M. D., & Pietrantoni, L. (2023). Assessing the Transparency and Explainability of AI Algorithms in Planning and Scheduling tools: A Review of the Literature. Human Interaction & Emerging Technologies (IHIET 2023): Artificial Intelligence & Future Applications, 111(111). https://doi.org/10.54941/ahfe1004068
Mozelius, P., & Humble, N. (2024). On the Use of Generative AI for Literature Reviews: An Exploration of Tools and Techniques. European Conference on Cyber Warfare and Security, 161–168. https://www.proquest.com/docview/3092061589/abstract/536C39CD10744142PQ/1
Ngwenyama, O., Toronto Metropolitan University / University of Cape Town, Rowe, F., & University of Nantes. (2024). Should We Collaborate with AI to Conduct Literature Reviews? Changing Epistemic Values in a Flattening World. Journal of the Association for Information Systems, 25(1), 122–136. https://doi.org/10.17705/1jais.00869
Ofori-Boateng, R., Aceves-Martins, M., Wiratunga, N., & Moreno-Garcia, C. F. (2024). Towards the automation of systematic reviews using natural language processing, machine learning, and deep learning: A comprehensive review. Artificial Intelligence Review, 57(8), 200. https://doi.org/10.1007/s10462-024-10844-w
Okoli, C. (2015). A Guide to Conducting a Standalone Systematic Literature Review. Communications of the Association for Information Systems, 37. https://doi.org/10.17705/1CAIS.03743
Onuoha, C. E. (2025). Impact of Artificial Intelligence (AI) on the Quality, Efficiency, and Transparency of the Scholarly Publishing Process.
Oyelude, A. A. (2024). Artificial intelligence (AI) tools for academic research. Library Hi Tech News, 41(8), 18–20. https://doi.org/10.1108/LHTN-08-2024-0131
Pan, S. L., Nishant, R., Tuunanen, T., & Nah, F. F.-H. (2023). Literature review in the generative AI era—How to make a compelling contribution. The Journal of Strategic Information Systems, 32(3), 101788. https://doi.org/10.1016/j.jsis.2023.101788
Paré, G., Tate, M., Johnstone, D., & Kitsiou, S. (2016). Contextualizing the twin concepts of systematicity and transparency in information systems literature reviews. European Journal of Information Systems, 25(6), 493–508. https://doi.org/10.1057/s41303-016-0020-3
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015). Synthesizing information systems knowledge: A typology of literature reviews. Information & Management, 52(2), 183–199. https://doi.org/10.1016/j.im.2014.08.008
Pritting, S., Doran, P., Covington, H., Kennedy, J., & Sun, H. (2025). AI-integrated literature discovery tools: A guide to reviewing for institution wide use. Journal of Electronic Resources Librarianship, 0(0), 1–15. https://doi.org/10.1080/1941126X.2025.2497736
Saunders, M. N. K., & Rojon, C. (2011). On the attributes of a critical literature review. Coaching: An International Journal of Theory, Research and Practice, 4(2), 156–162. https://doi.org/10.1080/17521882.2011.596485
Schryen, G., Marrone, M., & Yang, J. (2024). Adopting Generative AI for Literature Reviews: An Epistemological Perspective.
Schryen, G., Marrone, M., & Yang, J. (2025). Exploring the scope of generative AI in literature review development. Electronic Markets, 35(1), 13. https://doi.org/10.1007/s12525-025-00754-2
Shor, R., Greene, E. A., Sumberg, L., & Weingrad, A. B. (2025). AI Tools in Academia: Evaluating NotebookLM as a Tool for Conducting Literature Reviews. Psychiatry, 0(0), 1–10. https://doi.org/10.1080/00332747.2025.2541531
Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104, 333–339. https://doi.org/10.1016/j.jbusres.2019.07.039
Sparkman, M., & Witt, A. (2025). Claude AI and Literature Reviews: An Experiment in Utility and Ethical Use. Library Trends, 73(3), 355–380.
Stahl, B. C. (2008). Researching Ethics and Morality in Information Systems: Some Guiding Questions.
Stone, J. A. M. (2023). Artificial Intelligence–Generated Research in the Literature: Is It Real or Is It Fraud? Medical Acupuncture, 35(3), 103. https://doi.org/10.1089/acu.2023.29231.editorial
Susarla, A., Gopal, R., Thatcher, J. B., & Sarker, S. (2023). The Janus Effect of Generative AI: Charting the Path for Responsible Conduct of Scholarly Activities in Information Systems. Information Systems Research, 34(2), 399–408. https://doi.org/10.1287/isre.2023.ed.v34.n2
Templier, M., & Paré, G. (2015). A Framework for Guiding and Evaluating Literature Reviews. Communications of the Association for Information Systems, 37. https://doi.org/10.17705/1CAIS.03706
Tingelhoff, F., Brugger, M., & Leimeister, J. (2024). A Guide for Structured Literature Reviews in Business Research: The State-of-the-Art and How to Integrate Generative Artificial Intelligence. Journal of Information and Technology. https://doi.org/10.1177/02683962241304105
Tomczyk, P., Brüggemann, P., & Vrontis, D. (2024). AI meets academia: Transforming systematic literature reviews. EuroMed Journal of Business. https://doi.org/10.1108/EMJB-03-2024-0055
Van Dinter, R., Tekinerdogan, B., & Catal, C. (2021). Automation of systematic literature reviews: A systematic literature review. Information and Software Technology, 136, 106589. https://doi.org/10.1016/j.infsof.2021.106589
Wagner, G., Lukyanenko, R., & Paré, G. (2022). Artificial intelligence and the conduct of literature reviews. Journal of Information Technology, 37(2), 209–226. https://doi.org/10.1177/02683962211048201
Walsham, G. (1996). Ethical theory, codes of ethics and IS practice. Information Systems Journal, 6(1), 69–81. https://doi.org/10.1046/j.1365-2575.1996.00105.x
Walter, Y. (2024). Managing the race to the moon: Global policy and governance in Artificial Intelligence regulation—A contemporary overview and an analysis of socioeconomic consequences. Discover Artificial Intelligence, 4(1), 14. https://doi.org/10.1007/s44163-024-00109-4
Webster, J., & Watson, R. (2002). Analyzing the Past to Prepare for the Future: Writing a Literature Review. MIS Q.
Yawson, R. M. (2025). Perspectives on the promise and perils of generative AI in academia. Human Resource Development International, 28(3), 476–487. https://doi.org/10.1080/13678868.2024.2334983
Yue Yim, I. H. (2024). A critical review of teaching and learning artificial intelligence (AI) literacy: Developing an intelligence-based AI literacy framework for primary school education. Computers and Education: Artificial Intelligence, 7, 100319. https://doi.org/10.1016/j.caeai.2024.100319
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Dipesh Jalui, Mary Tate, Jocelyn Cranefield
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
