Exploring metacognitive regulation activities and their roles in prompt literacy development among humanities and social science researchers in GenAI
DOI:
https://doi.org/10.47989/ir31iConf64194Keywords:
Prompt literacy, Metacognitive regulation activities, Humanities and social science researchers, Human-AI interactionAbstract
Introduction. Generative artificial intelligence (GenAI) has profoundly influenced research paradigms in the humanities and social sciences (HSS) while imposing greater demands on users’ prompt literacy. There is an urgent need for empirical inquiry into how HSS researchers leverage metacognitive regulation activities to develop prompt literacy.
Method. Seventeen HSS researchers were recruited to participate in semi-structured interviews designed to examine their metacognitive activities and prompting behaviors when interacting with GenAI and to explore the role of metacognition in developing prompt literacy.
Analysis. The interview data were analysed using thematic analysis, with initial coding and theme identification conducted through iterative clustering.
Results. Our research shows that metacognitive regulation activities—planning, monitoring and evaluation—enhance prompt literacy in HSS researchers through four distinct stages. Prompt literacy manifests as a normative attitude toward prompts, the ability to design prompts, the ability to evaluate prompts, and the ability to think critically about prompting.
Conclusions. These findings reveal how metacognitive regulation shapes prompt literacy and provide theoretical and practical insights for cultivating prompt literacy in the GenAI era.
References
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
Braun, V., Clarke, V., Hayfield, N., & Terry, G. (2019). Thematic analysis. In Handbook of research methods in health social sciences (pp. 843-860). Springer, Singapore.
Federiakin, D., Molerov, D., Zlatkin-Troitschanskaia, O., & Maur, A. (2024, November). Prompt engineering as a new 21st century skill. In Frontiers in Education (Vol. 9, p. 1366434). Frontiers Media SA. https://doi.org/10.3389/feduc.2024.1366434
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906-911. https://doi.org/10.1037/0003-066X.34.10.906
Gagnière, L., Betrancourt, M., & Détienne, F. (2012). When metacognitive prompts help information search in collaborative setting. European Review of Applied Psychology, 62(2), 73-81. https://doi.org/10.1016/j.erap.2011.12.005
Gattupalli, S., Maloy, R. W., & Edwards, S. A. (2023). Prompt literacy: A pivotal educational skill in the age of AI. College of Education Working Papers and Reports Series, 6, 1-5. https://doi.org/10.7275/3498-wx48
Grossmann, I., Feinberg, M., Parker, D. C., Christakis, N. A., Tetlock, P. E., & Cunningham, W. A. (2023). AI and the transformation of social science research. Science, 380(6650), 1108-1109. https://www.science.org/doi/abs/10.1126/science.adi1778
Han, Z., Ying, R., Huang, C., Tsai, C.-C., Wang, X., & He, T. (2025). Identifying students’ metacognition patterns by their needs for cognitive closure in human-GenAI collaboration. Computers & Education, 105422. https://doi.org/10.1016/j.compedu.2025.105422
Hwang, Y. (2023). The emergence of generative AI and PROMPT literacy: Focusing on the use of ChatGPT and DALL-E for English education. Journal of the Korea English Education Society, 22(2), 263-288. http://dx.doi.org/10.18649/jkees.2023.22.2.263
Hwang, Y., Lee, J. H., & Shin, D. (2023). What is prompt literacy? An exploratory study of language learners' development of new literacy skill using generative AI. arXiv preprint arXiv:2311.05373. https://doi.org/10.48550/arXiv.2311.05373
Ji, J., Qiu, T., Chen, B., Zhang, B., Lou, H., Wang, K., Duan, Y., He, Z., Zhou, J., & Zhang, Z. (2023). Ai alignment: A comprehensive survey. arXiv preprint arXiv:2310.19852. https://doi.org/10.48550/arXiv.2310.19852
Jia, M., Zhao, Y. C., & Zhang, X. (2025). Will potential threats persist? Attitudes and behavioral responses of creative professionals toward GenAI. Information Research an International Electronic Journal, 30(iConf), 1073-1083. https://doi.org/10.47989/ir30iConf47098
Juhaňák, L., Juřík, V., Dostálová, N., & Juříková, Z. (2025). Exploring the effects of metacognitive prompts on learning outcomes: An experimental study in higher education. Australasian Journal of Educational Technology, 41(1), 42-59. https://doi.org/10.14742/ajet.9486
Julien, H. (2018). Digital literacy in theory and practice. In Encyclopedia of Information Science and Technology, Fourth Edition (pp. 2243-2252). IGI Global Scientific Publishing. https://www.igi-global.com/chapter/digital-literacy-in-theory-andpractice/183937
Kim, J., Yu, S., Lee, S.-S., & Detrick, R. (2025). Students’ prompt patterns and its effects in AI-assisted academic writing: Focusing on students’ level of AI literacy. Journal of Research on Technology in Education, 1-18. https://doi.org/10.1080/15391523.2025.2456043
Knoth, N., Tolzin, A., Janson, A., & Leimeister, J. M. (2024). AI literacy and its implications for prompt engineering strategies. Computers and Education: Artificial Intelligence, 6, 100225. https://doi.org/10.1016/j.caeai.2024.100225
Krakowska, M., & Zych, M. (2025). (Un) conventional ways of dialogic information retrieval using prompt engineering and the role of AI literacy. Information Research an International Electronic Journal, 30(CoLIS), 121-140. https://doi.org/10.47989/ir30CoLIS52243
Lee, H. P., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025, April). The impact of generative AI on critical thinking: Self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (pp. 1-22). https://doi.org/10.1145/3706598.3713778
Lee, H.-Y., Chen, P.-H., Wang, W.-S., Huang, Y.-M., & Wu, T.-T. (2024). Empowering ChatGPT with guidance mechanism in blended learning: Effect of self-regulated learning, higher-order thinking skills, and knowledge construction. International Journal of Educational Technology in Higher Education, 21(1), 16. https://doi.org/10.1186/s41239-024-00447-4
Liu, V., & Chilton, L. B. (2022, April). Design guidelines for prompt engineering text-to-image generative models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (pp. 1-23). https://doi.org/10.1145/3491102.3501825
Lo, L. S. (2023a). The CLEAR path: A framework for enhancing information literacy through prompt engineering. The Journal of Academic Librarianship, 49(4), 102720. https://doi.org/10.1016/j.acalib.2023.102720
Lo, L. S. (2023b). The art and science of prompt engineering: a new literacy in the information age. Internet Reference Services Quarterly, 27(4), 203-210. https://doi.org/10.1080/10875301.2023.2227621
Long, D., & Magerko, B. (2020, April). What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-16). https://doi.org/10.1145/3313831.3376727
Lund, B. (2023). The prompt engineering librarian. Library Hi Tech News, 40(8), 6-8. https://doi.org/10.1108/LHTN-10-2023-0189
Matysek, A. (2023). The use of one-on-one interviews in library and information science: A scoping review. Journal of Information Science, 01655515231171088. https://doi.org/10.1177/01655515231171088
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualising AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, 100041. https://doi.org/10.1016/j.caeai.2021.100041
Nguyen, A., Hong, Y., Dang, B., & Huang, X. (2024). Human-AI collaboration patterns in AI-assisted academic writing. Studies in Higher Education, 49(5), 847-864. https://doi.org/10.1080/03075079.2024.2323593
Saks, K., & Leijen, Ä. (2019). The efficiency of prompts when supporting learner use of cognitive and metacognitive strategies. Computer Assisted Language Learning, 32(1-2), 1-16. https://doi.org/10.1080/09588221.2018.1459729
Schraw, G., & Moshman, D. (1995). Metacognitive theories. Educational Psychology Review, 7(4), 351-371. https://doi.org/10.1007/BF02212307
Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A. E., Sarkar, A., Sellen, A., & Rintel, S. (2024, May). The metacognitive demands and opportunities of generative AI. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (pp. 1-24). https://doi.org/10.1145/3613904.3642902
Terry, G., Hayfield, N., Clarke, V., & Braun, V. (2017). Thematic analysis. The SAGE handbook of qualitative research in psychology, 2(17-37), 25. Sage.
Urban, M., Děchtěrenko, F., Lukavský, J., Hrabalová, V., Svacha, F., Brom, C., & Urban, K. (2024). ChatGPT improves creative problem-solving performance in university students: An experimental study. Computers & Education, 215, 105031. https://doi.org/10.1016/j.compedu.2024.105031
Wang, Y., & Zhao, Y. (2023). Metacognitive prompting improves understanding in large language models. arXiv preprint arXiv:2308.05342. https://doi.org/10.48550/arXiv.2308.05342
White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., & Schmidt, D. C. (2023). A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382. https://arxiv.org/abs/2302.11382
Yan, L., Greiff, S., Teuber, Z., & Gašević, D. (2024). Promises and challenges of generative artificial intelligence for human learning. Nature Human Behaviour, 8(10), 1839-1850. https://doi.org/10.1038/s41562-024-02004-5
Yang, Y., Zhang, Y., Sun, D., He, W., & Wei, Y. (2025). Navigating the landscape of AI literacy education: insights from a decade of research (2014–2024). Humanities and Social Sciences Communications, 12(1), 1-12. https://doi.org/10.1057/s41599-025-04583-8
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023, April). Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (pp. 1-21). https://doi.org/10.1145/3544548.3581388
Zhao, Y., Huang, H., Li, Q., & Zhao, Y. C. (2025). How user information search patterns impact learning outcomes across different stages of learning-related search tasks. Library & Information Science Research, 47(2), 101354. https://doi.org/10.1016/j.lisr.2025.101354
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Yutian Jing , Yuxiang Chris Zhao , Durong Wang , Siqi Liu , Qinghua Zhu
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
