Social bot literacy and responses to bot-generated information in AI-mediated environments: the roles of self-efficacy and perceived threat
DOI:
https://doi.org/10.47989/ir31263122Abstract
Introduction. Social bots constitute an increasingly prominent form of artificial intelligence within online information environments, shaping how information is generated, circulated, and assessed. This study investigates how social bot literacy relates to responses to bot-generated information, with particular attention to the mediating role of social bot self-efficacy and the moderating role of perceived threats from bots.
Method. A mixed-methods design was employed. Survey data were collected from 1,159 Sina Weibo users in China to examine associations among social bot literacy, social bot self-efficacy, perceived threats, and information response strategies. In addition, in-depth interviews were conducted to explore how bot-generated information is interpreted and evaluated in everyday information environments.
Analysis. Quantitative data were analysed using confirmatory factor analysis and moderated mediation models. Qualitative interview data were thematically analysed to contextualise information evaluation and response processes in AI-mediated settings.
Results. The results indicate that higher social bot literacy is associated with greater confidence in handling bot-generated information and with more active information response strategies. Social bot self-efficacy mediates this relationship, while perceived threat moderates the strength of the indirect effects. Interview findings further show how social bots are understood as information actors and how informational risks are negotiated.
Conclusion. The study advances Information Science by clarifying how social bot literacy shapes information evaluation and use in AI-mediated environments, with implications for information literacy development and responsible AI governance.
References
Ajzen, I. (2002). Perceived behavioral control, self‐efficacy, locus of control, and the theory of planned behavior. Journal of Applied Social Psychology, 32(4), 665-683. https://doi.org/10.1111/j.1559-1816.2002.tb00236.x
Al-Rawi, A., Kane, O., & Bizimana, A. (2021). Topic modelling of public Twitter discourses, part bot, part active human user, on climate change and global warming. Journal of Environmental Media, 2(1), 31-53. https://doi.org/10.1386/jem_00039_1.
Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 37(2),122. https://doi.org/10.1037/0003-066X.37.2.122.
Bandura, A. (1986). The explanatory and predictive scope of self-efficacy theory. Journal of Social and Clinical Psychology, 4(3), 359-373. https://doi.org/10.1521/jscp.1986.4.3.359.
Bossu, R., Corradini, M., Cheny, J., & Fallou, L. (2023). A social bot in support of crisis communication: 10-years of@ LastQuake experience on Twitter. Frontiers in Communication, 8. https://doi.org/10.3389/fcomm.2023.992654
Confessore, N., Dance, G.J., Harris, R., & Hansen, M. (2018). The follower factory. The New York Times, 27. https://www.nytimes.com/interactive/2018/01/27/technology/social-media-bots.html.
Cresci, S., Yang, K., Spognardi, A., Di Pietro, R., Menczer, F., & Petrocchi, M. (2023). Demystifying misconceptions in social bots research. ArXiv Preprint ArXiv:2303.17251. https://doi.org/10.48550/arXiv.2303.17251.
Daume, S., Galaz, V., & Bjersér, P. (2023). Automated framing of climate change? the role of social bots in the twitter climate change discourse during the 2019/2020 Australia Bushfires. Social Media+ Society, 9(2). https://doi.org/10.1177/20563051231168370
DeVrio, A., Eslami, M., & Holstein, K. (2024). Building, Shifting, & Employing Power: A taxonomy of responses from below to algorithmic harm. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency. 1093–1106. https://dl.acm.org/doi/10.1145/3630106.3658958.
Duan, Z., Li, J., Lukito, J., Yang, K.-C., Chen, F., Shah, D. V., & Yang, S. (2022). Algorithmic agents in the hybrid media system: Social bots, selective amplification, and partisan news about COVID-19. Human Communication Research, 48(2), 1-18. https://doi.org/10.1093/hcr/hqac012
Fan, R., Talavera, O., & Tran, V. (2020). Social media bots and stock markets. European Financial Management, 26(3), 753-777. https://doi.org/10.1111/eufm.12245.
Fang, W., & Nie, C. (2024). Social media use, social bot literacy, perceived threats from bots, and perceived bot control: a moderated-mediation model. Behaviour & Information Technology, 43(13), 3271-3287. https://doi.org/10.1080/0144929X.2023.2276801
Ferrara, E. (2023). Social bot detection in the age of ChatGPT: Challenges and opportunities. First Monday, 28(6). https://doi.org/10.5210/fm.v28i6.13185
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016). The rise of social bots. Communications of the ACM, 59(7), 96–104. https://doi.org/10.1145/2818717.
Gorwa, R., & Guilbeault, D. (2020). Unpacking the social media bot: A typology to guide research and policy. Policy & Internet, 12(2), 225-248. https://doi.org/10.1002/poi3.184.
Hagen, L., Neely, S., Keller, T.E., Scharf, R., & Vasquez, F.E. (2022). Rise of the machines? Examining the influence of social bots on a political discussion network. Social Science Computer Review, 40(2), 264-287. https://doi.org/10.1177/0894439320908190.
Hajli, N., Saeed, U., Tajvidi, M., & Shirazi, F. (2022). Social bots and the spread of disinformation in social media: the challenges of artificial intelligence. British Journal of Management, 33(3):1238-1253. https://doi.org/10.1111/1467-8551.12554.
Hayes, A. F. (2017). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. Guilford Publications.
Hays, C., Schutzman, Z., Raghavan, M., Walk, E., & Zimmer, P. (2023). Simplistic collection and labeling practices limit the utility of benchmark datasets for Twitter bot detection. In Proceedings of the ACM web conference 2023. https://doi.org/10.48550/arXiv.2301.07015
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118.
Huang, C. (2013). Gender differences in academic self-efficacy: A meta-analysis. European Journal of Psychology of Education, 28, 1-35. https://doi.org/10.1007/s10212-011-0097-y.
Kelfve, S., Kivi, M., Johansson, B., & Lindwall, M. (2020). Going web or staying paper? The use of web-surveys among older people. BMC Medical Research Methodology, 20, 252. https://doi.org/10.1186/s12874-020-01138-0
Keller, F.B., Schoch, D., Stier, S., & Yang, J. (2020). Political astroturfing on twitter: How to coordinate a disinformation campaign. Political Communication, 37(2), 256-280. https://doi.org/10.1080/10584609.2019.1661888.
Kenny, R., Fischhoff, B., Davis, A., Carley, K.M., & Canfield, C. (2024). Duped by bots: why some are better than others at detecting fake social media personas. Human Factors, 66(1), 88-102. https://doi.org/10.1177/00187208211072642
Kušen, E., & Strembeck, M. (2019). Something draws near, I can feel it: An analysis of human and bot emotion-exchange motifs on Twitter. Online Social Networks and Media, 10, 1-17. https://doi.org/10.1016/j.osnem.2019.04.001.
Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., Schudson, M., Sloman, S. A., Sunstein, C. R., Thorson, E. A., Watts, D. J., & Zittrain, J. L. (2018). The science of fake news: Addressing fake news requires a multidisciplinary effort. Science, 359(6380), 1094–1096. https://doi.org/10.1126/science.aao2998.
Livingstone, S., & Helsper, E. (2010). Balancing opportunities and risks in teenagers’ use of the internet: The role of online skills and internet self-efficacy. New Media & Society, 12(2), 309–329. https://doi.org/10.1177/1461444809342697.
Marlow, T., Miller, S., & Roberts, J. T. (2021). Bots and online climate discourses: Twitter discourse on President Trump’s announcement of US withdrawal from the Paris Agreement. Climate Policy, 21(6), 765–777. https://doi.org/10.1080/14693062.2020.1870098
Martin, A. (2006). A European framework for digital literacy. Nordic Journal of Digital Literacy, 1(2), 151–161. https://doi.org/10.18261/ISSN1891-943X-2006-02-06
Martini, F., Samula, P., Keller, T. R., & Klinger, U. (2021). Bot, or not? Comparing three methods for detecting social bots in five political discourses. Big Data & Society, 8(2), 54050782. https://doi.org/10.1177/20539517211033566
Ngo, T., Wischnewski, M., Bernemann, R., Jansen, M., & Krämer, N. (2023). Spot the bot: Investigating user's detection cues for social bots and their willingness to verify Twitter profiles. Computers in Human Behavior, 146, 107819. https://doi.org/10.1016/j.chb.2023.107819.
Nightingale, S. J., & Farid, H. (2022). AI-synthesized faces are indistinguishable from real faces and more trustworthy. In Proceedings of the National Academy of Sciences, 119(8), e2120481119. https://doi.org/10.1073/pnas.2120481119.
Sax, L. J., Gilmartin, S. K., & Bryant, A. N. (2003). Assessing response rates and nonresponse bias in web and paper surveys. Research in Higher Education, 44(4), 409–432. https://doi.org/10.1023/A:1024232915870
Schmuck, D., & von Sikorski, C. (2020). Perceived threats from social bots: The media's role in supporting literacy. Computers in Human Behavior, 113, 106507. https://doi.org/10.1016/j.chb.2020.106507.
Sharma, K., Ferrara, E., & Liu, Y. (2022). Characterizing online engagement with disinformation and conspiracies in the 2020 US presidential election. In Proceedings of the International AAAI Conference on Web and Social Media, 16(1), 908–919. https://doi.org/10.1609/icwsm.v16i1.19345.
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., & Virk, G. (2023). Sociotechnical harms of algorithmic systems: Scoping a taxonomy for harm reduction. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3600211.3604673
Shi, W., Liu, D., Yang, J., Zhang, J., Wen, S., & Su, J. (2020). Social bots’ sentiment engagement in health emergencies: A topic-based analysis of the COVID-19 pandemic discussions on Twitter. International Journal of Environmental Research and Public Health, 17(22), 8701. https://doi.org/10.3390/ijerph17228701
Starbird, K. (2019). Disinformation's spread: Bots, trolls, and all of us. Nature, 571(7766), 449–450. https://doi.org/10.1038/d41586-019-02235-x
Stocking, G., & Sumida, N. (2018). Social media bots draw public’s attention and concern. Pew Research Center. https://www.pewresearch.org/journalism/2018/10/15/social-media-bots-draw-publics-attention-and-concern/
Suarez-Lledo, V., & Alvarez-Galvez, J. (2022). Assessing the role of social bots during the COVID-19 pandemic: Infodemic, disagreement, and criticism. Journal of Medical Internet Research, 24(8), e36085. https://doi.org/10.2196/36085
Tan, Z., Feng, S., Sclar, M., Wan, H., Luo, M., Choi, Y., & Tsvetkov, Y. (2023). BotPercent: Estimating bot populations in Twitter communities. arXiv Preprint. https://doi.org/10.48550/arXiv.2302.00381
Thomala, L.L. (2023). Most Popular News on Weibo in China 2022. Statista. https://www.statista.com/statistics/1377073/china-most-popular-news-on-weibo/ #statisticContainer.
Wagner, C., Mitter, S., Körner, C., & Strohmaier, M. (2012). When social bots attack: Modeling susceptibility of users in online social networks. In Proceedings of the 2012 IEEE/ACM International Conference on Multimodal Search and Mining. https://ceur-ws.org/Vol-838/paper_11.pdf
Wald, R., Khoshgoftaar, T.M., Napolitano, A., & Sumner, C. Predicting susceptibility to social bots on twitter. In Proceedings of the 2013 IEEE 14th international conference on information reuse and integration. https://doi.org/10.1109/IRI.2013.6642447.
Weibo. (2021). Weibo 2020 User Development Report. https://data.weibo.com/report/reportDetail?id=456.
Wischnewski, M., Ngo, T., Bernemann, R., Jansen, M., & Krämer, N. (2024). "I agree with you, bot!" How users (dis)engage with social bots on Twitter. New Media & Society, 26(3), 1505–1526. https://doi.org/10.1177/14614448211072307
Yan, H. Y., Yang, K., Menczer, F., & Shanahan, J. (2021). Asymmetrical perceptions of partisan political bots. New Media & Society, 23(10), 3016–3037. https://doi.org/10.1177/1461444820942744
Yan, H. Y., Yang, K., Shanahan, J., & Menczer, F. (2023). Exposure to social bots amplifies perceptual biases and regulation propensity. Scientific Reports, 13(1), 20707. https://doi.org/10.1038/s41598-023-46630-x
Yang, K. C., Varol, O., Davis, C. A., Ferrara, E., Flammini, A., & Menczer, F. (2019). Arming the public with artificial intelligence to counter social bots. Human Behavior and Emerging Technologies, 1(1), 48–61. https://doi.org/10.1002/hbe2.145.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Wei Fang, Chen Nie
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
