Workplace user engagement with a semantic AI-based search tool
DOI:
https://doi.org/10.47989/ir31263125Keywords:
AI-based search tool, AI metadata, AI literacy, information seeking behaviour, search behaviour, Enterprise searchAbstract
Introduction. This study investigates non-professional searchers’ engagement with and challenges in using an AI-based workplace search tool. The tool included twelve AI-assigned, domain-specific metadata types and information from internal and external sources. Drawing on AI literacy perspectives, the study identifies competencies and support needed for effective workplace use.
Method. A case study was conducted using an exploratory sequential mixed-methods design. Data included an interview with a corporate information specialist responsible for AI metadata development, a questionnaire with seventy searchers, and semi-structured interviews with nine searchers.
Analysis. Preparatory interviews provided context for analysis. Questionnaire data were examined using uni- and bivariate statistics, while interview data were thematically analysed to explore searcher engagement and challenges.
Results. The metadata types were found relevant for both exploratory and known-item searches. Users generally found the tool easy to use but identified challenges: domain knowledge was required; associated information needed enrichment; AI metadata roles were not immediately clear; and some routines had to be adapted.
Conclusions. Searchers demonstrated AI competencies and were able to manage challenges. Effective use of AI-based search tools requires experience, domain knowledge, an explorative mindset, and continuous interaction. AI competence development can be understood as the coordinated deployment of tangible, intangible, and human resources supporting advanced information-seeking practices.
References
Allard, S., Levine, K. J., & Tenopir, C. (2009). Design engineers and technical professionals at work: Observing information usage in the workplace. Journal of the American Society for Information Science and Technology, 60(3), 443–454. https://doi.org/10.1002/asi.21004
Asula, M., Makke, J., Freienthal, L., Kuulmets, H.-A., & Sirel, R. (2021). Kratt: Developing an Automatic Subject Indexing Tool for the National Library of Estonia. Cataloging & Classification Quarterly, 59(8), 775–793. https://doi.org/10.1080/01639374.2021.1998283
Bloom, B. S. (1956). Taxonomy of Educational Objectives: The Classification of Educational Goals. D. McKay.
Cetindamar, D., Kitto, K., Wu, M., Zhang, Y., Abedin, B., & Knight, S. (2024). Explicating AI Literacy of Employees at Digital Workplaces. IEEE Transactions on Engineering Management, 71, 810–823. IEEE Transactions on Engineering Management. https://doi.org/10.1109/TEM.2021.3138503
Chen, R.-C., Dewi, C., Huang, S.-W., & Caraka, R. E. (2020). Selecting critical features for data classification based on machine learning methods. Journal of Big Data, 7(1), 52. https://doi.org/10.1186/s40537-020-00327-4
Cleverley, P. H., & Burnett, S. (2015). Retrieving haystacks: A data driven information needs model for faceted search. Journal of Information Science, 41(1), 97–113. https://doi.org/10.1177/0165551514554522
Cleverley, P. H., & Burnett, S. (2019). Enterprise search and discovery capability: The factors and generative mechanisms for user satisfaction. Journal of Information Science, 45(1), 29–52. https://doi.org/10.1177/0165551518770969
Cleverley, P. H., Burnett, S., & Muir, L. (2017). Exploratory information searching in the enterprise: A study of user satisfaction and task performance. Journal of the Association for Information Science and Technology, 68(1), 77–96. https://doi.org/10.1002/asi.23595
Corrado, E. M. (2021). Artificial Intelligence: The Possibilities for Metadata Creation. Technical Services Quarterly, 38(4), 395–405. https://doi.org/10.1080/07317131.2021.1973797
Domke, E. M., Leidig, J. P., Schymik, G., & Wolffe, G. (2018). Query Expansion in Enterprise Search. Proceedings of the ACM Symposium on Document Engineering 2018, 1–4. https://doi.org/10.1145/3209280.3229111
Elliott, A. (2019). The Culture of AI: Everyday Life and the Digital Revolution. Routledge. https://doi.org/10.4324/9781315387185
Fagin, R., Kumar, R., McCurley, K. S., Novak, J., Sivakumar, D., Tomlin, J. A., & Williamson, D. P. (2003). Searching the workplace web. Proceedings of the 12th International Conference on World Wide Web, 366–375. https://doi.org/10.1145/775152.775204
Freund, L. (2015). Contextualizing the information-seeking behavior of software engineers. Journal of the Association for Information Science and Technology, 66(8), 1594–1605. https://doi.org/10.1002/asi.23278
Furnas, G. W., Landauer, T. K., Gomez, L. M., & Dumais, S. T. (1987). The Vocabulary Problem in Human-system Communication. Commun. ACM, 30(11), 964–971. https://doi.org/10.1145/32206.32212
Given, L. M., Case, D. O., & Willson, R. (2023). Looking for Information: Examining Research on How People Engage with Information. Emerald Group Publishing.
Golub, K. (2021). Automated Subject Indexing: An Overview. Cataloging & Classification Quarterly, 59(8), 702–719. https://doi.org/10.1080/01639374.2021.2012311
Gunadi, E., & Albayrak, S. (2015). Information Aggregation in an Enterprise. In F. Hopfgartner (Ed.), Smart Information Systems: Computational Intelligence for Real-Life Applications (pp. 99–122). Springer International Publishing. https://doi.org/10.1007/978-3-319-14178-7_4
Kadhim, A. I. (2019). Survey on supervised machine learning techniques for automatic text classification. Artificial Intelligence Review, 52(1), 273–292. https://doi.org/10.1007/s10462-018-09677-1
Kruschwitz, U., & Hull, C. (2017). Searching the Enterprise. Foundations and Trends® in Information Retrieval, 11(1), 1–142. https://doi.org/10.1561/1500000053
Long, D., & Magerko, B. (2020). What is AI Literacy? Competencies and Design Considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–16. https://doi.org/10.1145/3313831.3376727
Lowe, D. B., Dollinger, I., Koster, T., & Herbert, B. E. (2021). Text Mining for Type of Research Classification. Cataloging & Classification Quarterly, 59(8), 815–834. https://doi.org/10.1080/01639374.2021.1998281
Author (2022)
Marchionini, G. (2006). Exploratory search: From finding to understanding. Communications of the ACM, 49(4), 41–46. https://doi.org/10.1145/1121949.1121979
Marques, A., & Murphy, G. C. (2022). Evaluating the Use of Semantics for Identifying Task-relevant Textual Information. 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), 240–251. https://doi.org/10.1109/SANER53432.2022.00039
Mathieu, C. (2022). Defining knowledge workers’ creation, description, and storage practices as impact on enterprise content management strategy. Journal of the Association for Information Science and Technology, 73(3), 472–484. https://doi.org/10.1002/asi.24563
Moulaison-Sandy, H., Adkins, D., Bossaller, J., & Cho, H. (2021). An Automated Approach to Describing Fiction: A Methodology to Use Book Reviews to Identify Affect. Cataloging & Classification Quarterly, 59(8), 794–814. https://doi.org/10.1080/01639374.2021.1992694
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review—ScienceDirect. Computers and Education: Artificial Intelligence, 2, 100041. https://doi.org/10.1016/j.caeai.2021.100041
Ng, D. T. K., Wu, W., Leung, J. K. L., Chiu, T. K. F., & Chu, S. K. W. (2024). Design and validation of the AI literacy questionnaire: The affective, behavioural, cognitive and ethical approach. British Journal of Educational Technology, 55(3), 1082–1104. https://doi.org/10.1111/bjet.13411
Author (2005). Special Libraries and Specialized Vocabularies in the Digital Age. 8. https://doi.org/120024989
Russell-Rose, T., Chamberlain, J., & Azzopardi, L. (2018). Information retrieval in the workplace: A comparison of professional search practices. Information Processing & Management, 54(6), 1042–1057. https://doi.org/10.1016/j.ipm.2018.07.003
Samuel, J., Ali, G. G. M. N., Rahman, M. M., Esawi, E., & Samuel, Y. (2020). COVID-19 Public Sentiment Insights and Machine Learning for Tweets Classification. Information, 11(6), 314. https://doi.org/10.3390/info11060314
Schuff, D., Louis, R. S., Corral, K., & Schymik, G. (2016). Word Ambiguity and Search: Implications for Enterprise Performance Management. AMCIS 2016 Proceedings. https://aisel.aisnet.org/amcis2016/Decision/Presentations/11
Schymik, G., Corral, K., Schuff, D., & St. Louis, R. (2015). The Benefits and Costs of Using Metadata to Improve Enterprise Document Search. Decision Sciences, 46(6), 1049–1075. https://doi.org/10.1111/deci.12154
Stocker, A., Richter, A., Kaiser, C., & Softic, S. (2015). Exploring barriers of enterprise search implementation: A qualitative user study. Aslib Journal of Information Management, 67(5), 470–491. https://doi.org/10.1108/AJIM-03-2015-0035
Author (2024). Searching for people in the workplace: Aims, behaviour, and challenges. Information Research an International Electronic Journal, 29(2), Article 2. https://doi.org/10.47989/ir292848
Thangaraj, M., & Sivakami, M. (2018). Text Classification Techniques: A Literature Review. Interdisciplinary Journal of Information, Knowledge, and Management, 13, 117–135.
Townsend, R., & Mathieu, C. (2018). Improving Enterprise Content Findability through Strategic Intervention. The Code4Lib Journal, 42. https://journal.code4lib.org/articles/13877?utm_source=rss&utm_medium=rss&utm_campaign=improving-enterprise-content-findability-through-strategic-intervention
White, M. S. (2020). Information flows – Or does it? The complexity of enterprise information management. Business Information Review, 37(3), 103–110. https://doi.org/10.1177/0266382120950114
Wu, M., Brandhorst, H., Marinescu, M.-C., Lopez, J., Hlava, M., & Busch, J. (2025). Automated metadata annotation: What is and is not possible with machine learning. Data Intelligence, 5(1), 122–138. https://doi.org/10.1162/dint_a_00162
Yang, H., Wang, N., Yang, L., Liu, W., & Wang, S. (2023). Research on the Automatic Subject-Indexing Method of Academic Papers Based on Climate Change Domain Ontology. Sustainability, 15(5), 3919. https://doi.org/10.3390/su15053919
Zhou, D. X., Liu, L., Anubhai, A., Shandilya, M., Sigalas, S., Wang, W. Y., & Huang, Z. (2023). Beyond Accurate Answers: Evaluating Open-Domain Question Answering in Enterprise Search. Proceedings of the 2023 Conference on Human Information Interaction and Retrieval, 308–312. https://doi.org/10.1145/3576840.3578314
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Marianne Lykke
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
