The emergence of generative Artificial Intelligence (AI) has made Large Language Model (LLM) powered chatbots such as ChatGPT, Claude, Gemini, and Perplexity a viable alternative for information seeking. AI chatbots do not simply retrieve existing content (like that which could be done with Google, for example) but also generate new content on a topic (Schuetzler et al., 2024, p. 7504). With its remarkable natural language understanding and generation capabilities, AI chatbots complement search engines by organising answers in a natural and easily digestible format (Liu et al., 2024, pp. 103-104). The extent to which people trust in AI-generated content is dependent on how they evaluate its perceived accuracy, bias, personalisation and content explainability (e.g. Zhong and Fang, 2026). A recent study conducted by Yun and Bickmore (2025) found that while search engines and institutional health websites continue to be the dominant sources of online health information, 21% of the respondents had used LLM-based chatbots for medical purposes. In a national survey conducted in 2023, Mendel et al. (2025) found that 32% of U.S. adults reported using AI chatbots for health-related queries, including interpreting symptoms, understanding treatment options and reviewing test results.

On the other hand, the use of generative AI tools is not without problems because chatbots may expose users to undesirable consequences such as hallucinations. They occur when chatbots generate responses that appear plausible and factually grounded but are in fact misleading, inaccurate or entirely fabricated (Lim et al., 2025). Hallucinations occur because the contemporary generative AI models are probabilistic in nature. They make a prediction about what a meaningful answer might look like without applying any forms of logical or contextual reasoning (Grimes et al., 2023, pp. 1619-1620). Different from traditional search engines that present a variety of information resources for users to choose from, AI chatbots offer a single response based on what LLMs deem most relevant to the user’s query (Ray, 2023). This means that the success or failure of information seeking hinges on the AI model’s ability to understand and accurately gather information, thereby limiting user autonomy in evaluating diverse information sources (Lund et al., 2025). AI chatbots can perform well in expert evaluations for accuracy, clarity and clinical usefulness but the responses often lack proper source citations and sometimes include outdated information (Sezgin et al., 2025). Although LLM-powered chatbots can access a vast repository of information on the open internet, their capacity to generate accurate and contextually appropriate responses may be limited (Kim et al., 2025).

Given the above limitations of current LLMs, the credibility of information generated by AI chatbots has become a significant issue in diverse fields such as information research (Choi, Bak and An, 2025), medicine (Wang et al., 2025) and health communication (Lim and Hong, 2025). So far, investigations on this topic have resulted in mixed findings. On the one hand, as discussed in more detail in the literature review below, there are observations suggesting that AI chatbots can offer trustworthy and accurate responses. On the other hand, there is opposite evidence about incorrect and misleading information generated by generative AI tools. The present study contributes to information behaviour research by elaborating the picture of the pros and cons of AI-generated information by exploring how a relatively large number of participants in online discussion assess the credibility of responses offered by AI chatbots. To this end, an empirical study was made by analysing 1464 posts on the above topic, submitted by 919 individual participants to 30 discussion threads in Reddit - a major social media forum. The findings elaborate the picture of credibility assessment in two ways. First, the results deepen our understanding about the significance of diverse criteria used in the credibility assessments focusing on chatbots as information sources. Second, the findings specify the ways in which credibility assessments are grounded in online discussion.

The literature review identified a number of criteria such as accuracy, reliability and currency by which people evaluate the credibility of responses offered by AI chatbots. Overall, the findings of prior investigations suggest that chatbots can generate relatively accurate information in certain domains of medicine. In opposite cases, information can be incorrect and misleading, particularly due to hallucinations generated by LLM powered tools. The present investigation aims at deepening our understanding about how people assess the credibility of information generated by chatbots in online discussion. To achieve this, the study was focused on discussions occurring in Reddit discussion threads. They were chosen for the study for two reasons. First, as Reddit is a global social media forum, its discussion threads attract a wide variety of participants’ views on the research topic. Second, as a publicly open platform, Reddit offers free access to discussion threads containing rich empirical data.

To specify the research framework further, it is assumed that online participants employ diverse criteria while making credibility assessments. Moreover, it is assumed that the participants ground their assessments by making use of various answer strategies manifesting themselves in diverse answer types. In the present study, credibility is approached as a subordinate category generally denoting the quality of being trusted and believed in. The nature of credibility can be specified by identifying a set of subcategories indicative of criteria by which the credibility of information generated by chatbots is assessed. To this end, the findings of prior studies reviewed above were used. The criteria thus identified include the following: accuracy of information, currency of information, unbiasedness of information (Choi, Bak and An, 2025), correctness of information (Zhang et al., 2025), and coverage of information (Kim et al., 2025). The list of criteria was substantiated by inductively identifying categories from the preliminary analysis of the empirical data used in the present investigation. These criteria include trustworthiness of information, consistency of information, verifiability of information, usefulness of information and chatbot’s capability to generate relevant information. The ways in which the above 10 criteria were defined for empirical analysis will be specified below in the methodology section.

Second, in the identification of relevant answer types indicative of answer strategies, the findings of Savolainen (2025) and Westbrook (2015) were used. Westbrook (2015) specified the category of explanation answer, while Savolainen (2025) identified opinion answers. The list of relevant answer types was substantiated by inductively identifying two categories from the empirical data of the present investigation, that is, experience answers and comparison answers. The empirical definitions of the above four answer types will be specified in the methodology section. The research framework is presented in Figure 1.

Figure 1 suggests that online participants assess the credibility of information generated by AI chatbots by answering the opening question or claim submitted by the initiator of the discussion thread. While assessing the credibility of information, multiple criteria can be used, ranging from trustworthiness and accuracy to currency and usefulness of information. While grounding their assessment, the participants may use diverse answer strategies, by drawing on their opinions about the trustworthiness of such information. Alternatively, the participants may describe their experiences about the usefulness of answers offered by chatbots. The contributors can also compare the credibility of information obtained from chatbots and other sources, as well as explain why chatbots tend to offer incorrect information.

Drawing on the above framework, the present study seeks answers to the following research questions.

The empirical data were gathered from Reddit - a major social media platform headquartered in San Francisco. Reddit consists of individual forums called subreddits, which are mostly user-created and organised by topic of discussion. Subreddits are managed by volunteer moderators who enforce rules for posting. Reddit users (Redditors) can join a forum to participate in posting and commenting on others’ posts (Eldridge, 2025). As of May 2025, Reddit had an estimated 91 million daily active users all over the world (Kumar, 2025).

The empirical data were downloaded in the beginning of October 2025 from nine subreddits focusing on the discussion about the trust in AI chatbots and credibility of information generated by them. The subreddits of this kind included the following:

These subreddits were chosen because they offer the most relevant repertoire of discussion threads for the topic of the present investigation. To obtain an overall picture of the posts submitted to the above subreddits, relevant discussion threads were tentatively identified using the search terms AI chatbot, trust in chatbot, and credibility of information. At the beginning of October 2025, the search resulted in the identification of 132 potentially relevant discussion threads from the above subreddits. The preliminary reading indicated that many of the discussion threads appeared to be of low informational value because they contained only the opening post or a small number (1-9) of posts submitted by fellow contributors. In addition, there were discussion threads focusing on a narrow topic, for example, trust in chatbots in digital marketing. After having excluded 92 irrelevant threads of this type, the remaining sample of 30 discussion threads was chosen for closer analysis. Examples of the opening posts indicative of the topics of the discussion threads include ‘Can we trust AI that’s trained on potentially flawed information sources?’, ‘How can we trust that any specific thing an AI says is accurate?’ and ‘How do you all trust ChatGPT’?

Altogether 919 individual participants submitted 1464 posts to the 30 threads during the period of October 2023 - September 2025. Of the 1464 posts, 30 opened the discussion, while 1434 posts were answers and comments to the opening question or claim presented by the initiator of the thread. The number of posts per discussion thread varied from 10 to 368. On average, a thread contained 48 posts. Of the contributors, the most active wrote no less than 64 posts, while the majority of the participants, that is, 66% submitted 1-2 posts. Although the discussion topics attracted a relatively high number of participants, most of them were occasional contributors who did not delve deeper into the debate about the credibility of information generated by AI chatbots. Importantly, however, the sample of 30 discussion threads appeared to be sufficient for the needs of qualitative analysis because the data became saturated. Data saturation occurs at the point at which collecting any further data will not produce value-added insights about the matter under investigation (Saunders et al., 2018, p. 1895). Therefore, it became evident that the analysis of additional discussion threads initiated after the beginning of October 2025 would not essentially change the qualitative picture of how the Redditors assess the credibility of information generated by AI chatbots. The sample of 30 discussion threads was downloaded in a separate file and coded by making use of the categories specified in Table 1.

The coding was an iterative process in which the data were scrutinised several times by the author. The pre-defined categories specified in Table 1 were used to code all the data - while still allowing new codes to emerge. However, the content of all posts submitted by the Redditors fit into the existing categories defined in Table 1 and no new categories were needed to cover the data. The 1464 posts were assigned with 2111 codes. Of them, 935 dealt with the criteria used in the credibility assessment, while 1176 identified the answer type. In the coding, a post was coded only once for a criterion category, once it was identified for the first time in the post. In long posts in particular, it was not unusual that the same criterion was identified in several segments of the same post. In these cases, a post was coded for a criterion category, for example, accuracy (credibility criterion) and opinion (answer type) when the category appeared for the first time in the text; other instances of the same category were simply ignored. However, a post could be assigned with several criteria indicative of credibility assessment and answer type, for example, accuracy and usefulness of information, as well as opinion and explanation. To strengthen the reliability of the coding, the initial coding was refined by repeated reading of the data. Miles and Huberman (1994, p. 65) recommended that check-coding the same data is useful for the lone researcher and that code-recode consistencies should be at least 90%. To achieve this, several iterations were executed to ensure that the codes appropriately describe the data and that there are no anomalies.

In order to examine the relative share of the categories specified in Table 1, percentage distribution was calculated for individual items indicative of the criteria used in the credibility assessment and diverse answer types. To this end, the number of codes assigned to a category, for example, correctness of information (n = 305) was divided by the total number of the codes dealing with credibility criteria, 935. Second and more importantly, the data were scrutinised by means of qualitative content analysis. To achieve this, the constant comparative method was used to capture the variety of the participants’ articulations (Lincoln and Guba, 1985, pp. 339-344). More specifically, the qualitative analysis concentrated on the participants’ articulations dealing with the assessment of the credibility of information generated by chatbots. The participants’ responses concerning these assessments were systematically compared per individual instances of credibility assessments of diverse kind. In this way, it was possible to identify similarities and differences in the ways in which the Redditors, for example, assessed the trustworthiness of information generated by chatbots and how they grounded their assessments by depicting their opinions.

Although the posts submitted to Reddit are freely accessible to all readers and can be used as research material, the anonymity of the Redditors was protected while presenting illustrative extracts from their posts. This was done by deleting all information identifying the usernames of individual contributors, as well as the dates for their posts. Instead, individual posts were identified using technical codes. For example, in R628-T4, R628 refers to the Redditor who appears in the 628^th place in the alphabetical list of the 919 participants, while T4 identifies discussion thread 4 where his or her post appeared.

To provide background for the qualitative findings, the quantitative overview of the criteria and answer types used in the credibility assessments is presented first. Table 2 specifies how the online participants preferred diverse criteria while judging the credibility of information generated by AI chatbots.

Table 2 indicates that the participants most frequently devoted attention to the correctness of information while assessing the credibility of answers offered by chatbots. This emphasis is mainly due to the frequently expressed doubts that hallucinations generated by chatbots can result in the provision of incorrect information. Trustworthiness of information was another criterion frequently used by the Redditors. In their assessments, they also often drew on the chatbot’s capability to generate relevant information. In addition, the usefulness of information was a significant criterion for many contributors. Other criteria such as verifiability, accuracy, consistency, coverage and currency of information were used less frequency.

A more detailed analysis of the frequencies presented in Table 2 revealed that the majority of the 935 codes assigned to the credibility assessments, that is, 72.8% reflected a critical approach to information generated by AI chatbots, while only 27.2% of the codes were positive in this regard. For example, of the assessments dealing with the correctness of information, no less than 92.5% were negative, that is, indicative of the judgment that information generated by chatbots tend to be incorrect rather than correct. Similarly, it was believed that the information generated by chatbots tends to be untrustworthy (71.1%) rather than trustworthy (28.9% of the codes assigned to this criterion). The only exceptions were the criteria of capability to generate relevant information and usefulness of information. For example, of the codes assigned to the latter criterion, 77.8% were positive suggesting that chatbots offer useful information, despite the fact that it may be not always be fully correct.

The ways in which the participants grounded their credibility assessments by drawing on various types of answers are specified in Table 3.

Table 3 demonstrates that while grounding the credibility assessments, the Redditors mainly depicted their personal opinions about the credibility of chatbots as sources of information. The assessments were also often based on the description of concrete experiences obtained from the chatbot use. In contrast, analytical approaches drawing on comparison and explanation were employed quite seldom. All in all, due to the popularity of opinion and experience answers, the discussions were characterised by the dominance of a descriptive approach to the topic. This means that the online discussions were broad but not particularly deep in the analytical sense.

The main contribution of the present study is the elaboration of the criteria by which people assess the credibility of information content generated by AI chatbots and how such assessments are grounded by making use of diverse answer strategies. The findings draw on the quantitative and qualitative analysis of credibility assessments made by 900+ participants contributing to Reddit discussion threads. Considering that the online participants represent a wide range of views on AI chatbots and experiences obtained about their use, the findings offer a realistic picture of how people assess the credibility of information generated by current LLM applications.

In sum, the findings suggest that correctness, trustworthiness and usefulness of information, as well as the chatbot’s capability to generate relevant information are particularly important credibility criteria. Moreover, criteria such as accuracy, verifiability, coverage and currency of information are used to assess the information content of answers offered by chatbots. The importance of correctness and trustworthiness of information may be due to how AI chatbots are seen as newcomers which have not yet established their position as credible sources information, as compared to Google search engine and Wikipedia. The findings suggest that overall, people tend to take a critical view on AI chatbots, as far as their credibility as information sources is concerned. The critical stance is mainly due to the risk of the provision of hallucinated data generated by chatbots. Even though chatbots are seen capable of offering useful information in certain situations, it is doubted that such information may not be fully trustworthy and correct. Reflecting the limitations of current LLMs, critical assessments also dominate while weighing the credibility of information by means of other criteria such as accuracy, coverage, currency, consistency and verifiability of information. Overall, the main recommendation arising from the Redditors’ assessments of the credibility of AI-generated information can be crystallised as ‘trust with reservations and always verify’.

While assessing the credibility of information, the participants mainly grounded their judgments in personal opinions. Assessments expressed in opinions deal with presumed credibility of information, that is, they pertain to the degree of belief a perceiver holds due to general assumptions in their mind (Tseng and Fogg, 1999). Redditors also often grounded their assessments in their use experiences. Assessments of this kind are indicative of experienced credibility: the degree of belief formed through direct, firsthand experience about an object (Tseng and Fogg, 1999). The popularity of experience answers is understandable because they enable people to express what is most familiar and concrete to them when talking about chatbots. It is easier to report one’s recent experience about a useless answer offered by ChatGPT than make use of answer strategies that are cognitively more demanding. Such strategies include the comparison of accuracy of answers offered by diverse chatbots and the explanation of why information generated by chatbots is difficult to verify.

The reflection of the novelty value of the above findings is rendered more difficult, due to the paucity of similar studies. However, a few comparative notions can be made. First, the observations of the present investigation support the findings of prior studies in that information generated by chatbots is deficient in some regard. Such deficiencies manifest themselves in incorrect or erroneous answers (Choi et al., 2025; Shen et al., 2023), the provision of inaccurate data (Kim et al., 2025) and the presentation of outdated or incomplete information (Murashko, 2025). On the other hand, the results of the present study lend support to conclusions drawn by prior investigations in that chatbots can generate information whose credibility is sufficient in certain situations (Barbosa-Silva et al., 2024). Lim and Hong (2025, pp. 137-138) demonstrated that people’s concern about the risk of obtaining misleading information from AI chatbots does not necessarily mean that their use is abandoned. Many people continue seeking information from chatbots, despite recognising their potential for factual errors. This counterintuitive pattern may reflect users’ growing familiarity with AI chatbots’ limitations and a willingness to tolerate minor inaccuracies in exchange for convenience and speed of information seeking. People do not require complete credibility from chatbots to continue using them, but rather credibility that is good enough.

The above conclusions concur with observations about reflexive mistrust (Citrin and Stoker, 2018). Reflexive mistrust deals with the critical engagement with information available in diverse sources. Given the occurrence of hallucinations, the Redditors exhibited mistrust in that they questioned, verified and cross-referenced information generated by AI chatbots in order to ensure that such information would not contain fabricated elements. Reflexive mistrust of this kind can be understood as a form of information resilience - the capacity to sustain informed decision-making under conditions of contradiction and uncertainty. Rather than undermining trust, the critical stance drawing on reflective mistrust enables individuals to maintain control over their information environment, balancing openness to new data with protective scepticism. On the other hand, the Redditors also exhibited distrust, that is, a settled belief that an information source is inherently untrustworthy (Citrin and Stoker, 2018). Distrust appeared most clearly in cases in which it was believed that LLM powered chatbots draw on low-level training data originating from social media forums such as blogs and online discussion threads.

While considering the possibilities to enhance the credibility of information generated by chatbots, the Redditors mainly drew attention to the need to check the correctness, accuracy, coverage and currency of answers offered by AI tools. To compare, the participants seldom reflected the potential of prompt engineering, that is, the process of designing, crafting and refining instructions (prompts) to guide generative AI models toward producing desired, accurate, and relevant outputs (Lo, 2023). One of the promising generative AI techniques is retrieval augmented generation (RAG). It can enhance the responses generated by LLM so that it references an authoritative knowledge base outside of its training data sources. Kelly et al. (2025) in their study on the effectiveness of AI chatbots in answering questions about type 2 diabetes found that the RAG aspect of the LLM (when a source is cited) resulted in 94% of responses being completely appropriate in one-off questions and 100% being appropriate in a test conversation. This suggests that the formulation of the prompt is critical to ensure that chatbots provide only the requested information, do not stray from the topic of interest, and do not fabricate information. On the other hand, as Magesh et al. (2025) demonstrated, RAG is not necessarily a panacea for the elimination of hallucinations. The findings indicate that while RAG appears to improve the performance of language models in answering legal queries, the hallucination problem persisted at significant levels. This is because RAG may encounter its limitations while generating more specific responses to legal issues (Magesh et al., 2025, p. 220-221). We may think that the above reservations are also relevant in other domains while making attempts to enhance the credibility of information generated by AI chatbots.

The present investigation contributed to empirical research on credibility assessment by focusing on information generated by AI chatbots. The findings highlight that in this regard, people’s opinions and experiences about the correctness, trustworthiness and usefulness of information are particularly important. As the present study focused on a sample of Reddit discussion threads, the findings cannot be generalised to concern credibility assessments made in other contexts. This is because the critical tone of most assessments presented by the Redditors may be due to the particular features of online discussion culture. It may encourage the articulation of critical or negative rather than positive assessments while debating controversial issues in particular (Savolainen, 2023). Thus, there is a need to expand the research approach by making use of empirical data gathered from other groups of people. To this end, interviews with diverse groups of chatbot users, as exemplified by Choi, Bak and An (2025) can be particularly fruitful. It is evident that interviews - particularly if substantiated with the analysis of critical incidents dealing with chatbot use in information seeking - would offer a more nuanced picture of the ways in which people assess the credibility of information generated by AI chatbots.