IR Information Research 1368-1613 University of Borås ir30iConf47155 10.47989/ir30iConf47155 Research article Enhancing vaccine communication in social Q&A: identifying readily applicable factors for answer acceptance on medical sciences stack exchange FuHengyi Hengyi is an Assistant Professor in the School of Library and Information Studies, at the University of Alabama. Her primary research interests are online peer production, health information behaviour/interaction, and AI implementation in libraries. 06052025 2025 30 i 960 969 2025 © 2025 The Author(s). This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), permitting all non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract

Introduction. Despite the efficacy of vaccines, online misinformation contributes to vaccine hesitancy. This study investigates factors influencing the acceptance of answers to vaccine-related questions on social Q&A platforms, aiming to improve online vaccine communication.

Method. We analysed 511 vaccine-related questions and 875 associated answers on the medical sciences stack exchange platform. Eleven variables were identified as potential predictors of answer acceptance, including citation practices, use of personal experiences, and recognition of asker’s concerns.

Analysis. Logistic regression was employed to determine which variables significantly predict the likelihood of an answer being designated as the ‘accepted answer’ by the asker.

Results. Personal experience sharing, recognition of asker’s concerns, citing data sources, and including calls-to-action emerged as significant predictors of answer acceptance. Notably, professional credentials had a modest effect, while factors such as the use of analogies or citing health authorities were not significant predictors.

Conclusions. Our findings suggest that effective vaccine communication on social Q&A platforms should balance factual information with personal narratives, address user concerns empathetically, and provide clear, actionable advice. These insights can inform strategies for health communicators and platform designers to enhance the quality and acceptance of vaccine information online.

Introduction

Vaccines have prevented millions of deaths and substantially reduced the burden of infectious diseases globally. However, despite their proven efficacy, vaccine hesitancy has emerged as a significant global health concern. The World Health Organization has identified vaccine hesitancy as one of the top ten threats to global health, highlighting the urgent need for effective vaccine communication strategies (World Health Organization, 2019).

The advent of social media has dramatically altered the landscape of health information dissemination, particularly regarding vaccines. Social Q&A communities have evolved into significant platforms where users seek and share advice, knowledge, and experiences on various health topics, including vaccination (Sharon et al., 2020). The exchange of vaccination-related questions and answers has become particularly prominent during the COVID-19 pandemic, reflecting increased concern for understanding vaccination protocols and their effects (Wang et al., 2022, Fu and Oh, 2023). In this context, the ability to craft convincing and accurate responses to vaccine-related questions on social Q&A platforms has become crucial. Health professionals, public health communicators, and concerned citizens engaging in these spaces face the challenge of not only providing factual information but also addressing emotional concerns, countering misinformation, and fostering vaccine confidence (Chou and Budenz, 2020).

Thus, the fundamental question this project aims to answer is: how can we provide convincing and persuasive responses to online vaccine-related information seeking? Our focus is on identifying factors that are easily recognized and implemented by general users when writing responses on social Q&A platforms and other online forums, and also can be easily incorporated into community’s policy system or health education materials.

In this paper, we report our preliminary findings on accepted answers selected by askers for vaccine-related questions on the medical science stack exchange Q&A site. Specifically, we explore what variables can predict whether askers will designate a given answer to a vaccine-related question as the ‘accepted answer.’ Unlike features such as readability score, which can be difficult for ordinary users to consistently improve, we focus on factors that answerers can easily be conscious of and implement, and that can be readily applied by health professionals, online community designers, and concerned citizens engaging in online vaccine discussions (Oh et al., 2012, Kienhues et al., 2011).

By focusing on these readily applicable factors, we aim to provide practical insights that can be put into action by individuals seeking to contribute positively to online vaccine discussions. This research not only contributes to improving public health communication strategies but also empowers individuals to play an active role in combating misinformation and promoting vaccine confidence in digital spaces (Vraga and Bode, 2017).

It’s important to interpret the results of this paper carefully. Right now, our study doesn’t suggest a cause-effect relationship. We’re not saying that the variables/factors identified are the only or the full set behind the difference of accepted or non-accepted answers.

 Previous studies

Despite the proven efficacy of vaccines in preventing disease and saving lives, vaccine hesitancy has emerged as a significant concern, leading to delays and refusals in vaccination. This scepticism is often combined with mistrust and suspicion of health providers and medical companies, potentially compromising their impartiality (Sharon et al., 2020). During the COVID-19 pandemic, such dynamics have intensified, with misinformation significantly influencing vaccination decisions.

Research has shown that users evaluate answers based on multiple criteria, including completeness of information, solution feasibility, and the answerer’s attitude (Kim et al., 2009), with answer quality influenced by the answerer’s reputation and supporting references (Shah & Pomerantz, 2010). Harper et al. (2008) found that answer length, references, and the answerer’s history were significant predictors of answer acceptance. Zhang (2014) found that users value both expertise indicators and relatable personal experiences when evaluating health-related answers. These findings complement Moran et al.’s (2016) analysis of persuasive techniques in anti-vaccine websites, particularly noting the impact of personal narratives and scientific evidence.

Kienhues et al. (2011) highlighted the effectiveness of balanced information presentation, including potential risks and benefits, which informed our examination of factors like recognizing asker’s concern and citing multiple sources. This aligns with Vraga and Bode’s (2017) findings that corrections were more effective when they explained why misinformation was incorrect rather than simply stating it was wrong.

Oh et al. (2016) found that users seeking health information value emotional support alongside factual information. Sharon et al. (2020) found that while few users explicitly sought expert testimony, answers from self-identified health professionals were twice as likely to be designated as ‘best answers.’ Jang et al. (2022) and Xu and Guo (2018) demonstrated how emotional language and narrative approaches influenced vaccine information acceptance. Chou and Budenz (2020) emphasized that effective vaccine communication should address both factual information and emotional concerns, while Lunz Trujillo et al. (2021) found that stories highlighting non-vaccination consequences were particularly effective.

While these studies have addressed various factors influencing health information acceptance, no comprehensive model on predictive features for vaccine-related information has been tested in social Q&A platforms. Some predictive features, even when proven effective, are not easily applicable by ordinary users or community designers. Our study contributes by providing empirical evidence on factors influencing vaccine-related answer acceptance in a specialized Q&A community.

 Method Studying site and dataset

The medical sciences stack exchange (MSSE) (https://medicalsciences.stackexchange.com/) is a specialized online health Q&A platform within the broader Stack Exchange network, which hosts over 200 subject-oriented Q&A communities. Launched in 2015, MSSE has grown to become one of the most prominent platforms dedicated to health and medical sciences, with a community of 19,527 registered users as of January 2024. Unlike other Q&A platforms such as Yahoo! Answers or Quora, MSSE prioritizes community-driven development and self-governance, with members playing an active role in managing and moderating the platform. Additionally, MSSE actively encourages participation from health professionals to align with its mission of disseminating accurate medical knowledge. Another key advantage of MSSE is its commitment to data transparency and accessibility. The platform publishes all user-generated content, making it publicly available through its interface and application programming interface (API).

In this study, we extracted all questions posted on MSSE between March 31, 2015, and December 31, 2022, using the official API. The quantitative and interactive question features, including question score, view count, answers, answer count, and accepted answer status, were collected until March 31, 2023. A total of 6,760 questions and 6,505 answers were collected. Out of these, 511 vaccine-related questions were identified by retrieving questions containing the keywords ‘vaccine(s)’, ‘vaccination’ or related synonyms in the questions and were used for further analysis. There were in total 875 answers associated with the 511 vaccine-related questions. Among those answers, 323 answers (36.91%) were selected as accepted answers by the asker. A question can have only one accepted answer selected by the asker, and some questions do not have any accepted answer. It is important to note that on MSSE, the acceptance of an answer is decided solely by the question asker. While this does not necessarily reflect a collective assessment of answer quality, it indicates whether the asker’s information needs have been met.

 Variable selection

The dependent variable represented whether the answer appeared as the ‘accepted answer’ to the question. It was coded as 0 or 1 to signify ‘no’ and ‘yes’. As a starting point, when choosing the independent variables, we began with those can be consciously controlled or easily incorporated into answers, including citing of research (Oh et al., 2012, Diviani et al., 2015), citing of data resources (Oh et al., 2012, Sbaffi and Rowley, 2017), citing health authorities (Vraga and Bode, 2017), citing other answers (Oh et al., 2012), use of hedging language (Mayweg-Paus and Jucks, 2015, Dunn et al., 2015), use of formatting for clarity (Meppelink et al., 2015, Lazard and Mackert, 2014), indicating him/her is a health professional (Hu and Shyam Sundar, 2010, Huh et al., 2013), including personal experience (Pian et al., 2020, Ziebland, 2012), including call-to-action (Ludolph et al., 2016, Korda and Itani, 2013), use of analogies and metaphors (Pelaccia et al., 2011, Reyna, 2012), recognizing asker’s concern (even if the concern is not scientifically valid) (Goldstein et al., 2015, Amin et al., 2017). All these variables have been examined and proven effective in serving online health information needs in previous research. Table 1 lists explanations, measurements, and examples of each variable (We use a list of 111 terms of hedges in Bordignon et al. (2021)).

Variable description, example, and value

Variable Description Example Value
if_accepted If an answer is an accepted answer or not N/A (determined by platform) 0 or 1
n_citing research Number of research citations in the answer ‘A study by ABC et al. (2022) found that vaccine efficacy remains high after six months…’ Integer ≥ 0
n_citing_datasource Number of data sources cited in the answer According to the CDC’s COVID Data Tracker (https://covid.cdc.gov/covid-data- tracker), vaccination rates have increased by 15% since last month…” Integer ≥ 0
n_citing_health_authorities Number of times health authorities mentioned ‘CDC recommended annual flu vaccines for everyone over 6 months old…’ Integer ≥ 0
n_citing_other_answer Number of references to other answers ‘To add to what @DrSmith said about vaccine safety in their answer above...’ ‘Building on the point about herd immunity raised by (answer link) in his response...’ Integer ≥ 0
n_hedging term/word Number of hedging terms an answer includes ‘It’s possible that new variants could affect vaccine efficacy…’ Integer ≥ 0
if_formatting If the answer uses formatting (bullet points or numbered lists) ‘Common vaccine side effects:\n1. Soreness at injection site\n2. Mild fever\n3. Fatigue…’ 0 or 1
if_health_pro If the answerer indicates s/he is a health professional ‘Speaking from my perspective as a family physician…’ 0 or 1
if_personal_exprience If the answer includes personal experience ‘After getting my COVID-19 vaccine, I experienced mild arm soreness for also three days…’ 0 or 1
if_call_to_action If the answer includes call to action ‘If you’re concerned about vaccine side effects, I recommend keeping a symptom diary for a few days after receiving the vaccine…’ 0 or 1
if_analogies If the answer uses analogies or metaphors ‘The mRNA in vaccines is like a blueprint. It doesn’t build anything itself…’ 0 or 1
if_recognazing_concern If the answer recognizes the asker’s concern, even if it’s not scientifically valid ‘While I can assure you that vaccines do not contain microchips for tracking, your question reflects a broader concern about privacy and trust in medical interventions...’ 0 or 1

We employed a combination of automatic and manual coding methods to code variables for each answer. For the automatic coding process, we used Python to develop Regex (regular expression) rules for the initial screening of certain variables. These rules identified links or symbols associated with citing research, data resources, and other answers, as well as specific formatting elements such as bullet points and numbered lists. Additionally, Regex was utilized to detect keywords or phrases indicative of hedging language, such as ‘possibly’ or ‘it seems’ (Bordignon et al., 2021).

Manual coding was conducted to comprehensively annotate variables that required nuanced interpretation, such as recognizing the use of personal experiences or assessing whether an answer addressed the asker’s concerns. This process involved two researchers who worked collaboratively to refine coding guidelines and resolve ambiguities in the annotation process. Both researchers independently coded a subset of the data to calculate intercoder reliability, achieving a Cohen’ s κ of 0.856, which indicates a high level of agreement. Discrepancies were resolved through discussion to ensure consistency across the dataset.

Automatic coding was also performed using the GNU ‘Style and Diction’ software package (https://www.gnu.org/software/diction/) following Fu and Oh (2019) for more sophisticated writing style features, such as sentence complexity and vocabulary usage. However, these features were not included in our final analysis as they do not align with our focus on readily applicable factors.

 Preliminary results and discussion

A logistic regression analysis was conducted using SPSS to examine variables influencing the likelihood of an answer being selected as the ‘accepted answer’ in vaccine-related questions on Medical Science Stack Exchange. The model demonstrated moderate predictive power, with a Nagelkerke R² of 0.474, indicating that it explains 47.4% of the variance in accepted answer selection. The Omnibus Test of Model Coefficients indicated that the accuracy of the model exceeded that of the baseline model when adding the predictors (χ² (11) = 204.623, p < 0.001). The Hosmer & Lemeshow test of goodness of fit suggested the model was a good fit to the data (p = 0.279).

Logistic regression predicting the likelihood of selecting as ‘accepted answer’

Variable β S.E. β Wald’s χ² df p e^β (odds ratio)
Constant -1.885 0.145 168.905 1 <.001 *** 0.152
if_personal_experience 0.391 0.079 24.503 1 <0.001*** 1.478
if_recognizing_concern 0.287 0.082 12.252 1 <0.001*** 1.332
n_citing_datasource 0.269 0.072 13.958 1 <0.001*** 1.309
if_call_to_action 0.230 0.078 8.686 1 0.003** 1.259
n_citing_research 0.182 0.068 7.164 1 0.007** 1.200
if_health_pro 0.184 0.073 6.356 1 0.012* 1.202
n_citing_health_authorities 0.156 0.081 3.710 1 0.054 1.169
if_formatting 0.143 0.094 2.315 1 0.128 1.154
if_analogies 0.128 0.099 1.670 1 0.196 1.137
n_citing_other_answer 0.117 0.082 2.035 1 0.154 1.124
n_hedging_term -0.074 0.051 2.105 1 0.147 0.929
Overall model evaluation χ² df p
Omnibus Test of Model Coefficients 204.623 11 <0.001
Goodness-of-fit Test χ² df p
Hosmer & Lemeshow 9.8 8 0.279
Nagelkerke R²: 0.474

The results show several factors that powerfully predict the likelihood of an answer being chosen as an accepted answer. Personal experience emerged as the strongest predictor (OR = 1.478, p < 0.001), suggesting that answers incorporating the writer’s personal experiences are 47.8% more likely to be selected as accepted. This finding aligns with previous research that highlighted the value users place on personal experiences in online health information seeking. Recognizing the asker’s concerns also proved to be a significant factor (β = 0.287, OR = 1.332, p < 0.001). While Medical Stack Exchange is a health Q&A community that emphasizes professional health information sharing and in general, does not recommend emotional support and too much personal experience (Fu and Oh, 2022), this highlights the importance of empathy and direct engagement with the asker’s perspective in effective health communication.

The citation of data sources (OR = 1.309, p < 0.001) and research (OR = 1.200, p = 0.007) both significantly increased the odds of an answer being selected. This indicates that users value evidence-based responses, with each additional data source cited increasing the odds by 30.9% and each research citation increasing the odds by 20%. These findings emphasize the importance of backing claims with credible sources in vaccine-related discussions. These findings align with previous research emphasizing the importance of backing claims with credible sources in health- related online discussions.

Interestingly, including a call to action in the answer also positively influenced selection (OR = 1.259, p = 0.003). Answers that provided clear, actionable advice were 25.9% more likely to be chosen as best, suggesting that users appreciate practical guidance alongside information.

The identity of the answerer as a health professional had a modest effect (OR = 1.202, p = 0.012). Unlike other studies, this indicates that while professional credentials are valued, they are not the dominant factor in determining the perceived quality of an answer in this context.

Notably, several factors that might be expected to influence answer selection were found to be non-significant. These included citing health authorities, the use of formatting for clarity, employing analogies, referencing other answers, and the use of hedging language. The non-significance of these factors suggests that users may prioritize content and relevance over stylistic elements or appeals to authority. The non-significance of references to public health authorities may be caused by the distrust of public authorities during the pandemic but requires further investigation.

 Implications, limitations, and future research

Our study identifies several key factors that predict the acceptance of vaccine-related answers on social Q&A platforms, which have implications for various stakeholders.

For ordinary users, implementing the identified factors can enhance the credibility and engagement of their responses. Our finding that personal anecdotes significantly increase the likelihood of answer acceptance empowers users to confidently share their own vaccine experiences. The fact that answer acceptance is not solely dependent on professional credentials but also on factors like citing data sources, encourages more users to contribute their knowledge. These insights are particularly valuable for users who might hesitate to contribute to vaccine discussions due to a perceived lack of professional credentials. By focusing on these accessible factors, ordinary users can craft more effective responses and play a more active role in online vaccine discussions.

While personal narratives can make information more relatable and engaging, they also present potential risks in health communication, particularly around vaccine information. Personal experiences might inadvertently spread misinformation if they present outlier cases as typical or contradict scientific evidence. One potential solution, supported by our findings, is to encourage the combination of personal experiences with data citations and research citations. This combined approach could help maintain scientific accuracy while preserving the engaging aspects of personal narratives.

For platform designers, the insights from this study can directly inform the design of social Q&A platforms to facilitate more effective vaccine communication. Specific features could be integrated into the answer submission process, such as prompts encouraging users to share relevant personal experiences, built-in citation tools to easily reference credible sources, and suggestion systems that remind users to address the asker’s specific concerns. Additionally, platforms could refine their algorithms to prioritize answers exhibiting these characteristics. For example, answers that include data citations and address user concerns could be given higher visibility, potentially improving the overall quality of information presented to users.

Health communicators can tailor their messages by incorporating those factors to increase relatability, consistently citing reputable data sources to enhance credibility. These strategies can be incorporated into health communication curricula, with specific modules on effective online communication. For instance, training exercises could focus on crafting responses that balance personal experience, scientific evidence, and empathetic addressing of concerns. This approach could lead to a new generation of health communicators better equipped to engage in digital health discussions, particularly around vaccines.

This study has several limitations. Firstly, our analysis is based on data from a single platform, Medical Science Stack Exchange, which may not be representative of all social Q&A environments or user demographics. Secondly, the study focuses on a specific time period, which may not capture long-term trends or changes in user behavior, especially given the rapidly evolving nature of vaccine discourse during the COVID-19 pandemic. Future research could address these limitations by expanding the analysis to multiple platforms and conducting longitudinal studies to track changes over time. Additionally, qualitative research methods, such as interviews or focus groups with users, could provide deeper insights into the reasoning behind answer acceptance and the nuanced ways in which users interpret and value different answer characteristics.

 References AminA. B.BednarczykR. A.RayC. E.MelchioriK. J.GrahamJ.HuntsingerJ. R.OmerS. B.2017Association of moral values with vaccine hesitancyNature Human Behaviour112873880 BordignonF.ErmakovaL.NoelM.2021Over‐promotion and caution in abstracts of preprints during the COVID‐19 crisisLearned Publishing344622636 ChouW.-Y. S.BudenzA.2020Considering emotion in COVID-19 vaccine communication: addressing vaccine hesitancy and fostering vaccine confidenceHealth communication351417181722 DivianiN.Van Den PutteB.GianiS.van WeertJ. C.2015Low health literacy and evaluation of online health information: a systematic review of the literatureJournal of Medical Internet Research175e112 DunnA. G.LeaskJ.ZhouX.MandlK. D.CoieraE.2015Associations between exposure to and expression of negative opinions about human papillomavirus vaccines on social media: an observational studyJournal of Medical Internet Research176e4343 FuH.OhS.2019Quality assessment of answers with user-identified criteria and data- driven features in social Q&AInformation Processing and Management5611428https://doi.org/https://doi.org/10.1016/j.ipm.2018.08.007 FuH.OhS.2022Online community development in the early stages: the life cycle model application to Medical Sciences Stack ExchangeAslib Journal of Information Management74612141232https://doi.org/10.1108/AJIM-11-2021-0345 FuH.OhS.2023Topics of questions and community interaction in social Q&A during the COVID‐19 pandemicHealth Information & Libraries Journalhttps://doi.org/10.1111/hir.12506 GoldsteinS.MacDonaldN. E.GuirguisS.2015Health communication and vaccine hesitancyVaccine333442124214https://doi.org/https://doi.org/10.1016/j.vaccine.2015.04.042 HuY.Shyam SundarS.2010Effects of online health sources on credibility and behavioral intentionsCommunication research371105132 HuhJ.Yetisgen-YildizM.PrattW.2013Text classification for assisting moderators in online health communitiesJournal of biomedical informatics4669981005 JangS. H.GerendM. A.YoumS.YiY. J.2022Understanding coronavirus disease 2019 (COVID-19) vaccine hesitancy: Evidence from the community-driven knowledge site QuoraDigital Health820552076221145426https://doi.org/10.1177/20552076221145426 KienhuesD.StadtlerM.BrommeR.2011Dealing with conflicting or consistent medical information on the web: When expert information breeds laypersons’ doubts about expertsLearning and Instruction212193204 KimS.OhS.OhJ. S.2009Best-answer selection criteria in a social Q&A site from the user-oriented relevance perspectiveProceedings of the American Society for Information Science and Technology451115 KordaH.ItaniZ.2013Harnessing social media for health promotion and behavior changeHealth promotion practice1411523 LazardA.MackertM.2014User evaluations of design complexity: The impact of visual perceptions for effective online health communicationInternational journal of medical informatics8310726735 LudolphR.AllamA.SchulzP. J.2016Manipulating Google’s knowledge graph box to counter biased information processing during an online search on vaccination: Application of a technological debiasing strategyJournal of Medical Internet Research186e137 Lunz TrujilloK.MottaM.CallaghanT.SylvesterS.2021Correcting misperceptions about the MMR vaccine: Using psychological risk factors to inform targeted communication strategiesPolitical Research Quarterly742464478 Mayweg-PausE.JucksR.2015Evident or doubtful? How lexical hints in written information influence laypersons’ understanding of influenzaPsychology, health & medicine208989996 MeppelinkC. S.van WeertJ. C.HavenC. J.SmitE. G.2015The effectiveness of health animations in audiences with different health literacy levels: an experimental studyJournal of Medical Internet Research171e3979 MoranM. B.LucasM.EverhartK.MorganA.PrickettE.2016What makes anti-vaccine websites persuasive? A content analysis of techniques used by anti-vaccine websites to engender anti-vaccine sentimentJournal of Communication in Healthcare93151163 OhS.YiY. J.WorrallA.2012Quality of health answers in social Q&AProceedings of the American Society for Information Science and Technology OhS.ZhangY.ParkM. S.2016Cancer information seeking in social question & answer services: Identifying health-related topics in cancer questions on Yahoo! AnswersInformation Research213n3 PelacciaT.TardifJ.TribyE.CharlinB.2011An analysis of clinical reasoning through a recent and comprehensive approach: the dual-process theoryMedical education online1615890 PianW.SongS.ZhangY.2020Consumer health information needs: A systematic review of measuresInformation Processing & Management572102077 ReynaV. F.2012Risk perception and communication in vaccination decisions: A fuzzy-trace theory approachVaccine302537903797 SbaffiL.RowleyJ.2017Trust and credibility in web-based health information: a review and agenda for future researchJournal of Medical Internet Research196e218 ShahC.PomerantzJ.2010Evaluating and predicting answer quality in community QAProceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval411418 SharonA. J.Yom-TovE.Baram-TsabariA.2020Vaccine information seeking on social Q&A servicesVaccine381226912699https://doi.org/10.1016/j.vaccine.2020.02.010 VragaE. K.BodeL.2017Using expert sources to correct health misinformation in social mediaScience communication395621645 WangL.XianZ.DuT.2022The public information needs of COVID-19 vaccine: A study based on online Q&A communities and portals in ChinaFrontiers in psychology13961181https://doi.org/10.3389/fpsyg.2022.961181 World Health Organization2019Ten threats to global health in 2019https://www.who.int/news-room/spotlight/ten-threats-to-global-health-in-2019 XuZ.GuoH.2018Using text mining to compare online pro-and anti-vaccine headlines: Word usage, sentiments, and online popularityCommunication studies691103122 ZhangY.2014Beyond quality and accessibility: Source selection in consumer health information searchingJournal of the Association for Information Science and Technology655911927 ZieblandS.2012Health and illness in a connected world: How might sharing experiences on the internet affect people’s health? The Milbank Quarterly902219249https://doi.org/https://doi.org/10.1111/j.1468-0009.2012.00662.x