In the last decade, artificial intelligence has become increasingly impactful in both socio-cultural life and humanities research, from generating poetry that competes against ‘the work of professional writers’(Köbis and Mossink, 2021) to deciphering ancient scrolls (Marchant, 2024). However, tensions persist between humanities methodologies and AI technologies (Chun and Elkins 2023). One key question is AI applications’ capabilities to depict the complexities and nuances of humanities data and questions (Chun and Elkins 2023; Windhager and Mayr, 2024). This is a particularly pressing problem for digital humanities research on social media data ‘produced within the fabric of sociomaterial’ (Wu, 2023). Due to factors ranging from the platform owners’ restricted data access to heterogeneous user behaviour, rarely can scholars comprehensively evaluate the limitations, biases, and representativeness of the datasets available for research. This lack of understanding of the datasets remains a bottleneck for critical usage of large humanities datasets, despite advances in AI (Nguyen et al. 2020).

To address this problem, digital humanities and information sciences scholars have proposed multiple theories and methods, such as data archaeology (Lee, 2021), predictive modelling (Underwood 2019), and speculative bibliography (Cordell, 2020). It is increasingly acknowledged that a nuanced understanding of the data as it is (e.g., how the data has been created, curated, and used within social, cultural, and technical systems) is key to computational modelling that adequately reflects the dataset’s complexities and contexts.

Digital humanities research on online book review studies provides a modelling research case. Although advanced AI models have been applied to reviews across various platforms and languages, answers to some research questions remain unclear (Herrmann et al., 2024). For example, are all reviewers merely book readers (Landow, 2006)? To what extent are reviews incentivized (Hu et al., 2023), copied (David and Pinch, 2006), or generated by AI? Why do some reviewers rate their favourite books with 1 star and less favoured ones with 5 stars (citation anonymized for blind review)? Without addressing these questions, AI applications can hardly make reliable and insightful interpretations of millions of book reviews available.

Among various book reviews, those for children’s books are particularly challenging to interpret computationally as they tend to involve multiple stakeholders and perspectives. For example, many self-identified parent reviewers provide opinions of their own and those of their children in the same review; therefore, opinions and sentiments presented in the same review might diverge drastically. Yet only one numeric rating is associated with each review. Such features pose practical questions to data analysis: to what extent and how are children readers’ opinions represented by online book reviews, and how can researchers identify them? When multiple opinions are presented in one review, how can the computer disentangle the discourses or enrich the single-valued ratings?

It is essential to explore these questions for more advanced computational data analysis, which advances humanities understanding of children’s literature and yields practical implications. For decades, a number of heated discussions on children’s literature remains theoretical or anecdotal, partially due to the lack of reading records (Hu, et al. 2023): Who is in a position to judge children’s literature, adults, children, or both (Abate, 2020)? What are the perceived differences between literature for children and young (Gubar, 2011; Trites, 2001)? Unprecedented online book reviews enable scholars to empirically understand the reception of children’s literature from different perspectives and with varied granularities, answering existing research questions while raising new ones.

Given such backgrounds, we conducted exploratory data analysis on 37,408 reviews on 17,407 children’s books collected from the social reading platform Goodreads. This is designed as an exemplar case study of inspecting characteristics of social media datasets for advancing cultural data analytics. We leveraged both computational models and qualitative analysis of content to explore three questions: Who engages with reading and reviewing children’s literature? How are topics and opinions presented in the reviews? Based on the first two questions, what are the practical implications for AI-infused research on children literature reviewing and datasets alike? Specifically, our research focuses on reviews that explicitly mention family members, as reading and reviewing children’s books within families is a prominent setting identified in our dataset. In the remainder of this paper, first, we introduce the research dataset and method. Then we present our data analysis and outcomes. Finally, we summarise our findings and their limitations, along with proposals for future work.

We leveraged existing research datasets called Goodreads book graph datasets (Wan et al. 2019; Wan and McAuley 2018). The data was scraped from Goodreads, a social reviewing platform based in the U.S. and one of the most prominent data sources for digital humanities research on online book reviewing (Walsh and Antoniak 2021). Our analysis focused on its subset of reviews for children’s books. We pre-processed the reviews by dropping the reviews that are not in English or too short to be semantically meaningful (with less than 50 English characters). This process returned 297,222 reviews. Then, we developed a small dictionary for computationally identifying the reviews that mentioned family members. Based on our preliminary data annotation and testing, the regular expression of ’my’ followed by a singular or plural form of a noun for a family member effectively and efficiently retrieves reviews of our research interest. Please see Appendix 1 for the family members included in the dictionary. We include ‘friends’ as an exceptional ‘family member’ because our pilot research shows that friends are often referred to with similar intimacy and significance as one’s family members. With this dictionary and regular expression matching, we retrieved 37,408 reviews that mention family members. It is to be noted that neither our dictionary nor the regular expression-based method can exhaustively identify all reviews mentioning family members, as our goal was to identify sufficient instead of all reviews in this category for exploratory analysis.

We applied two complementary analyses to the reviews. First, we conducted a computational overview of the review regarding (1) mentions of family members and (2) topics that emerge from the reviews. Second, we conducted a qualitative analysis of content (Wildemuth, 2017) on a stratified sample of 400 reviews. For topic modelling, we used the topic coherence measure implemented in Gensim (Rehurek and Sojka, 2011) to optimise the number of topics: using LDA modelling and searching across a range from 10 to 90 topics, the topic coherence optimised at 40. Therefore, we set the number of topics to 40. Please see the Appendix 2 for details of the topics. For content analysis, we randomly selected an even number of reviews associated with each of the family members appearing in the reviews. After deduplication, we got a sample of 400 reviews which we manually annotated and analysed.

Our research proposal was reviewed by the Institutional Review Board (IRB) of our current institution. A decision was made that the application for non-human subjects’ research or IRB application was not required. Nevertheless, our research complies with the guidelines proposed by latest research on the legal and ethical use of online book reviews in research (Hu, Layne- Worthey, et al. 2023). For instance, reviews quoted in this paper were all paraphrased and further anonymized to minimise the risks that might be caused to reviewers (e.g., reidentification).

First, our topic modeling on the 37,408 reviews provides an overview of topics discussed. In addition to family members, topical words center on book genres and features (e.g., illustration, version, and graphic novel), sentiment and emotions (e.g., amazing, humorous, and thrill), education and schooling (e.g., teacher, school, and study), as well as reading behaviour and scenarios (e.g., reread, bedtime, and listen). For instance, words associated with fairytales and fantasy stories are also commonly seen (e.g., magic, mystery, and enchant), which aligns with genre features of children’s literature. Comparing mentions of family members with the topics, we find that reviews involving senior family members, particularly grandparents, are more likely to involve nostalgia emotions, childhood memories, and retrospective reflections. It is also noted that many topic words are conventionally associated with female characteristics, such as actress, witch, ballet dancer, and princess, which might reflect gender stereotypes in children’s literature.

Second, we applied close reading and qualitative analysis to the sample of 400 reviews. 70 reviews explicitly mentioned children’s reactions to the book being reviewed, while none of the reviewers self-identified themselves as children. Most of the reviews were created (1) to share personal opinions about a children’s book as a child supervisor and/or care provider; (2) to express retrospective opinions about a book they had read in their childhood; and (3) to express their thoughts about a book they recently got introduced to or re-read.

These suggest that the reviews might be dominated by adults’ opinions. Specifically, according to the reviews written by self-identified parents, parents tend to make decisions for their children, and sometimes even impose their preferences on their children. For instance, one parent wrote:

if I cannot bear with a book, I’m not going to read it to my kids by giving it away or hiding it somewhere they won’t find or request the book.

Children’s opinions might be underrepresented in numeric book ratings as well. For instance, a parent gave a 3-star rating to a book and commented ‘I read this book to my girls who are 5 and 11. They both loved it; but I was bored’. While acknowledging their children’s love for the book, this review did not give it 4-star or 5-star ratings, which are generally regarded as positive. A few parents further explained why they failed to relate to their children’s favor for a book, such as ‘my children love this book, probably because it contains the word fart, but I do not think this is one of my favourite books on the shelves’.

Another parent commented:

my children love this book because I cannot help crying when reading it aloud to them, then they point at me and laugh, when their father weeps, trying to get the words out without turning into a sobbing mess. sigh. I do love my damn kids.

Such disagreements and divergences of opinions across reviewers and their children also exist between reviewers and their more senior family members. For instance, a reviewer wrote ‘I will never get why my grandmom was so fond of this book. Who needs an upsetting story about cruelty to animals? I am not’.

Nevertheless, there are also cases where opinions about books converge across generations and books get to be passed on in a family, as the review below shows:

my grandma used to read this book to me when I was a kid every time I stayed overnight at her place. I still vividly remember how I sat on her lap in a recliner and listened to her voice. I can still hear it. One day I will read this book to my own kids.

Out of the 70 reviews, only two offer perspectives from multiple children, as shown in Figure 3. The first review includes the opinions of two child readers but does not reference the adult reviewer’s perspective. The second review presents ratings and feedback from each family member, including two adults and three children. For such reviews that feature multiple sets of opinions and ratings from different individuals, the standard approach of treating one review as representative of a single reviewer no longer applies. A more nuanced data analysis, accommodating varying granularities of opinions, is needed for a better interpretation of the data.

This case study of 37,408 Goodreads reviews on children’s books has three primary findings. First, a variety of family members were mentioned in online book reviews on children’s books, which suggests their broad participation in reading children’s literature. Parents and female family members played especially prominent roles in reading such books in households and reviewing them online. Second, while various topics were discussed in the reviews, the reviews were predominantly adults’ voices. Children’s feedback on books appears to be underrepresented in both review texts and numeric ratings. Third, findings of our study, such as the gender-specific topic words and some parents’ irrelevance to children’s love for certain books, speak to problems identified by critical scholarship on children’s literature, such as sexism and gender issues in children’s literature (Varga-Dobai 2013), paternalism (Gubar, 2020) and censorship (Jenkins, 2010; MacLeod, 1983) in children’s reading activities. Such connections demonstrate the promise of enriching critical scholarship on children’s literature with online book reviews and empirical methods. Questions worthy of future investigation also emerge from our preliminary study, such as (1) how cross-generation readers of the same family negotiate with each other, given divergent opinions about the same books; and (2) how female family members’ participation might have shaped online book reviewing.

As an exploratory analysis of a specific dataset, these preliminary findings are subject to further examination and might not apply to other datasets. Therefore, we intend further examine the prominent roles played by female family members through hypothesis testing and analysis on other datasets. Nevertheless, existing findings demonstrate the complexities of online reviews of children’s books and have practical implications for cultural data analysis. For instance, this case study demonstrates how mixed-method analysis contributes to a more comprehensive understanding of online cultural datasets; such understanding should help researchers identify suspicious data instances such as fake reviews and AI-generated content, so as to improve data quality in scholarly research. In the future, we plan to deepen our analysis by conducting more qualitative investigations and testing our findings on other datasets. We are currently working on surveys on online book reviews and interviews on reading children literature at home, to advance our understanding of the dissemination and evaluation of children’s literature.