Developing a teacher agent to explore the role of gaze direction on young school children’s comprehension

Författare

  • Jonas Brännström Lund University
  • Sebastian Waechter Lund University
  • Jens Nirme Lund University
  • Susanna Whitling Lund University
  • Viveka Lyberg-Åhlander Åbo Academy and Lund University
  • Birgitta Sahlén Lund University

DOI:

https://doi.org/10.58986/al.2024.13972

Nyckelord:

listening comprehension, classroom, gaze behaviour, teacher agent

Abstract

Teachers’ gaze and school children’s language proficiency play a role for listening comprehension in the classroom. The influence of multimodal conditions on listening comprehension may be hard to study in a systematic manner with real speakers. The aim was to test the concept and feasibility of studying the influence of a speakers on passage comprehension performance in children using a digitally animated character. Twenty-four primary-school children were tested. A narrative passage was presented to the participants by a teacher agent. Half of the participants were randomized to a direct gaze condition and half to an averted gaze condition. The method is feasible with the intended population. We detected no differences in listening comprehension of the teacher agent’s gaze behavior in our sample. The concept works and it is feasible to test the effect of speaker gaze on children using the proposed methodology with some further development.

Nedladdningar

Nedladdningsdata är inte tillgängliga än.

Referenser

Bente, G., Eschenburg, F. and Kramer, N. 2007. Virtual gaze. A pilot study on the effects of computer simulated DOI: https://doi.org/10.1007/978-3-540-73335-5_21

gaze in avatar-based conversations. In: International Conference on Virtual Reality 2007. Berlin, Germany: Springer, pp. 185–194.

Boerma, T., Chiat, S., Leseman, P., Timmermeister, M., Wijnen, F. and Blom, E. 2015. A quasi-universal nonword repetition task as a diagnostic tool for bilingual children learning Dutch as a second language. J Speech Lang Hear Res, 58, 1747–1460. DOI: https://doi.org/10.1044/2015_JSLHR-L-15-0058

Brannstrom, K.J., Rudner, M., Carlie, J., Sahlen, B., Gulz, A., Andersson, K. and Johansson, R. 2021. Listening effort and fatigue in native and nonnative primary school children. J Exp Child Psychol, 210, 105203. DOI: https://doi.org/10.1016/j.jecp.2021.105203

Carlie, J., Sahlen, B., Nirme, J., Andersson, K., Rudner, M., Johansson, R., Gulz, A. and Brannstrom, K. J. 2021. Development of an auditory passage comprehension task for Swedish primary school children of cultural and linguistic diversity. J Speech Lang Hear Res, 64, 3883–3893. DOI: https://doi.org/10.1044/2021_JSLHR-20-00611

Chiat, S. 2015. Non-word repetition. In: Armon Lotem, S. and De Jong, J. (eds.) Assessing Multilingual Children: Disentangling Bilingualism from Language Impairment. Bristol, UK: Multilingual Matters, pp. 125–150. DOI: https://doi.org/10.21832/9781783093137-008

Dockrell, J.E. and Marshall, C.R. 2015. Measurement issues: assessing language skills in young children. Child Adolesc Ment Health, 20, 116–125. DOI: https://doi.org/10.1111/camh.12072

Garau, M., Slater, M., Vinayagamoorthy, V., Brogni, A., Steed, A. and Sasse, M.A. 2003. The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 529–536. DOI: https://doi.org/10.1145/642611.642703

Helminen, T.M., Pasanen, T.P. and Hietanen, J.K. 2016. Learning under your gaze: the mediating role of affective arousal between perceived direct gaze and memory performance. Psychol Res, 80, 159–171. DOI: https://doi.org/10.1007/s00426-015-0649-x

Holler, J., Schubotz, L., Kelly, S., Hagoort, P., Schuetze, M. and Ozyurek, A. 2014. Social eye gaze modulates processing of speech and co-speech gesture. Cognition, 133, 692–697. DOI: https://doi.org/10.1016/j.cognition.2014.08.008

Jokinen, K., Nishida, M. and Yamamoto, S. 2010. On eye-gaze and turn-taking. In: Proceedings of the 2010 Workshop on Eye Gaze in Intelligent Human Machine Interaction. pp. 118–123. DOI: https://doi.org/10.1145/2002333.2002352

Developing a teacher agent to explore the role of gaze direction 31 https://doi.org/10.58986/al.2024.13972 Acta Logopaedica is published according to open access under a Creative Commons CC-BY 4.0 license

Lanthier, S.N., Zhu, M.J., Byun, C.S., Jarick, M. and Kingstone, A. 2022. The costs and benefits to memory when observing and experiencing live eye contact. Visual Cognition, 30, 70–84. DOI: https://doi.org/10.1080/13506285.2021.1926381

Lewis, J.P., Anjyo, K., Rhee, T., Zhang, M., Pighin, F.H. and Deng, Z. 2014. Practice and theory of blendshape facial models. Eurographics (State of Art Rep), 1, 2.

Mattys, S.L., Davis, M.H., Bradlow, A.R. and Scott, S.K. 2012. Speech recognition in adverse conditions: a review. Lang Cognitive Proc, 27, 953–978. DOI: https://doi.org/10.1080/01690965.2012.705006

Nirme, J., Haake, M., Lyberg Ahlander, V., Brannstrom, J. and Sahlen, B. 2019. A virtual speaker in noisy classroom conditions: supporting or disrupting children’s listening comprehension? Logopedics Phoniatrics Vocology, 44, 79–86. DOI: https://doi.org/10.1080/14015439.2018.1455894

Nirme, J., Sahlen, B., Ahlander, V.L., Brannstrom, J. and Haake, M. 2020. Audio-visual speech comprehension in noise with real and virtual speakers. Speech Commun, 116, 44–55. DOI: https://doi.org/10.1016/j.specom.2019.11.005

Novick, D.G., Hansen, B. and Ward, K. 1996. Coordinating turn-taking with gaze. In: Proceedings of the Fourth International Conference on Spoken Language Processing, pp. 1888–1891. DOI: https://doi.org/10.21437/ICSLP.1996-485

Otteson, J.P. and Otteson, C.R. 1980. Effect of teacher's gaze on children's story recall. Perceptual Motor Skills, 50(1), 35–42, 50. DOI: https://doi.org/10.2466/pms.1980.50.1.35

Rudner, M., Lyberg-Ahlander, V., Brannstrom, J., Nirme, J., Pichora-Fuller, M.K. and Sahlen, B. 2018. Listening comprehension and listening effort in the primary school classroom. Front Psychol, 9, 1193. DOI: https://doi.org/10.3389/fpsyg.2018.01193

Sherwood, J.V. 1987. Facilitative effects of gaze upon learning. Perceptual Motor Skills, 64, 1275–1278. DOI: https://doi.org/10.2466/pms.1987.64.3c.1275

Sumby, W.H. and Pollack, I. 1954. Visual contribution to speech intelligibility in noise. J Acoust Soc Am, 26, 212–215. DOI: https://doi.org/10.1121/1.1907309

Swerts, M. and Krahmer, E. 2008. Facial expression and prosodic prominence: effects of modality and facial area. J Phonet, 36, 219–238. DOI: https://doi.org/10.1016/j.wocn.2007.05.001

Downloads

Publicerad

2024-04-25

Referera så här

Brännström, J., Waechter, S., Nirme, J., Whitling, S., Lyberg-Åhlander, V., & Sahlén, B. (2024). Developing a teacher agent to explore the role of gaze direction on young school children’s comprehension. Acta Logopaedica, 1, 23–31. https://doi.org/10.58986/al.2024.13972

Nummer

Vol 1 (2024)

Sektion

Forskningsartiklar

Licens

Copyright (c) 2024 Jonas Brännström, Sebastian Waechter, Jens Nirme, Susanna Whitling, Viveka Lyberg-Åhlander, Birgitta Sahlén

Creative Commons-license

Det här verket är licensierat under en Creative Commons Erkännande 4.0 Internationell-licens.