We live in an age of algorithms in which the technologies that we interact with are gaining in intelligence through neural nets, machine learning techniques, and the use of different statistical procedures. In this paper I take a broad view of intelligence and propose that it reflects a machine or software’s ability to perform tasks that are typically associated with human intelligence. One type of intelligent technology that is entering society is social robots which may operate using the latest techniques of artificial intelligence (AI). For example, using computer vision robots can navigate the environment and avoid obstacles (Russell & Norvig, 2020), and using machine learning algorithms robots can detect a person’s face, and to some extent, their emotional state (Liu et al., 2017). Robots also have the ability to communicate with users through text or spoken language (Edwards et al., 2019). Surprisingly, with the increased use of algorithms creating smart technologies, there has been relatively little research to determine the effectiveness of AI-driven techniques from a usability perspective. To address this gap in the literature this paper explored how people evaluated different techniques of AI when used to guide the behaviour of a social robot.

As background, past studies have shown that different AI abilities controlling a robot’s performance can influence the user’s perception of the robot and its problem-solving ability (Flores-Fuentes et al., 2014; McKee, 2003). For example, Dou et al. (2021) found that the use of a robot equipped with natural language processing was effective for shopping, education, and as a home companion. Further, Makibuchi et al. (2010) showed that an algorithmic-based problem-solving approach combined with a humanoid robot was effective for solving problems encountered in a real-world setting. In a different application, Chen et al. (2021) investigated the effectiveness of AI for customer experiences when chatbots were used. Using a quantitative approach, they found that the usability of the chatbot had a positive influence on extrinsic values of customer experience, whereas the responsiveness of the chatbot had a positive impact on intrinsic values of customer experience.

Evaluating another AI technique, Roundtree and Moallem (2021) investigated the use of facial recognition ability for young adult and adolescent populations. In a review of the literature they concluded that facial recognition raised important questions about the clinical and ethical applications of the technology. They also noted that prior studies had not adequately addressed the complexities of facial change over time and for different ethnicities thus influencing the accuracy of facial recognition software (Roundtree & Moallem 2021). They concluded that while facial recognition is a promising technology, usability studies were mostly lacking for facial recognition applications. For a different application, Riaz et al. (2022) looked at the use of facial recognition as a biometric platform and particularly in the context of employment in a real-world setting. Specifically, they evaluated the usability and vulnerability of biometric technologies implemented in the UAE public transportation system to boost security and public service delivery. They noted that usage of facial recognition technology had raised numerous concerns about biometric data security and privacy. Based on participants responses collected using a survey instrument they concluded that public transport users had a poor impression of facial recognition technology compared to iris recognition and fingerprints authentication.

Considering the usability of robots, in addition to the use of AI to control the behavior of robots, the physical design of the robot has been shown to influence user evaluations of the robot. For example, using the shooter bias paradigm, Bartneck et al. (2018) found that people would shoot faster at a robot that had a darker surface colour than a lighter colour robot. Additionally, Sparrow (2020) commented that people racialized a robot based on its design, and particularly surface colour. Barfield (2021) also found that a robot’s surface colour influenced the task that robots would be selected to perform with lighter-coloured robots selected for tasks which required problem solving ability and darker-coloured robots selected for tasks which were more labor intensive. From the above brief discussion, missing in the literature are studies which looked at features of a robot’s design in the context of evaluating AI techniques guiding the robot’s performance. This led to the following question addressed in the current study- would the sur-face colour of a robot, thought by Sparrow (2020) to racialize a robot, influence its evaluation when equipped with different techniques of AI? More specifically, based on the above discussion of the literature, in this study the primary objective was to evaluate an individual’s perception of robot intelligence as a function of different AI techniques controlling the robot. To reiterate, the main interest was to determine if robot surface colour influenced the evaluation of robot intelligence when combined with different AI abilities. The research is guided by the following research questions.

The following is a preliminary analysis of the data as further analysis are being performed. Participants were asked to rate robot intelligence using 1-7 Likert items, from this an ANOVA was performed on robot surface colour crossed with the different types of AI enablement presented in the four narratives.

For judgment of robot intelligence, the ANOVA procedure indicated that the main effect for robot surface colour was not statistically significant (p > .05). Thus, there was no significant difference among participants in the perception of robot intelligence as a function of whether the participant viewed a black or white-coloured robot. However, the main effect for the different AI enablement narratives was highly significant (p < .001). The Tukey HSD multiple comparison test indicated that voice enablement resulted in the perception of a more intelligent robot than a robot that communicated with text (p < .01). Further, facial recognition ability led to the perception of more perceived intelligence than text communication (p < .01), and similarly, the ability to detect the user’s emotions was evaluated as more useful for judging robot intelligence than the ability to communicate with text (p < .01). Interestingly, the two-way interaction between AI enablement and robot surface colour was not statistically significant (p > .05); indicating that the black or white colour of the robot did not affect the evaluation of the robot’s intelligence when the different AI techniques were considered.

Another question asked how intelligent robots were thought to be (with black and white robots combined in one group) when considering the AI techniques of voice communication, facial, and emotion recognition. A one-way ANOVA with voice, facial recognition, and emotion recognition abilities was not statistically significant (p > .05). More broadly, Figure 2 shows a graphical depiction of the participant’s evaluation of robot intelligence as a function of voice or text communication, and facial and emotion recognition ability. Supporting the results of the multiple comparison test presented earlier, the figure shows that text communication was rated lowest in the evaluation of robot intelligence.

Overall, the results indicated that emotion detection ability, facial recognition, and natural language ability increased the ratings of robot intelligence comparted to text communication. However, it is interesting to note that these AI techniques were not judged to be significantly different from each other when participants evaluated robot intelligence. But consistently, participants evaluated a robot that communicated with text as being less intelligent. It was also shown that colorising a robot as black or white did not affect the evaluation of robot intelligence. In fact, across the different types of AI enablement, the evaluation of robot intelligence was remarkedly similar for white and black-coloured robots (Figure 3). old heading, normal text.

As a discussion point, I should note that these results are based on the viewing of a static robot image and not based on interacting with a robot to perform a task; for that reason, I postulate that the type of task performed could influence the participants rating of robot intelligence and will be the topic of a future study (Ciocirlan et al 2019). As another point to emphasize, the current study should be considered exploratory in that this study reflects a beginning effort by the author to investigate a relatively unexplored topic: whether different techniques of AI influence the user’s evaluation of a social entity. In the current study, the use of voice was a strong cue to increase the perception of robot intelligence and with large language models beginning to be used with social robots, going forward, it is likely that robots will be perceived as an even more intelligent entity during social interactions. Thus, voice communication seems to be a factor in usability for social robots which leads to another point- the perception of intelligence for an artificial entity could be an important factor to consider when evaluating the usability of an AI-equipped entity. From this observation perhaps a robot intelligence scale should be developed and used in usability studies. It was also an interesting finding that robot surface colour had no effect on the perception of robot intelligence. If robots are racialized as indicated by Sparrow (2020a) it is possible that racial stereotypes may not be a factor in the evaluation of robot intelligence. This conclusion warrants further exploration. Finally, given the viewing of a static robot image, facial and emotion recognition, robot mobility, and natural language ability were not different in user evaluations of robot intelligence. Whether the results presented here are replicated in future studies using different robots and tasks, and operating with more interactivity between user and robot represents a future direction of my research. old heading, normal text.