What if conversational agents’ answers appear correct but are judgmental? Piloting the potential effect on people’s experience

With this study we examined how fairness in chatbot responses affects user experience and willingness to continue using the chatbot. Methods: A two-phase study was conducted. A pre-experimental phase involved 10 people in designing and 30 participants in testing a set of fair and unfair chatbot responses related to a fictional master’s program. Six pairs of highly discriminable fair and unfair answers to a set of six questions about the master’s were included for the experiment. The experimental phase involved 75 participants who interacted with chatbots featuring different appearances (male, female, neutral) and programmed to provide responses to the questions about the master’s program with varying levels of fairness, namely: completely unfair, completely fair, or partially fair (i.e., 50% fair and unfair answers). User experience (UX) was assessed before and after the interaction, focusing on satisfaction, trust, competence, and helpfulness. The intention to use was measured with the Net Promoter Score. Results: Fairness significantly influenced users’ perception of the chatbot’s fairness. Fairness strongly impacted experience, F(2, 72) = 14.47; p < .001, which significantly affected intention to use, F(3, 71) = 49.2; p < .001. Chatbot fairness did not directly affect usage intention. Appearance and prior experience did not significantly influence UX or usage intention. Conclusion: Fairness plays a crucial role in shaping UX, influencing chatbot usage. Unfair elements in responses of chatbots can negatively impact users’ decision to use, emphasizing the importance of chatbots’ unbiased communication. Nevertheless, people show a certain level of acceptability of judgmental statements if there are no direct consequences of unfairness.