Artificial Intelligence - Partner in research, or the fifth wheel

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

This article is not in any list yet, why not save it to one of your lists.
Log in to save this article

Abstract

In social research qualitative analysis of interviews by verbatim coding is a quite common, yet a labour-intensive task. Artificial Intelligence, e.g. ChatGPT, may be helpful here.Previous research showed that qualitative researchers tend to self-monitor, especially when expertise is required. We also found that the application of Artificial Intelligence in qualitative analysis is unknown. The latter is reinforced by doubts on consistently trustworthy responses from the Artificial Intelligence systems studied. On trustworthiness we found that transparency and robustness are the two relevant components to address on this issue. As humans have, Artificial Intelligence may be found biased perhaps introduced by ‘previous experiences’; data used to train the Artificial Intelligence systems.This paper explores the performance in qualitative analysis of interviews of four generally available and interesting contemporary Artificial Intelligence systems: 1) Bard (Google), 2) Bing-AI (Microsoft), 3) ChatGPT (OpenAI), and 4) WriteSonic (WriteSonic). Therefore, we used one set of quotations to assess the summarizing performance of these four Artificial Intelligence systems.Over the four assess Generative AI systems we noticed wide variety on performance with respect to trustworthiness as well as on bias. Our research shows that ChatGPT may be a useful sparring partner in summarizing quotations emerging from interview analysis. Responses returned by ChatGPT showed a better performance on trustworthiness and a recognizable bias. In general, a supportive role could be seen for Generative AI as a co-pilot in coding and qualitative analysis. Whereas the response time in ample seconds, may be considered as a significant advantage. To conclude our study, we may argue that the variation in quality of ChatGPT responses, the use of ChatGPT can merely be considered as inspirational. The use of con-textually rich words in sentence structure also made us aware of a pitfall as the change of meaning of the sentence may occur.

Article activity feed