Gold Penguin Logo with Text

New Study Finds Writing by ChatGPT Makes Complaints More Convincing

A new study shows that consumer complaints written with the assistance of ChatGPT are more likely to receive relief from financial firms, providing some of the first evidence that AI can improve persuasiveness in human communication.
The center of a grocery store where a man is complaining to automated robot, made with DALL-E 3
The center of a grocery store where a man is complaining to automated robot, made with DALL-E 3
December 5, 2023 1:06 am

A new study published in arXiv provides some of the first empirical evidence that large language models like ChatGPT can be used to improve the persuasiveness of messages in human communication. The research focused specifically on whether consumers were using ChatGPT to write more effective complaints to financial companies.

Analyzing a database of over 780,000 consumer complaints filed with the Consumer Financial Protection Bureau (CFPB), the authors found that the number of complaints likely written with assistance from AI surged after the release of ChatGPT in late 2022. Using an AI detection tool called Winston AI, they identified which complaints had a high probability of being written with ChatGPT.

The study then compared the linguistic features between complaints deemed as "Likely-AI" versus those by "Likely-Humans." Key findings showed Likely-AI complaints were more coherent and polite, conveyed more negative sentiment, and were less readable.

Financial firms were over 5% more likely to provide relief for Likely-AI complaints, even after controlling for the complaint sub-issues and linguistic qualities.

To test whether using ChatGPT could directly cause an increase in complaint persuasiveness, the researchers conducted an online experiment. They had participants evaluate financial complaints either in their original form or after being edited by ChatGPT to enhance qualities like clarity, coherence, and professionalism.

The results revealed that hypothetical compensation likelihood was higher for ChatGPT-edited complaints compared to original ones.

This points in the direction that complaints edited by ChatGPT are more effective in achieving compensation than complaints in their original, unedited form.

This early research highlights ChatGPT's ability to improve the persuasiveness of human communication in real-world settings like consumer finance. The authors say that large language models could have transformative impacts on communication outcomes ranging far beyond just writing more compelling complaints.

However, more studies are still needed to fully understand the breadth of these potential influences on human behavior.

Stay Updated On The Latest AI Tools
Get notified with the biggest releases and tips in Artificial Intelligence delivered straight to your email. We don't sell your data.
Written by Justin Gluska
Justin is the founder of Gold Penguin, a business technology blog that helps people start, grow, and scale their business using AI. The world is changing and he believes it's best to make use of the new technology that is starting to change the world. If it can help you make more money or save you time, he'll write about it!
Notify of

Inline Feedbacks
View all comments