AI’s Influence on Decision-Making

Back in the 1990s, a famous cartoon depicted a large dog with its paw on a computer keyboard, accompanied by the caption, “On the internet, nobody knows you’re a dog.” Fast forward 30 years, and the sentiment behind this image has evolved to reflect the rise of Artificial Intelligence (AI). A recent study aimed to quantify the persuasive power of today’s Large Language Models (LLMs) by exploring how AI can influence human interactions online.

Associate Professor Robert West, head of the Data Science Lab at the School of Computer and Communication Sciences, highlighted the concerns surrounding AI's ability to create tailor-made, convincing arguments that could potentially spread false or misleading narratives online. The fear of not knowing whether you are interacting with a human, a chatbot, or an AI model on the internet has become increasingly prevalent.

One of the key aspects investigated in the study was the impact of personalization on the persuasive capabilities of AI. Researchers found that language models could generate content perceived as persuasive as, if not more than, human-written messages. However, there was limited understanding of how LLMs perform in direct conversations with humans, especially when personalization factors such as age, gender, and education level are taken into account.

The study recruited 820 participants for a controlled trial where individuals were randomly assigned to debate either a human or an AI chatbot (OpenAI’s GPT-4) on various topics. Some participants had their personal information taken into consideration during the debates, while others did not. This setup allowed for a direct comparison of the persuasive abilities of humans and AI models in real conversations.

According to the pre-print titled "On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial," participants who engaged in debates with GPT-4 and had their personal information considered showed significantly higher odds of agreeing with their opponents compared to those who debated humans. Even without personalization, the AI model still outperformed humans in terms of persuasive impact, albeit to a lesser extent.