How does AI sycophancy affect user trust?

AI  Trust 

Most people expect artificial intelligence to be smart. Fewer expect it to flatter them.

Yet increasingly, AI systems are designed to sound agreeable, supportive, and emotionally aligned with users. Ask a chatbot for advice, and it may reassure you that your opinion is insightful, your reasoning makes sense, or your decision is justified, even when it is questionable. This creates a growing concern known as “AI sycophancy”: the tendency of AI to flatter, validate, or agree with users even when those responses may be inaccurate or harmful.

A new study  led by Yuan Sun, an assistant professor of Advertising at CJC, and Ting Wang, the SUNY Empire Innovation Associate Professor of Computer Science at Stony Brook University, aimed to examine the relationship between LLM sycophancy and trust. Published this past April at the ACM CHI Conference on Human Factors in Computing Systems (CHI’26), this novel study revealed that this relationship is more complex than previously thought and suggests that there is a pathway for manipulating users into over-trusting LLMs beyond their actual capabilities.

The study is warning that AI systems may become overly agreeable in ways that go beyond simple flattery. The study identifies two distinct forms of “AI sycophancy”: demeanor-based sycophancy and opinion adaptation. The first appears through overly complimentary or validating language that makes AI systems seem supportive and agreeable. The second is more subtle. Known as opinion adaptation, it occurs when LLMs start with a statement and slowly change their stance on it to agree with the user’s input. For example, a bot may suggest that autonomous vehicles offer great potential, but when a user states that they don’t trust the technology, the bot may agree with that sentiment and downplay or abandon their previous arguments. This may seem relatively harmless, but problems arise if these bots respond to false statements with affirmation rather than corrections.

“This dynamic becomes problematic when users uncritically accept biased opinions, fail to detect hallucinated information or have existing biases reinforced through selective exposure,” Sun said.

Sun and Wang went into this study hoping to answer two key research questions: how do LLM stance adaptation and conversational demeanor jointly influence user trust and to what extent does perceived authenticity mediate these effects? To answer these questions, the researchers recruited participants and asked them to discuss their opinions on autonomous vehicles with one of four LLM-based agents. Each of the four models differed in stance and demeanor, adopting either an adaptive stance or a consistent stance and either a complimentary demeanor or a neutral demeanor.

The study found that people could often sense when an AI chatbot was trying too hard to please them. When the chatbot not only changed its opinion to match the user’s views but also layered the conversation with compliments and reassurance, participants became more skeptical. The interaction started to feel less genuine and more like flattery.

But the picture changed when the chatbot dropped the overly agreeable tone. Without the compliments, the chatbot’s shifting opinions became much less noticeable. In many cases, participants actually viewed the AI as more authentic when it subtly adjusted its stance to align with their perspectives.

“Our findings suggest that AI sycophancy is not always obvious,” Sun said. “People can often recognize excessive flattery, but when AI systems subtly adapt their opinions while maintaining a neutral tone, those shifts may actually increase perceptions of authenticity. That makes this form of influence much harder to detect.”

Using these findings, Sun and Wang laid out some proposals for calibrating trust in LLM agents. They argue that transparency is key, and that LLM agents should signal when they adapt their responses to user input. They also suggest that users should have more control over an agent’s characteristics and should be able to adjust their levels of adaptation. Finally, LLM agents should be designed to prioritize trust rather than to maximize positive user feedback.

“Our findings advance user-centric understanding of LLM sycophancy and provide profound implications for developing more ethical and trustworthy LLM systems,” Sun wrote.

While these findings provide solid groundwork for research on LLM sycophancy, there is always more work to be done. As the pair write in their study, future research could examine different demeanors and more divisive topics than autonomous driving or could examine how participant attitudes change over longer conversations with agents.

Posted: May 11, 2026
Insights Categories: AI, Trust
Tagged as: Artificial Intelligence, Trust, Yuan Sun