What is Agreement Bias?
Agreement bias is the tendency to give ‘positive’ answers and agree with statements regardless of their content. This can lead to a skew towards agreement on answer scales such as this:
Example of agreement answer scale
You may have encountered this bias in your daily life. For example, imagine you’re trying to find the hottest new place to eat out at the weekend. On TripAdvisor, you find a quaint French restaurant with plenty of reviews and an average rating of 3.5. That’s a whole star above 2.5, which must surely be the average as it is the mid-point – this place must be great!
In your naïve joy, you announce that you have likely found Manchester’s next Michelin star contender. But then, as you sit, staring at your expensive selection of uninspiring and tasteless small plates, you realize you have made a terrible mistake – you have forgotten the agreement bias. In fact, the average score for hospitality businesses on TripAdvisor is much higher than the midpoint of 2.5, at 4.12 out of 5. While there are multiple causes of positive skews on sites like TripAdvisor, agreement bias is undoubtedly at play here.
3 Causes of Agreement Bias
Social Conditioning
One of the main suggested causes of agreement bias is social conditioning. Over time, we have been conditioned by social norms to be polite and agreeable, as this encourages social cohesion. As a result, we feel much less cognitive strain when giving positive responses as opposed to ‘disagreeable’ ones. Think about how often in daily conversation we ‘agree’ to avoid causing conflict, insult, and misunderstanding.
Survey Fatigue
The tendency to agree because of social norms can be exacerbated by ‘survey fatigue’. Completing surveys requires respondents to use cognitive energy when assessing questions and selecting answers. Often, as respondents move through surveys, they become increasingly tired of expending this mental effort and therefore put less thought into the answers they give. As a result, they are much more likely to default to positive answers, which require less energy because this usually requires less critical analysis and aligns with aforementioned social norms.
Desire to Conform
Some suggest that the impetus to be ‘agreeable’ is heightened by the desire to conform to the opinions of higher authority individuals – in this case, the researcher or interviewer. This can be particularly pronounced when an interviewer is physically present at the time a survey is taken but can also be caused through poor question design, which might subtly imply a preferred or correct answer.
For example, the question, “How much would implementing this new policy enhance team productivity?” implicitly assumes that the respondent agrees that the new policy would enhance productivity at all, further increasing the likelihood of answers in agreement.
How to Avoid Agreement Bias
There are several ways to combat agreement bias:
Balanced Question Design
Ensure that any questions with answer scales are balanced and use equivalent terminology. By having positive and negative answers of equal number and ratio, respondents are less likely to be swayed in any particular direction.
Example of an unbalanced scale
Benchmarking
Other methods attempt to correct for agreement bias after data has been collected. A great way to do this is through benchmarking. Knowing that the average TripAdvisor score is 4.12, we can use this as a benchmark to compare other restaurants against. Through our Superpowers Index, we at B2Bi can derive average scores for a multitude of common satisfaction metrics by industry and country, enabling performance comparisons while accounting for agreement bias.
Net Promoter Scores (NPS)
Other common metrics account for agreement bias in their design. For example, Net Promoter Scores (NPS) ask how likely customers are to recommend a product or service to others on a scale from 0 to 10. NPS mitigates against agreement bias by setting a high bar for positive sentiment. Only answers in the 9-10 range are counted as positive sentiment, and anyone who gives a score below 9 is deemed to be neutral or disloyal. This approach ensures the score reflects genuine enthusiasm, excluding moderate ratings (like 7 or 8) that might stem from agreement bias.
Summary
While agreement bias can skew research towards positive results, benchmarking and accounting for the bias when interpreting results helps mitigate its effects, preserving the research’s integrity and validity.
Readers of this article also viewed:
B2B Insights Podcast #62: How to Achieve World-Class Quality in B2B Market Research Addressing the Problem with Data Quality in B2B Research How to Ensure Representative Samples in Customer Research How To Overcome The Challenge Of Cultural Bias With Multi-Country B2B Research
To discuss how our tailored insights programs can help solve your specific business challenges, get in touch and one of the team will be happy to help.