en
Cloudbeds

Articles

How to Analyze Your Reviews for Context and Bias

Subscribe for More Content
Posted by Alex Gaggioli
March 21, 2016

Introduction

Recently, many people have been pretty cynical when writing about TripAdvisor reviews. Many have even said they are no longer trustworthy. Hoteliers can learn many things from online reviews about their property — even from those that are fake, angry, sarcastic, and unfair. Analyzing reviews for context and bias, beyond numbers, is important. Here, we will help you find the most effective ways to judge your online reviews.

Previously, we explained how trustworthy user-generated reviews, such as those found on TripAdvisor, really are. In our post, we discussed how and why user-generated reviews may not be the best place for travelers to research their trip. But, hoteliers should not take TripAdvisor’s quick response lightly. TripAdvisor points out that millions of people visit their site every single day, and that if traveler’s didn’t find value, they would go elsewhere.

TripAdvisor has a point. If millions of people are doing something, you should pay attention. Your property’s reviews may contain bias and guests may judge your property on an unfair scale. But, their prominent presence demands attention.

A Recent Cornell Study

A recent Cornell study took a look at over 50,000 reviews, according to Tnooz. Cornell’s goal was to categorize the different types of guest reviews by sentiment. For example, the most helpful reviews were the longer ones that only brought up a few points. The major takeaway: a review’s content is much more valuable to other guests than its point rating.

When Cornell analyzed the reviews, they determined the following general trends:

  1. If a hotel was of a higher tier, the guest’s experience was the most important feature of the review.
  2. Long reviews that focused on a just a few of the property’s attributes had lower ratings. In contrast, reviews that were short and took a big picture view of the property had higher ratings.
  3. In the highest rated hotels, 70% of reviews discussed the guest’s experience. In middle tier rated hotels, 45% of reviews mentioned the guest’s experience and in lowest rated properties, the guest experience was only mentioned 32% of the time. Patrons of top tier hotels are paying for luxury and attention, and are satisfied by an experience that meets these requirements. Patrons of lower tier hotels are more concerned with price.
  4. Negative reviews often focus on a few particular reasons why their experience was bad (i.e. noise, lacking a specific amenity, etc.).
  5. Reviews that give a higher rating are often short and to the point while not focusing on anything specific.
  6. Negative comments cause a steeper decrease in a review’s rating than positive reviews. The uneven scoring system means taking an average may not provide a clear representation of a guest’s opinion of the property.

Most hoteliers check their online reviews to get an idea of what people are saying. But often, hoteliers don’t analyze reviews in the correct context. There are two major ways hoteliers should use reviews:

  1. Identify their most positive attributes to better market themselves
  2. Identify their problem areas so they can implement changes

Number two is trickier than number one, because negative reviews are often biased.  

How to Make the Most of Your Reviews

Cornell’s new insights can help hoteliers more efficiently audit their reviews. The study advises you take a more macro approach. Hoteliers should analyze reviews in relation to the whole, rather than on an individual basis.For example, a lengthy review that only discusses two negative features of your property is helpful. But, it’s more helpful when compared to 100 other reviews.

When you compare reviews for trends, you gain a much more balanced point of view. For example, if only one review mentions noise, then that particular guest may just be sensitive. However, if 50% of your reviews mention noise levels, it may be a viable concern.

Based on Cornell’s research, we came up with some tips on how to make the most of your reviews.

Find Context

When you analyze reviews, try and gain as much context as possible. When we discussed TripAdvisor review bias in our previous blog post, the number one concern was that reviews lack real context. Everyone has their own expectations and therefore has bias. People make claims based on their situation and point of view. Empathize with their situation, good or bad, to pull better insights.

For example, if there is a lengthy negative review that focuses on noise level and a sub-par continental breakfast, find out a little bit more about the guest. Check their reservation, or see if they’ve left other reviews on the site (TripAdvisor, Yelp, etc). If you see their reservation was for two adults and two young kids, you will better understand why noise was an issue.

If you can find other reviews, see if your property was out of their normal comfort zone. If a guest usually stays in quiet bed and breakfasts, and you’re a downtown hotel above a bar, they may not be your ideal guest. Managing guest expectations is important. Make sure your website and booking profiles reflect your property’s true identity.

In the future, you can adjust by (1) managing expectations and (2) taking extra care when it comes to more sensitive parties, such as families.

Identify Biases

Most guest reviews are biased. Unless they are professional travel writers or avid bloggers, guests don’t know how to control their biases. Bias is a necessary evil in user-generated reviews and hoteliers need to recognize it when they see it. Cornell suggests taking a broader view of your reviews to gain the most accurate insights. If a long review only focuses on two facets of your business, it may or may not represent a real problem.

However, when you can compare one negative review to 100 other reviews, you paint a bigger picture. Your breakfast may not be an issue because only 8% of negative reviews mention it. But 76% of negative reviews mentioned noise, so you have some work to do there.

Understanding how context and bias work together will help you separate the real issues from one-time events.

False Reviews

Analyzing reviews on a macro scale will also help you identify outliers and fake reviews. If someone leaves a review discussing something that could not have happened (i.e. huge hotel party, loud wedding party, etc.). You can flag them for removal or publicly respond to them. Identifying reviews that weren’t left by real guests will help you deal with real problems and mitigate the online damage they cause.

As mentioned in our previous post, there are companies who exist solely to leave false reviews on TripAdvisor.  They leave positive reviews on their client’s profile and negative reviews on their competitor’s. In a fraudulent system, hoteliers need to be aware of incoming reviews to deal with any possible cases.

It’s much easier to identify false reviews when you understand the typical reviews your property gets.

Don’t Only Focus on Numbers

User-generated review rating systems are poor indicators of guest experience. Cornell states that negative comments influence a score more than a positive one. For example, a negative review may have a really low score. But, they’re unfairly weighting the negative attribute to the overall property experience.

While the numeric value influences your property’s rating on user-generated sites, you should try to look beyond it. Analyzing the context and bias of your reviews will prove much more effective than harping on a review’s rating.

Conclusion

Managing your online reviews is a tedious process. In a perfect world, reviews would objectively reflect a guest’s experience. But too often, a guests context and bias influences their review, muddling the real experience. Analyze your reviews from a macro scale and discover the overarching trends.