Examples Of Biases in Customer And User Reviews

Here at buywise.com, we are consumers too. We often purchase a product that has a five star customer review, get the product home, and then discover the product is really bad. As professional product designers and usability experts, we have the tools and processes to evaluate most products and discover why a product fails to meet the user’s needs. For a detailed discussion see Product Testing (Testing Methods).

Buywise.com has also put together a list of issues based on media articles, see Top 4 Reasons Most Customer and User Reviews Are Not Reliable.

Documenting Problems With Customer Reviews

We use this page to document our real world examples of problems with customer reviews as they occur. Check back as we’ll keep it updated.

Some Sites Only Publish The Positive Reviews

YLighting Customer Reviews - October 8, 2008

Our search was to find a new desk lamp that was functional and beautiful. We found a desk lamp that had two customer reviews and both were 4 stars, shown here:

We purchased the 4-star rated product, the “Berenice Table Lamp (Small)” from YLighting for $320.00, + $57.60 for the blue shade, on September 4, 2008. The product arrived and we assembled it. 

 

Due to the purchase, on October 4, we were invited by email to write a review on the product. 

 

 

Here’s what we wrote:

Rating: 1-star

Headline: Good aesthetics, Horrible functionality.

Full Review: The photo of the product is great. The lamp looks sleek, has a glass dome, and nice light output.

But, the functionality and usability is horrible. The customer has to assemble the product and that is hard! To assemble the cover, the user has to deform a plastic ring and snap everything into place. The instructions are telling because there are several warnings about being careful to avoid breaking parts… crappy design.

The plastic ring looks cheap. There are plastic tabs hanging down where the mold wasn’t tight. I’ll need to go buy some sand paper and remove the burs.

Finally, the switch is right by the hot light bulb, instead of down by the base (like most desk lamps). So, you have to watch out to make sure you don’t burn your hand when trying to turn off the light.

If I had not already thrown away the packaging, I would return the light. Oh well.

The review was submitted on October 4. We got the $25 gift certificate on October 5. However, the review was never published on the YLighting web site.

This is another example where companies are only publishing the positive reviews. This unfairly biases the customer reviews to indicate a higher customer rating. That’s not fair!

Last edited by on October 8, 2008 at 10:55 am