The Anatomy of Online Reviews: Evidence from the Steam Store

Online reviews are now a ubiquitous part of the modern consumer's information set. But how good are reviews as signals of product quality for consumers? Using a dataset derived from the popular `Steam' gaming platform I investigate two linked issues in the generation of reviews: the self-selection of reviewers and the `priming' of quality judgements as based on pre-existing consumer assessments. A policy reform on Steam in 2019 both lowered the transaction cost of reviewing and changed the average level of exposure to pre-existing consumer quality ratings, and this randomly occured within a game and reviewer's life cycle. I find that the new individuals elicited to review by the policy change are 4% more likely to rate any game positively, leave 20% shorter reviews and are less experienced both within and across games. In addition, reducing the exposure of a reviewer to a product's average rating leads to a substantive drop in the dependency of their review on such a rating. This effect is asymmetric, with worse-rated games receiving more positive reviews, suggesting that negative opinions can compound to bias ratings downward. Overall, the policy led to a game's rating better reflecting its popularity with players, but these new reviewers were rated as less helpful by their peers, implying that more accurate review scores come at the cost of less helpful reviews.