Rothbart’s Rant #41 - “Rating Reviews”

October 24th, 2007 · 5 Comments

BPlusWe’ve all seen it before. A hot new game comes out and one review gives it a 10/10 while another review gives it a 7/10. I personally have a system I’ve never really taken the time to fully describe that I use when it comes to looking at reviews. I generally toss out the highest (and lowest) review scores and take a look at what’s left. Sometimes that step needs to be done multiple times until I start to home in on a small enough range of review scores to call it a consensus. Metacritic.com is a great tool for doing this.

I’m personally very excited for for Ratchet & Clank Future and was relieved to see the reviews start coming in; a few low-to-mid 90’s, a couple 100’s, then an 85. 1UP gave the game an 85. So I read their review and couldn’t really fault them for what they said, there was nothing outlandish. Everything was fine, I just accepted their score as one of many opinions. Not to be outdone, GameSpot gave the game a 7.5/10. GamePro (yeah, I know, don’t judge me I’m just listing some places we’ve all heard of), PSM, IGN, PSM3, Game Informer, Gaming Age, and GameTrailers all rated the game a 90 or higher (by Metacritic’s rating conversions). How does one decide to draw the line and ignore the ratings you’re not comfortable with?I’m of the school that you either play loose with your reviews and not take them too seriously or you take them very seriously and (likely) never give out 100’s or 10/10’s. I don’t have a problem with a 5 point scale (that usually includes half points). This lack of granularity helps hide some of the nitpickiness that will draw fire when people don’t agree with your reviews, or worse yet, you’re stuck arbitrarily defending them. In the example above, PSM gave R&CF a 95, but IGN gave it a 94. You can never “prove” one correct or incorrect. But if you were to round both of those reviews to a 5 point scale (including halves — which in reality makes it a 10 point scale), you’d see one be a 4.5 and the other be a 5.0 and there would be much less to talk about (in my opinion less to argue about).

So I ask you, what is your personal method for looking at game reviews? Yes, I know we all probably have a close group of friends that we value their input, but for me personally, my friends play genres I don’t and vice versa, so that’s not always an option. Do you also subscribe to the toss the highest and lowest scores until they narrow down to a pretty small spread? I’d be curious to hear your thoughts (as well as if you think reviewers should be generous with or extremely stingy with perfect scores). Actually, while we’re on that subject, should review scores change over time? Certainly comparing a 10/10 game from the SNES era won’t have the same impact on some people as say a 10/10 game review for Halo 3. Should the larger organizations periodically go back and “adjust” their reviews or is it just our responsibility to realize that it’s pointless to compare Halo 3 to a SNES game?



Related Posts:
  • Rothbart’s Rant #45 - “Turn It To Eleven”
  • Rothbart’s Rant #24 - "Parental Expectations" Edition
  • Rothbart’s Rant #26 - "Microsoft, I Hate You Back!" Edition
  • Previously on Sarcastic Gamer - July 8, 2008
  • Rothbart’s Rant #21 - "Exclusively Non-exclusive" Edition
  • Tags: Rothbart's Rant

    5 responses so far ↓

    • 1 spekulyst // Oct 24, 2007 at 1:53 pm

      I am pretty much the opposite. I read the highest and the lowest reviews…With this method I get to know all the absolutely fantastic features and the ones that suck…From there I can sift through and figure out which things on that list I actually care about(example: sometimes the story sucks…but you know before you play that you will skip through the cut scenes anyway).

    • 2 Nicholie // Oct 24, 2007 at 3:17 pm

      I’ve always tried to keep an eye on exactly WHO was reviewing the game, not paying much attention as to who it comes from.

      Its better to find a reviewer that your more in tune with, than a magazine. For example; Luke Smith was my go-to guy when he worked for 1up, I knew I could trust pretty much what he said to be a very mutual feeling.

    • 3 Sean "rothbart" Workman // Oct 24, 2007 at 4:41 pm

      When you look at the 1UP staffer that reviewed R&CF though, his favorite games are listed as: PQ2: Practical Intelligence Quotient, Cooking Mama: Cook-Off, Carcassonne. I wonder if it was best to put someone like that to review a platformer? It at least made me think for a minute. I’m not saying you have to pick your most fanboyish reviewer based on the genre or franchise, but I could do a decent review of an RPG, RTS, or MMO because I don’t play or enjoy those games. I could review the best of all those genres and they wouldn’t get 10’s in MY book…

    • 4 sinisterff // Oct 25, 2007 at 1:00 am

      I think the reviews that are more exact. The ones that are in 100 or 10 with decimals, just my opinion tough.

      I agree with you in metacritic is a great site for this. I read some of the reviews, and see what points they have in common, and asume those are the pros and cons. But I don’t let influence it in my decision, if I’m curious about a game I’ll buy it or rent it to see.

      “my friends play genres I don’t and vice versa”
      same here.

      Finally, I really think some site really suck when assigning reviewers to games. I would find a guy like me (in gaming tastes) and read his/her reviews,to ensure I’m getting an opinion similar to mine.

    • 5 sinisterff // Oct 25, 2007 at 1:01 am

      In the first paragraph of the previous comment replace think with like, my bad.

    Leave a Comment