Review scores are completely meaningless to me, and pretty much always have been.
Part of that is just that I have always been a bit "weird" (I don't know the right word). I've never been one of the crowd, part of the masses, and I've never participated in pop culture. So, it's always been highly unlikely that I share opinions with the average gamer, and thus review scores aren't targeted at me.
Part of that is I have a reasonably good understanding of statistics. An average game should get a score of 5.0, but in reality the average game gets a score of about 7.5. This undermines all review scores.
Part of that is that there is no standardisation, nor much of a breakdown of how the scores are calculated.
For example, does 10/10 mean the game is perfect? Or does it mean it's the best game ever? Or just the best game in that genre?
What I'd love to see is review scores broken down into standardised parts so that I can pick out the bits that are actually important to me. E.g.
Graphics Quality
Performace
Story / theme
Duration
Gameplay Quality / complexity / depth
Meta-Game
Then get into genre-specific things, like scale of multiplayer (for MMOs), amount of roleplaying, importance / depth to the roleplaying. Creativity for builders. Crispness of the action in an action combat game etc.
An interesting question I have about doing reviews professionally is how you deal with reviews for different genres? Final Fantasy is a good one to think about. These are high quality games, huge fanbase, generally an iterative design from game to game. But they are listed as Roleplaying Games......but they tend to have virtually no roleplaying in them!
So, if you are reviewing a FF game as an RPG, you'd have to give it a low score, because the roleplaying portion of the game sucks. But if you look at it from a wider gaming POV, then you can give it a higher score.
Same sort of thing with the Diablo games. They are marketed as Action RPGs......but they have hardly any roleplaying and the action is simplistic and undermined by the meta-game. So, as an ARPG, Diablo sucks hard. But as a generic game? Obviously has a lot of appeal.
Currently Playing: WAR RoR - Spitt rr7X Black Orc | Scrotling rr6X Squig Herder | Scabrous rr4X Shaman
Review scores are completely meaningless to me, and pretty much always have been.
Part of that is just that I have always been a bit "weird" (I don't know the right word). I've never been one of the crowd, part of the masses, and I've never participated in pop culture. So, it's always been highly unlikely that I share opinions with the average gamer, and thus review scores aren't targeted at me.
Part of that is I have a reasonably good understanding of statistics. An average game should get a score of 5.0, but in reality the average game gets a score of about 7.5. This undermines all review scores.
Part of that is that there is no standardisation, nor much of a breakdown of how the scores are calculated.
For example, does 10/10 mean the game is perfect? Or does it mean it's the best game ever? Or just the best game in that genre?
What I'd love to see is review scores broken down into standardised parts so that I can pick out the bits that are actually important to me. E.g.
Graphics Quality
Performace
Story / theme
Duration
Gameplay Quality / complexity / depth
Meta-Game
Then get into genre-specific things, like scale of multiplayer (for MMOs), amount of roleplaying, importance / depth to the roleplaying. Creativity for builders. Crispness of the action in an action combat game etc.
An interesting question I have about doing reviews professionally is how you deal with reviews for different genres? Final Fantasy is a good one to think about. These are high quality games, huge fanbase, generally an iterative design from game to game. But they are listed as Roleplaying Games......but they tend to have virtually no roleplaying in them!
So, if you are reviewing a FF game as an RPG, you'd have to give it a low score, because the roleplaying portion of the game sucks. But if you look at it from a wider gaming POV, then you can give it a higher score.
Same sort of thing with the Diablo games. They are marketed as Action RPGs......but they have hardly any roleplaying and the action is simplistic and undermined by the meta-game. So, as an ARPG, Diablo sucks hard. But as a generic game? Obviously has a lot of appeal.
I’m at TwitchCon right now, but just saving this because I’d love to respond more fully tonight when I’m home.
Part of that is I have a reasonably good understanding of statistics. An average game should get a score of 5.0, but in reality the average game gets a score of about 7.5. This undermines all review scores.
Yes and you can't score anything as perfect as a 10 as far as I am concerned but most would not agree. These problems are found in awards the world over in what I think of as "score inflation" and "award creep". For scores the only direction is ever higher, for awards the only direction is more and more awards. You find this in gaming scores, noble prizes, the Olympics, education exam grades, the prize for "best cabbage" at you local fete. It is a problem every organization faces no matter how small or what they are awarding.
Score inflation comes from the pressure to award higher scores originating from the industry and its advocates. In gaming it is a commercial pressure from the industry, in education that pressure comes from the government, parents and schools wanting higher grades.
Creating new awards occurs for three reasons, firstly in that year there are a few titles the organization feels must get an award but only one title can get Best "whatever". So they make up a new category to ensure an award is given, as we see most typically in gaming awards and other entertainment awards. Once you have the new category you can't put it back in the bottle. Secondly, for political reasons like we see in the noble prizes; finally to simply become a bigger event like we see in the Olympics.
There are systems to stop this happening, for example in education we used to have a system where only a certain percentage each year would earn an "A" and so on, no matter how high the scores only that percentage would get an "A". Now we have one where you only have to get a certain score to get an "A", this has resulted in huge grade inflation.
For gaming you could not do this, but what you could do is to standardize the system as much as possible across the industry. Which is what Joseph decided to do in house. The problem is once you get other organizations all trying to standardize together you will have a quagmire of issues. And you could never fully address the central problem of a review, that it is a personnel take on a game. But still it would help if there was more standardization, look at how varied those reviews on Metacritic are.
For the player there is a solution, look at the below par reviews the ones that say the game is not that good. Here you will find the flaws that may mean it is a no go for your purchase. These will differ from player to player, my main ones are "out of step" humour, survival gameplay where having to survive makes you feel like it is stopping you doing anything else, games which are good as co-op but poor solo, P2W and a few others. High scoring reviews do not tend to highlight problems so they don't tend to show you anything that may be a show stopper for you. So I start by a scan of a few lower scoring reviews looking for problems, if that is OK then on to a full read of a higher score review from a reviewer I trust.
Reviews are just opinions. Opinions are like assholes, everyone has one.
Stop listening to reviews. Form your own opinion on things. Quit being followers.
Except that....life is too short to waste a moment playing bad games.
It is impractical, if not impossible to play every game being made these days so reviews (opinions) of like minded people can be quite helpful in determining which ones to try.
Of course it's a matter of degree and it's usually a good idea to not rely on one opinion but rather take an aggregate or average across a broader spectrum, at least before digging in to the details.
I won't bother much if a game is getting 6's and 7's across the board but if the overall is 8 to 10, probably worth at least a look.
I have no problem forming my own opinions, my issue is getting everyone else to agree with mine, for some reason most people hold totally different views from me.
I just assume they enjoy being wrong most of the time.
Just trying to live long enough to play a new, released MMORPG, playing New Worlds atm
Fools find no pleasure in understanding but delight in airing their own opinions. Pvbs 18:2, NIV
Don't just play games, inhabit virtual worlds™
"This is the most intelligent, well qualified and articulate response to a post I have ever seen on these forums. It's a shame most people here won't have the attention span to read past the second line." - Anon
Just trying to live long enough to play a new, released MMORPG, playing New Worlds atm
Fools find no pleasure in understanding but delight in airing their own opinions. Pvbs 18:2, NIV
Don't just play games, inhabit virtual worlds™
"This is the most intelligent, well qualified and articulate response to a post I have ever seen on these forums. It's a shame most people here won't have the attention span to read past the second line." - Anon
Review scores are a scourge and anathema to me and the hobby. I loathe them to my core.
My guess is you weren't a big fan of essay tests in school either.
I mean how can a professor objectively assign a letter grade to what their students write?
Oh, maybe based on certain criteria they are looking for in the answer.
Hmmm?
I think some gamers see the problems with scores but they don't think through the problems we would have without scores. We would be in a worse position without scores, have a think about it.
Comments
Score inflation comes from the pressure to award higher scores originating from the industry and its advocates. In gaming it is a commercial pressure from the industry, in education that pressure comes from the government, parents and schools wanting higher grades.
Creating new awards occurs for three reasons, firstly in that year there are a few titles the organization feels must get an award but only one title can get Best "whatever". So they make up a new category to ensure an award is given, as we see most typically in gaming awards and other entertainment awards. Once you have the new category you can't put it back in the bottle. Secondly, for political reasons like we see in the noble prizes; finally to simply become a bigger event like we see in the Olympics.
There are systems to stop this happening, for example in education we used to have a system where only a certain percentage each year would earn an "A" and so on, no matter how high the scores only that percentage would get an "A". Now we have one where you only have to get a certain score to get an "A", this has resulted in huge grade inflation.
For gaming you could not do this, but what you could do is to standardize the system as much as possible across the industry. Which is what Joseph decided to do in house. The problem is once you get other organizations all trying to standardize together you will have a quagmire of issues. And you could never fully address the central problem of a review, that it is a personnel take on a game. But still it would help if there was more standardization, look at how varied those reviews on Metacritic are.
For the player there is a solution, look at the below par reviews the ones that say the game is not that good. Here you will find the flaws that may mean it is a no go for your purchase. These will differ from player to player, my main ones are "out of step" humour, survival gameplay where having to survive makes you feel like it is stopping you doing anything else, games which are good as co-op but poor solo, P2W and a few others. High scoring reviews do not tend to highlight problems so they don't tend to show you anything that may be a show stopper for you. So I start by a scan of a few lower scoring reviews looking for problems, if that is OK then on to a full read of a higher score review from a reviewer I trust.
It is impractical, if not impossible to play every game being made these days so reviews (opinions) of like minded people can be quite helpful in determining which ones to try.
Of course it's a matter of degree and it's usually a good idea to not rely on one opinion but rather take an aggregate or average across a broader spectrum, at least before digging in to the details.
I won't bother much if a game is getting 6's and 7's across the board but if the overall is 8 to 10, probably worth at least a look.
I have no problem forming my own opinions, my issue is getting everyone else to agree with mine, for some reason most people hold totally different views from me.
I just assume they enjoy being wrong most of the time.
"True friends stab you in the front." | Oscar Wilde
"I need to finish" - Christian Wolff: The Accountant
Just trying to live long enough to play a new, released MMORPG, playing New Worlds atm
Fools find no pleasure in understanding but delight in airing their own opinions. Pvbs 18:2, NIV
Don't just play games, inhabit virtual worlds™
"This is the most intelligent, well qualified and articulate response to a post I have ever seen on these forums. It's a shame most people here won't have the attention span to read past the second line." - Anon
I mean how can a professor objectively assign a letter grade to what their students write?
Oh, maybe based on certain criteria they are looking for in the answer.
Hmmm?
"True friends stab you in the front." | Oscar Wilde
"I need to finish" - Christian Wolff: The Accountant
Just trying to live long enough to play a new, released MMORPG, playing New Worlds atm
Fools find no pleasure in understanding but delight in airing their own opinions. Pvbs 18:2, NIV
Don't just play games, inhabit virtual worlds™
"This is the most intelligent, well qualified and articulate response to a post I have ever seen on these forums. It's a shame most people here won't have the attention span to read past the second line." - Anon