Honestly i can't complain much about the reviews on MMORPG.COM , seen worst but not better. I am not saying that there is not room for improvement. On some ocasions i find the revies a bit subjective , and if we are talking about some form of jurnalism , that would imply that they should be done on a more objective style. This way the reader can decide for himself what letter or number to attribute to the game.
As for grading games , it's not an all that bad idea , i just hope people will still read the reviews and not only look at numbers, but they are not real fans of the genre if they do that anyway.
One neat idea i thought of, or so it think it's neat, is to have 2 ratings , one given by MMORPG.COM and one by the user. The way i see it only registered users can see the ratings and only after they rated a game atleast once. And a user should be able to change his rating choice on a game. I can give you my latest example with Rift , at some point i wouldn't rate the game more than 7/10 , but now after a month of playing i could go up as far as a 9/10
First of all it's really great to see honest reads like this once in a while, and i really appreciate them + it explains things at "our level" not an editorial level.
If u don't mind i have a few ideeas that i think might be of use related to the scoring system.
As said, it's almost imposible to give a good editor rating for a game which hasn't even been released yet SO my ideea was this
1. Pre-release score
2. Review score (2-3 months after the game has been officially released)
3. User score (which in my opinion shouldn't even be enabled till the game is released. After the game is released, the option for users to score is enabled, this can solve some missunderstandings)
Now we have the 3 scores but how do we exactly score in them? Well this is a bit of a tough question, because normal user score will always be dynamic and probably based on his personal standards (game is good but he doesn't like that conveniently placed rock outside the city, so he scores it bad) or because of the fanatic perspective (user goes omg the game is made by X? don't care what ppl say, i vote it 10), so considering these factors and that the number of useras that actually give a very thought over vote is very limited, user votes should be ignored as a pylon for deciding if the game is good or not, but should still be displayed.
"makes people happy"
Editor/Publisher vote, this is were the "magic" should happen. For a more accurate voting, instead of voting directly from 1 to 10, i suggest u make some clear categories of what is ACTUALLY being rated.
For example:
1. Game performance [1..10]
2. Game sound [1..10]
3. Game graphics [1..10]
4. Stuff that players actually want to hear! like if the game is based more on grinding or leveling [low...ok'ish...medium...high...grindfest]
5. Story [1..10]
6. Customer Support [1..10]
After an editor selects the right scores, an average of the score is taken and give to the game. This will clearly illustrate a more accurate score, because the final number won't be decided by just 1-2 factors, and a random click. After the general score is given, the user should have the possibility (by clicking on the score of each game, in the list) to see what exactly led to that score.
The rest depends on the mentality and training of editors/publishers i guess. U can't leave out the human factor.
ARTICLES: Articles should strictly reflect the score each publisher gives a game. For example, an article consists of "bla bla" witch most ppl scroll very fast, then title: Performance [3] and explained why the game got that score, this is what ppl are looking for, not "oh i saw a pritty little bunny so i gave it a 11 out of 10, because bunnies are cool!
The number steps explained above are just to offer an example.
Hope u read this post, and find something of use through it. If i find other sollutions i will let u know.
For a person that has seen alot of strange reviews over the past 5-6 years I think we will have to look at MMOs in a diffrent light.
Now - I dont know how MMORPG.COM reviews Free to play games. Considering they are free to play - ppl are free to try themout for themselfs and find out. And that should be pretty much the standard in the MMO comunity - even for payed for games and subs. Sadly its not and thats why the Reviews have been hit with so much negative talk when in most cased its rated way to high cause its only played for few hours (RIFT was 30 hours and no class above 25? ). Considering that lower level content is the mostly played and beta tested content - thats never gonna give a right perspective.
I feel that big part of the reviewers job is to find out information how the potential players can get to experience the game without paying the full price for it. That should become a standard question to ask the devs - Will there be a free trial and when will it be released ? Games that can not answer the question when or are not even releasing free trials should loose 1 whole raiding point. Simply because its not acceptable to ask ppl to pay for something like a MMO game considering the history of crap releases in the past.
Since very few reviewers can actually judge the game from an endgame perspective - Its should aslo be noted specially - and the final verdict should always have a +- 1 variation for later changes.
Both games and reviewers need to come together and realise that its no good for anyone to give a game wrong raiting. Thus a number should never be the true mesurement of the game - rather it should be rated based on cons and pros.
So as I promised I gave it thought. As I read over all the post here I get the feeling this has not been overly helpful to you, and in fact maybe somewhat you regret it, LoL. I want to be useful but I think you have a handle on this. Unless we can revolutionize the format it seems A,B and C type grades improves the formula.
HOWEVER! The biggest complaint I would have may be most useful to you. Others have touched on this so I echo it hoping it gets attention.
Scores need to be updateded more often. For me the biggest reason I leave MMO's is because developers change things for the worse. This seems like an epidemic. I mean every game I played except for one I left because the developers or caretakers of the game ruined it enough to spoil my fun. So if I reviewed World of Warcraft 6 moths after release it would get a much higher review then if I reviewed it today.
Thus when I am looking for a new game using your reviews I have to go to forums and other sites redering your review fairly useless unless it is recent. I suppose my dream job would be to get paid to play MMO's and keep reviews fresh. (even if the pay was fairly minimum, hint heh )
But yes I think this is my biggest issue. Other then that I have never relied on one review thus I mentioned having user reviews easy to find would be a huge plus. So having more then one staff member review a game would rock, esp. if you had 2 reviews in conflict!
Everyone has their own view on good and bad. A super hot girl in my eyes rates a 9.5/10. In my brothers eyes, he might give her an 8 even though she's the hottest girl he's ever seen too. So getting it just right for everyone just doesn't work and never will. The trick is to be consistent as a site. Often times people, including me, will just glance at the overall score. If it's low, they won't bother reading why. If it's high, they might just jump at the game without reading why also. If it's decent, but maybe not quite, they might look over it to see if it fits in their idea of fun.
MMORPGs are trickier to rate than other games. I've played World of Warcraft for 6 1/2 years. Those first years I would have given the game a 9.2/10. I loved it! Then we played on a PvP server. That experience gets just a 7/10 for me, borderline not worth bothering. There are many aspects of an MMORPG and there are a lot of players that don't want to experience all of them. Large group raiding for me is a waste of time, so if you give a game a 9/10 because the large group raiding is fantastic, that 9/10 means nothing to me. If you give it a 9/10 because the incredible PvP, same thing, I won't bother. If you give it a 9/10 because of the solo play, or the small group experience, now you've got my attention. An MMORPG usually has all of these. Perhaps that's how you should review the game.
Section it off as SOLO, SMALL GROUP, LARGE GROUP, PVP, maybe some others in there too. I remember reading on here some face offs I think with like World of Warcraft and Everquest II and you went over each section of the game, crafting, world, PvE, PvP, etc. and you gave each area a rating. I really liked that and I think you should do that for each game now, and then in the summary (summaries are very important to me, and others "TLDR") list the type of gamer and whether you think they'd get a good experience out of it. PvP player - not much to see here. Solo player - a ton of content not to miss.
Here's an idea: Get your reviewers some actual time in the game that they're reviewing. For example, the terrible mess that was this week's WoW v Rift comparison: This reviewer probably played for three hours before declaring Rift superior to WoW. Even though said reviewer had positive points, they are meaningless because (s)he has no actual experience with the game.
First impressions are fine. But you can do First Impressions in two paragraphs. It is not a review.
I appreciate that you want to write something up immediately, but it serves no purpose. New and shiny is going to be new and shiny no matter what. MMOs (especially of the Eastern influence) put TONS of focus on the first 1/3rd of the levelling process just for people like you who base your reviews on the first 3 hours.
I found a thought-provoking article about game reviews in general written by a gaming journalist at this link: http://nohighscores.com/node/508
I have no idea whether or not this guy's experience is typical, but it was an interesting read. As for the reviews here at MMORPG.COM, I'm old school. I always found it more informative when various categories were rated, and then used to calculate an overall score.
Good find. I prefer reviews broken down by categories as well just for the reason you stated. For example, PvP may not be high on my list, but Crafting would.
I'd also prefer to see the occassional review by community member along side the site reviewer. Obviously this wouldn't work for every game, but the larger or premier titles could warrant such partnership. Reviews done in this manner involve the community to a greater degree and may act as a counter-balance to any creditibility (or bias) issues.
I've been a reader of the site for a long time now and would like to see more substantive reviews and scoring.
Originally posted by TK-420 Good find. I prefer reviews broken down by categories as well just for the reason you stated. For example, PvP may not be high on my list, but Crafting would. I'd also prefer to see the occassional review by community member along side the site reviewer. Obviously this wouldn't work for every game, but the larger or premier titles could warrant such partnership. Reviews done in this manner involve the community to a greater degree and may act as a counter-balance to any creditibility (or bias) issues. I've been a reader of the site for a long time now and would like to see more substantive reviews and scoring.
I have to agree, having ratings across different categories would be a nice addition. Same for having Reader Reviews, or something similar. However, the question for that is, by what criteria should readers be selected for reviews?
My personal choice with regards to reviews would be to assign an individual writer, or writers, to do write-ups over a period of time. Have them do a quick first impression write-up detailing their first 5 - 10 hours of the game, with an ongoing weekly follow up for 30 to 60 days. Given how much MMO's can change in the first couple of months, or not change in some cases, there needs to be some sort of system by which the games progress out of the gate can be taken into account.
When looking for a new game, I often start with those games with higher ratings on this site. The ratings are useful.
The next step is to determine whether it is the type of game I like to play.
What is often missing is enough information to tell me whether it is the type of game I like to play. For example, a game can be excellent, but if it has non-consentual PvP, I most likely will not like the game. Personal game preferences should not be part of the rating. However, it is an important addiional filter for players looking for a game they have not already played. That additional information may include a description of:
PvP (Battle grounds, open world non-consentual, open world consentual)
PvE
Raid content
Group content
Solo content
Crafting
Housing
Instances, open world or both
Quests
Combat mechanics
Of the above, the 2 that I personally would like in the game description is
enough information to guage whether or not PvP is optional; and
whether or not the game is solo friendly. For example, are there progression points that force you to group/raid?
I agree with the several comments about different types of gamers. It would be most useful to me to have 3 or 4 professional reviewers writing their responses to the same topic/subtopic and perhaps taking the average of their scores by topic.
"Never met a pack of humans that were any different. Look at the idiots that get elected every couple of years. You really consider those guys more mature than us? The only difference between us and them is, when they gank some noobs and take their stuff, the noobs actually die." - Madimorga
Use the same small handful of people to write the reviews. The fewer the better.
The writers can have a small "team" of people to assist in playtesting. The team must submit their "sub-reviews" to the primary writer who combines their own experience with that of their team's.
Once a year have a little pow wow and decide what is working and what isn't. Make adjustments.
Oh, and hire some writers with traditional Journalism training. Not just folks who like to write editorials non-stop.
Objectivity is something you still have to keep in mind when writing a review.
Opinion does drive a review, however, objectivity is the only way to compare two features in two seperate games.
And have someone who plays the game in question write your quizes. At this point it is painfully obvious that it is presently an intern + 15 minutes + the game in question's website; Which = boring quiz.
Why not poll your users on the several points of this topic?- Examples--
What sort of grading system do you find most effective for game reviews?
1-10, 1-2~ being a terrible game, 9-10 being the place you should spend the rest of your life.
1-10. Current System; 5 is bottom of the barrel, anything below is unplayable or perhaps an April Fool's joke.
A+-F. Not as prcise as say 9.6 but perhaps better for a universally understood rank.
"Play", "Meh", "Run Away". A pretty simple pass/fail system.
Should we summarize the rating of the game or let each part of the game stand on it's own merit? IE, if we rank sound 5, graphics 8, and crafting 10, do you want to see "Sound: 5, Graphics.." or a combination score based on the three?
Yes, one summary rating is exactly what I want.
No, let the various aspects stand or fall on their own merit
Should we use multiple reviewers for the same title, and should they be lind to, or awayre of, each other's identities?
Yes, and they should be blind to each other. Each person's honest and raw opinion would be more beneficial to the site's users.
Yes, but they should collaborate, together. Perhaps each can shed some light on an aspect for the other and get to the true value of the game.
No, one review tells me as much as I need to know about the game as two would, four would, or six would.
--and from there, you might ask other questions like "Do you consider [x] important to a review?" listing various aspects.
Spec'ing properly is a gateway drug. 12 Million People have been meter spammed in heroics.
Comments
Honestly i can't complain much about the reviews on MMORPG.COM , seen worst but not better. I am not saying that there is not room for improvement. On some ocasions i find the revies a bit subjective , and if we are talking about some form of jurnalism , that would imply that they should be done on a more objective style. This way the reader can decide for himself what letter or number to attribute to the game.
As for grading games , it's not an all that bad idea , i just hope people will still read the reviews and not only look at numbers, but they are not real fans of the genre if they do that anyway.
One neat idea i thought of, or so it think it's neat, is to have 2 ratings , one given by MMORPG.COM and one by the user. The way i see it only registered users can see the ratings and only after they rated a game atleast once. And a user should be able to change his rating choice on a game. I can give you my latest example with Rift , at some point i wouldn't rate the game more than 7/10 , but now after a month of playing i could go up as far as a 9/10
Hey there,
First of all it's really great to see honest reads like this once in a while, and i really appreciate them + it explains things at "our level" not an editorial level.
If u don't mind i have a few ideeas that i think might be of use related to the scoring system.
As said, it's almost imposible to give a good editor rating for a game which hasn't even been released yet SO my ideea was this
1. Pre-release score
2. Review score (2-3 months after the game has been officially released)
3. User score (which in my opinion shouldn't even be enabled till the game is released. After the game is released, the option for users to score is enabled, this can solve some missunderstandings)
Now we have the 3 scores but how do we exactly score in them? Well this is a bit of a tough question, because normal user score will always be dynamic and probably based on his personal standards (game is good but he doesn't like that conveniently placed rock outside the city, so he scores it bad) or because of the fanatic perspective (user goes omg the game is made by X? don't care what ppl say, i vote it 10), so considering these factors and that the number of useras that actually give a very thought over vote is very limited, user votes should be ignored as a pylon for deciding if the game is good or not, but should still be displayed.
"makes people happy"
Editor/Publisher vote, this is were the "magic" should happen. For a more accurate voting, instead of voting directly from 1 to 10, i suggest u make some clear categories of what is ACTUALLY being rated.
For example:
1. Game performance [1..10]
2. Game sound [1..10]
3. Game graphics [1..10]
4. Stuff that players actually want to hear! like if the game is based more on grinding or leveling [low...ok'ish...medium...high...grindfest]
5. Story [1..10]
6. Customer Support [1..10]
After an editor selects the right scores, an average of the score is taken and give to the game. This will clearly illustrate a more accurate score, because the final number won't be decided by just 1-2 factors, and a random click. After the general score is given, the user should have the possibility (by clicking on the score of each game, in the list) to see what exactly led to that score.
The rest depends on the mentality and training of editors/publishers i guess. U can't leave out the human factor.
ARTICLES: Articles should strictly reflect the score each publisher gives a game. For example, an article consists of "bla bla" witch most ppl scroll very fast, then title: Performance [3] and explained why the game got that score, this is what ppl are looking for, not "oh i saw a pritty little bunny so i gave it a 11 out of 10, because bunnies are cool!
The number steps explained above are just to offer an example.
Hope u read this post, and find something of use through it. If i find other sollutions i will let u know.
For a person that has seen alot of strange reviews over the past 5-6 years I think we will have to look at MMOs in a diffrent light.
Now - I dont know how MMORPG.COM reviews Free to play games. Considering they are free to play - ppl are free to try themout for themselfs and find out. And that should be pretty much the standard in the MMO comunity - even for payed for games and subs. Sadly its not and thats why the Reviews have been hit with so much negative talk when in most cased its rated way to high cause its only played for few hours (RIFT was 30 hours and no class above 25? ). Considering that lower level content is the mostly played and beta tested content - thats never gonna give a right perspective.
I feel that big part of the reviewers job is to find out information how the potential players can get to experience the game without paying the full price for it. That should become a standard question to ask the devs - Will there be a free trial and when will it be released ? Games that can not answer the question when or are not even releasing free trials should loose 1 whole raiding point. Simply because its not acceptable to ask ppl to pay for something like a MMO game considering the history of crap releases in the past.
Since very few reviewers can actually judge the game from an endgame perspective - Its should aslo be noted specially - and the final verdict should always have a +- 1 variation for later changes.
Both games and reviewers need to come together and realise that its no good for anyone to give a game wrong raiting. Thus a number should never be the true mesurement of the game - rather it should be rated based on cons and pros.
So as I promised I gave it thought. As I read over all the post here I get the feeling this has not been overly helpful to you, and in fact maybe somewhat you regret it, LoL. I want to be useful but I think you have a handle on this. Unless we can revolutionize the format it seems A,B and C type grades improves the formula.
HOWEVER! The biggest complaint I would have may be most useful to you. Others have touched on this so I echo it hoping it gets attention.
Scores need to be updateded more often. For me the biggest reason I leave MMO's is because developers change things for the worse. This seems like an epidemic. I mean every game I played except for one I left because the developers or caretakers of the game ruined it enough to spoil my fun. So if I reviewed World of Warcraft 6 moths after release it would get a much higher review then if I reviewed it today.
Thus when I am looking for a new game using your reviews I have to go to forums and other sites redering your review fairly useless unless it is recent. I suppose my dream job would be to get paid to play MMO's and keep reviews fresh. (even if the pay was fairly minimum, hint heh )
But yes I think this is my biggest issue. Other then that I have never relied on one review thus I mentioned having user reviews easy to find would be a huge plus. So having more then one staff member review a game would rock, esp. if you had 2 reviews in conflict!
Everyone has their own view on good and bad. A super hot girl in my eyes rates a 9.5/10. In my brothers eyes, he might give her an 8 even though she's the hottest girl he's ever seen too. So getting it just right for everyone just doesn't work and never will. The trick is to be consistent as a site. Often times people, including me, will just glance at the overall score. If it's low, they won't bother reading why. If it's high, they might just jump at the game without reading why also. If it's decent, but maybe not quite, they might look over it to see if it fits in their idea of fun.
MMORPGs are trickier to rate than other games. I've played World of Warcraft for 6 1/2 years. Those first years I would have given the game a 9.2/10. I loved it! Then we played on a PvP server. That experience gets just a 7/10 for me, borderline not worth bothering. There are many aspects of an MMORPG and there are a lot of players that don't want to experience all of them. Large group raiding for me is a waste of time, so if you give a game a 9/10 because the large group raiding is fantastic, that 9/10 means nothing to me. If you give it a 9/10 because the incredible PvP, same thing, I won't bother. If you give it a 9/10 because of the solo play, or the small group experience, now you've got my attention. An MMORPG usually has all of these. Perhaps that's how you should review the game.
Section it off as SOLO, SMALL GROUP, LARGE GROUP, PVP, maybe some others in there too. I remember reading on here some face offs I think with like World of Warcraft and Everquest II and you went over each section of the game, crafting, world, PvE, PvP, etc. and you gave each area a rating. I really liked that and I think you should do that for each game now, and then in the summary (summaries are very important to me, and others "TLDR") list the type of gamer and whether you think they'd get a good experience out of it. PvP player - not much to see here. Solo player - a ton of content not to miss.
Here's an idea: Get your reviewers some actual time in the game that they're reviewing. For example, the terrible mess that was this week's WoW v Rift comparison: This reviewer probably played for three hours before declaring Rift superior to WoW. Even though said reviewer had positive points, they are meaningless because (s)he has no actual experience with the game.
First impressions are fine. But you can do First Impressions in two paragraphs. It is not a review.
I appreciate that you want to write something up immediately, but it serves no purpose. New and shiny is going to be new and shiny no matter what. MMOs (especially of the Eastern influence) put TONS of focus on the first 1/3rd of the levelling process just for people like you who base your reviews on the first 3 hours.
Good find. I prefer reviews broken down by categories as well just for the reason you stated. For example, PvP may not be high on my list, but Crafting would.
I'd also prefer to see the occassional review by community member along side the site reviewer. Obviously this wouldn't work for every game, but the larger or premier titles could warrant such partnership. Reviews done in this manner involve the community to a greater degree and may act as a counter-balance to any creditibility (or bias) issues.
I've been a reader of the site for a long time now and would like to see more substantive reviews and scoring.
RIP TK-421. Best imperial guard ever.
I have to agree, having ratings across different categories would be a nice addition. Same for having Reader Reviews, or something similar. However, the question for that is, by what criteria should readers be selected for reviews?
My personal choice with regards to reviews would be to assign an individual writer, or writers, to do write-ups over a period of time. Have them do a quick first impression write-up detailing their first 5 - 10 hours of the game, with an ongoing weekly follow up for 30 to 60 days. Given how much MMO's can change in the first couple of months, or not change in some cases, there needs to be some sort of system by which the games progress out of the gate can be taken into account.
When looking for a new game, I often start with those games with higher ratings on this site. The ratings are useful.
The next step is to determine whether it is the type of game I like to play.
What is often missing is enough information to tell me whether it is the type of game I like to play. For example, a game can be excellent, but if it has non-consentual PvP, I most likely will not like the game. Personal game preferences should not be part of the rating. However, it is an important addiional filter for players looking for a game they have not already played. That additional information may include a description of:
PvP (Battle grounds, open world non-consentual, open world consentual)
PvE
Raid content
Group content
Solo content
Crafting
Housing
Instances, open world or both
Quests
Combat mechanics
Of the above, the 2 that I personally would like in the game description is
enough information to guage whether or not PvP is optional; and
whether or not the game is solo friendly. For example, are there progression points that force you to group/raid?
I agree with the several comments about different types of gamers. It would be most useful to me to have 3 or 4 professional reviewers writing their responses to the same topic/subtopic and perhaps taking the average of their scores by topic.
"Never met a pack of humans that were any different. Look at the idiots that get elected every couple of years. You really consider those guys more mature than us? The only difference between us and them is, when they gank some noobs and take their stuff, the noobs actually die." - Madimorga
Have one set of criteria and stick to it.
Use the same small handful of people to write the reviews. The fewer the better.
The writers can have a small "team" of people to assist in playtesting. The team must submit their "sub-reviews" to the primary writer who combines their own experience with that of their team's.
Once a year have a little pow wow and decide what is working and what isn't. Make adjustments.
Oh, and hire some writers with traditional Journalism training. Not just folks who like to write editorials non-stop.
Objectivity is something you still have to keep in mind when writing a review.
Opinion does drive a review, however, objectivity is the only way to compare two features in two seperate games.
And have someone who plays the game in question write your quizes. At this point it is painfully obvious that it is presently an intern + 15 minutes + the game in question's website; Which = boring quiz.
Why not poll your users on the several points of this topic?- Examples--
What sort of grading system do you find most effective for game reviews?
1-10, 1-2~ being a terrible game, 9-10 being the place you should spend the rest of your life.
1-10. Current System; 5 is bottom of the barrel, anything below is unplayable or perhaps an April Fool's joke.
A+-F. Not as prcise as say 9.6 but perhaps better for a universally understood rank.
"Play", "Meh", "Run Away". A pretty simple pass/fail system.
Should we summarize the rating of the game or let each part of the game stand on it's own merit? IE, if we rank sound 5, graphics 8, and crafting 10, do you want to see "Sound: 5, Graphics.." or a combination score based on the three?
Yes, one summary rating is exactly what I want.
No, let the various aspects stand or fall on their own merit
Should we use multiple reviewers for the same title, and should they be lind to, or awayre of, each other's identities?
Yes, and they should be blind to each other. Each person's honest and raw opinion would be more beneficial to the site's users.
Yes, but they should collaborate, together. Perhaps each can shed some light on an aspect for the other and get to the true value of the game.
No, one review tells me as much as I need to know about the game as two would, four would, or six would.
--and from there, you might ask other questions like "Do you consider [x] important to a review?" listing various aspects.
Spec'ing properly is a gateway drug.
12 Million People have been meter spammed in heroics.