Actually in both the games you mentioned, the issue was a combination of design and service - refusal to adapt design based on customer feedback until well after much damage had already been done. In both UO and EQ, there was huge "churn" - players buying the game, playing it for a while, then quitting. Both games were financially successful in spite of that churn (largely because the concept of MMOs itself was new and different) but could easily have been more so.
I think this is easy to say in retrospect. But honestly, adapting design based on customer feedback only works if the feedback is the right thing to do. The fundamental issue with "listening to player feedback" as a general principle is that the feedback is more often than not quite conflicting. Even if players are in general agreement that a particular feature is disliked, the proposed remedies will pretty much run the gamut of opposites. Part of a designer's job is to make sure the right thing happens design-wise (and there are hundreds of factors here). Being in touch with the player is key - no arguments. But then we are back to designers picking the correct course of action out of list of infinite hypotheticals, and this us puts on square 1 in terms of adapting design based on customer feedback.
Designers make games to be played, after all. But producers make games to be purchased, which is slightly different . As you say "both games were financially successful" ... and I ask, who is higher on the food-chain in a studio: designers, or producers?
fully agree its easy to see on the discussions they have about battlegrounds in EQ2 at the moment, there is atleastt 10 diffrent views on how it should be done, and those can be splitted up in 3....ppl that wish it never got implemented, ppl that think its ok as long it doesnt affect PvE progression in ANY way and the 3rd that wish to just do the "PvP" to get their gear.
so its already been changed afew times how it works. and well in end SOE need to take THEIR decision, but they do waste alot of time on this project - because they listen to player feedback...
I've never worked on an MMO, but I have worked on several big productivity packages (little known accounting app called Glasspac, and a well known program called Adobe Reader/Acrobat) and much of what can be said about making a MMO successful can be said for making any program successful - doubly so as many of these apps have online services now (like acrobat.com).
TL;DR version, we brought it upon ourselves and they're feeding us what they think we want. If you want to break the trend, stop buying it and they'll stop making it. Numbers and $ speak louder than words.
One trend that is disturbing too - only hand out beta's to people who pre-ordered the game, or people who signed up for fileplanet on a certain day. In other words - MMO companies are literally selling the beta.
I have no clue, but I suspect only the most hardcore devoted players take this route - those people will buy the game and a good majority of them will stick it out no matter the problem.
I think if most games did a much more formal beta process at least initially (applications, reviewing applicants etc etc) you might get a clearer picture of how your game will stack up to the general public.
After all - if the general public can't play your beta (or don't want to) you might have big problems ahead.
Designers make games to be played, after all. But producers make games to be purchased, which is slightly different . As you say "both games were financially successful" ... and I ask, who is higher on the food-chain in a studio: designers, or producers?
THIS.. straight from the get go is so very important to consider.. and imo simply not touched upon enough at all in this entire thread. Common mistake to generalize the developer/designer as the culprit for reasons of game failure. When the failure starts WELL before the game even begins development. Not in all cases.. but a good deal of them, especially as of late.
best example (weak I realize) without going ape wild with walls of text and links.
Publisher to Dev house: "Great game/name/concept, love it, great story, awsome board, inovative features. we can make it work perfect for the IP Aquired the funding, whats the damage estimate Fred?"
Dev to Pub: "Well Bob, looks like we will need around 10 mil for the project we put on your desk., 3 years min, this includes free turkeys and hams for thanksgiving and xmass"
Pub to Dev: "Ohman, Oh Fred, Fred Fred Freddy... Seriously Fred, the most we could get for that long of time is 6mil, What can you cut or edit to make it work"
Dev to Pub: "Well, we really need the contract Bob, I guess we could lose a few features, double up on the work commits, and offer free donuts. I'll be honest with you Bob, Can't hire more crew, but feel bad about overworking my folks and releasing something less than what you used to aquire the funding to begin with"
Pub to Dev: You let me worry about the tape Fred, you just work your magic and I'll get those donuts to ya"
LATER that night:
Bob. " so honey, closed the deal, where you wanna go this wknd?"
Fred Wife: "whats wrong babe?" Fred: "I hope you don't have any plans for the next 3 years"
Now, I realize that this little dramatization, is a bit silly and in some cases way off the mark. But, you get the idea. There are three main entities that handle concept to finished product. Dev/Design house, Producer, Distributor. And some of the more successfull projects are those that have run all three in the same camp. yes SOE, EA., (thats right I said it) Blizzard/Vivendi, CCP, Tubine..to name a few.
The crux is that each one of these entities has thier own Scope, thier Own concept/idea of technology (or how it should run) and least of all.. usually as an after thought... thier own Idea of Service. The marriages and relationships formed are very similar to IPO/preIPO culture when it comes to sustaining long term support for each other. And the "push" is more important than the final product or the consumer's viewpoint, before, during or after the lifetime of the product. It's like the concept of "one hit wonder" to music. It has become so resourced out, that it costs more resources to communicate the initial concept and release a solid combined effort, with respect to consumer in mind.
I really enjoyed the article. It's well written, insightful and smart. I just feel that it weights on the Dev/Designer much, and doesn't bare enough light to Publisher pressure/marketing/timelines and the effects on said topics of Scope, Tech and Service.
meh.. just my opinion.
d
"He who reigns within himself and rules his passions, desires, and fears is more than a king."
"Where there is much desire to learn, there of necessity will be much argruing, much writting, many opinions; for opinions in good men is but knowledge in the making."
Yeah.. Sell-out... I mean, he could have been working for EA, which is EVERY MMO developer's dream, considering their long and illustrious track record with MMO's.
You say Sell-out, I say one of the few people at Mythic with a brain.
Matt Frior was another one - He left to work for Zennimax (Possibly Elder Scrolls MMO), then there's Walter "Copper" Yarbrough (Turbine). Can you imagine if Sanya Weathers had to be the community rep for WAR, towing the EA party line?
Get real. EA is where MMO's (and their developers) go to die. More like Mark Jacobs sold out Mythic for Warhammer.. Might have made sense at the time, but look how it ended.. Even Mark got fired.
One of the smartest things Scott ever did was leave. How you can see it as otherwise is baffling.
This!
Bioware was a good company 10 years ago now is just another cash cow for EA. Unfortunatly they are lots of xbox and pc weak minded players that they are praising them like the next mesia, so overall they dont give a frack. They release a lot of games with chessy graphics and presentation and with a weak gameplay, and they sell milions. What a surprise these days? People are dumb, uninformed, patetic, and that hurts all of us. Since if they get away it the first time, you can bet your ass they will do it the 10th time and they are doing it. They are not the only one but it hurts to see them sell out like that. Oh well
I am sorry that we all cannot be as sophisticated as you and the other god like beings that hate EA. I am also sorry that by me liking Dragon Age Origins and Mass Effect 2 is dooming the gaming industry that you love so much. After all I am just a weak minded mortal, from now I will ask you and the other God like beings that hate EA what I should like and play before I spend money. sarcasm off
Dude get over yourself, I like Dragon Age as much as I liked Ice winddale and Baldur’s Gate 2, as you said Bioware from 10 years ago. Just because you hate something does not mean everyone is going to see it the same as you. So saying that your opinion of Bioware and EA is a fact and we all are hurting the gaming industry is pretty low. To me it is the people that is blinded by their opinions and will not even consider anything else, that are hurting and destroying this industry; both the haters and the fanboys are to blame for this. But that is still just my opinion not fact!
Hating EA has nothing to do with being sophisticated or god-like, it just has to do with being informed. It is the graveyard for mmorpgs and that is simply a reality: check their track record. While I might not agree with everything the original poster above wrote, I do see where he is coming from, but perhaps from a different angle. I agree with Scott's discussion on scope, but perhaps in a more directed application. For the past 5ish years, the scope of mmorpgs has been expanded to included things originally limited to the fps arena. Personally, I hate the fps influence on this genre and feel that it has done nothing but destroy gameplay and invite a more impatient, instant-gratifying-seeking playerbase. This also ties in with discussions about players rushing to level cap and lack of end game content.
Now it seems we are in the era of single player rpg influence. I also dont like the effect this is having on the genre; whether it be almost prodiminately single player-driven content or lack of community. This is where I agree with the comments about Bioware. Bioware has made some great games, but alot of their games have been largely copy and paste jobs. I have very little hope for SWTOR now after several months of following development. Forcing a single player model on a mmorpg wont work in my opinion. This is where scope comes in to play. Too many companys are designing a fps mmorpg or a single player mmorpg. Why can we not have a mmorpg mmorpg again? Why is it so unthinkable to limit your scope to actually designing a game for the genre that you are, well, designing it for? To me, that is where many of the games listed in the first article tripped up. If you want to make a mmorpg, do it. If you want to make a fps, do it. Trying to expand your customer base by bastardizing the two into a hybrid may work. But is the result really a mmorpg? Should we be pointing to things like Eve and WoW as mmorpg successes without analyzing whether they are really even mmorpgs?
Maybe thats the problem, maybe the definition or accepted definition of an mmorpg has changed and maybe instead of injecting new games into a vague genre, the focus should shift to a new genre. A genre filled with games whose scope is more in line with the UOs, the EQs, and the DAoCs of old. Imperfect games with problems, but games that for the most part knew what they wanted to do and did that one thing pretty darn well.
One of the points mentioned in the article is the server lag, crashes etc, and the poor investement often done.
Hype and expectations I imagine can't make it easy for devs to estimate the true need. Take CO for examples. Gazillion copies sold, and finally how many? maybe a quarter remain. So you build up the architecture for a gazillion, then in the end you don't need it. Now comes STO. Everyone knows it is hyped due to the IP. Will there be as many people left after launch as the sales predict? Or will it be like CO?
I am not envious of those types of decision, but then again, as customer, I simply don't care, I just want it to work!
TL;DR version, we brought it upon ourselves and they're feeding us what they think we want. If you want to break the trend, stop buying it and they'll stop making it. Numbers and $ speak louder than words.
One trend that is disturbing too - only hand out beta's to people who pre-ordered the game, or people who signed up for fileplanet on a certain day. In other words - MMO companies are literally selling the beta.
I have no clue, but I suspect only the most hardcore devoted players take this route - those people will buy the game and a good majority of them will stick it out no matter the problem.
I think if most games did a much more formal beta process at least initially (applications, reviewing applicants etc etc) you might get a clearer picture of how your game will stack up to the general public.
After all - if the general public can't play your beta (or don't want to) you might have big problems ahead.
The real problem is that many designers are incredibly invested in their creations. So much in fact that they tend to ignore their Closed Beta testers. That's what happened to Champions Online. From the beginning of Closed Beta (heck even in the pre-Closed beta), the testers told devs that the game was boring. That the "action MMO" was not fun. Spamming 1,1,1,1,1,3,2,1,1,1,1 was not fun. Nothing changed, and the game is now partially F2P. Failed because the designers couldn't/wouldn't listen to their player base.
EQ2 failed because SOE both misread the change in MMO public. They also either ignored or didn't know the word of mouth going on from the WoW Beta. SOE thought that a game grounded in huge grind like EQI and SWG was what the players wanted. They assumed that grind = Epic feel to the game. They were wrong, They got strong numbers for their first month or so, but when WOW shipped word of mouth got around that the new game with the primitive cartoon graphics was actually quite a bit of fun.
So when Devs build a new game. Like Scott said, they really need to make sure it's fun. They should bring in other teams to try out prototypes (Possibly from other studios owned by the company), bring in players from communities like MMORPG.com, get as many different eyes on the game as possible.
When I read the comment that went something like "Check your ego. You are not a god." All I could think of was John Smedly from Verant, back when he was working on EQ1. That tagline for Everquest, "You're in our world now!" haha, they sure thought they were gods it seemed. Eventually egos were checked and changes were made, as those in charge took a long hard look at their game and decided some aspects were just too much (or conversely, too little). If only the introspection and ego checking could have come sooner...
Honestly, I think you summed up most of my gripes towards the MMO community. Yes, I said gripes towards the MMO community.
I've been saying the scope of games and the expectations of players have been growing so rapidly and irrationally that its almost impossible for game makers to make a true MMO to please the masses.
Finally, somebody who been in the business stated the problems with MMO development. With MMO's you are always starting from scratch. No matter how many times somebody calls a game in development a wow clone, the god honest truth is they are starting from scratch.
The player base has to realize you are not (at least with today's programs and technology) ever going to have a WOW or a Eq, or (insert favorite MMO thats been out for year) type of game launch and be that in depth and that bug free.
I know some games that have launched in the past few years have been complete crap. But there also has been some very good ones that just needed to have the same chance to "bloom" as WoW did. Eve had a chance to bloom and look at how good that game is.
I also agree with the "scope" thing. I beleive this is what went wrong with WAR. EA buying Mythic was the worst thing for this game. But that doesn't mean its EA's fault. It was Mythic's fault for not knowing what to do when it suddenly had a huge budget for a huge game. so their scope went out of whack. And before they knew it, they had to cut a lot of wha that media mahcine was already promising to the players. Also, with an out of whack scope they could not finish the basics of the game.
I've always said Mythic probably would have had a hit if it had its own small little budget like DAOC.
Great information, good reading. The 4 things you mentioned are applicable to all new product or service development, not just MMOs or software. You've got to do those things if you want to make any thing new at all. Interestingly, as a result, we've got thousands of years of experience at it. But somehow, when dealing with the relatively new field of software, people just don't raise their heads and take advantage of those lessons learned.
Therefore, you get a lot of wheel-reinventing, and that kind of burden will kill your project momentum. When that happens, you lose the energy to make it over the hurdles, and you lose the creative initiative too - people start just getting things off of their plate instead of giving their best.
This sums up the only thing I've learned about projects that matters - every project is all about the people making it happen. Not the plan nor the budget - those are just tools to enable people to be somewhere doing something. The most important thing you can do is keep the people up about the project, keep them connected with the customer, and they'll keep up the speed to make it over the hurdles and deliver their best.
When I read the comment that went something like "Check your ego. You are not a god." All I could think of was John Smedly from Verant, back when he was working on EQ1. That tagline for Everquest, "You're in our world now!" haha, they sure thought they were gods it seemed. Eventually egos were checked and changes were made, as those in charge took a long hard look at their game and decided some aspects were just too much (or conversely, too little). If only the introspection and ego checking could have come sooner...
heh, I got banned from the EQ forums (think it was the beta 4 forums, actually) for complaining too "impolitely". A bug with the guards let a friend duel me with spells, but when I whacked his low-level butt lightly with my stick, I was instaganked by the nearby guards. I had just run across the entire continent of Antonica to meet him so we could play together, and boom - I was back in my hometown (level 13, no bind yet) and unable to play with my friend.
So I complained about the fact that we couldn't play together unless we were on the phone together when we made our charactors or had accidentally picked the same starting areas (impossible if you picked certain races btw). My point was that in an MMO, isn't playing with your friends kinda a good thing to support? And I complained about the guard bug. And about the whole "bind" mechanic just making no sense. And a few other things that had caused me trouble in the new player experience.
And I got banned by Brad McQ himself calling the mechanics stupid, counterproductive, etc. while being upset after losing an entire Saturday afternoon running across the friggin' continent so I could play the game with my friend. Side note - we'd bought the game specifically to play together, as we'd been playing the same Elder Scrolls and other games for awhile at that time.
Originally posted by wootin . This sums up the only thing I've learned about projects that matters - every project is all about the people making it happen. Not the plan nor the budget - those are just tools to enable people to be somewhere doing something. The most important thing you can do is keep the people up about the project, keep them connected with the customer, and they'll keep up the speed to make it over the hurdles and deliver their best.
You must have not done a lot of projects then. Its all about planning and budgeting. The bigger the project the more planning and budgeting you need. When a lot of people are working on a project they tend to look at the parts that concerns themselves. A plan and a budget keeps people in check. With no plan, the more people you have, the bigger the cluster **ck.
Sure a game will only be as good as the designers. But the designers can only reach their potential if there is good budgeting and planning done before hand. Also, a good system of exchanging ideas and troubleshooting between different people and different departments is crucial and also requires much planning before hand.
You're not going to read this: another reason why MMO's fail.
I've played a lot of video games and I've spent a lot of time playing them. My last estimate put me at somewhere over 30,000 hours of game time. No, that's not 3,000, that's Thirty Thousand hours. Does anyone care? Nope, why should they? I don't make games after all, I just play them.
I've written to game developers, posted in forums, screamed at the top of my lungs and have done everything short of walking into a studio office in an attempt to tell developers what I think of their games. It took me a little while to figure it out, but I now know why they ignore me: I'm not the average gamer.
It's a paradox see, as a developer, you want to listen to your customers. The problem is, your customers give you horrible feedback. One of the biggest reasons MMOs fail is because they're developing a product for a temperamental, unpredictable kids. I think that developers need to stop placating people and make games that allow the customer to decide what is fun and what isn't.
That's right, no more cookie-cutter, overbalanced MMOs. No more gankfests, no more item based game play, no more static, boring, predictable BS. Give us something we can sink our teeth into and call our own! I know, the technology isn't there yet... and even if it was, nobody is willing to risk millions of dollars on something that's so different. Gaming is a business after all and it's one I've stopped supporting.
You're not going to read this: another reason why MMO's fail.
I've played a lot of video games and I've spent a lot of time playing them. My last estimate put me at somewhere over 30,000 hours of game time. No, that's not 3,000, that's Thirty Thousand hours. Does anyone care? Nope, why should they? I don't make games after all, I just play them.
I've written to game developers, posted in forums, screamed at the top of my lungs and have done everything short of walking into a studio office in an attempt to tell developers what I think of their games. It took me a little while to figure it out, but I now know why they ignore me: I'm not the average gamer.
It's a paradox see, as a developer, you want to listen to your customers. The problem is, your customers give you horrible feedback. One of the biggest reasons MMOs fail is because they're developing a product for a temperamental, unpredictable kids. I think that developers need to stop placating people and make games that allow the customer to decide what is fun and what isn't.
That's right, no more cookie-cutter, overbalanced MMOs. No more gankfests, no more item based game play, no more static, boring, predictable BS. Give us something we can sink our teeth into and call our own! I know, the technology isn't there yet... and even if it was, nobody is willing to risk millions of dollars on something that's so different. Gaming is a business after all and it's one I've stopped supporting.
You made me laugh. The reason MMO fail is because the average gamer doesn't know what they like and give Dev's bad information? But if they would just listen to you with all your gaming experience everything would be fine?
I found the article very informative and interesting, kuddo's to you LumtheMad
I am guessing everything was from a development side of things. Some things I believe could be part of your 'scope' problem. For example, marketing, like telling the playerbase one thing, then doing something very different. Not listening to the player base can be huge as you stated, some of these company's do not even think of the customers, they only see their bottom line. When something like that happens it is doomed to failure. But at the same time, you really need to know your stuff as well because you need to realize just who you are listening too. Majority of the player's will always be people that never go to or even post on the forums, so exactly what kind of feedback are you getting, and which ones should you listen too. Having a poll in game sounds like a great idea, but that isn't going to help when people don't buy your game and can't log in.
I remember what got me to play EQ originally, not sure if anybody else remembers, but they had a demo that you could play, was like a trial, it was a great learning tool. Now I didn't play the game from the start, it was my first MMORPG but my brother bugged me to play, and I was able to kind of test the game out to see if I liked it. Now if you could combine that with an in game poll, before ever having to buy the game, you could be doing your company a better service in the long run, getting info from all of the people who either play, or potentionally play your game. One of the biggest downfall's of EQ, like you said, was the lack of attention to the player base. Pretty much everybody who ever played that game can agree that spawn spots, the rarity of certain items even dropping from that ultra long spawn, and the constant farming, and 'trains' in that game was bad, but EQ never listened until it was too late, and people were just fed up with it. Back then people stayed with the game because there really wasn't much else to do, there was DAoC, or....nothing really, a lot of people went to DAoC, so they could get away with it, but as more MMORPG"s came out, well lets just say they should have known what was going to happen.
Another big issue company's have now, they are all trying to be cutting edge, in technology, graphics, etc. And like you said, they need to stick with what works, but expand on it or change it to make it their own. They never think (at least it seems that way), that the potential customers may not buy their game because guess what, they will have to buy a whole new computer too, or upgrades, and those customers add that money up. Most of us live on a budget, so what makes them think that Joe and Mary Sue are going to shell out not just the 50 bucks to buy your game, but the extra 50-100 bucks to even be able to play your game. I wish more companies would look at these issue's more before they go off the deep end of advanced state of the art tech, especially in today's economic times.
Taking bad advice from your customers has contributed to the downfall of many MMO's. If you're not aware of this, then you have no right to even be discussing the issue.
As is mentioned in this very article, developers suffer from serious tunnel vision. They are incapable of seeing their game from the right perspectives. Experienced gamers like myself can easily see flaws in their designs. I'm guessing you don't have anywhere near the game testing experience I have, so I wouldn't expect you to understand what I'm talking about. When I said I play games, I didn't mean it in any trivial way. I'm guessing you don't know what putting 30,000 hours of work into something means. You probably never will.
Don't misunderstand me, I can't tell you that I can single-handedly solve every problem that every MMO has. It takes a lot of work to fix certain issues and some are just not solvable. I can tell you that I have predicted the solutions to problems that many MMOs have implemented and I have even predicted the very RPG systems that some MMOs have come to use. If I wasn't tied up with other issues in my life, I'd probably be an MMO developer myself.
BTW, you should probably learn to write better, people might actually take you seriously.
Tunnel vision is kind of a paradox isn't it? On one hand, players want developers to jam-pack an MMO with features until it's overflowing at the seams. And due to this sort of expectation it's not unreasonable in the game industry to have a several-month long crunch of 12+ hour work days, plus weekends, when doing a push for a release. And this is just creating all those glorious features that players come to expect and take for granted because "WoW does it."
On the other hand, players want developers to see be able to experience a game through a player's eyes. And not just any player's eyes, but the hardcore player that logs 30 hours a week doing endgame content. Which is why the objections to MMOs I often read on forums is rarely "this game isn't casual friendly," but rather it's: "this game doesn't get the needs of the hardcore player."
So how would you advise the paradox be reconciled?
An excellent article! My favourite comment is, "This also means that once the game launches, your players are your customers. Calling them customers is critically important. It implies that you are there to serve them, and not the other way around. You are not the god of your game world, you are a customer service professional, and if you want to keep those customers contributing to your paycheck, you had damned well better act like it." Man, that is priceless. Why do so many MMO execs seem to completely and utterly forget this basic reality?
Great article, Scott!
I found it particularly amusing that you chose Dragon Age as an example in the Customer Service part of this article. Having played this game since launch, I've experienced terrible customer service from Bioware/EA with this game.
Of the two questions I sent in, the first reply took almost 2 weeks to receive and was a "canned" response that didn't address or answer my actual concerns. The second was ignored totally.
A great example of what NOT to do as a Customer Service Manager is what Mr. Priestly did regarding the delay of the DA:O DLC "Return to Ostagar." In the official forum, Priestly posted about the delay in releasing the DLC and, to his credit, indicating the reasons for the delay and that there was no new release date, but would keep us posted, locking the thread soon after and not providing ANY updates to the community for weeks. The only indication to the customers as to what was going on was provided by "good Doctor M" in an interview two-three weeks later, where he indicated the DLC would likely be released the following week. In my view, an EPIC FAIL by their Customer Service folks and a clear textbook example of how NOT to handle it.
Great article. The scope portion is the part I hope every mmo developer reads and rereads. It's so bloody obvious that every one of these games suffers from massive scope-creep. Some handle it better than others while some (I'm looking at you Mortal) seem like they didn't even bother to hire a project manager and instead just flung a bunch of ideas at the wall to see what stuck. It seems, from the outside looking in, that there are very few competent PMs in the mmo industry. Yes I know development is hard, but so was going to the moon, and they somehow managed to pull it off using slide rules. Dreamers and developers with huge imaginations will only get you so far. Having these guys serve as your PM is a recipe for disaster (McQuaid and Vanguard comes to mind). You've got to have some hardline anal-retentive detail guys in charge to bring the whole project in for a smooth landing.
With regard to PMs in the software industry. I think that there is more of a problem with who is told to wear the PM hat and associated expectations.
IE, is the PM really a PM or a working developer with the PM hat, that does not really want the job or know it?
All to often the executives of a coporation just throw titles around like candy from a Pinnata and then expect all requests to be honored without consequence.
Stupid is what stupid does and the working developer has no real influence in most cases. A real PM has authority to make decisions of consequence. Corporations tend to castrate this process.
Fail is Fail and fact is that most people are still too ignorant about technology and it's associated development. Scope creap is just the result of bad decisiosns by humans.
Seems to me that all the points made in this artical are good ones.
I have a follow-up comment, however.
I've been involved with playing games since 1974. Starting with face to face rpg and boardgames, and now mmorpgs. I've been an office worker for a face to face game design company. I've been an office worker for a play by mail computer game company (the first one, the oldest one, and, arguably, the one that made all these MMORPG's a possibility by inspiring a whole bunch of gamer geeks to think of better ways to do it than by mail...you don't know how many of your designers were our customers way back when they were in high school and such, how many of your employees were our employees at one time... it might scare you (refering to game design companies in general).
My first love was boardgames and face to face rpgs, so that's where I stayed, but I played computer games a whole lot too. Starting on an apple-2C, then a comodor, then an IBM pc... Lets just say I played when Text based games were popular and if you got stick figures on a cubist background, you were lucky.
Somewhere in there I got to help playtest a boardgame. I worked on playtesting a couple of RPG's. I got to edit some scenario packs. I got to write my own scenario packs. I even managed to help design a boardgame. I'm saying all this to give you some indication that I might actually have some basis for an informed opinion.
I've seen the playtest method you guys use. I think you need to add another layer of testing.
In the time I worked as a playtester on face to face games, I noticed that the designers are sometimes lost in the forest of details. They get to where they know what is 'supposed to be' and that's what they see. I got lost in that while working on one of the games I was working on, a spectacular failure that never got published, despite being a wonderful idea. It failed because of expanding scope. Got too long, would not fit in budget. All that work for naught.
I turned it over to a friend, who editted it down to a viable product. Still didn't get published, the company went out of business who had optioned it. Sigh. But we had a good product, at least.
In face to face games, that sort of failure means a few hundred hours work lost, maybe some cash outlay for art or story content. Not a really big deal except to the poor guys working on it.
In mmorpgs a similar failure would cost millions.
The suggestion I have is to put in a new layer of playtester. Somebody who isn't a designer, isn't a programmer, isn't somebody who works in shipping. All they do is play the game and give feedback. Let you know what they like, what they don't, and all that. OK, you do that in Alpha-testing and Beta-testing. But where do you draw your alpha testers from? Usually in-house people who, even if they work in the mail department, know what is supposed to be happening. I'm suggesting recruiting totally unconnected, out of the loop, never been anywhere near the office, don't know the inside scoop people.
Get them involved from the beginning, keep them compartmentalized, and anylize the heck out of what they do.
If I am barking up a wrong tree here, if you already do that, then chalk it up to my lack of familiarity with the computer mmorpg segment of the industry. I'm only a player over here, and I don't read every newsletter or board on development. All I know is, an independent, segregated playtesting group is much more valuable than a connected, in the know group, when it comes to figuring out what is actually wrong with a face to face game. They have no preconcieved ideas, no subconscious understanding of what is supposed to happen. They are, in fact, on the same level as the customer who picks up the game because they liked the packaging or read a review.
Do you already do this? I dunno. But I think you should be.
The suggestion I have is to put in a new layer of playtester. Somebody who isn't a designer, isn't a programmer, isn't somebody who works in shipping. All they do is play the game and give feedback. Let you know what they like, what they don't, and all that.
You're not going to read this: another reason why MMO's fail.
I've played a lot of video games and I've spent a lot of time playing them. My last estimate put me at somewhere over 30,000 hours of game time. No, that's not 3,000, that's Thirty Thousand hours. Does anyone care? Nope, why should they? I don't make games after all, I just play them.
I've written to game developers, posted in forums, screamed at the top of my lungs and have done everything short of walking into a studio office in an attempt to tell developers what I think of their games. It took me a little while to figure it out, but I now know why they ignore me: I'm not the average gamer.
It's a paradox see, as a developer, you want to listen to your customers. The problem is, your customers give you horrible feedback. One of the biggest reasons MMOs fail is because they're developing a product for a temperamental, unpredictable kids. I think that developers need to stop placating people and make games that allow the customer to decide what is fun and what isn't.
That's right, no more cookie-cutter, overbalanced MMOs. No more gankfests, no more item based game play, no more static, boring, predictable BS. Give us something we can sink our teeth into and call our own! I know, the technology isn't there yet... and even if it was, nobody is willing to risk millions of dollars on something that's so different. Gaming is a business after all and it's one I've stopped supporting.
You made me laugh. The reason MMO fail is because the average gamer doesn't know what they like and give Dev's bad information? But if they would just listen to you with all your gaming experience everything would be fine?
Seriously? Thats your argument?
I'll back him on this statement alone "Give us something we can sink our teeth into and call our own!". That is the essence of any long-term product - you need to give the customer the perception that THEY own it, not you.
Comments
I think this is easy to say in retrospect. But honestly, adapting design based on customer feedback only works if the feedback is the right thing to do. The fundamental issue with "listening to player feedback" as a general principle is that the feedback is more often than not quite conflicting. Even if players are in general agreement that a particular feature is disliked, the proposed remedies will pretty much run the gamut of opposites. Part of a designer's job is to make sure the right thing happens design-wise (and there are hundreds of factors here). Being in touch with the player is key - no arguments. But then we are back to designers picking the correct course of action out of list of infinite hypotheticals, and this us puts on square 1 in terms of adapting design based on customer feedback.
Designers make games to be played, after all. But producers make games to be purchased, which is slightly different
. As you say "both games were financially successful" ... and I ask, who is higher on the food-chain in a studio: designers, or producers?
fully agree its easy to see on the discussions they have about battlegrounds in EQ2 at the moment, there is atleastt 10 diffrent views on how it should be done, and those can be splitted up in 3....ppl that wish it never got implemented, ppl that think its ok as long it doesnt affect PvE progression in ANY way and the 3rd that wish to just do the "PvP" to get their gear.
so its already been changed afew times how it works. and well in end SOE need to take THEIR decision, but they do waste alot of time on this project - because they listen to player feedback...
I've never worked on an MMO, but I have worked on several big productivity packages (little known accounting app called Glasspac, and a well known program called Adobe Reader/Acrobat) and much of what can be said about making a MMO successful can be said for making any program successful - doubly so as many of these apps have online services now (like acrobat.com).
Well done article
.
One trend that is disturbing too - only hand out beta's to people who pre-ordered the game, or people who signed up for fileplanet on a certain day. In other words - MMO companies are literally selling the beta.
I have no clue, but I suspect only the most hardcore devoted players take this route - those people will buy the game and a good majority of them will stick it out no matter the problem.
I think if most games did a much more formal beta process at least initially (applications, reviewing applicants etc etc) you might get a clearer picture of how your game will stack up to the general public.
After all - if the general public can't play your beta (or don't want to) you might have big problems ahead.
QFT.
This is a great article and bang on the money.
The initial discussion of scope should be the first chapter of a 'how to write an MMO' guide for anyone in the industry to read.
THIS.. straight from the get go is so very important to consider.. and imo simply not touched upon enough at all in this entire thread. Common mistake to generalize the developer/designer as the culprit for reasons of game failure. When the failure starts WELL before the game even begins development. Not in all cases.. but a good deal of them, especially as of late.
best example (weak I realize) without going ape wild with walls of text and links.
Publisher to Dev house: "Great game/name/concept, love it, great story, awsome board, inovative features. we can make it work perfect for the IP Aquired the funding, whats the damage estimate Fred?"
Dev to Pub: "Well Bob, looks like we will need around 10 mil for the project we put on your desk., 3 years min, this includes free turkeys and hams for thanksgiving and xmass"
Pub to Dev: "Ohman, Oh Fred, Fred Fred Freddy... Seriously Fred, the most we could get for that long of time is 6mil, What can you cut or edit to make it work"
Dev to Pub: "Well, we really need the contract Bob, I guess we could lose a few features, double up on the work commits, and offer free donuts. I'll be honest with you Bob, Can't hire more crew, but feel bad about overworking my folks and releasing something less than what you used to aquire the funding to begin with"
Pub to Dev: You let me worry about the tape Fred, you just work your magic and I'll get those donuts to ya"
LATER that night:
Bob. " so honey, closed the deal, where you wanna go this wknd?"
Fred Wife: "whats wrong babe?" Fred: "I hope you don't have any plans for the next 3 years"
Now, I realize that this little dramatization, is a bit silly and in some cases way off the mark. But, you get the idea. There are three main entities that handle concept to finished product. Dev/Design house, Producer, Distributor. And some of the more successfull projects are those that have run all three in the same camp. yes SOE, EA., (thats right I said it) Blizzard/Vivendi, CCP, Tubine..to name a few.
The crux is that each one of these entities has thier own Scope, thier Own concept/idea of technology (or how it should run) and least of all.. usually as an after thought... thier own Idea of Service. The marriages and relationships formed are very similar to IPO/preIPO culture when it comes to sustaining long term support for each other. And the "push" is more important than the final product or the consumer's viewpoint, before, during or after the lifetime of the product. It's like the concept of "one hit wonder" to music. It has become so resourced out, that it costs more resources to communicate the initial concept and release a solid combined effort, with respect to consumer in mind.
I really enjoyed the article. It's well written, insightful and smart. I just feel that it weights on the Dev/Designer much, and doesn't bare enough light to Publisher pressure/marketing/timelines and the effects on said topics of Scope, Tech and Service.
meh.. just my opinion.
d
"He who reigns within himself and rules his passions, desires, and fears is more than a king."
"Where there is much desire to learn, there of necessity will be much argruing, much writting, many opinions; for opinions in good men is but knowledge in the making."
John Milton 1608-1674
Yeah.. Sell-out... I mean, he could have been working for EA, which is EVERY MMO developer's dream, considering their long and illustrious track record with MMO's.
You say Sell-out, I say one of the few people at Mythic with a brain.
Matt Frior was another one - He left to work for Zennimax (Possibly Elder Scrolls MMO), then there's Walter "Copper" Yarbrough (Turbine). Can you imagine if Sanya Weathers had to be the community rep for WAR, towing the EA party line?
Get real. EA is where MMO's (and their developers) go to die. More like Mark Jacobs sold out Mythic for Warhammer.. Might have made sense at the time, but look how it ended.. Even Mark got fired.
One of the smartest things Scott ever did was leave. How you can see it as otherwise is baffling.
This!
Bioware was a good company 10 years ago now is just another cash cow for EA. Unfortunatly they are lots of xbox and pc weak minded players that they are praising them like the next mesia, so overall they dont give a frack. They release a lot of games with chessy graphics and presentation and with a weak gameplay, and they sell milions. What a surprise these days? People are dumb, uninformed, patetic, and that hurts all of us. Since if they get away it the first time, you can bet your ass they will do it the 10th time and they are doing it. They are not the only one but it hurts to see them sell out like that. Oh well
I am sorry that we all cannot be as sophisticated as you and the other god like beings that hate EA. I am also sorry that by me liking Dragon Age Origins and Mass Effect 2 is dooming the gaming industry that you love so much. After all I am just a weak minded mortal, from now I will ask you and the other God like beings that hate EA what I should like and play before I spend money. sarcasm off
Dude get over yourself, I like Dragon Age as much as I liked Ice winddale and Baldur’s Gate 2, as you said Bioware from 10 years ago. Just because you hate something does not mean everyone is going to see it the same as you. So saying that your opinion of Bioware and EA is a fact and we all are hurting the gaming industry is pretty low. To me it is the people that is blinded by their opinions and will not even consider anything else, that are hurting and destroying this industry; both the haters and the fanboys are to blame for this. But that is still just my opinion not fact!
Hating EA has nothing to do with being sophisticated or god-like, it just has to do with being informed. It is the graveyard for mmorpgs and that is simply a reality: check their track record. While I might not agree with everything the original poster above wrote, I do see where he is coming from, but perhaps from a different angle. I agree with Scott's discussion on scope, but perhaps in a more directed application. For the past 5ish years, the scope of mmorpgs has been expanded to included things originally limited to the fps arena. Personally, I hate the fps influence on this genre and feel that it has done nothing but destroy gameplay and invite a more impatient, instant-gratifying-seeking playerbase. This also ties in with discussions about players rushing to level cap and lack of end game content.
Now it seems we are in the era of single player rpg influence. I also dont like the effect this is having on the genre; whether it be almost prodiminately single player-driven content or lack of community. This is where I agree with the comments about Bioware. Bioware has made some great games, but alot of their games have been largely copy and paste jobs. I have very little hope for SWTOR now after several months of following development. Forcing a single player model on a mmorpg wont work in my opinion. This is where scope comes in to play. Too many companys are designing a fps mmorpg or a single player mmorpg. Why can we not have a mmorpg mmorpg again? Why is it so unthinkable to limit your scope to actually designing a game for the genre that you are, well, designing it for? To me, that is where many of the games listed in the first article tripped up. If you want to make a mmorpg, do it. If you want to make a fps, do it. Trying to expand your customer base by bastardizing the two into a hybrid may work. But is the result really a mmorpg? Should we be pointing to things like Eve and WoW as mmorpg successes without analyzing whether they are really even mmorpgs?
Maybe thats the problem, maybe the definition or accepted definition of an mmorpg has changed and maybe instead of injecting new games into a vague genre, the focus should shift to a new genre. A genre filled with games whose scope is more in line with the UOs, the EQs, and the DAoCs of old. Imperfect games with problems, but games that for the most part knew what they wanted to do and did that one thing pretty darn well.
One of the points mentioned in the article is the server lag, crashes etc, and the poor investement often done.
Hype and expectations I imagine can't make it easy for devs to estimate the true need. Take CO for examples. Gazillion copies sold, and finally how many? maybe a quarter remain. So you build up the architecture for a gazillion, then in the end you don't need it. Now comes STO. Everyone knows it is hyped due to the IP. Will there be as many people left after launch as the sales predict? Or will it be like CO?
I am not envious of those types of decision, but then again, as customer, I simply don't care, I just want it to work!
Now if we could only get Richard Garriot to read this article.
One trend that is disturbing too - only hand out beta's to people who pre-ordered the game, or people who signed up for fileplanet on a certain day. In other words - MMO companies are literally selling the beta.
I have no clue, but I suspect only the most hardcore devoted players take this route - those people will buy the game and a good majority of them will stick it out no matter the problem.
I think if most games did a much more formal beta process at least initially (applications, reviewing applicants etc etc) you might get a clearer picture of how your game will stack up to the general public.
After all - if the general public can't play your beta (or don't want to) you might have big problems ahead.
The real problem is that many designers are incredibly invested in their creations. So much in fact that they tend to ignore their Closed Beta testers. That's what happened to Champions Online. From the beginning of Closed Beta (heck even in the pre-Closed beta), the testers told devs that the game was boring. That the "action MMO" was not fun. Spamming 1,1,1,1,1,3,2,1,1,1,1 was not fun. Nothing changed, and the game is now partially F2P. Failed because the designers couldn't/wouldn't listen to their player base.
EQ2 failed because SOE both misread the change in MMO public. They also either ignored or didn't know the word of mouth going on from the WoW Beta. SOE thought that a game grounded in huge grind like EQI and SWG was what the players wanted. They assumed that grind = Epic feel to the game. They were wrong, They got strong numbers for their first month or so, but when WOW shipped word of mouth got around that the new game with the primitive cartoon graphics was actually quite a bit of fun.
So when Devs build a new game. Like Scott said, they really need to make sure it's fun. They should bring in other teams to try out prototypes (Possibly from other studios owned by the company), bring in players from communities like MMORPG.com, get as many different eyes on the game as possible.
When I read the comment that went something like "Check your ego. You are not a god." All I could think of was John Smedly from Verant, back when he was working on EQ1. That tagline for Everquest, "You're in our world now!" haha, they sure thought they were gods it seemed. Eventually egos were checked and changes were made, as those in charge took a long hard look at their game and decided some aspects were just too much (or conversely, too little). If only the introspection and ego checking could have come sooner...
Posting in the most legendary thread ever.
Honestly, I think you summed up most of my gripes towards the MMO community. Yes, I said gripes towards the MMO community.
I've been saying the scope of games and the expectations of players have been growing so rapidly and irrationally that its almost impossible for game makers to make a true MMO to please the masses.
Finally, somebody who been in the business stated the problems with MMO development. With MMO's you are always starting from scratch. No matter how many times somebody calls a game in development a wow clone, the god honest truth is they are starting from scratch.
The player base has to realize you are not (at least with today's programs and technology) ever going to have a WOW or a Eq, or (insert favorite MMO thats been out for year) type of game launch and be that in depth and that bug free.
I know some games that have launched in the past few years have been complete crap. But there also has been some very good ones that just needed to have the same chance to "bloom" as WoW did. Eve had a chance to bloom and look at how good that game is.
I also agree with the "scope" thing. I beleive this is what went wrong with WAR. EA buying Mythic was the worst thing for this game. But that doesn't mean its EA's fault. It was Mythic's fault for not knowing what to do when it suddenly had a huge budget for a huge game. so their scope went out of whack. And before they knew it, they had to cut a lot of wha that media mahcine was already promising to the players. Also, with an out of whack scope they could not finish the basics of the game.
I've always said Mythic probably would have had a hit if it had its own small little budget like DAOC.
Great information, good reading. The 4 things you mentioned are applicable to all new product or service development, not just MMOs or software. You've got to do those things if you want to make any thing new at all. Interestingly, as a result, we've got thousands of years of experience at it. But somehow, when dealing with the relatively new field of software, people just don't raise their heads and take advantage of those lessons learned.
Therefore, you get a lot of wheel-reinventing, and that kind of burden will kill your project momentum. When that happens, you lose the energy to make it over the hurdles, and you lose the creative initiative too - people start just getting things off of their plate instead of giving their best.
This sums up the only thing I've learned about projects that matters - every project is all about the people making it happen. Not the plan nor the budget - those are just tools to enable people to be somewhere doing something. The most important thing you can do is keep the people up about the project, keep them connected with the customer, and they'll keep up the speed to make it over the hurdles and deliver their best.
heh, I got banned from the EQ forums (think it was the beta 4 forums, actually) for complaining too "impolitely". A bug with the guards let a friend duel me with spells, but when I whacked his low-level butt lightly with my stick, I was instaganked by the nearby guards. I had just run across the entire continent of Antonica to meet him so we could play together, and boom - I was back in my hometown (level 13, no bind yet) and unable to play with my friend.
So I complained about the fact that we couldn't play together unless we were on the phone together when we made our charactors or had accidentally picked the same starting areas (impossible if you picked certain races btw). My point was that in an MMO, isn't playing with your friends kinda a good thing to support? And I complained about the guard bug. And about the whole "bind" mechanic just making no sense. And a few other things that had caused me trouble in the new player experience.
And I got banned by Brad McQ himself calling the mechanics stupid, counterproductive, etc. while being upset after losing an entire Saturday afternoon running across the friggin' continent so I could play the game with my friend. Side note - we'd bought the game specifically to play together, as we'd been playing the same Elder Scrolls and other games for awhile at that time.
You must have not done a lot of projects then. Its all about planning and budgeting. The bigger the project the more planning and budgeting you need. When a lot of people are working on a project they tend to look at the parts that concerns themselves. A plan and a budget keeps people in check. With no plan, the more people you have, the bigger the cluster **ck.
Sure a game will only be as good as the designers. But the designers can only reach their potential if there is good budgeting and planning done before hand. Also, a good system of exchanging ideas and troubleshooting between different people and different departments is crucial and also requires much planning before hand.
You're not going to read this: another reason why MMO's fail.
I've played a lot of video games and I've spent a lot of time playing them. My last estimate put me at somewhere over 30,000 hours of game time. No, that's not 3,000, that's Thirty Thousand hours. Does anyone care? Nope, why should they? I don't make games after all, I just play them.
I've written to game developers, posted in forums, screamed at the top of my lungs and have done everything short of walking into a studio office in an attempt to tell developers what I think of their games. It took me a little while to figure it out, but I now know why they ignore me: I'm not the average gamer.
It's a paradox see, as a developer, you want to listen to your customers. The problem is, your customers give you horrible feedback. One of the biggest reasons MMOs fail is because they're developing a product for a temperamental, unpredictable kids. I think that developers need to stop placating people and make games that allow the customer to decide what is fun and what isn't.
That's right, no more cookie-cutter, overbalanced MMOs. No more gankfests, no more item based game play, no more static, boring, predictable BS. Give us something we can sink our teeth into and call our own! I know, the technology isn't there yet... and even if it was, nobody is willing to risk millions of dollars on something that's so different. Gaming is a business after all and it's one I've stopped supporting.
You made me laugh. The reason MMO fail is because the average gamer doesn't know what they like and give Dev's bad information? But if they would just listen to you with all your gaming experience everything would be fine?
Seriously? Thats your argument?
I found the article very informative and interesting, kuddo's to you LumtheMad
I am guessing everything was from a development side of things. Some things I believe could be part of your 'scope' problem. For example, marketing, like telling the playerbase one thing, then doing something very different. Not listening to the player base can be huge as you stated, some of these company's do not even think of the customers, they only see their bottom line. When something like that happens it is doomed to failure. But at the same time, you really need to know your stuff as well because you need to realize just who you are listening too. Majority of the player's will always be people that never go to or even post on the forums, so exactly what kind of feedback are you getting, and which ones should you listen too. Having a poll in game sounds like a great idea, but that isn't going to help when people don't buy your game and can't log in.
I remember what got me to play EQ originally, not sure if anybody else remembers, but they had a demo that you could play, was like a trial, it was a great learning tool. Now I didn't play the game from the start, it was my first MMORPG but my brother bugged me to play, and I was able to kind of test the game out to see if I liked it. Now if you could combine that with an in game poll, before ever having to buy the game, you could be doing your company a better service in the long run, getting info from all of the people who either play, or potentionally play your game. One of the biggest downfall's of EQ, like you said, was the lack of attention to the player base. Pretty much everybody who ever played that game can agree that spawn spots, the rarity of certain items even dropping from that ultra long spawn, and the constant farming, and 'trains' in that game was bad, but EQ never listened until it was too late, and people were just fed up with it. Back then people stayed with the game because there really wasn't much else to do, there was DAoC, or....nothing really, a lot of people went to DAoC, so they could get away with it, but as more MMORPG"s came out, well lets just say they should have known what was going to happen.
Another big issue company's have now, they are all trying to be cutting edge, in technology, graphics, etc. And like you said, they need to stick with what works, but expand on it or change it to make it their own. They never think (at least it seems that way), that the potential customers may not buy their game because guess what, they will have to buy a whole new computer too, or upgrades, and those customers add that money up. Most of us live on a budget, so what makes them think that Joe and Mary Sue are going to shell out not just the 50 bucks to buy your game, but the extra 50-100 bucks to even be able to play your game. I wish more companies would look at these issue's more before they go off the deep end of advanced state of the art tech, especially in today's economic times.
Great post Scott
Tunnel vision is kind of a paradox isn't it? On one hand, players want developers to jam-pack an MMO with features until it's overflowing at the seams. And due to this sort of expectation it's not unreasonable in the game industry to have a several-month long crunch of 12+ hour work days, plus weekends, when doing a push for a release. And this is just creating all those glorious features that players come to expect and take for granted because "WoW does it."
On the other hand, players want developers to see be able to experience a game through a player's eyes. And not just any player's eyes, but the hardcore player that logs 30 hours a week doing endgame content. Which is why the objections to MMOs I often read on forums is rarely "this game isn't casual friendly," but rather it's: "this game doesn't get the needs of the hardcore player."
So how would you advise the paradox be reconciled?
Great article, Scott!
I found it particularly amusing that you chose Dragon Age as an example in the Customer Service part of this article. Having played this game since launch, I've experienced terrible customer service from Bioware/EA with this game.
Of the two questions I sent in, the first reply took almost 2 weeks to receive and was a "canned" response that didn't address or answer my actual concerns. The second was ignored totally.
A great example of what NOT to do as a Customer Service Manager is what Mr. Priestly did regarding the delay of the DA:O DLC "Return to Ostagar." In the official forum, Priestly posted about the delay in releasing the DLC and, to his credit, indicating the reasons for the delay and that there was no new release date, but would keep us posted, locking the thread soon after and not providing ANY updates to the community for weeks. The only indication to the customers as to what was going on was provided by "good Doctor M" in an interview two-three weeks later, where he indicated the DLC would likely be released the following week. In my view, an EPIC FAIL by their Customer Service folks and a clear textbook example of how NOT to handle it.
With regard to PMs in the software industry. I think that there is more of a problem with who is told to wear the PM hat and associated expectations.
IE, is the PM really a PM or a working developer with the PM hat, that does not really want the job or know it?
All to often the executives of a coporation just throw titles around like candy from a Pinnata and then expect all requests to be honored without consequence.
Stupid is what stupid does and the working developer has no real influence in most cases. A real PM has authority to make decisions of consequence. Corporations tend to castrate this process.
Fail is Fail and fact is that most people are still too ignorant about technology and it's associated development. Scope creap is just the result of bad decisiosns by humans.
IMHO
Seems to me that all the points made in this artical are good ones.
I have a follow-up comment, however.
I've been involved with playing games since 1974. Starting with face to face rpg and boardgames, and now mmorpgs. I've been an office worker for a face to face game design company. I've been an office worker for a play by mail computer game company (the first one, the oldest one, and, arguably, the one that made all these MMORPG's a possibility by inspiring a whole bunch of gamer geeks to think of better ways to do it than by mail...you don't know how many of your designers were our customers way back when they were in high school and such, how many of your employees were our employees at one time... it might scare you (refering to game design companies in general).
My first love was boardgames and face to face rpgs, so that's where I stayed, but I played computer games a whole lot too. Starting on an apple-2C, then a comodor, then an IBM pc... Lets just say I played when Text based games were popular and if you got stick figures on a cubist background, you were lucky.
Somewhere in there I got to help playtest a boardgame. I worked on playtesting a couple of RPG's. I got to edit some scenario packs. I got to write my own scenario packs. I even managed to help design a boardgame. I'm saying all this to give you some indication that I might actually have some basis for an informed opinion.
I've seen the playtest method you guys use. I think you need to add another layer of testing.
In the time I worked as a playtester on face to face games, I noticed that the designers are sometimes lost in the forest of details. They get to where they know what is 'supposed to be' and that's what they see. I got lost in that while working on one of the games I was working on, a spectacular failure that never got published, despite being a wonderful idea. It failed because of expanding scope. Got too long, would not fit in budget. All that work for naught.
I turned it over to a friend, who editted it down to a viable product. Still didn't get published, the company went out of business who had optioned it. Sigh. But we had a good product, at least.
In face to face games, that sort of failure means a few hundred hours work lost, maybe some cash outlay for art or story content. Not a really big deal except to the poor guys working on it.
In mmorpgs a similar failure would cost millions.
The suggestion I have is to put in a new layer of playtester. Somebody who isn't a designer, isn't a programmer, isn't somebody who works in shipping. All they do is play the game and give feedback. Let you know what they like, what they don't, and all that. OK, you do that in Alpha-testing and Beta-testing. But where do you draw your alpha testers from? Usually in-house people who, even if they work in the mail department, know what is supposed to be happening. I'm suggesting recruiting totally unconnected, out of the loop, never been anywhere near the office, don't know the inside scoop people.
Get them involved from the beginning, keep them compartmentalized, and anylize the heck out of what they do.
If I am barking up a wrong tree here, if you already do that, then chalk it up to my lack of familiarity with the computer mmorpg segment of the industry. I'm only a player over here, and I don't read every newsletter or board on development. All I know is, an independent, segregated playtesting group is much more valuable than a connected, in the know group, when it comes to figuring out what is actually wrong with a face to face game. They have no preconcieved ideas, no subconscious understanding of what is supposed to happen. They are, in fact, on the same level as the customer who picks up the game because they liked the packaging or read a review.
Do you already do this? I dunno. But I think you should be.
This is called "QA"
Actually, there are MMOs which have launched using middleware.
Dark Age of Camelot used Renderware
Warhammer Online used Gamebryo
Vanguard used Unreal
Darkfall used Bigworld (ha!)
and I'm sure there are others as well. Yes, the codebases were modified from the originals, but nevertheless, each game still used middleware.
You made me laugh. The reason MMO fail is because the average gamer doesn't know what they like and give Dev's bad information? But if they would just listen to you with all your gaming experience everything would be fine?
Seriously? Thats your argument?
I'll back him on this statement alone "Give us something we can sink our teeth into and call our own!". That is the essence of any long-term product - you need to give the customer the perception that THEY own it, not you.