Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Quick Links
General: Wood: Closed Betas For the Dogs
Stradden
Managing EditorMember CommonPosts: 6,696
MMORPG.com's Jon Wood uses his column this week to talk about the Closed beta process and its importance to a well developed game.
As I'm sure that anyone reading MMORPG.com often enough to be following my column probably already knows, we've spent the last week in the Closed Beta test for Cryptic's Star Trek Online working on a series that we're cleverly calling: Inside Beta. This has got me pondering the idea of Closed Beta and its overall role in a game's development.
As with my now eleven week old puppy, at no time in a game's life is it changing and adapting and learning more than it does in the closed beta stage, and I'm not just talking about Star Trek Online here, but betas in general. The information that is collected in Closed Beta and the changes that are made because of it will go on to create a foundation for the game that will most likely last as long as it does.
Cheers,
Jon Wood
Managing Editor
MMORPG.com
Comments
Jon I loved this article. Being a few very good closed betas, I have to say I agree with you completely. The interaction with the devs is great if the closed beta is used properly. I have been in a few closed betas (names withheld to protect the guilt *cough* Mythic *cough*) that have had very little interaction with the players. I was in the DDO and LOTRO and can honestly say the devs were in constant dialogue with the players which the devs for DDO continued to do after release. A good dev/player relationship developed in closed beta carries over to release and on. I personally think the more the devs work with the players, the better the game will be in the long run.
"Closed Beta though, is an entirely different beast provided that it hasn't become too firmly entrenched in the company's marketing plan as well. "
By, for example, selling a promise of closed beta access to people who pre-purchased a long enough subscription to another game? Hypothetically speaking.
While I don't agree with the practice, I don't think that's mucked things up too badly yet. it just... sets a scary precedent. Hypothetically.
Cheers,
Jon Wood
Managing Editor
MMORPG.com
well I have been in soem very well run closed beta's such as EQII were we had a schedual as to what we were test and when. Even had a few times here we set up in a zone and they ported us into another zone to see how many they could get into it before it crashed. Then I have been in others were it was basically just a free play of the game . Currently, I am in a closed beta that has set times the servers are online and there are devs standing by so when we bugg something they can instantly contact us to look at the bugs. I have been in others were we had to fill out detail survies everytime we completed a quest. I am not against closed beta slots being given out to long time players of a game. SOE did this with EQII giving people who were on the premium server auto invites to the EQII beta. However, their reasoning was that these were hardcore players and they wanted part of their test group to include those types of players.
Magic is impressive, but now Minsc leads! Swords for everyone!
Very nice article Jon. I have been in a few betas and I can say for the most part it is the best time for communicating with the devs and others behind the scene so to speak.
I have been in a couple it wasn't that way but for the most part it has. I suppose that has a lot to do with me liking to do beta. I get to be where changes are made and the game develops and fine tunes a bit. It is fun and gives one a really good feeling.
Gikku
Some has total failors when it come to close beta's like Darkfall:(
Short close beta with only few testing.
Then the devs did not fix the real bugs and exploits discovered in beta.
And launched the game buggy and unfinished, it was rushed out after short close beta with no testing at all and no stress test eather.
They still pay for there mistakes with low pops.
2 servers maybe max 6 to8k paying subs lol.
Its realy a shame Darkfall is now a rather polished game and way better then most mmo's out there, and i dont mean becouse of pvp ffa.
I mean stable no bugs and rather complete and smooth gameplay.
Macro/exploit is main couse of low pops after 9months they should have close beta longer so they could have fix that before launch it.
Close beta should have last at least for 6 months.
If they would have launched now with last expansion Darkfall would been a succes guarenteed.
Games played:AC1-Darktide'99-2000-AC2-Darktide/dawnsong2003-2005,Lineage2-2005-2006 and now Darkfall-2009.....
In between WoW few months AoC few months and some f2p also all very short few weeks.
While I don't agree with the practice, I don't think that's mucked things up too badly yet. it just... sets a scary precedent. Hypothetically.
Hypothetically, it can be bad if someone unsuited to testing or a testing mentallity gets into a closed beta weither by fiscal purchase or by the Greate RMD of Forum Goers. Honestly, it would be nice to have psych tests on testers, before they are put into a pool that could go into testing such products.
I agree, with some of the absolute garbage that goes through chat windows in a released game, you certainly do not want to bring that into a Beta Test environment where the purpose of the the thing is to find and repair bugs, not bash the game because it is not as good as say a market leading game.....
One of our frequent forum posters recently implied that it was a bad thing that MMO developers ask their fans how they can improve their product. Personally, I don't understand that point of view. I think that the more the people who are doing the entertaining look back at those of us that they are trying to entertain and ask, "What can I do better?" the better off we're all going to be.
I can't go 100% in agreement with either side of this point. Yes, I agree that developers do indeed need to make a seriously better effort in communicating with their customers to find out what they want. That said, I don't think it needs to necessarily be the "behind closed doors" angle of information gathering that was the case when SOE supposedly had met with a panel of "players" to find out what they want and "voila", the NGE.
No, it needs to be in a very open and honest practice on a moderated forum where all interested can weigh in as ultimately we are ALL affected. Developers, after launch, need to include the players more on information dissemination as far as what they are "thinking" about doing to the game as far as system and mechanics changes.
This genre isn't The wonderful land of Oz anymore. Patience with the man behind the curtain is about gone. People don't like having to throw away years of their lives and friendships just because some guy in a company has a "bright idea" that turns out being contrary to what the overwhelming majority of your players want or are even open to trying out.
"Many nights, my friend... Many nights I've put a blade to your throat while you were sleeping. Glad I never killed you, Steve. You're alright..."
Chavez y Chavez
It is a sad thing that when beta goes live and not one of the things that you turned in as a bug got fixed. Not one. So you have to even ask yourself why do we do a beta, even a closed one.
They have become more of a publicity stunt than anything else.
Like most people have already said,betas are becoming more and more of an extension of the game. Especially in games where an extra item or incentive is offered for being one of the "chosen few" who were in the early stages of ther game. That's not even taking into account F2P games that have 57 betas that they use as an excuse to relaunch the game over and over again with new promotions and hooks.
On a happier note, this article had a picture of a puppy in it which makes up for having to think about all the hucklebucks that go on with betas nowadays.
Gimmicked Betas =
Puppies =
One of our frequent forum posters recently implied that it was a bad thing that MMO developers ask their fans how they can improve their product. Personally, I don't understand that point of view....
I think you have to be specific here about whether you are asking 'fans' or 'fanbois'.
A genuine Fan of a game wants to see the game improve for the benefit of everyone, himself, other gamers and the developer.
A Fanboi however has a different agenda and may be pushing the game in a certain direction for purely selfish reasons?
In addition, Fanbois will often disregard legitimate criticism and even 'shout it down' at the expense of the game. The big concern is that Developers can become Narcissitic and believe what the Fanbois are saying (because after all they are telling the Developer that they are right and stroking their egos) to the point of not looking at potential issues.
My most memorable experience of this was with PotBS on the year prior to release.
There were a couple of notable Fans of that game on the boards.
There were also hordes of Fanbois.
A number of issues were raised which were shouted down.
Fanbois pushed the game in certain directions that they liked.
FLS listened of course, because there were so many 'fans' who wanted things done this way or that.
One issue that is particularly relevant (again) now was the balancing in instanced battles.
Several of us argued that just because the battles were instanced did NOT guarentee balance.
This was shouted down and Fanbois screamed that this was wrong even though they couldn't justify why.
The game went live and many of the instanced battles (including the pre announced and critical Port Battles) were totally unbalanced to the point where in some cases - nobody came.
Instancing did NOT guarentee balance.
Interesting that this comment was made in relation to STO then:
"All PvP in Star Trek Online, initially, will take place in instances. This, for one, eliminates the problems of population imbalance, which could really get out of control in a single shard game. "
Nothing says irony like spelling ideot wrong.
If it's same company I'm thinking about, then don't worry. It doesn't matter who gets in beta, how many get in beta, how long the beta will be or if it's closed or open.. that company has made history with repeated "interesting" launches.
The problem with closed betas is that the "average gamer" is not giving their feedbacks. If you want to take part in closed beta you have to be intresed in a concept that devs are "forcing" upon you. The average gamer picks games based on diffrent things and thus - closed betas are in my opinion of no real use for the future development progress of the game (other than reporting bugs).
AOC went through closed beta with their no stats item values and nothing was done about it...
Warhammer went through 4 tiers of seperated testing just to find out that they actually split the game into 4 diffrent games that very few ppl bothered to play more than 1 -2 tiers before giving up when realising they had crap ablites when starting on the next tier...
Closed betas are just bugfinding phase... And sadly in many cases.. many of the bugs are not even fixed.
Forgot to tell you to take a good care of your dog
Its better than any MMORPG game can ever be - closed beta or live !!
Just make sure your shoes are will hidden for the next 6 months or so !!!
In my opinion the single largest contributor to a good closed beta is communication. Well, that and convincing certain testers that closed betas are a way to fix issues, not just whine about the shoddy condition of the game.
Cute puppy, great article! Oh, and stock up on slippers... they are great munchies late at night.
Ken
www.ActionMMORPG.com
One man, a small pile of money, and the screwball idea of a DIY Indie MMORPG? Yep, that's him. ~sigh~
I am the type of person who would deliver a lot of feedback, testing to find bugs and describing them exactly, but I am demotivated by having to compete with the 1000s of people who are in closed beta just to play the game.
Perhaps the application process could include a multiple-choice quiz about game-design, to seperate beta testers who have some knowledge or intuition about making good games, from the testers who share poorly thought out overly selfish opinions that won't make the game better.
Cryptic is trying a Customer Development approach to MMO creation.
Cute doggy! Aww I wish I had one too, but no time. ^^()
I love being in closed betas for the same reason; I like to see a game grow and change over time, being one of the first to see things in the making is something very exciting. Yeah I too wish developers would listen more or maybe filter better the various sayings what is founded critique and what isnt.
People don't ask questions to get answers - they ask questions to show how smart they are. - Dogbert
Some beta's still ask this. I remember when middle earth now called lotro had that type of application. Asked about guild size, hours of playtime, questions about what would you do if you found the bug. I know KOTROL had an application that asked questions like that as well as hardware design so they could get a good cross section of players.
I reported tons of bugs back in the early days of middle earth SOA, and we got to see those fixed as A2 through a4 were rolled out, LOTRO itself when SOA got here did not have many bugs. I wish I could say the same about SOM. I was in the beta. I reported many bugs, and I can say that I have actually had a good 10 or more of those bugs hit me when we went live. There was little effort made to change thigns as by the time we got to beta we were only 2 months out and basicly the product was already finished at that point.
There used to be a day when you could say quilty at turbine, and probably other games. Quality has been bypassed by speed of the release, and the intent to let folks get into your closed beta, knowing that they are going to tell their friends what they see hoping folks buy into it.
Asheron's Call does this via their main forums, and some nice things have come out of it. Fortunatly they seem to notice the one issue with this and act accordingly. A very small amount of the player base of an MMO goes to the forums. And certain play styles that are common in the game are never represented on the forums because those types of players don't want to get that involved in their game, they want it to be a game and just log in and play for a bit and that is it.
That has to be taken into account and sometimes there will be issues that a group of forum posters can make seem as if it should definetly be changed, but if you were to ask all the players in game it clearly shouldn't be changed. Taking feedback is good but it has to be taken very carefully. Also you don't want to get into a state where players start thinking they run the show, that will only lead to anger when their idea isn't listened to.
And having been in the closed beta of Ultima Online and almost every major MMO since, I have a tiny amount of experience in this.
The evidence does not support the theory that Beta testing improves games, particularly MMOs.
If I (and you) had a dollar for every MMO that has done one or all of the following, Id be typing this from my beach side villa in Miami.
I could go on.. but anyone whos been around this a while knows... many many games that were not publically beta tested have come on the market in great shape and done well. Many extensively public beta tested games have totally tanked inside 18 months, Tabula, DDO, almost everything by NCSoft etc.
The notion you put forward John is spot on!
If the industry understood and used Beta in the way you describe it, the industry and players would be better off. Sadly Beta is rapidly becoming a pay to test product phase that people seem only too willing to engage in STO took it to a new level... and IMHO not a high one.
+-+-+-+-+-+
"MMOs, for people that like think chatting is like a skill or something, rotflol"
http://purepwnage.com
-+-+-+-+-+-+
"Far away across the field, the tolling of the iron bell, calls the faithful to their knees. To hear the softly spoken magic spell" Pink Floyd-Dark Side of the Moon
One thing few developers seem to fully realize (or perhaps more accurately, realize, but have other demands which make it unavailable) is how resource-intensive good testing is. It is very easy to overlook bugs that are found by new users/players in minutes, because you know how to 'do things the right way' and you know that way works. However, if you fill your testing pool with people who are similarly skilled and have lots of gaming and testing experience, many of those people will also do things 'the right way'.
But at the same time, it can be very labor and time intensive to take a sampling of 'average gamers' and put them through essentially supervised play periods, where you can immediately reproduce errors and the like. It is nearly always one of the first areas to be cut in a project plan as dates slip for other reasons (and often with good intentions - Oh well if we change how we do this, it will improve the overall performance and clear up all these other things that we already know are wrong, so if we cut a bit of time from the test periods we should still be fine) and that time is rarely made up in actual fixes, as there is almost always another bug somewhere when you start making changes to a big system, or unintended consequences several steps down the road.
Its definitely a difficult issue, and one that may be well known to the developers, but ignored by those in charge of 'making the date' as even with the many examples they can look to of decent games which never lived up to their potential due to poor launches, they are able to convince themselves that 'that won't happen to us because (we're smarter, better funded, have better name recognition, etc etc)