• Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity
  • Far Beyond My Capacity

Friday, August 12, 2011

Beta Testing - "Unexpected this is. And unfortunate... Incomplete is your training."

1 comment
With the recent announcement that EA/Bioware will be holding large-scale beta-testing weekends for Star Wars: The Old Republic in September I thought it might be interesting to discuss the kind of things you could expect to experience assuming you are lucky enough to receive an invite.

Over the past 11 years I've been involved in beta testing for a lot of MMOs including (but not limited to) Dark Age of Camelot, Everquest 2, Age of Conan, Vanguard SoH, Rift and Lord of the Rings Online. There are a few things that always seem to crop up no matter who the developer or what the game, so for those of you going into a beta test for the first time here is a few of the things you can expect. In fact the vast majority of these points will apply to all MMO beta tests.

Performance

The first thing to take into account is that the tests will still be using a beta client, probably one that that is a few versions behind the current developer client. Beta clients traditionally have poor performance compared to release clients for several reasons. They are not yet fully optimised and many game systems will not be in their final state. Additionally most beta clients contain a good deal of extra code for measuring metrics (see my post on Bioware's Skynet system for an idea of what I mean). This extra code will be logging and reporting on just about everything you do as a beta tester - where you go, where and how often you die, what skills you are using, what UI windows you open and how often, etc. It is important feedback for the developers but it will impact the performance of the client. When the game is finally released much of this code will be removed from the game (though not all of it as it can be used to track gold sellers and bots etc).

Performance will also be affected by unoptimised graphics drivers. Expect to see both nVidia and AMD release new driver profiles for the game shortly after release that have been optimised for performance in SWTOR. Things like anti-aliasing routines are often only included or optimised just before launch.

Finally there will be huge amounts of lag. The purpose of the September beta weekends from the developers point of view is not to give people a free try at the game, but to run server stability and population tests and to check out their queueing and sharding systems before launch. Often server populations are deliberately over stacked during this phase of stress testing, which means a lot of people playing in the same place at the same time, leading to both graphics lag and latency issues.

The servers will probably be coming up and down a lot. They may need resetting due to bugs and lock-ups and if the developer is on the ball they may even be getting patched quickly too. Each time this happens you'll need to re-log back into the game.

There is nothing you will be able to do about this. Performance issues will happen to some degree.

Don't let Beta stress turn you into Angry Wookiee!
Population

The September test weekends will form part of the stress testing phase for the game. This part of the test is normally done near the end of the development cycle to test the server stability and to test systems designed to manage server populations. What this means for you as a tester however is that there will likely be queues to get into the game, sometimes very long ones. This is often done deliberately to test the queueing systems themselves.

The large population for the stress tests will likely bring other issues as well. Obviously there could be a lot of graphics lag and latency due to the high number of players in a particular area, but you may also find that there are issues with mob density. What I mean by that is  if you are given a quest to "kill 10 rats" (to use an MMO-specific meme) then many, many other people could be doing the same quest at the same time, which will lead to a big problem as there won't be enough rats to go round. This is unlikely to be such a large problem once the game goes live, but I've often found it an issue in beta stress tests; quests that would normally take 10 minutes might take an hour as hundreds of players jump on every rat as soon as it pokes it's nose out if it's hole.

Community

Unfortunately in my experience beta communities go from being helpful and friendly to bitchy and trolling once open beta begins. Ok, so the September testing isn't an open beta (you still need an invite), but it's the closest thing SWTOR will have, with thousands of new players invited to test the game. So, expect to see lots of moaning, comparisons to existing games and a lot of "omg this game is so crap!" type comments. Just ignore them.

The Game

The first thing to realise is that if you are lucky enough to get invited to one of the September tests you are going to have a bloody huge download to get through before you can even start to think about playing. With the huge size of the game, plus hours of music and hundreds of thousands of lines of recorded dialogue the game client is going to be large. The preview photos of the Collector's Edition show that the game comes on four DVD disks. This gives the installed client a size of between 19GB and 35GB depending on whether the disks are single or double-layered and assuming there is no compression on the disks. It could be even larger! Like I said, that's a big download!

The game itself is likely still going to have a lot of bugs. Hopefully the vast majority of game-breaking bugs will have been ironed out at this stage, but there will still be bugs and graphical glitches. If you find any be sure to report them (they will eventually get fixed, if not in time for launch).

The game you see during the September testing programme is likely to be very, very close to the game that will be released. There will not be any miracle patch added before launch that will resolve all issues, and there will not be any huge unannounced features. At this stage the game is likely to be feature-locked and undergoing bug testing and polish more than anything else.

The final thing to point out is that your characters from the test will be deleted prior to the game going live. This may seem obvious but I think that in every single beta test I have participated in this question has been asked on more than one occasion.

You have to be signed up for a chance to be invited to the September tests.

After all that you might be wondering if playing during the September testing is worth it! I can honestly say that I have enjoyed every beta test I have been a part of, and your participation will go towards making a better game, even if you do not diligently file bug reports and feedback.

Finally, for an interesting insight into what MMO developers themselves think about beta testing programmes I suggest reading this excellent article over at Massively.

1 comment:

  1. Very interesting stuff. Quite a few snippets I didn't know or consider. Cheers Bro.

    ReplyDelete

Related Posts Plugin for WordPress, Blogger...