Continuation

posted in A Keyboard and the Truth for project 96 Mill

Published August 03, 2005

So, we have made a decision to stop external testing.

Mainly because we are not getting what we hopped out of it.
We found most testers to be extreamly under-zealous, yet more than willing to bash us, as if the free beta copy they were playing was actually the Final 'I-paid-30-bucks-for' copy.

So the majority of testing on the MW will continue as in-person testing sessions, AND choice remote people who have shown good beta testing skills in the past.

But as far as plucking people out of the vast blue sea of GDNet in hopes that they will be good testers, that has come to an end.

If anyone here still wants to be a beta tester, they will need to prove to me, somehow, that they will be a good beta tester.

With that out of the way, we can move on to:

Stuff I've done recently:

My time on MW recently has consisted mainly of bug fixing, and thus far things have gone well.

It seems that we have tamed the savage save-game beast, and it is now working very well.

The Phantom Bug:

Unfortunetly we are experiencing a phantom bug,
it comes without a warning and at different times and crashes the game to the desktop.

Due to the severity I am wagering it is a System-Level bug, and my spider-sense tells me it is the audio adaptor.

The Audio Adaptor

Oh what horror. Everything was wonderful in AudioAdaptor land until a long while back we needed to integrate audio streaming (keeping a bunch of 40 MB PCM wave data files in RAM is not smart =D)

I implemented streaming using multi-threading, and the DirectSound notification system (waiting on HEVENTs), and it's always been flaky.

Somtimes out of the blue, WaitForMultipleObjectsEx will just fail to wait, and at times the adaptor will just cause an illegal exception and die.

So in short, it's got issues.

I am going to implement a new system in accordance with one of the DirectSound articles here in GDNet, which seems to be more efficant and less prone for error (all of the potential synchronization gotchas).

Anyone who has ever worked will filling buffers with raw data, knows the potential hazards of a slight miscalculation.

So for the moment I am going to say the bug is that.
It better be that.
Better not be anything graphical wise.
Or I'll kill it.
Yeah. [grin]

Previous Entry hehe

Next Entry Screenshot

0 likes 8 comments

Comments

TraderJack

So, uh... where r t3h pikt4rz?

-IV

August 03, 2005 09:05 AM

EDI

What would you like pict4rs of?
Don't want to spoil the game do you?

August 03, 2005 09:07 AM

Jesse Chounard

I had a thought that might help you. If you need to test a bigger variety of hardware configurations without shipping the current beta out to a bunch of potential customers, you could build a very simple level, and send that out instead. (So, basically change your code for when a person clicks on New Game to load "simple_test.level" instead of "the_real_game.level".)

If you decide to do something like that, I'd be glad to take part in that sort of test for you.

August 03, 2005 10:17 AM

johnhattan

Sorry to hear you're having problems with testers. I wrote a big post about that a while back (that I can't find now).

Basically I stated that if you have ten testers, you can assume that six of them will completely flake out on you and won't do anything. Three will do the minimum necessary, but only if you constantly beg 'em to do what they agreed to do. One will go above and beyond the call and will be worth his weight in gold.

One thing you need to do with your testers is to foster a sense of community. At the very least, make a Yahoo group where they can share bugs and impressions. People will be more candid if they know that they've got a community that feels the same way about something as they do.

You can also try offering a cash prize to foster some competition. Unfortunately, I had middling results with this. When I did level-testing for Duck Tiles, I ended up with about 12 testers and offered $50 for the best tester. As expected, seven did nothing. Three gave me the minimum I required (playing ten levels and putting 'em in order of difficulty). One tested 50 levels, and one tested all 200 (even the ones that other people had already tested).

In short, keep plugging away. Expect to sift through lots of flakes before you find one person who's worth his weight in gold.

August 03, 2005 10:21 AM

EDI

Yeah, I would say that is about right.

Thankfully we do have one guy who is worth his weight in gold.
And a few others worth thier salt =D

I am just worried that we won't find all the bugs unless we have some signifigant 'different player' coverage.

I am _very_ dissapointed at this fact, though I should have expected it, people in general are notorious for not doing what they say =)

As for testing a level, that is a good idea, and I had considered it. Though we arn't so worried about hardware compatibility as we are actual gameplay bugs.

But I agree, having an 'engine test' application would probably be a very smart idea. =D

We have already found a few issues pertaining to WindowsXP Tablet Edition, and it seems to flake a bit on laptops.

Not sure if it is the crappy graphics hardware, or what yet.

August 03, 2005 10:31 AM

johnhattan

My level testing was a bit different, as I wasn't testing a complete product but just ordering levels. The problem I had was that I had 200 levels that I needed tested. Duck Tiles is a bit different from your game in that it doesn't have a plot per se. I just needed to get 'em roughly ordered from least to most difficult so level 192 isn't easier than level 12.

I ended up breaking 'em into twenty groups of ten, zipping 'em up, and letting people put their ten in order from least to most difficult (along with pointing out any broken levels). I'd then fold all those lists together and hopefully have a reasonable curve of difficulty.

Turns out having one person do all 200 helped, because then I got one person's impression of easy versus difficult.

Keep plugging at it. Finding reliable testers isn't an easy process. At least take comfort in the fact that you don't have to pay 'em. I've got pals who wrote productivity apps and had to pay quite a bit of money to get their stuff tested.

August 03, 2005 11:53 AM

Rob Loach

I'm really sorry about the testing. I wish I could've helped you out more...

August 03, 2005 10:09 PM

Seriema

hmm I guess I'm a bit surprised since I did replay the game several times, although always ending up with a crash making me stop =P
(I'm the Win XP Tablet Edition user! MWUAHAHAAA)

EDI, if you feel that I bashed to hard, then I'm really sorry.. Don't hate me edi! =( hehe
I kinda whined about stuff I was expecting as a paying customer, because going "well this was free so OK" isn't gonna cut it since after it has passed me and the other testers it _will_ end up in paying customers hands. So I thought, hey better me giving creative critisism (that's how I feel I reported, although I might sound a bit negative and grumpy :P) than customers wanting their money back.

Hope I still get to test the save'able version!

Cheers
/JP

August 03, 2005 11:35 PM

You must log in to join the conversation.

Don't have a GameDev.net account? Sign up!

EDI

Author

🎉 Celebrating 25 Years of GameDev.net! 🎉

Continuation

Comments

EDI

Latest Entries

Whoa, it's dusty in here

Porting Static:IT - Part 2- Engine Differences

Porting Static:IT - C++ to HTML5

What am I up to?

96 Mill is now on Steam!

First Youtube Review of 96 MILL

First Youtube Review of 96 MILL

96 MILL Released!

96 Mill product page goes live, new info & gameplay video

96 Mill Nearing Completion

🎉 Celebrating 25 Years of GameDev.net! 🎉

Continuation

Comments

EDI

Latest Entries

Whoa, it's dusty in here

Porting Static:IT - Part 2- Engine Differences

Porting Static:IT - C++ to HTML5

What am I up to?

96 Mill is now on Steam!

First Youtube Review of 96 MILL

First Youtube Review of 96 MILL

96 MILL Released&#33;

96 Mill product page goes live, new info &#38; gameplay video

96 Mill Nearing Completion

Reticulating splines

96 MILL Released!

96 Mill product page goes live, new info & gameplay video