My first failure as a Video Game Tester
My first job as a Video Game Tester was in a small company making Video Game on Mobile phones – back when smartphones didn’t exist (wow, I feel like a dinosaur!).
We were a QA team of 4 in a company of about 30 employees. All working together in a small open space office, including the CEO. One day he comes around QA team desks and kicks the trash can:
-“Guys, how the hell are you testing? The phone carrier [Who validates our games] found yet another bug in one of the games we sent. This is costing us money EVERY time it happens!”
And then we wondered: How are we actually testing games? Well we would typically receive a given game for a given phone model, play for 3 hours (short arcade games) looking for bugs, and call it tested. The actual content of the test was left at the discretion of each tester.
This was obviously not working. And when asking more details to the CEO, it turns out most of the bugs we missed were not directly related to the gameplay. They were either problems in the menu or issues related to using the phone’s hardware during the game (receiving calls for example).
When you think about it it makes a lot of sense. If I ask you to test a racing game, how likely is it that you will start by checking every option in the menu and test all the option settings?
This little story taught us a lesson: There was a lot of value in planning and documenting the testing.
The way we solved our problem was creating a generic checklist of all the tests we could perform on any game and any phone.
These generic verifications included: receiving a text message while playing, deactivating the sound in the option menu, checking that the highscores are saved from a game to another, etc. The checklist actually became quite long, and we would make custom versions of it for each game so that we could add specific verifications.
The checklists helped us a lot in catching more bugs, and that’s the point when I realized testing professionally is more than playing the game randomly.
And after all, it’s not about how hard you fail, but about how hard you keep trying. I’m sure Nintendo would agree.