Wednesday, March 25, 2009

Software Testers - Ah, my own job security

Great article here that shows the importance of my job.

Basically, a school district paid $4 million for a software system that has been down 4 times since launch (once for ~17 hours), duplicated student ID numbers, double registered students, misrepresented grades (would you like to receive an "E" instead of an "A"?) and provided parents with access to the wrong students' grades.

Unfortunately, even if people agree as to the importance of having someone (preferably a team of someones) test the software, there are a couple of factors that get in the way.
  • Time - There's never enough time to exhaust every possible scenario. Even to get a good feel for 80% of the most common scenarios, it's impossible to be certain that every real world scenario has been covered.
    • Furthermore, added time crunch usually comes into play as stakeholders determine that a product must release by a specified date regardless of readiness. Thus, major issues are either never found or are swept under the rug.
  • Money - Money drives the Time issue in that the longer it takes to develop a product, the longer before you can start making money on it. Add to that the fact that the people working on the project want to get paid. Corners will inevitably cut. Quality will invariably be sacrificed in lieu of getting the project out so it can start making money...under the assumption that any remaining problems can be fixed in the future...without acknowledging the truth that any issue fixed once in production can cost 5, 10, 20 times or more to fix than it would if it was fixed while in the development life cycle.
  • Information - Even if all the time/money/people were available, there still exists the problem that most projects (at least every one I've been involved in) never has all of the information required to adequately design, plan, develop, or test the project. No matter how much planning, brainstorming, and creativity goes into the test plan, without full information about the design, function, user set, environment, etc., there is no way to ensure 100% confidence that the product will get out there without some major issue...some of which may be very noticeable to someone working under a different information set than the testers.

Whenever I see an article like this discussing the failing of some major software first instinct is to laugh at the awful bugs that made it into the real world.

After that, I want to cry, because I know that somewhere, somebody with my job title, is likely getting reamed for not finding these bugs before they made it to the world. I only hope that the test team involved has good records as to issues reported, tests planned (including whether they were conducted or not due to time/budget constraints) and notes from triage discussions related to areas where quality was compromised.

On the plus side, as long as these stories make it into the papers, there will always be a need for somebody to hire me. :)

1 comment:

Kevin said...

I would hate to be a software tester. I wouldn't wish that on my worst enemy. Uh, wait...