Why testing doesn’t really happen until after D-Day

It happened to all of you. You work on a project and keep delivering features at a steady pace. You test your own work, and get constant confirmation from your project managers that each feature has been tested, and you have a green light to move on from the customer.

The release date is closing in, you manage to wrap everything up, and launch. You lean back in your chair and think to yourself “ahh, another job well done”.

Then, out of nowhere, after just a few hours in, your task list / inbox gets filled with things like “How can I do X?”, “We have an issue with Y” and the dreaded “Z isn’t working”. You open the list, scan the first few items, and it hits you:

“Wait a minute, we’ve been over these, this is the way this feature was supposed to work, it’s been tested.”.

Ahh, but you see, it wasn’t.

But how could this happen?

Here’s the thing: your customers don’t really test. Your project managers don’t really test. You don’t really test. Not until the lauch date that is. That’s when everyone really starts testing.

“But, we had at least 6 people involved in this project at all times. Surely between them they tested everything”.

Well, not really, no. You see, every person involved in your project has a certain way of  looking at it.

You have a list of features that needs implemented. While you churn out feature after feature, you only tend to test the specific behavior you built into each one. “Right, so the user clicks this, fills out this form, and ends up here.”.

You don’t really try to break your own application, so you only interact with it in the way it’s meant to work. And of course, it works … “on your machine”.

Your project managers don’t test because they don’t have the time for it, because they have that meeting about that other project with that other client in 10 minutes, just pop them an email and they’ll look it up later … They won’t. They’ll be too busy.
They’ll rephrase parts of your email (because really, you suck at writing to corporate clients), and forward it to the customer. And they will not get a chance to test it again … until after the lauch date, when …

… the customer will start testing. The customer, the person you build the entire project for, doesn’t even remotely start testing until after the project has set sail. Why? Because until then, to them, the project is not real yet. It’s “under construction”.

They will open it up, have a peek, and unless something is downright wrong, they will not notice it. They will not follow the individual interaction flows, they will not start using the project like they will use it for real. “Oh, I can see some products, that probably means the entire administering, checkout and billing processes must be working.”.

Most of the problems that will rise from the customer’s lack of testing are those when he expects a feature to work a certain way, but he described it to your project manager at first in a totally different way. And she described it to you in yet another way. And you developed it “your way”.

Who’s fault is it?

No one’s really. All of yours.

Who should test?

First of all, nothing beats professional testing. Ideally, you should have dedicated testers, but odds are you will have to make do with what you have available.

So there is good news, and bad news:

The good news is that, when searching for bugs, it’s easy! Grab a few members of your team that don’t work on this particular project. Best to do this with all types of colleagues: other developers, account managers, even the office secretary.

Sit down with them, and politely ask them if they can spare any amount of time to test your product. Then assign them some user roles, and let them loose.

The bad news is that application bugs are the easiest to find and fix. The biggest issues come from what I said earlier: miscommunication.

My bet is that shorter release cycles, more customer involvement (the project managers really have to step up here), and enhanced communication can at least lessen the impact of these problems on the typical project. Also, the User Stories and Acceptance Tests agile development concepts fit perfectly.

June 8, 2010 | Filed Under Development, www 

Comments

2 Responses to “Why testing doesn’t really happen until after D-Day”

  1. add
    on June 8th, 2010 08:18

    The client does not “start testing” the project. He starts “using” it. That is the point issues come up.

  2. ketamynx
    on June 10th, 2010 16:42

    This is why we have application testers and beta stages.

Leave a Reply