How do you do Acceptance Testing?

There are two points at which the user is intimately involved with application development. At the beginning of the project, the user has to explain what he wants the software to do, so that requirements can be created -- whether that specification is as simple as, "I need a utility to back up these files" or it requires a document as long as some novels (though infinitely less interesting).

While some programming methodologies encourage the user participation throughout the development process, the other point where the end user HAS to get involved is at the end of the project: acceptance testing. That project phase has a lot of names, though my own working definition is, "the last phase of the testing cycle, right before the team goes out to have a celebratory beer."

I'm writing my column for Software Test & Performance magazine about "the acceptable Acceptance Test," and I'd like your input. (I'd also like to quote you, so please give me your full name, company, geography, or at least an e-mail ID where I can ask you for that info later.)

How do you go about doing Acceptance Testing? What do you do to prevent the users from coming back AFTER your team has had that celebratory beer to say, "Oh, we decided we didn't need that, so can you take it out? And we realized we wanted it to work another way. Fixing that isn't a problem, is it?"

I'm not asking only about QA process, though I'm interested in that, too. (Do you have a formal way of doing acceptance testing?) The transition from "bits of code that might work" to "an application about to go into production" sounds a lot more political than it does technical. This is the last opportunity for someone outside the QA/development team to stick his thumb in the pie. All the people for whom the word "deadline" doesn't seem to apply ("I know I was supposed to identify problems last month, but gosh this is really important"); the political shenanigans when management wants to take credit or cover their butts; I'm sure you can supply more examples than I can.

What works for you? What have you tried to do for the Acceptance Testing phase that DOESN'T work?

For me to deliver my article on time (and get my own Acceptance Testing signed off by my editor!), I need responses by August 10. I'll check back here frequently, but it'd help me out if you also cc me privately at [email protected].

Esther Schindler
Contributing editor, Software Test & Performance Magazine (http://www.stpmag.com)

We specify requirements with the user - absorbing the paper flow and office procedure behind the aspect we are to automate.

We deliver early, pre-production code - for demonstration and further definition.

We go back to the shop - re-tool around the aspects found during the latest discovery session.

This can be repeated - based on the scope of the project - several times.

After delivery of production code, we accept requests for change, bug-fix and enhancement. For small projects, this is informal. For large projects we use a specific form for Requests/Changes. These get numbered, prioritized and processed. Weekly release of complete items for approval - sign off - by the client.

A project we are currently working on was delivered for production 2 months ago, after 3 or 4 months of demonstration/re-tooling. There have been roughly 300 Requests/Change orders submitted in the past 2 months - about 100 completed to date. The size of the project will probably mean that the list of Request/Changes will never disappear - but become more "enhancement" oriented as time goes on.

That's really good feedback, slamany. Do you have any good before-and-after examples to show how this has helped? Or a horror story that demonstrated why you needed to put in that process? (Everybody loves to read about other people's horror stories.)

In other words: what do you think would happen if you DIDN'T do all that?

(Be sure to let me know, privately if necessary, how I can quote you. My editor will want more than a handle. <grin>)

--Esther

I've been on vacation for a week - first time in three years without a laptop :)

Are you still interested?

Nah. I already handed in the article!