“What happens if that file is locked when the process tries to write to the file?”
“Um, I don’t know…”
“Do you think maybe we should find out?”

There’s an interesting conversation happening right now on the Agile Testing mail list under the heading “How many test cases?” The discussion centers around an interview question in which a candidate was asked to design test cases for a program that writes data to a file.

What’s particularly interesting to me about the conversation is how widely it is ranging. There are now comments on the thread about the role of testers, the nature of requirements, and the existence (or not) of specifications. All this from a simple question around the sort of test cases that might come from a given requirement. It’s almost as though testing is inextricably intertwined with the whole software development process.

Oh. Wait. It is.

And if testing is an inherent part of developing software, the very first tests begin long before there’s software to test. In fact, I think the first tests executed on any given project are questions project team members start asking when a requirement is little more than a hazy notion in someone’s head that the software should do something.

Let’s imagine for a moment that we are conversing with a stakeholder about a program that will read and write a file, much like the interview question described on the Agile Testing mail list.

We might ask questions about error conditions that would interfere with the program operating on the file:

• What should the program do if the file is locked and cannot be read?
• What if the file is not found?
• What if the current user does not have read permission on the file?

We might also ask questions about conditions we imagine the program might not handle well:

• Should the program be able to handle any file name that the operating system can create?
• Should the program be able to handle file names and paths with spaces?
• What about extremely long paths or deeply nested paths?

We might ask about possible contents of the file:

• Should the program work on any file format?
• Any contents?
• Any language?
• Multibyte?
• What if the file has nothing in it (0 length)?

And we might ask questions about the effect the program should have on its operating environment:

• Should the program lock a file on which it is operating so other software cannot access the file?
• Should the program always close any file it opens?
• Even if the program encounters an error condition?

Finally, we might ask questions to determine what information is interesting to our stakeholders:

• Do you want to know about memory leaks?
• If there were a situation in which this program hogged all the CPU cycles, would you want to know?
• Do you want to know if I can make this program crash?
• If there were a way for this program to corrupt the file, would you want to know about it?

Testers tend to be particularly good at having conversations like this because we have mental catalogs of risks and possible bugs.

It’s best to have this conversation early. Unfortunately, that’s not always possible, even on theoretically Agile teams. I actually had a tester on a Scrum team tell me, not long ago, “I just don’t see the point in going to the Scrum kickoff meeting.” Eeek!

The point of going to the meeting is that we can start discussing potential tests early. In doing so, we’ll prevent a bunch of bugs caused by misunderstood requirements, miscommunications, or blind spots. We’re providing the earliest feedback possible. Feedback early and often is a hallmark of an Agile team.

Of course, we’re not testing actual software that early, and that’s why the tester saw no point in attending the meeting. Instead, we’re testing ideas. We’re testing for understanding. We’re testing requirements. Occasionally we may be testing patience. But our questions will help the team arrive at a better understanding of what, exactly, they’re building, and what it will look like when it’s done.

In Lessons Learned in Software Testing, Cem Kaner, James Bach, and Bret Pettichord say that “All tests are an attempt to answer some question” (lesson 24).

Yes, and let’s start posing those questions early. If we don’t, the first time we ask our questions, they will be in the form of tests, and that’s too late.