Test Obsessed

A participant on the agile-testing news group recently asked, “How can you plan for the unexpected?” He was asking about how you design negative tests, and his question reminded me of an exercise I use in my testing classes for doing just that. Continue reading The Nightmare Headline Game: Planning for the Unexpected

“Our testing leaves a lot to be desired,” the Project Manager shot an accusatory glance at the QA Manager. The QA Manager glared back. I was seated between them at the conference table, and felt trapped in the middle.

“What makes you say that?” I asked, shifting my chair so I could see both managers at once.

“The software stays in test for too long. Our ship dates just slip and slip,” the PM shook his head. “Testing takes too long!”

“I see,” I replied. “And why is it that the software stays in test so long?”

The QA Manager cocked his head expectantly, apparently curious about the PM’s answer too.

“The testers find too many bugs!” complained the PM.

Continue reading Testing: Not a Phase, but a Way of Life

One of the reasons some organizations spurn Exploratory Testing is the notion that “All Testing Should Be Repeatable.”

To this, I say: “Bah.”

Repeatability is an overrated attribute for testing. Repeating a given test for a specific purpose, like for regression testing, is one thing. But insisting all testing be repeatable is an unnecessary constraint that results in more expensive, less powerful testing. Consider: Continue reading Repeatability is Overrated

“What happens if that file is locked when the process tries to write to the file?”
“Um, I don’t know…”
“Do you think maybe we should find out?”

There’s an interesting conversation happening right now on the Agile Testing mail list under the heading “How many test cases?” The discussion centers around an interview question in which a candidate was asked to design test cases for a program that writes data to a file.

What’s particularly interesting to me about the conversation is how widely it is ranging. There are now comments on the thread about the role of testers, the nature of requirements, and the existence (or not) of specifications. All this from a simple question around the sort of test cases that might come from a given requirement. It’s almost as though testing is inextricably intertwined with the whole software development process.

Oh. Wait. It is. Continue reading Can You Phrase That As a Question?

When I took my first job in QA many years ago, I thought that my role was to ensure quality.

I was dead wrong.

I figured out how wrong I was pretty fast.  A developer stonewalled on fixing a bug until it was just too late to get the bug fixed for the release.  I fought for that bug fix, and fought hard.  In the end, I learned that I could lobby all I wanted, but getting bugs fixed was outside a my scope of control.  Quality was barely within my sphere of influence.  My title may have been “QA” but my real role was testing.  I’ve since discovered that’s normal: many (most?) organizations use the term “QA” and “Test” interchangeably.

My new insight was the classic: Testing can tell us about the absence of quality, but cannot ensure it.  I still think that’s true, but that insight doesn’t guide what or how I should test.  It fails to inspire me.  It’s accurate, but not helpful. Continue reading Questions and Answers

“You can observe a lot by just watching” — Yogi Berra

I’d just joined a newly formed software team. After a kickoff meeting, we all headed out to eat. “When did you get back from the East Coast?” I asked one of my new coworkers while we were browsing the menu.

“Huh?”

His puzzlement was understandable. We’d just met. I knew nothing about him. I didn’t actually know that he’d been on the East Coast. But I did know that 1) he worked for a consulting firm and traveled quite a bit, 2) that the newly-formed team was having lunch in San Francisco, and 3) that his watch was set for three hours later, East Coast time. I explained my observations and conclusion.

He looked stunned. “Yesterday,” he mumbled. Continue reading Noticing the Little Things

Exploratory Testing is a style of testing in which you explore the software while simultaneously designing and executing tests, using feedback from the last test to inform the next. Exploratory Testing helps us find surprises, implications of interactions that no one ever considered, and misunderstandings about what the software is supposed to do. Cem Kaner . . . → Read More: Rigorous Exploratory Testing