In a comment on my last post, Shrini asked:
“While hitting hard at ‘Jargon’ based software testing experts, you also appear to give the impression that ‘testing’ is ‘everyone’s job’ (as quality) and seem to dilute the importance and role of testing in software world – you might want to clarify. Don’t you think that you can hit at these jargon creators without diluting the role of testing?”
Yes, thank you Shrini. I appreciate your question, and I would like to clarify.
I do believe that testing is so crucial that it is everyone’s job.
And I also believe that software development teams need people who have taken the time to become really good at testing.
I believe that these two ideas are compatible. Let me ‘splain.
On more than one occasion I have worked with software development teams that were under such time pressure to write code, that they felt that they had no time available to test. They left the testing to a designated team of independent testers. The result in each case was code that was so bug-riddled that stabilizing it took months. During these long months of playing whack-a-bug, the testers and developers were constantly battling, management was perpetually screaming, and no one was happy. Predictably, even when the software did ship, it was prone to failure. In weekly meetings that followed the release, Tech Support let us all know that the quality was unacceptable and that the customers were cranky. Subsequent new development efforts were hampered by the need to patch critical bugs in the field. We were in technical debt up to our eyeballs.
(At this point, I’m guessing that at least one person I have never met in person is reading this and thinking “Wow! She worked here!” I probably didn’t; it’s an all-too-universal experience. Kinda like Dilbert.)
These experiences led me to write “Better Testing, Worse Quality,” a paper that explores the system effects that lead improvements in testing to yield even more fragile software.
And then I started working with Extreme Programming teams and discovered what I’d been missing. I experienced how Test Driven Development and Continuous Integration and Collective Code Ownership and Paired Programming and continuous Refactoring led to a solid code base. Project after project, I observed that the XP teams I was working with were achieving significantly better results than the code-and-fix teams of my past. Oh, the XP projects weren’t perfect. We still had bugs. But we didn’t play whack-a-bug for months on software that was supposed to be done-except-for-testing. We didn’t need stabilization phases. The software was always stable. It might not do everything yet, but what it did, it did well.
I also witnessed the power of Customer Acceptance Testing. I realized that the people responsible for defining the requirements are in the best position to determine whether or not the implementation matches their vision.
But at the same time, I recognized that I had something to offer the team. Compared to the professional programmers, my programming skills were meager. Compared to the product managers, my understanding of the product vision was superficial. But compared to any of the other team members, my understanding of where the risks and vulnerabilities were likely to hide was superb.
I used my skills to provide feedback to the team, to point out implementation bugs and requirements ambiguities and risks. I explored the implementation and reported what I found back to the team. I thought up tests no one else had thought up before.
But I did all these things in collaboration with the team. We pooled our knowledge. And we shared responsibility for testing activities. Testing became an integral part of the software development process. We could no more separate the development and testing activities than we could have separated our hands from our bodies, leaving our hands to type on the keyboard while the rest of our bodies took a break.
Perhaps paradoxically, spreading responsibility for testing to the whole team increased the overall test effectiveness. The testing mindset became pervasive. The test effort wasn’t diluted; instead it grew and flourished.
(I could write bad prose with corny analogies involving a comparison between diluting a drop of colored dye in water and spreading seeds in the wind. I could point out that inanimate objects become weak or disintegrate when spread, while living things – like ideas and knowledge – grow. But I won’t. Oh dret. I just did. Um, never mind.)
So anyway…bad analogies aside…
I do believe that everyone is responsible for testing, even on teams that don’t claim to be doing Agile or XP. The developers test that the code does what they expect and intend, and that new changes don’t break existing expectations. The business stakeholders test that the implementation is what they had in mind. Testers – those who have studied testing long enough and hard enough to get good at it – bring specialized skills to the table to support the rest of the team.
Testers apply critical thinking skills and analytical abilities, coming up with new questions to ask the software. “What if a user does this after that?” we ask. “What if the system is in this state when that happens? What if the data looks like this? What if we configure it like that?”
But testers are most effective when they do this with, not for, the team.