Some years ago I had lunch with a QA specialist who invited me to visit him at work. He wanted to show off how he had used a macro recorder to automate his testing. Over lunch I offered the opinion that test automation is a programming activity. The QA specialist vehemently disagreed with me.
“I don’t need to program to automate my tests!” he said, waving his fork. “And I don’t want to program!” His fork punctuated his words. “All those FOR loops. Never could get the hang of FOR loops. And what’s the point anyway!” The fork was becoming dangerous. I considered ducking.
I couldn’t help but notice that he seemed angry that FOR loops were too complicated, and yet he was trying to automate tests. I haven’t visited that company again and I have no idea how they’re doing. But that guy? He scared me. No, it wasn’t the fork that scared me. It was the attitude about programming and automated tests.
Not Everyone Has to Code
I am often asked whether QA/Test people need to know how to program.
I always answer the same: QA/Test people do not need to know how to program, even on Agile projects. Everyone on a team brings something to the table. QA/Test people my not bring programming skills, but that’s OK, they don’t need to. The programmers already have that covered. Rather, QA/Test people bring testing skills and analysis skills and critical thinking skills and communication skills. Some bring other non-programming technical skills like database admin or network admin or system admin skills. And some do bring programming skills, and that’s great too. Whatever their skills, everyone brings something to the table.
And non-programming testers can collaborate very effectively with programmers to create awesome test automation (just ask Lisa Crispin).
But someone who is scared of FOR loops doing the coding part of test automation in isolation by recording and playing back stuff? Seems to me like that’s a good way for everyone on the project to waste a huge amount of time and become very frustrated.
Is Record-and-Playback a Solution to the Wrong Problem?
I’ve been thinking about that guy today as I’ve been thinking about record-and-playback test automation. I have had several conversations about record-and-playback test automation over the last few months with a wide variety of people.
Some people have said, “Yes, we discovered that what you are saying is true. So while we may start by recording a script, we modify it so much that it is not recognizable as a recorded script when we’re done.”
Others have said, “I only use the record-and-playback for quick and dirty test-assistance scripts. I know it can’t create real, robust test automation.”
Still others – usually vendors – have said, “Yes, I know you don’t like record-and-playback. But others do, and they want that capability. They want to automate their tests without programming.”
So individual contributors generally recognize that record-and-playback can be a helpful stepping stone but it is not a strategy. Yet at least some vendors still are very attached to record-and-playback, and customers apparently still demand it.
I wonder if record-and-playback is an attempt to solve the problem that QA/Test groups think they have rather than the real, underlying problem? It seems to me that the reasoning usually goes something like this:
- We need automated tests to provide fast feedback.
- Someone needs to write those tests.
- It has the word “test” in it, so it must belong to the QA/Test group.
- The QA/Test group doesn’t have much in the way of programming skills.
Therefore, the reasoning concludes, we must need something that will allow QA/Test groups to automate tests without knowing how to program. Or we must make all QA/Test people learn to code. That second thing isn’t gonna happen, so we need a tool to take care of the first solution.
The problem is that item #3 in the list is a total fallacy. Just because something has the word “test” in it does not automatically mean that it should be assigned to a designated, independent tester. If we get rid of the original constraint #3, we simplify the problem and open a world of alternative solutions. So let’s state the situation instead as:
- We need automated tests to provide fast feedback.
- Someone needs to write those tests.
- The QA/Test group may not have much in the way of programming skills.
So the people who are really good at coding can do the programming part of automating tests, and the non-programming testers can collaborate with them to make sure the automated tests are useful and effective. And I do mean collaborate. Not “hand off a specification to…”, not “make a request of…”, and certainly not “supervise.” I mean collaborate as in work together on, at the same time.
Worse, is Record-and-Playback a Substitute for the Real Solution?
So if the reason customers demand record-and-playback capability in test automation tools is that it enables people who don’t know how to code to automate tests, it makes me wonder why they’re making non-programmers do programming work.
The most common reason I hear from QA/Test people is that the programmers won’t automate tests, so the testers have to do it. The most common reason I hear from the programmers is that they don’t have time, and besides, there is a whole QA/Test group assigned to that kind of work.
But it seems to me like the real issue here is that we’re trying to use a tool to as a substitute for something else. We’re using it as an alternative to real collaboration.
So now when I hear someone tell me that they’re using record-and-playback a lot in their test automation strategy, it suggests that perhaps the test effort and development effort aren’t integrated, that the organization is still operating in silos, and that we need to work on breaking down barriers to collaboration before we start talking seriously about tools.