In case you haven’t noticed, there’s been a big shift in the world of system-level test automation over the last few years. These free/open source tools are rapidly gaining traction:
What’s happening? Why are open source frameworks gaining such popularity?
Perhaps this trend started as a result of dissatisfaction with the big commercial test automation tools. In the late 1990’s, the heavyweight specialized test automation vendors positioned their Record-and-Playback tools as a way of enabling non-technical testers to leverage testing expertise into automation.
There was only one problem: Record-and-Playback didn’t work.
Wait. That’s not fair. The tools “worked” in that it was possible to record a script and play it back. Usually. Unless the recorder failed to capture a mouse gesture or two.
But even when the tools “worked,” they were painful and produced totally unmaintainable tests. This was not the silver bullet the vendors had promised. Many organizations discovered that the cure of test automation was worse than the disease of manual testing. The ones that succeeded did so because they realized that no matter what a vendor promises, test automation is a form of programming and test scripts are code.
But the commercial tools didn’t just cause testers pain: they divided organizations. People who considered themselves Real Programmers shunned the specialized commercial tools, leaving the tools to the ghetto backwater of disenfranchised QA teams. The result was predictably tragic: the teams that adopted commercial test automation solutions had to fight everyone and everything in their organization to keep their scripts alive. They had to fight for improved testability in the software under test; they had to fight against changes that would break the automated scripts; they had to fight their management to get budget for sufficient licenses and training; they had to fight for time and resources to maintain the existing scripts; and they had to fight the tools to get useful results. Too much fighting, not enough productive working.
Some individuals began creating free alternatives to commercial solutions. One such alternative that remains popular is Win32::GUI, a Perl module that allows programmers to drive Win32 GUI interfaces. Another is a Perl module for driving Web applications: WWW::Mechanize.
But just having free drivers available didn’t result in a fundamental shift in the industry. That’s because a comprehensive test automation solution requires more than just a way to drive the application under test (as I explain in my post “Do We Need Specialized Test Automation Tools?”).
I think the current trend in open source tools is due to:
- Key people in the industry who have generously contributed their time, effort, and insight, including Kent Beck, Ward Cunningham, Rick Mugridge, Bob Martin, Micah Martin, Bret Pettichord, lots of Thoughtworks people, and a bunch of folks I’m forgetting to mention.
- The combined effect of open source testing and continuous integration tools.
- The increased adoption of Agile practices encouraging automated testing at all levels in a project.
The result is that the latest crop of open source test automation frameworks are better than ever. Teams are discovering that there are viable alternatives to buying a high-priced specialized tool that divides the team. Instead, they’re discovering that the open source frameworks foster collaboration between programmers and testers and subject matter experts. And that collaboration is helping to make teams more adaptable.
So, what does this all mean to the commercial world of test automation tool vendors? Be afraid. Be very afraid.