On the toolsmith-guild discussion forum, Russ DeWolfe asked me what problem I was trying to solve with all this next generation tools talk.
Russ brings up a good point. I talked a lot about what I want, but not about the core problem that I’d like to see solved. So here’s what I hope a next generation of functional tools could do.
JUnit revolutionized unit testing by creating an incredibly simple yet powerful model: setup/test/teardown. Further, it provided a bare bones framework for implementing and executing your tests. In the past, a developer wanting to automate unit testing would have to write some kind of harness, would have to think about results reporting, and would have to think through how to structure the tests. JUnit simplified all that and allowed the developer to focus on just writing one test at a time. The result: a developer can write a unit tests within minutes. Thus JUnit significantly lowered the barrier to automated unit tests.
Fully automated functional testing still has numerous barriers. And Agile teams that want to integrate automated acceptance testing with their continuous integration process need fully automated tests.
Watir is an excellent example of something that helps break through barriers…it’s possible to create a lot of tests very quickly with Watir.
But after a while it becomes difficult to manage those tests. And reporting is still a challenge. And the business people who must agree that the automated tests reflect their acceptance criteria have a hard time (sometimes) understanding what all that Ruby code is doing.
Various people have tried various structures including table driven things (like FIT/FITnesse) and Domain Specific Languages. But none of these approaches has quite succeeded in transforming the state of the practice the way JUnit did for unit testing.
So I’d like a thing, or more likely, a collection of things, that break down the barriers to effective, fully automated end-to-end tests in a lightweight way, like what JUnit accomplished. I’d like it to be general enough that it could be used anywhere. But like JUnit, I’d like to see that general applicability achieved through simplicity rather than through gold plating, tack-another-feature-on-the-beast.