Better Automating
November 5th, 2006
Filed under Test Automation
Testers often ask me how they can learn how to automate tests or improve their test automation skills. Here’s what I tell them:
Do you know how to program in any language? If not, start there.
When I say “any language,” I really mean any language. Writing Perl, VBScript, or even DOS batch files or Shell scripts counts. Cobol, Fortran, Ada, or even JCL: they count too. More modern languages like C++, Java, and Ruby, are handy, but any programming experience you have will help you. Your school experience counts even if you think you don’t remember any of it.
The point is that if you have any programming experience at all, you probably understand basic control-of-flow constructs (for, if, else, case/switch statements), structured programming techniques such as creating reusable functions and passing parameters, and maybe even object-oriented concepts. That’s an important start.
If you have a bit more programming experience, you might understand the concepts of encapsulation, coupling, and cohesion. These concepts become important when you want to make your test automation more maintainable.
If you don’t have any programming experience at all, start growing your test automation skills by learning to program. Ruby is a fabulous language to start with for several reasons:
- It’s free (see http://www.ruby-lang.org/en/).
- It’s an interpreted language, so it’s easy to get started.
- Ruby syntax allows you to write code that reads like English.
- There’s an extensive community and plenty of resources to support newbies. Check out Chris Pine’s book Learn to Program and Brian Marick’s forthcoming Everyday Scripting with Ruby.
- The Watir framework allows you to put your new programming skills to work right away on test automation.
Do you have any experience at all with test automation? If not, start writing automated tests.
Having experience with test automation, whether with a commercial tool, a roll-your-own test harness, or open source frameworks, gives you experience with the pitfalls of test automation. When you start writing automated tests, you discover how it’s easier to write fragile tests that don’t provide useful information than it is to create reliable, maintainable tests that tell you something important about the software under test.
Watir is a great framework to start with. Even if the software you’re testing isn’t web-based, use Watir to write tests against a random website. The point is to gain practice automating tests. You’ll quickly discover the pain of hard coded data or object identifiers, and will just as quickly discover ways to make your scripts more maintainable.
Alternatively, you could start testing your real application using Ruby, Perl, VB, or Shell scripts. All you need to write tests against your application is a scripting language and an interface to your app. The interface could be a user interface, an http interface, a SOAP/Web services interface, a COM interface, or a programmatic (API) interface. If you can script it, you can automate tests against it.
The first program most people write in any given language is a “hello, world” program: it outputs something. The “hello, world” equivalent for an automated test performs some simple manipulation of the application under test, verifies something, and reports a result. Perhaps it logs in, clicks a link, or pushes a button, then verifies that the resulting page has the expected title. Whatever you choose, make sure it’s something simple.
From there you can expand: make your scripts automate tedious tasks like data entry or verifying attributes for elements on a page.
Got both experience with programming and test automation? Apply good programming practices to your test automation.
I gave a test automation demonstration at a conference a while back. I showed how to make a long, messy script more readable and maintainable using common refactorings (extracting methods, renaming, parameterizing). One person in the audience commented, “Wait a minute. You’re just showing us good programming practices.” Well, um, yes. That’s the point.
To be really good at test automation, you have to be good at programming, and use what you know about programming to make your automated tests better programs. For example, look for opportunities to:
- Encapsulate changing interfaces. The reason test automation tends to be fragile is that whenever the interface against which the tests are written changes, the tests must change. Encapsulating the parts of the automation vulnerable to those changes keeps the maintenance overhead down to a minimum. (Another way of saying “encapsulate” is “build an abstract layer.” Commercial tools do this: Segue Silk has the notion of a test frame while Mercury’s tools have a GUI map. You don’t need a commercial tool to do this: any scripting language will support this kind of structure.)
- Extract common sequences into reusable functions or methods. For example, instead of repeating all the commands for entering the user name and password and for clicking the “login” button, create a function or method called something like “login_as” that takes two parameters: username, and password. The result: if the login screen changes, you only need to update one place, the login function, instead of updating every script that requires a login.
- Separate the data and the logic. Another reason test automation tends to be fragile is that it mushes together data and logic. The result is that every time the application flow changes, the scripts all need to change. Put the logic in one place and the data in another. The Fit and Fitnesse frameworks (and other automation frameworks that borrow ideas from Fit, like Selenium) put all the data into tables.
You’ve done all this and still want ideas to improve?
Two more suggestions:
- Continue honing your testing skills. No matter how long you’ve been testing, no matter how much you know, there is always more you can learn.
- Make your automated tests smarter. See Harry Robinson’s model-based-testing.org site for some ideas on how to do that.
And if you’ve done all this, you’re a rare bird indeed. I hope you’ll consider reaching out to the broader community through local interest groups, conferences, and writing.
5 Comments
Leave a Comment
Because of the rise in blog-spam, I've turned on comment moderation. If it takes a while before your comment appears, I hope you understand.
Moderation Policy: I approve substantive comments. I reject ads. And if I don't know whether it's substantive or advertising, it sits in my moderation queue until I get sick of looking at it, at which point I reject it, kind of like the questionable meatloaf in the fridge. But please be assured that I think long and hard before clicking that reject link. I really am grateful for every comment any human takes the time to make. (Spambots, not so much. But if you're reading this, you're probably human.) So please contribute to the conversation...
Nov 07, 2006
5:53 pm
With Selenium RC, your tests don’t have to written in FIT-like HTML tables. In fact, we officially support support Ruby! And with Ruby + Selenium RC, you can test IE, Firefox, and Safari on Windows, Mac, or Linux.
Nov 07, 2006
6:06 pm
Hi Jason,
Thanks for pointing that out! I’ve only used the JavaScript flavor of Selenium (and found it fabulous for automating tests for Web 2.0 apps). Some day I’ll find time to try automating the same tests in Ruby…
Elisabeth
Nov 08, 2006
5:53 pm
Hey, no problem.
Lots of folks first learn of and only know of Selenium though the lense of the “table-style” tests– it’s the simplest to learn and setup. Selenium Remote Control (RC), however, is more complex, but once setup, is quite powerful, since you’re using a “real” language, instead of Selenium’s table-style test “macro” language. Also, having the additional language support in Selenium of the “big” languages (C# and Java) can’t be underestimated… I’ve seen communication become easier when the developers and testers write with the same programming language.
On the Ruby front, the big difference you’ll see between Watir/Ruby tests and Selenium RC/Ruby tests is a noticable object vs functional emphasis in the APIs. For example, with Watir, to click a button, you’d say “ie.button(:id, ’submit’).click”. With Selenium, you’d say “selenium.click(’submit’)”. Deciding between the two is a matter of personal preference, though one of my wishful features is to also support a Watir-style API for those who want it. Probably sometime after version 1.0, though.
Also, you’ll find the setup and install of Watir is a far nicer experience than setting up Selenium RC. But we’re working on that. 
Nov 19, 2007
9:25 am
Nice article. Please add a “Printview” or “Print to PDF” button to your blog. I like to share this with my team.
Thanks,
Allan
Nov 19, 2007
10:07 am
Hi Allan,
Thanks for your suggestion. I’m delighted that you want to share these articles with your team!
I think the best way to share my articles is by sharing the URL to the original article.
However, if the only way you can share the articles is by printing, or printing to a file, the browser print option produces something that has the right width for printing even if it does have a bunch of extraneous stuff (like the search field and blog roll) that make it look a little ugly. If I made it not print the extraneous bits, would that come close to what you want, or were you looking for something even fancier?
Thanks,
Elisabeth