So You're Trying to Automate Tests for a Legacy Web Application…

…and it isn’t cooperating?

I sympathize.

I recently fought my way through the process of automating a test to reproduce a bug on a legacy(*) web application that had no IDs on any of the elements that I wanted to address. And I thought it might be helpful to capture some of my lessons learned here in case they help someone else. Also, I’m putting this here so I remember what I did.

Lesson 1: SeleniumRC Rocks!

I’ve been extolling the virtues of SeleniumRC for quite a while. This project gave me the perfect opportunity to refresh my Selenium skills. And I’m delighted to report that SeleniumRC is even better than I remember it being. First, I can write my tests in my programming language of choice (Ruby). Second, it has a wide variety of ways to locate these pesky non-ID’d elements. Third, every time I ran up against a road block, I discovered that the smart folks who make Selenium had already anticipated the problem and found a solution.

Lesson 2: Selenium Server Flags can Solve Common Execution Problems

In my particular case, the app I was testing didn’t play nicely in IFrames. This is a problem: by default Selenium runs the web app in an IFrame in the same browser window where it displays its own status. Fortunately, it turns out that there is a -multiWindow flag to solve exactly this problem. I solved the IFrame problem by running the Selenium Server like so:

java -jar selenium-server.jar -multiWindow

There are a variety of other Selenium Server flags that address other common problems. See the Server Command Line Options documentation for a full list.

Lesson 3: ‘Permission Denied’ Errors Probably Mean the App Violates the ‘Same Origin Policy’

Once I’d gotten to the point where I could launch the app, I started encountering very puzzling Permission Denied errors. I vaguely recalled that such errors probably meant there was some problem with the domain names changing and browser security and cross-site scripting something-or-other.

So I checked the domains. Sure enough, the home page was at “www.example.com.” From there, when you log in, it goes to “app.example.com.” Bingo! The domain was changing in the middle of my test. I experimented a little and discovered there was no way around it: the app was going to redirect to a different domain no matter what.

It turns out I’m not the first person to have this problem. Fortunately, Selenium has a strategy for addressing the issue: experimental browsers. I tried the chrome browser for testing on FireFox and it worked perfectly.

Lesson 4: Firefox Rocks!

At this point I could launch the app and log in, but now I had another problem. After the login page, all the things I needed to click, check, or otherwise manipulate were buried deep in convoluted HTML. I realized that figuring out how to address these things was going to be non-trivial.

The most basic strategy for discovering the locator for an element is to view the HTML source. You can view the source for the whole page, but Firefox has a great feature that allows you to see just the source for just a selection. To use it: highlight a selection on the page, then right-click. One of the available menu options is View Selection Source. Choose it, and you get a window with just the relevant HTML.

However, if you’re dealing with something complex, viewing the source isn’t enough. You really need to look at the Document Object Model (DOM). The best way I know to do that is with the DOM Viewer included with the Web Developer plugin by Chris Pederick.

Web Developer also includes a feature that lets you see all the attributes for a given element. From the Information menu, choose “Display Element Information.” Now you can get the attributes for any element just by clicking on it. I love that feature.

Finally, the XPath Checker plugin by Brian Slesinsky is a very helpful tool for figuring out how to address those pesky non-ID’d elements. More on xpath in the next section.

Lesson 5: xpath Is Now My Good Friend

One of the hallmarks of legacy web apps is the annoying lack of IDs on important elements. Of course, web apps aren’t the first place where a lack of IDs is problematical. I recall struggling with Windows apps that lacked field IDs back in the 1990s.

The good news is that this is a much more tractable problem in web apps than in Windows apps. Xpath to the rescue!

Let’s take just one example. I needed to verify the text associated with a particular image. The image served as a kind of custom bullet in a bulleted list, so there were several identical ones. The text itself did not appear in the DOM immediately next to the graphic. Rather, its parent was a peer to the parent element of the image. (Got that straight? Yeah, me either. Seriously, this one took me a while.) In brief, the HTML around this thing looked kinda like this:

<div>
    <span>
        <img src='/path/to/images/check.gif'>
    <span>
    <font>
        item 1
    </font>
</div>
<div>
    <span>
        <img src='/path/to/images/check.gif'>
    <span>
    <font>
        item 2
    </font>
</div>

Mind you, the HTML didn’t look that clean. There was a lot of other random stuff in there, and every tag had a gazillion attributes, and there were hard-coded styles everywhere. But I digress. And I’m probably whining. I’ll stop that now.

So it turns out the only way I could grab the text associated with the second bullet in the list was with the following Selenium command:

get_text("xpath=(//img[contains(@src,'check.gif')])[2]/../../font")

Let’s all say it as a group: “EWWWWW!”

But let’s also appreciate that doing such a thing is actually possible. Selenium has a wide range of locator styles, and even allows you to add your own locator strategies. (I haven’t needed to do that yet, so I’m not quite sure how to use the feature, but I noticed from the documentation that it’s there.)

(For more on using xpaths with Selenium, I found the Help with XPath article on the openqa.org site useful. It’s hard not to like an article with headings like “How the $%^@$ do I locate an element?”)

Mind you, the world might be a better place if we couldn’t write such code. Every time I figure out how to automate tests against an untestable application, I feel a twinge of guilt. By automating tests against an untestable interface, I become an enabler of more untestable interfaces. For the sake of improved collaboration and more testable applications, perhaps it’s better if those of us who automate tests avoid resorting to using our xpath superpowers except in the service of wrapping legacy apps with tests so they can be refactored for testability.

But once again, I digress.

The point I really want to make is that SeleniumRC gives us the power to automate tests even against icky hard-to-test legacy apps, and to do it with real programming languages (pick your favorite: C#, VB.Net, Perl, PHP, Ruby, Java). And that means we can write maintainable automated tests using good programming practices. And *that* means we can automate regression tests for faster feedback. And ultimately, *that* means we can make changes to the legacy app to improve testability and maintainability.

So rock on, SeleniumRC. And huge thanks to everyone who’s ever worked on SeleniumRC or Selenium. Also, huge thanks to the Selenium community as a whole. When I went looking for answers to my questions I found numerous blog posts and forum messages with tips and tricks. This post would not have been possible without such a community that’s so open about sharing knowledge.

 

 

 

 

* Here I mean “Legacy” as Michael Feathers defines it: code that lacks automated unit tests. Web developers, please take note: if you write good unit tests for your web app, including JSUnit tests for the JavaScript bits, the application will be MUCH more testable and the QA people will stop whining at you so much. return to footnote reference

Subscribe

Subscribe to our e-mail newsletter to receive updates.

12 Responses to So You're Trying to Automate Tests for a Legacy Web Application…

  1. Kevin Lawrence May 15, 2008 at 3:45 pm #

    The Web Developer extension is quite magnificent. I have been using it for a couple of years.

    But for super magnificent, you should be sure to give Firebug a try.

  2. Chris McMahon May 15, 2008 at 8:32 pm #

    The Selenium IDE is pretty awesome too. It’s saved my bacon any number of times because it tells you exactly all the ways a particular element may be addressed– including the XPath if that’s the only option.

  3. Andy Tinkham May 15, 2008 at 8:54 pm #

    I second the vote for trying FireBug. It’s a beautiful plugin. Add YSlow (the Yahoo plugin to Firebug) and you can get some basic performance benchmarking too.

    Also, are you using a framework to handle your tests? I’ve got tests that use RSpec and tests that use Test::Unit, and a clear favorite has not yet emerged. Some tests seem to lead me to xunit syntax and some to the should syntax. Haven’t tried the story format of RSpec or shoulda yet, though I’d like to (and I finally get a second tester, so maybe I’ll even get a chance to do so soon…)

  4. Jean-Noël Rouvignac May 16, 2008 at 12:39 am #

    Chris is right: Selenium IDE is your friend in getting to the point where you are ready to write your unit tests. The methodology to follow is the following:
    1- record with Selenium IDE the browsing you want to automate (that will generate locators automatically for you, using Ids over xpath).
    2- export your selenese script into your favourite programming language (you can somewhat tune the export code template).
    3- paste that code into the unit test you want to write (In a maitainable way) and add the missins bits.
    4- execute. TADA!

    Of course the code generated is not perfect. However, that helps you a lot by doing the big, uninviting work and leaving to you the smaller details. For example, I remember Selenium IDE did not record clicks on a Javascript calendar. This was easy to go round by using the input field associated with this calendar. There are some minor discrepancies between the selenese script you recorded and the export in your favourite language: ClickAndWait became Click and the Wait disappeared. But this is easy to notice and fix.

    Firebug is another great tool I use more than the Web developers toolbar. With Firebug, you can “Inspect” any element on your web page and look or modify its content on the fly. This is great for understanding what is going on, which DOM attribute you need to change or even testing something before actually modifying the application code. I did some prototyping on web pages by modifying the content of a web page (changing text, adding / deleting elements, nearly everything), before actually delivering it in the code. It is a Javascript debugger as well. However, mastering Firebug is not always easy, but you can start small and learn little by little.

  5. Shrini Kulkarni May 16, 2008 at 5:06 am #

    Lesson 5: Please work with Developers and have them to add some GUI related information.

    Typical web GUI/HTML/CSS and other related development practices do not seem to consider “automation requirements”.

    Web GUI controls like Table, CheckBoxes, buttons, List boxes typically do come with identification information. Since these are not visible on the GUI and developers least care about it. When I asked developers – they said, it is not a standard practice to include ID, Names etc in GUI elements.

    Consider a web form with several OK buttons each for a specifi purposes but has one label “OK”. Automation tool gets confused in identifying this button.

    In some automation projects, I have worked with developers and have them add code with identification names/labels so that automation easier.

    In one cases, there used be several popups with no window title. I worked with developers and made them to add unique window titles ….

    hence the Lesson …

    EH, I am glad that you are picking up those issues that are faced in IT automation space where automation happens at GUI level. When working with automation of Web Apps – Testability has always been the issue. Keep blogging more on these lines.

    Shrini

  6. Speedmaster May 17, 2008 at 3:12 pm #

    I’m a longtime WinRunner user, we’re just making the switch to QTP. I’m a bit behind on it as I’ve been doing a lot of LoadRunner work lately. But I’ll take a good look at Selenium. Thanks for the heads-up!

  7. gunn May 22, 2008 at 12:21 pm #

    Hi..

    I’ve hitched on the Selenium bandwagon recently and I’ve been doing all the reading I can about it. I use Selenium IDE and my company does not use RC or Core yet..

    Now, somebody .. pleeeese pleeeess pleaaaase tell me how to do the following things –

    1) data drive/parameterize my tests in Selenium *IDE* (not core, not RC.. but IDE)
    2) report test results, collate and consolidate them in some form I can present to my supervisor.

    Are there already extensions somebody has written to accomplish the above. Or is there some simple but winded way to accomplish it? Or is there some not winded but quite painful way to do it? PLeeeeeeeeeeeeeese help.

    I need this information badly.. Elisabeth dear.. would you consider writing a definitive guide to accomplishing the above tasks.. I dont care how winded or complicated it is.. I just want to be able to parameterize my tests in Selenium *IDE*. Please please please help.

    Elisabeth replies…

    I don’t use the IDE, but perhaps one of the people who does who commented here could address the question. Chris? Jean-Noël? So instead of answering your question directly, I’ll answer it indirectly. My advice would be to use something like FIT/FITnesse with a fixture that uses SeleniumRC. That will take care of parameterizing/data driving tests. It can do reports too. Or use the reports from a CI tool like CruiseControl.

  8. Martin Braun May 22, 2008 at 3:10 pm #

    Hi,

    (in reply to gunn)
    i really don’t think Selenium IDE and the Selenese “language” is the right way to go when doing real test automation. The IDE is a very nice tool to do a quick demo about the basic stuff you can do with selenium, but as soon as it comes to data driven tests, reporting, continuous testing and all the stuff which makes testing more valuable, IDE is no option any more.

    Simple things (like Selenium IDE) are great for simple tasks, but it’s doomed to fail when it comes to more complex testing or integration to $you_favorite_tool. We also played around with the IDE at the start but realized that it is not powerful enough – hey, if it would that simple all the big testing vendors would have burned billions in developing their complex toolchains…

    In my opinion, a effective, reliable and valuable testing tool is a mix of a framework which removes the hickups between the test runner (xUnit/Selenium RC) and the application under test and a real powerful programming language backend like java or ruby to provide a solid and useable interface for testcase development.

    Anyway, Selenium IDE will not ne able to provide the intelligence of a Selenium RC with a powerful framework. “Click, click, click, run = test done” may be work of the most simple test-cases but not for complex stuff that’s required in many web applications.

  9. Ruslan Strazhnyk May 23, 2008 at 8:20 am #

    I am using Selenium nearly a year, but recently i’ve discovered that “iframes” is really a problem. soulution with multiwindow didn’t pass :(((.
    I don’t know what to do(.


    Elisabeth replies…

    Can you say more about what didn’t work about multiwindow? It worked great for me…

  10. Ruslan Strazhnyk May 23, 2008 at 8:56 am #

    Hi Elisabeth! I’ve read your solution about adding -multiWindow when start selenium server, it runs now in two windows, but html source, which selenium parses is still like that.

    \n \n Problem displaying iframes\n \n.

    You see, in this frame there is email message displayed, but i can’t tell you the name of the app that I am testing.

    So, I can consider this as a bug in Selenium RC.

  11. gunn May 23, 2008 at 10:21 am #

    Thanks all for your replies. So I gather that IDE cannot to what I want to do and that RC is perhaps my best bet.

    So then, can somebody please point me to or write me a “Definitive Guide to Selenium RC” which includes info atleast about “HOW TO” –

    a) Write a simple automated test.
    b) Add checkpoints, breakpoints
    c) Data driving
    d) Error handling
    e) Reporting

    Also, Elisabeth please elaborate on that Fit/Fitnesse solution you mentioned… preferably in a seperate blog post/section dedicated to it.

    Thanks in advance.

  12. Jean-Noël Rouvignac June 4, 2008 at 2:44 am #

    Sorry gunn,
    I do not have a definite answer for all your questions. You will have to try to find it by yourself. I know this takes time, but the reward is probably high enough for you to invest it.
    About your questions:
    1) I do not know how to “data drive/parameterize my tests in Selenium *IDE*”. Selenium RC can do that provided you build your own framework for it. Advatage: it can be as flexible as you want. Disadvantage, unless you find something already done, this takes time.
    2) “report test results, collate and consolidate them in some form I can present to my supervisor.”. This is something you can achieve by writting your Selenium RC Tests using xUnit-like testing frameworks that generates reports of the tests runs.

    So my very simple advice for a very minimal “HOW TO” are:
    a) Write a simple automated test. => Use xUnit testing tools (and resolve all the setup problems you have at the beginning)
    b) Add checkpoints, breakpoints
    => Do you mean debugging the tests? For example, if you write your tests in Java you can run them through Eclipse and use Eclipse debugging capabilities very easily.
    => Do you mean writing maitainable tests? This is classical code maintenance. Ask any good developer for this.
    c) Data driving
    => I do not know how to do that. I never did. It seems to me what Elisabeth said makes a lot of sense.
    d) Error handling
    => Do you mean when tests fail? This is given for free by xUnit testing frameworks.
    e) Reporting
    => Should be part of a good xUnit testing frameworks, like JUnit or TestNG in the Java world.