Flush Specific Stack Fiercely

Random Scenario Generation: a Tester Creativity Enhancer

Want to try randomly generated scenarios on your software?

Click the button below.

What is this nonsense? A bad translation, like the urban legend involving the name “Coca-Cola” being translated into Chinese as “bite the wax tadpole?” No, I didn’t get this phrase from some poorly translated marketing slogan. Could it be a shameless attempt to create an eye catching title? Perhaps, but that’s not the point.

Actually, this phrase is a randomly generated step for a test scenario.

Any moderately complex software application has so many features that might interact in unexpected ways that we can’t possibly cover all the combinations. Randomly generated scenarios can help nudge us out of a rut and prompt us to test combinations we might not otherwise think about.

If you’re eager to start using the technique now, try the random scenario generator at the right. Then read this column to understand how something like “Flush Specific Stack Fiercely” can inspire a great test case.

A Simple Example of Complexity

Consider an imaginary word processor that handles just four types of elements: plain text, tables, lists, and images. These elements can combine as follows:

  • Text may have embedded tables, lists, and images.
  • Lists may have embedded images.
  • Tables have 0 or more cells
  • Table cells may contain nothing or may contain any combination of text, images, and lists

Notice that in the interest of providing a simple example, we’re ignoring fonts, indenting, and other niceties that real word processors provide.

Now let’s consider the interactions. A document created in this imaginary word processor might contain:

  • Nothing
  • A single element: text, table, list, or image
  • Lots of images
  • Lots of text containing lots of images
  • A list embedded within text
  • A table embedded within text where the cells each contain something different: nothing, text, a list, an image, text with a list, text with an image, a list with an image, etc.

These are just a few examples. There are numerous other possibilities. I counted over 100 before I decided to stop counting. And I was only counting permutations of document contents, not all the user actions that went into creating the document. And this is a simplistic example. The software we all test on a regular basis is far more complex.

So if the software we’re testing is even more complex than this example, how can we possibly test all the possible feature interactions?

We can’t.

In some software we can’t even predict all the ways the features could combine. So let’s try a reframe. Instead of attempting to test all interactions, let’s focus on increasing the probability that if there’s an interaction problem, we’ll find it.

Random Story Generation to the Rescue!

This deserves repeating: our goal is to increase the probability that if there’s an interaction problem we’ll find it.

If we do the same things over and over again, we’ll discover the same information over and over again. That’s boring. Worse, it does nothing to increase the probability of finding an interaction problem. So let’s not rerun the same scenarios ad nauseum. Instead, let’s increase the probability of spotting a problem by exercising the software in more, different ways.

It’s not always easy to come up with new and different scenarios. Fortunately, randomly generated scenarios can spur our creativity. Take the nouns and verbs that describe objects and actions in your system. Add in adjectives to modify the nouns and adverbs to describe how a user interacts with the application. Mix randomly. Then use your imagination to turn a nonsensical sentence into something you might execute.

Let’s try this with the imaginary word processor example. Our nouns include Document, Text, Table, Cell, List, and Image. Our verbs include Save, Insert, and Delete. Adjectives might include Long, Short, Wordy, and Terse. Adverbs could include Frequently, Repeatedly, and Quickly. By choosing combinations of verbs, adjectives, nouns, and adverbs randomly from these word lists, I get:

  1. Insert Long Document Repeatedly
  2. Delete Wordy Table Quickly
  3. Save Terse Text Frequently

Next we’ll transform these nonsensical phrases from gibberish into something we could actually do.

How might we interpret “Insert Long Document Repeatedly?” First we need to create a document into which we might insert. If we imagine we’ve created a new document, then we can open another document, copy its contents, and paste it repeatedly into our new, blank document.

What about “Delete Wordy Table Quickly?” Before we can delete a Wordy Table, we have to create one. So let’s put one in that second document from which we’re copying and pasting.

Finally, what might “Save Terse Text Frequently” mean? What if we reduce the contents of our new document so it’s terse, then edit and save in short cycles?

Our randomly generated steps result in the following far more rational scenario:

  1. Create a new document
  2. Open an existing document with lots of text, images, and at least one very long (wordy) table
  3. Copy the entire contents of the existing document into our new document multiple times
  4. Delete the wordy tables from the new document
  5. Edit the contents of the new document to make it short (terse)
  6. Save the file
  7. Edit and save again. Repeat several times.

Is this scenario likely to find problems? I have no idea. That’s the point.

Potentially Surprising Results

My goal with this technique is to create scenarios where I don’t know for sure what will happen. There has to be some chance that I will be surprised by the outcome. If my new scenario looks pretty much like every other scenario I’ve ever written, I need to start over. It’s not increasing the probability that I’ll find something new.

The word “surprise” is the key here. Surprises represent new information. The more we’re surprised, the more new information we’re uncovering. And the more we do the unexpected, the more likely we are to be surprised.

So try the random scenario generator tool at the top of this page to spark ideas for new scenarios you might run. Transform the gibberish into something you could actually do in the software you’re testing. Then let me know what surprises you find…

Subscribe

Subscribe to our e-mail newsletter to receive updates.

One Response to Flush Specific Stack Fiercely

  1. James Tharpe August 31, 2007 at 4:53 am #

    Hey – you should put a redirect on your original website (qualitytree.com). I looked for this article there (it is mentioned in a book I am reading) and ended up having to Google… and find nothing. So then I used your site’s search feature and here I am. Too many steps ;).