With promises of double and triple digit increases to sales or signup conversion rates, small business webmasters who discover the value of split web page testing can be tempted to begin randomly, without direction or plan. Results can be disappointing.

Two simple guidelines can improve the effectiveness of A/B split web testing. First, test first things first. Second, use common sense to plan test sequences.

Guideline #1: Test First Things First

More than other types of testing, the order that split web page tests are done can have a powerful impact on the value of the results.

For most benefit, test variables in the sequence that visitors encounter them, unless there is a strong reason to depart from that order.

Test elements above the "first fold" first. (This area appears first in a visitor's browser window.) It makes no sense, for instance, to test a subhead on the second screen if most visitors do not get past the headline. Testing a variable farther down the page will take many times longer if an element above the first fold repels visitors -- a poor headline, an offensive background, tiny fonts, or any other thing -- or if it simply loses them.

In split web page testing, it is not the number of page visitors that determines how long a test takes to run, it is the proportion of those visitors that take the action measured by the test. Therefore, the fewer visitors that reach the action "hot spot" the longer the test will take.

Suppose two different "calls to action" -- positioned two screens down the page -- are pit against each other. If only 5 out of 100 visitors scroll down far enough to see that call to action, and to order (the action measured), it would take 2,000 visitors before 100 even see the call to action.

Out of those 100, how many would then order? One? Two? Five? Less? If 50 orders per version were required for a reliable result, it could take anywhere from 40,000 to 200,000 or more visitors to that page to run that one test. For some small internet businesses, that could equal a year for a single test.

On the other hand, if by testing "first things first" the percentage of visitors that reach the order button doubled, the number of visitors required to test that same call to action would be cut in half.

Guideline #2: Use Common Sense

Do not adhere rigidly to guideline #1. Use common sense to evaluate where to start, and when to move on to a different test.

If a major problem is suspected farther down the page, it may make sense to jump ahead to test it.

There is also a point of "diminishing returns" where more significant results are likely to come out of testing something new rather than continuing to "tweak" the same thing. Later on, it is always possible to backtrack or retest something earlier on the page.

The Key

The key that underlies these two guidelines is to test in the order that will do the most to increase the number of visitors reaching the "hot spot" where they choose to do/not do the action measured by the tests.

To decide the best order to do split web page testing, experience the page as a visitor might. If necessary, observe someone else visiting the page for the first time.

A Simple Strategy to Identify Best A/B Split Test Sequence for a Web Page

1. Print the web page. (Set the browser screen view to medium text size to create an 'average' view.) Lay out the printed pages into one long sequence. Refer back to the screen version during the next steps.

2. Draw a line across the location of the bottom of the first screen to mark the first fold. Repeat for the second screen and down the page until the page is broken down into screens.

3. Circle or highlight the "hot spot" on the page where the action the test will measure occurs. This action could be an order button, subscription form, link, etc.

4. Scanning the page, as if for the first time, notice what catches your eye, without getting too involved reading. Circle the one or two things the visitor would notice first above the first fold.

5. Continuing to scan, circle specific elements that lead the eye down the page from the first fold toward the action "hot spot." Try to limit it to the most important ones.

6. Looking at those circled items, decide which are most likely to influence whether or not the visitor reaches that action "hot spot." Pay particular attention to potential "decision points." Note these on the page.

7. Use the print out as a guide to identify, evaluate, and prioritize potential test variables.

=> Look for things that could send visitors to the back button.

=> Look for obstacles, distractions, or confusions on the visitor's path from the first thing seen to the action "hot spot," especially above the first fold.

=> Look for individual elements that could be strong candidates to test changes on. Common "first fold" test variables include headlines, subheads, first paragraph, overall messaging and appeals, emphasis devices, navigation, and graphics.

Tip: Once repelling items are eliminated, the headline is often the variable that determines how many visitors remain on a page long enough to consider taking an action.

8. Choose the first, second, and third variable to test. Reevaluate this order as results come in.

Start split testing web pages in a well thought out sequence and produce results that create believers in the value of web page split testing.