During my time as a usability consultant I have seen Usability Testing grow from being a working progress to a means of improvement for anyone with a web presence. In the old days Usability Testing was very much unmapped territory. When analysing and Testing websites they were filled with flaws and inconsistencies anyway and you didn't need to be a Usability Expert to spot even the harder to find issues.

The methodology involved at this time was only really as much as the available technology would allow. This wasn't very much and going back eight years at the time it was little more than a panel of Testing User participants in a classroom where their reactions and opinions were monitored. To draw meaningful conclusions still very much depended on some quite weak theories.

Anyway the point is that in 2003 I conducted Usability Testing studies for over 20 clients and was not asked once to include a comparative analysis in the Usability Test. I look back now and find it quite bizarre that at the time I didn't suggest or was asked for it. Why was this?

Well User Testing was very much in its infancy and I was only aware a couple of pieces of literature which even hinted at the possibility of Usability Testing as a comparative competitor analysis. Maybe I should have included it more from the off but at the time the only organizations remotely interested in Testing User analysis were e-commerce sites, and at the time very little trust was placed in e-business. If you had an e-commerce site the chances were that at least in the earlier you had automatic e-loyalty because those willing to give up their credit card details to the web in the first place were very unlikely to extend that trust to a competitors website. There was simply no point to compare your sites to a competitors. Now as time went on as we know this statement has become more and more invalid up until today where I sit writing this blog the opposite is true.

Comparative competitor analysis is essential. In depth competitor analysis forms a significant part of the Testing User experience and gives clients real insight into the successes and failures of a competitors website. User Testing to compare you and your competitors web presence is now a very logical move as unlike eight years ago a competitive website is now a necessity.

The Testing user analysis contains in-depth competitor analysis as part of a multiphase approach.

Of course the nature of your competition will vary from client to client with the added difference that unlike eight years ago it's no longer a 'maybe' for an e-commerce site, but a serious analysis for organizations with even basic reference and resource sites.

How valuable though is the competitor data you will receive back?

In all honesty clients find the data concerning their own site to be worse than their competitors. Yes they will probably get the better User Testing feedback, but remember the motives you have for wanting to compare the two in the first place is probably based on the competitor having or are perceived to be having more success.

Even in the event that you receive more negativity than positivity the chances are that through the Testing User analysis they are easy to remedy. It's true as well that no one in the Testing User team has ever had a client complain about this negativity because anyone who commissions a Testing User analysis in the first place is awake to the fact their product or service isn't in some bubble untouchable by anyone. They want to be told straight what the playing field looks like. Even before you've implemented any usability design changes you've discovered that your competitors web presence gets a better reaction from users. You have gained insight into how your competitor approaches their online presence and what works for them may be able to work for you. It's worth mentioning swell that in my experience clients have found negative feedback to be powerful motivators for their teams.

That is Usability Testing in its simplest form though. To identify and to improve.

On the other hand you may find that the Testing User report confirms that your web presence gets more positive feedback than your competitors. If so congratulations but like in business any gain or advantage is more often that not only temporary, Usability is the same and you must continue to monitor and innovate these gains.

When the Testing User report comes back from our usability panel the data is quantified and finalized into a bespoke, board room ready document. In general when looking at the competitor analysis we find that as a general rule the pendulum will swing between 60% to 65% in each direction. So respectively x% worse or x% than your competitor. Of course there have been many instances where the User Testing report confirms differences of one or two percent, in this case we direct you to the individual component scores in the Testing User report, which can often vary by 30% to 40%.

Keep in mind though the point I made about temporary gains through enhancements of your usability. You will now though have the information to monitor and improve the individual Usability components.

The comparative competitor analysis has become a crucial component of the Testing User report. I have seen results from usability changes that have been implemented over night. In fact I can think of one example where a simple design that was noted on a competitors website had allowed to client to reap a 29% sales increase in that business quarter.

Now as great as this sounds Usability Testing is not a magic wand solution so I wont quote all the positive results every Testing User client has achieved. Besides there's too many variables that can affect the outcome. It's not a dig deep solution neither but if want to avoid going through your competitor trash to get an insight and potential advantage this product may well be for you.