If there was one word that could sum up the IT sector as 2001 drew to an end, it would have to be 'uncertainty'. And not just in terms of economic downturns, but also the way companies measure Web-based business performance - uncertainty in how relevant measurements can be achieved when faced with myriad metric bases to choose from.
As a host of technology companies announced earning reductions for the fourth quarter of 2001, there was little sign of improvement on the horizon and the importance of producing accurate and meaningful business forecasts comes to the fore again. Yet without implementing the right tools and leveraging the 'best practise' factor, Net metrics remain as much of a dark art as ever - viewed with almost universal doubt and caution as a result.
Surely it doesn't have to be this way, not now that technology and awareness are at a level to move metrics into the mainstream? With the troubles of the media advertising sector and one high-profile dot com company having shares suspended while allegations that the CEO "exaggerated" user numbers are investigated, independent traffic auditing is likely to become an increasingly important factor in the race to measure online business performance. Or at least you would hope so.
But some industry observers remain cynical and suggest that there will likely remain a tendency for many companies to produce meaningless figures with no common baseline or third party verification. Workers in the metrics market describe some existing systems as a famine of answers dressed up as a feast of data. Of course, this may just be the mind of a marketing man leaking thoughts of positioning and profile into the real world, but there is no doubt that the Web metrics marketplace is a key segment in need of some serious attention.
Malcolm Duckett, vice president of corporate marketing at Web data collection expert Speed-Trap, says the cat is out of the bag and traditional server-side technologies and Web-bugs provide such questionable data that no one with any insight can continue to accept them. "Last week, we met a company CEO who had been severely embarrassed by a member of his own board," says Duckett. "The board member, when confronted with the latest Web statistics for their site, suggested half of them were probably the Nimda virus at work - a little investigation proved he was right."
Or as Speed-Trap CTO Jeremy Barker rather bluntly puts it: "The whole Web metrics business has been seriously muddied by constant marketing piffle. Moreover, the statistics provided are hardly business oriented - a graph showing percentage of images downloaded in GIF versus JPEG format doesn't really move you forward, does it? You need metrics which actually tell you something about what happened: eyeball-seconds for a page, reading time per 100 words, or correlation between download time and purchase value are somewhat more realistic."
All of which suggests there is no common quest to deceive on a grand scale, but rather just a muddled market that produces large volumes of confusing data which is difficult to interpret relevantly, let alone accurately. Ian Thomas, strategic development director at Webabacus, admits there used to be a common practice to inflate user numbers and traffic, but places this firmly in an historical context, largely because advertisers and investors have become a lot more savvy about the figures touted at them. "It used to be the case that a site could talk proudly about getting 10,000 hits a week," Thomas insists, "but it is now well understood that page impressions and unique user numbers are much better indications of a site's popularity. Companies can also create more credibility by reporting user numbers through online auditors such as ABCe, which has worked tirelessly to establish itself as a trusted standard for Web reporting."
At this point in the evolution of online business models, people are just starting to understand that flat visitor and page impression numbers do not give enough detail and that detailed information about the behaviour of different groups of users is of far greater value. However, moving to this new level of understanding is hampered by the mindsets of some in the industry, such as advertising agencies and analytic tool vendors, who have come from a broadcast media background where pure reach (in target demographics) was the only means by which advertising could be sold.
"Online business people need to understand that most Web sites are much more like a large department store than a TV channel," says Thomas. "It's not enough simply to get people through the door - they have to have a meaningful interaction with the site for their visit to have delivered value. Interestingly, this means the focus on absolute accuracy in reporting user volumes and page impressions will become irrelevant, to be replaced by analysis of online behaviour."
Yet however data is analysed, it still has to be collected in the first place and vast quantities of information are incredibly difficult to manage. Turning profuse but often baseless information into useful site statistics, well interpreted business intelligence or invaluable commercial analysis is challenging to say the least.
John Woods, CEO of Site Intelligence, believes part of the problem lays with off-the-shelf software which is over-sold and denies the companies that purchase it gaining the insightful knowledge of their online business performance they require. "The companies we work with had little knowledge of the volume of information that can be discovered from a Web site, but have realised the value of more sophisticated Web site analysis not provided by off-the-shelf packages," says Woods.
"Because of the difficulty of analysing the data, a company should look to gain some consultancy advice when first starting to use Web site visitor intelligence. We recommend a pilot project to start with, which will give the company a feel for the benefits that can be gained from sophisticated analysis," he adds. At this stage, an organisation should also vet its software vendor to ensure an understanding of the company's business objectives. Because of the vast amount of information available, tight objectives and reporting requirements should be defined and monitored from the start of the relationship
Setting the standard
Richard Foan, managing director of ABC Electronic, thinks the way companies view independent auditing will depend on what they believe the definition of 'meaningless' to be. "If you asked someone responsible for monitoring Web performance within a new media company if their figures were meaningless, I'm sure their answer would be no, whatever methodology they were using," he claims. "To address this issue, the new media industry is working to ensure traffic figures are meaningful when they are measured to agreed industry standards which are transparent and credible."
This would appear to be the only way to ensure a common currency on which to trade online space across all industry sectors. Many in the new media industry see ABCe as the industry standard for independently verified Web site traffic, not only because of its experience and brand trust, but also because it ensures traffic is audited to standards and definitions set by the Joint Industry Committee for Web Standards (JICWEBS). Also, ABCe is not a vendor of data logging and analysis software, but a certifier of software for use by member companies.
Whether approached from the site traffic and unique visitor perspective, reflecting circulation and bums on seats philosophies for potential advertisers, or analysing business models and determining customer behaviour patterns, it really makes no difference - at the end of the day, as Speed-Trap's Duckett says so eloquently: "If you can't measure it, you can't manage it."
ABCe case study: Guardian Unlimited