Research is an invaluable business tool, but is subject to the vagaries of corporate sponsorship and inaccurate briefings. Annie Gurton analyses the pros and cons of market analysts
Like "lies, damned lies and statistics", research has its place, provided you know where it is. It has become fashionable to criticise analysts, but even sponsored research without much pretence of independence can have some value. At its most compromised extreme, research and analysis are like public relations - you know the person is talking about their client and has a deeply vested interest but, nevertheless, they might still be able to tell you something interesting.
At the other extreme, however, when research and analysis is truly independent and not dressed up as anything else, it can be invaluable. As every product marketing manager knows, it is essential to undertake research before developing a new product or moving into a new market. Likewise, any business project should include a detailed analysis of the IT infrastructure requirements that could underpin it.
Research falls into several categories. There is that which is done speculatively by a research organisation or consultancy, but in this cruel capitalistic world there is no such thing as a free lunch and it is rare for a report to be undertaken without an eye to opportunities to sell it and benefit from the investment in effort. Sometimes speculative, unsponsored, uncommissioned research is undertaken to give a broad flavour of an industry or product trend, and as a promotional activity for the agency.
More often, research is conducted according to a commission from a specific client or group of clients that has a distinct purpose or outcome in mind. Analysts should bring their expertise to bear by determining the methods of research and questions to be asked.
However, it is not unusual for analysts to undertake research for a client who determines the range of questions and parameters of research. And he who pays the piper gets to hear the tune he wants.
Andy Baul-Lewis, associate director of the IT division of research and consulting agency Morpace International, has worked for both research agencies and companies that have used analysts, including Microsoft, and he defends the industry. He says that market research is well established and highly respected, if you are looking for research material to support your predetermined thesis then you don't have to look far to find an agency that will deliver what you need. "There are plenty of agencies that will take your money and give the answers that you are looking for."
But he adds that there are also some highly respected, reputable agencies that would not dream of compromising their integrity or reputation by putting their name to anything except entirely genuine research and conclusions.
"The value of the analyst's report and the research material depends on the credentials of the agency as well as whether the research has been properly conducted," says Baul-Lewis. All agencies should use a combination of qualitative and quantitative research. The first is 'softer', more in-depth information about an issue, usually collected in face-to-face interviews, focus groups or feedback sessions.
The latter is 'hard data' often collected by telephone or Web-based interviews, based on a set of specific questions. But agencies' areas of specialisation vary, and Baul-Lewis says that the first step for someone seeking to use research material or to commission an analyst, is to ensure that they understand the market being researched and the questions being asked.
"You have to be sure that the analysts understand your concerns about the area being researched," he says. "Any agency that doesn't have experience of researching IT markets and issues may make errors that undermine the validity of the research. By not fully understanding some broader aspects of the issues, or failing to grasp the wider implications of a broader IT issue, errors in interpretation as well as collection can be made.
For example, with a hot issue like application hosting, any agency unable to talk intelligently about the topic will be highly unlikely to develop a survey methodology to really providea salient view of how IT consumers view this development."
Ensuring that the analyst who will be handling the research project understands your objectives and the market is more than just a safety check. With the right analyst who knows the issues and facts, you can gain considerable value-add beyond the original brief. "The analyst can raise considerations that you have not even thought of, point you in obtuse directions, make recommendations that are extremely well- informed, as well an ensuring that the research and analysis is undertaken appropriately," says Baul-Lewis.
One problem is that, understandably, agencies are reluctant to mention past and current clients and ask clients to provide them with references. This is a problem for both analysts and their potential clients. However, agencies should be able to provide views of the research that they have undertaken, if not a top-level slice of the research but a taste of what they are capable of.
Most agencies have several clients in the same sector. Again, like PR and marketing companies, they gain expertise and a reputation in a particular sector and it makes sense to exploit that with more similar clients. As a result, they often have to create internal independence and work with 'Chinese walls', with varying degrees of success.
Managing director of AMD Research Mike Di Pietro says, "Many analysis firms do not staff themselves with experienced people at the client point of contact. You can therefore find yourself working with a facilitator who merely escalates the question to a more senior person, who may not be a full-time employee, and who may have other contracts." This can be an advantage because you can find a specialist working on your account, or it can be a disadvantage because of the lack of expert personal contact.
Some agencies are reluctant to undertake one-off jobs, preferring to work as part of an on-going project to monitor the changing demands of a market and to deal with longer-term issues such as customer experience, campaign impact and product reception. Such projects, as well as providing a long-term revenue stream for the analyst agencies, also allow them to provide greater in-depth analysis, more focused findings and to make more considered recommendations.
Di Pietro explains, "Many analysis firms work on an annual subscription contract. These relationships typically entitle the client to ask questions regarding their projects."
Sometimes analysts have a problem with their clients, and the way that they want to use the material generated by the research.
Morpace's Baul-Lewis explains, "Good research is not cheap and any client spending money with a research agency will want to use the results in as many ways as possible." This might mean for PR and promotional work as well as for internal strategic decision-making. In that case, if the research agency's name is used in the campaign, it is worth considering asking for a discount because the agency will be promoted indirectly through the publicity.
Di Pietro says that analysis firms have to be prepared to be honest with their clients and tell them their true findings, otherwise they lose credibility. "People come to an analysis firm to be given unbiased opinion based on facts," he says. He reiterates Baul-Lewis's view that it is crucial to use an agency with first-hand and in-depth knowledge of the specific area being researched. "It will have inside knowledge, know the right questions to ask to achieve the best results and be able to interpret the results more sagely," he says.
Kevin Chapman of Symantec has recently commissioned two different pieces of market research. "Market research and market analysis are only ever as good as the brief you set. Too often companies are guilty of simply commissioning an agency without giving enough time and thought to defining what they want to get out of it. If the brief is ambiguous, your investment will be worth less than the paper the report is printed on," he says.
Leading analyst groups
Leading US-based IT analyst and research group. Provides a range of services to technology providers and user groups, which it splits into vertical sectors such as finance or healthcare. Other offerings include research, consulting, measurement, decision evalutation, product selection and regular conferences.
0208 987 7100
Large US-based IT industry analyst and forecaster. Analyses IT products and suppliers, forecasts worldwide IT markets and technology trends. Has more than 500 analysts in 42 countries.
Dedicated to "helping IT professionals make better strategic and tactical decisions", the Meta Group provides several levels of service from newsletters to specialist individual pan-European consultancy. Meta's offerings and services include reports, consulting, benchmarking, analyst briefings and strategic tools.
0207 631 0202
A US-based research and analysis group that publishes many general reports about industry trends, as well as undertaking specific surveys for clients, covering the consumer, business-to-business and technical marketplace. Recently acquired UK Internet research organisation, Fletcher Research, which will be rebranded as Forrester UK.
0207 551 9000
European research and consulting company specialising in telecoms, software and e-commerce. Provides regular reports on hot technology topics such as knowledge management; continuous evaluation of mainstream topics such as enterprise resource planning software; and advisory and consulting services.
UK-based analyst and consultancy organisation offering a range of technology and IT management research papers on subjects such as Linux versus Windows NT. Also provides consultancy services to both users and IT suppliers in areas such as dispute resolution, IT auditing and marketing strategy. IT news analysis service.
UK-based analyst organisation that produces new reports on emerging technologies such as customer relationship management every three months and provides users with a variety of subscription-based analyst services. Butler Group also offers benchmarking advice, a seminar programme, a newswire and a number of Web portal sites.
Retailers use web muscle
Read how three retailers boosted their buying power thanks to the Web: "A problem shared"