Microsoft toughens up over MCSE examinations

Many believe that Microsoft's industry certification, the MCSE has become too easy, making it too soft an option. Philip Hunter...

Many believe that Microsoft's industry certification, the MCSE has become too easy, making it too soft an option. Philip Hunter finds out why the company is taking a tougher stand over its MCSE exams

"Minesweeper consultant, solitaire expert" is the jibe often levelled at Microsoft's MCSE qualification. But Microsoft is fighting back. The IT training scene has been transformed by the meteoric rise of supplier certifications, which are fast overtaking traditional qualifications such as computer science degrees in the esteem of many employers. So IT staff seek these certifications to enhance their job prospects, and some of the less academically orientated universities and colleges in turn have added them to their curricula.

One advantage of supplier certifications in a fast-moving industry is that they only confer temporary status, and require periodic refresher courses and further exams to update the skills. They, therefore, give a snapshot of relatively current, rather than extinct, skills. But the disadvantage is that the pressure to alleviate the skills shortage has lured some suppliers into pumping too many individuals through their courses too quickly, devaluing the certification.

Cisco has succeeded in avoiding this trap with its Cisco Certified Internetworking Expert certification, partly by ensuring that the exams evolve rapidly in line with the pace of the networking industry. As a result, Cisco has avoided the pass rate inflation that has afflicted some certifications as candidates become increasingly familiar with the material, and with the type and style of questions.

Microsoft though has been less successful with its Microsoft Certified Systems Engineer (MCSE) and Microsoft Certified Systems Developer (MCSD) certifications, which have been a little devalued as skills currency. Under pressure to alleviate the skills shortage, Microsoft and its training partners increased the rate of production of MCSEs and MCSDs without checking that the raw material was of sufficient calibre.

The problems were that the multiple-choice questions had become too easy and too predictable and susceptible to cheating. Unlike conventional exams which all candidates sit simultaneously, IT certifications are taken on a rolling basis, so there is the potential for passing on details of questions.

Microsoft's UK education manager Clare Curtis admits that the tests have become too predictable, but stresses that this is changing with the development of new online exam techniques. Furthermore, Microsoft has taken the opportunity presented with the arrival of Windows 2000 to beef up the exams. Windows 2000 is substantially larger and more complex than Windows NT 4, requiring more knowledge of enterprise level issues, with the introduction of distributed directory services (Active Directory) and security options, notably the Kerberos authentication mechanism.

As a result, existing MCSEs who qualified by passing NT 4 exams have to take some additional course modules to become certified in Windows 2000. This means Microsoft can weed out some of the less able MCSEs, claims Curtis. "I don't believe all the MCSEs today will acquire the new certification," she says.

Given the ever-escalating skills shortage, Microsoft is clearly relying on a new intake of potential MCSEs to replace those existing MCSEs who fail the exams for upgrading to Windows 2000. There is the danger that there will be renewed pressure to step up the certification rate and compromise on standards.

Meanwhile though, with the full Windows 2000 certification process yet to begin and the new exams still in beta test, the rate of production of MCSEs, who deal with systems management issues, is still increasing. During the year ending 30 June 1999, there were 6,000 new MCSEs certified in the UK, bringing the total up to about 10,000. This indicates that this was a period of rapid growth (although there is also an annual drop-out rate as older MCSEs fail to requalify). But in the eight months to 29 February 2000, some 6,000 more MCSEs were certified, again according to Microsoft figures. This brought the figure up to 15,000 with current certifications, after taking account of drop-outs.

Curtis expects that the new exams will have an impact on the MCSE production numbers later in the year when the new exams begin to bite. "Some of the MCSE retirement announcements will kick in then," she says.

These announcements concern the obligation on existing MCSEs to take upgrade exams. Although Microsoft is discontinuing the NT 4 exams at the end of 2000, MCSEs who qualified by passing them will remain certified until the end of 2001. It is only MCSEs who qualified in earlier versions of Windows who will have to take new exams this year. The new exams will not have a serious impact on the raw numbers of MCSEs until 2002.

Enterprises migrating to Windows 2000 are likely to start discriminating between MCSEs according to whether or not they have passed the new exams. To some extent the same will be true of MCSDs. But there are signs that some employers are reacting to the confusion by placing less store on certifications during the recruitment process, focusing instead on the potential of the candidate. One such employer is IT technical support firm Stream International, which set up an internal training centre. "We re-engineered our recruitment model, becoming more focused on the type of employees than their skills," says training manager Paul Duddy.

Despite a temporary devaluation in its status, MCSE certificationdoes still increase job prospects and salary levels. Some employers believe they can save money by hiring uncertified IT staff, and then retain them on lower salaries by keeping them uncertified. But as Phil Lawman, Hewlett-Packard's general manager for education in northern Europe, points out, this is a flawed strategy. "Some say, 'If I train my staff, they'll leave.' But if that's the only thing that's keeping them there, they'll look for another job anyway."

Employers will have to live with IT certifications such as MCSE, but should certainly not take it as a guarantee of competence.

How Microsoft is making the MCSE exams harder

Apart from the expanded curriculum of Windows 2000, Microsoft is introducing new techniques to make exam questions less predictable and more testing of practical skills. Firstly, case study-based tests are being introduced, in which candidates are presented with online simulations of networks or systems with specified configurations. They are then set management or administrative tasks and scored according to how accurately they perform them.

Microsoft then plans to introduce Adaptive Testing as well, perhaps later this year, to examine raw knowledge that cannot be assessed so readily with case studies. Currently this material is tested via fixed-form exams where candidates are given a set number of questions with multiple-choice answers and scored according to the number they got right.

Adaptive exams are more subtle, with successive questions being selected by the test engine according to whether the previous one was answered correctly or not. If a question was answered correctly, the next one is harder, and if not, the next one is easier. This relies on an online testing engine with access to a large database of questions sifted into different categories of difficulty. In order to determine how difficult particular questions are, they need to be tried out in traditional non-adaptive tests first, and this is currently taking place during the beta trials of Windows 2000 exams.

Adaptive exams will then be introduced when the database of questions has been refined to reflect the performance of candidates.



Read more on IT jobs and recruitment