Microsoft interface is watching you

Microsoft chief executive Steve Ballmer revealed future enhancements to its user interface technology, including e-mail...

Microsoft chief executive Steve Ballmer revealed future enhancements to its user interface technology, including e-mail prioritisers, under research in Microsoft labs, when speaking at the Association for Computing Machinery ACM1 Conference in San Jose last week.

Ballmer supervised a demonstration of Priorities, an artificial intelligence technology that uses data, such as header structure and information about relationships between different people, to work out which e-mails are important.

The significant part of the technology, which would otherwise be little more than a smart mail filter, is its ability to gauge what the user is doing by using scheduler information, and even sensor technology to gain information from the ambient acoustics in a room. This helps it to work out what the user is doing and whether or not it should deliver e-mail.

This filtering technology, without the ambient sensors, underlies Outlook Mobile Manager, which was unveiled a couple of weeks ago and is currently available as a beta download. The product is designed for use with the Microsoft Mobile Information 2001 Server.

Eric Horvitz, a researcher at Microsoft, demonstrated the Notification Platform, which takes this technology a stage further. It monitors actions like desktop activity and could even have the potential to use "accelerometer" data to see whether or not a person is moving.

Such data about the end users' situation would be passed to the central notification manager in an XML format, which Horvitz refers to as the notification schema.

"The system knows that I am facing front and can see what applications I am using. It is also looking at my calendar," said Horvitz, adding that the system can also conduct a voice trace or sense him gazing at the display. "It uses this information to compute my space. Am I high-focus solo or am I low-focus solo? Am I sleeping?"

This is by no means original research. IBM has been conducting similar work with its BlueEyes project, which was also on show at the conference, although, like the Microsoft project, no firm date has been given for a launch.

Other user interface technology on show from Microsoft included a 3D environment in which users could store different Internet Explorer sessions, pulling off different ones configured for different tasks.

It also demonstrated its Easyliving project, which used Soap-like access protocols to create an automated house environment in which devices were automatically controlled according to the position of the residents.

But the most interesting user interface technology is the most immediate - speech recognition, which is being rolled out as part of the .net initiative and will be included in the imminent Office XP product. However, there was a basic contradiction in the speech that Ballmer made at the show. He said, "Some ask whether Microsoft and standards go together. The answer is yes."

It is interesting, then, that the company is not a member of the VoiceXML consortium. The group is developing XML-based voice control mechanisms for user interaction with server-side applications online which, given the mix of server-centric services and speech recognition in .net, places it squarely on Microsoft's radar.

This, along with Microsoft's increasing interest in speech recognition, should worry smaller companies in this area, which could do without this type of competition in the current economic climate.

Read more on Operating systems software