The critical vulnerability, detailed in security bulletin MS03-026, affects a Windows component called the Distributed Component Object Model (DCOM) interface, which listens for traffic on TCP/IP port 135, Microsoft said.
A flaw in the way that DCOM handles messages sent using the Remote Procedure Call (RPC) protocol causes the RPC service to fail when an incorrectly formatted message is received.
RPC is a common protocol that software programs use to request services from other programs running on servers in a networked environment.
Attackers could send malformed messages specifically designed to crash the RPC service, resulting in the attacker's malicious code being run on the vulnerable machine.
The flaw affects many supported versions of the Windows operating system, ranging from Windows NT 4.0 to Windows XP and Windows Server 2003, Microsoft said.
Any Windows machine on which traffic to port 135 has not been blocked is susceptible to attack, the company warned.
Personal and corporate firewalls typically block traffic on Port 135, protecting those machines from exploitation. However, machines located behind a firewall on corporate Lans and intranets would be vulnerable to internal attack.
Blocking port 135 on machines connected to a corporate Lan or intranet would prevent those machines from acting as file servers, according to Jeff Jones, senior director of Trustworthy Computing security at Microsoft.
This is the second time this year that Microsoft has issued a critical security bulletin linked to RPC. In March, the company warned of a flaw in Windows' implementation of RPC that could enable attackers to launch denial of service attacks against machines running Windows.
So widespread was that problem that Microsoft told customers that it would be unable to release a patch for systems running NT 4.0. However, an NT 4.0 patch for the latest RPC flaw is available, Jones said.
The latest RPC flaw was reported to the company in late June by an independent security consulting company called The Last Stage of Delirium Research Group.
The problem failed to get noticed internally because a tool that would have found the problem had not been integrated into automated code review tools that Microsoft uses to locate vulnerabilities, Jones said.
Microsoft will now be taking a closer look at the Windows component that yielded the latest critical flaw as well as all similar components, Jones said.
Asked whether the vulnerability was easy to take advantage of, Jones noted that it had existed in operating systems as far back as NT 4.0 and had not yet been detected by Microsoft or by the hacking community.
"There were easier ones to find that people did find sooner," he said.
In addition to the RPC vulnerability, Microsoft warned of two other security flaws, both rated "Important" by the company.
According to bulletin MS03-027, an unchecked buffer in a function used by the Windows XP desktop could enable an attacker to use a specially crafted configuration file to crash a Windows system and potentially execute code on the system.
Only machines running Windows XP Service Pack 1 are affected by the vulnerability, and attackers would be limited to the current Windows user's level of system access. (See http://www.microsoft.com/technet/treeview/default.asp?url=/technet/security/bulletin/MS03-027.asp)
In MS03-028, Microsoft warned of a vulnerability in its Internet Security and Acceleration (ISA) Server 2000 that could enable an attacker to use an ISA server to send malicious code to another user.
The flaw affects HTML error pages used by ISA Server to send customised error messages to requesting client machines.
An attacker would first have to know the address of the ISA server and its access policies and be able to trick a victim into viewing an affected HTML error page and clicking a link on that page, Microsoft said.
In addition, the ISA Server would need to be acting as a gateway device to access the internet directly, something that Microsoft does not recommend, Jones said. (See http://www.microsoft.com/technet/treeview/default.asp?url=/technet/security/bulletin/MS03-028.asp)
Paul Roberts writes for IDG News Service