When X.509 security certificates fail, servers break

Why are X.509 certificates causing computers to stop dead?

Four years ago, a major computer outage at Japanese airline Nippon Airways occurred due to a relatively minor problem related to security authentication certificates. So why are certificates causing computers to stop dead?

Computer systems use authentication based on X.509 trusted certificates, issued by a certification authority, to enable secure data transfer. To maintain security and trust, the certificates have a limited lifespan, normally a year. But in 2008, at Nippon Airways, the cryptographic certificate issued for authentication of check-in terminals had expired, preventing the terminals from communicating with key systems at the airline.

In a similar incident, online retailer Target.com’s website was blocked, due to its SSL certificate expiring. In its X.509 Certificate Management report, analyst Gartner notes: “Expired X.509 certificates result in a number of system maladies, ranging from a simple error message on a screen to an abrupt termination of service. This can lead to abandoned e-commerce transactions or the loss of trust in a company's web presence.”

The report’s authors, Eric Ouellet and Vic Wheatman,  vice presidents at Gartner, note that companies that have an unplanned certificate expiry typically focus on other IT  issues first, such as hardware or software crashes, long before they begin to consider an expired X.509 certificate as the source of troubles. This typically results in significant delays in identifying and resolving the root cause of a system outage, according to Gartner.

“Businesses do not know how many certificates are in their organisations and where they are. It is difficult to manage certificates,” says Jeff Hudson, CEO of Venafi, a company specialising in certificate management.

SSL certificates are deployed on servers and web browsers, to enable authentication and provide encryption. Issued by a certification authority (CA) like Verisign, the SSL certificate tells the browser user that the server’s certificate can be trusted. Over the last 16 years, certificate use has exploded.  They are not only used externally, systems use certificates internally such as on routers, and within software.  Hudson says certificates are often managed in silos, using spreadsheets: “When they expire they need to be renewed. There are hundreds of millions of certificates, all of which are managed manually, using spreadsheets.”

Encryption keys

Encryption keys are formed of two parts: a private key is used to encrypt data, while the recipient can access the sender’s public key to decrypt the message. Private keys can unlock confidential data, so should be stored securely. But Hudson has seen cases where system administrators have walked out of companies with the private keys, or stored them on an intranet, which could be targeted by external hackers.

In a survey of 471 senior managers by Venafi  in 2011, 54% of respondents admitted their organisations had experienced either stolen or unaccounted for encryption keys.

“The private key is widely deployed to system admins who can easily take it out of the organisation,” says Hudson. “ I have also seen people put their private keys on an internal website. This is a bit like installing a really secure front door and leaving the key under the doormat.”

Clone certificates

Another security concern for IT managers and CIOs is when certificate authorities are compromised, such as the well-publicised incident last year at Ducth authority DigiNotar. “Last year three separate CAs were compromised. Hackers could create and issue phoney certificates, which meant they could intercept all the traffic coming to a website,” says Hudson.

With such a compromise, previously issued certificates from the public providers needed to be  revoked. Gartner recommends organisations to explicitly be aware of the potential for significant impact on their operations should they be associated with such an incident. 

“Knowing the specific provenance of each and every X.509 certificates in use within an organisation is critical in ensuring the timely re-issuance of certificates, thus minimising downtime,” he says.

Certificate management systems

Director 6 from Venafi, is one the specialised products designed to manage X.509 certificates. The Venafi product discovers certificates on the network, and looks inside key stores to  report on encryption strength and expiration date. Hudson says Director 6 will automatically renew certificates, and install them

Trustwave CLM , is another product offering X.509 certificate discovery. Gartner says the product will also renew certificates originally issued by any certificate authority.


Read more on Identity and access management products

Data Center
Data Management