Swift action by IT staff saves software firm from bandwidth-draining zombie server

Strong security-incident management saved a large software company in the UK severe business damage and possible legal action...

Strong security-incident management saved a large software company in the UK severe business damage and possible legal action after a criminal gained access to a server at administrator level.

The incident, described anonymously at the annual BCS Birmingham IT Security Conference, began when system administrators noticed a significant increase in traffic to a server. The server made available source code and updates for registered developers. The traffic increase was so significant that general network performance was starting to suffer.

As this followed a service launch, it was initially assumed the company had underestimated demand. But a review of the server revealed that the disc was reading close to 100% use and that the audit logs had been deleted. Closer examination found that an authorised directory was housing several hundred copyrighted music files - accounting for the increased disc use.

An immediate study of the server and related systems was initiated to determine the extent of the problem. The corporate incident management strategy deemed the server to be non-critical to the continuation of business and it was taken offline.

Despite the removal of the audit logs, a bit-level examination of the system revealed that a brute-force attack had passed unnoticed and allowed an attacker to gain administrator access.

With this unrestricted access the intruder had created a new archive on the server and transferred several gigabytes of multimedia content to the system - creating an illegal private music-sharing resource.

Anonymous access privileges had been assigned to the folder and hundreds of users were accessing the system each day to attempt to download the music.

Integrity checks on surrounding systems and the entry point into the internal network made it clear the compromise was limited to the single server. Having identified the point of compromise and the level of exposure, the company put eradication and recovery strategies into practice.

The server was rebuilt using trusted back-ups of the operating system, applications and data, with special effort made to ensure the original exposure was not reintroduced. The system was then validated by engineers and probed to verify its integrity. It was put back into service and a monitoring schedule created.

The company considered itself fairly lucky. A well-rehearsed incident management plan had enabled swift and positive action on discovery of the compromise. And although the stored material represented a breach of copyright, it had not significantly affected the company's image or led to any legal liability.

Other organisations have not been so fortunate, unwittingly hosting pornographic material on compromised servers.

Lessons from the security breach   

After the hacker's work had been removed from the company's server, review meetings identified several key lessons.

These were documented in the incident report, with the core elements passed on to management and integrated into the security incident management and policy:  

  •  Tightening of system security was needed, particularly in exposed areas of infrastructure, with accounts and services being run under the principle of least privilege. 
  • Poor communication throughout the incident created uncertainty which could have been avoided with more openness and discussion about the incident and response team actions. 
  • A critical phase of the incident management cycle, containment, was largely ignored, which could have resulted in the problem spreading. 
  • Incident detection was enhanced by regular reviews of audit logs, including analysis by the incident response team where possible.

Ross Patel is director of security and assurance services at Afentis and co-ordinator of the BCS Birmingham IT Security Conference

Read more on IT risk management

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.