Change control management is an integral part of any IT strategy, and is used to protect processes and systems. The problem with change control is that it often goes unused, or is misused, when implemented. An inefficient change control management system can be incredibly problematic, and this is also true when it comes to storage. Here are some examples of how the misuse of change management can potentially be very costly.
The first example involves a small firm with a fledgling IT department and immature processes. Backups for a new email server began failing regularly after a few mailbox migrations, and vendor support was dismissed after returning a seemingly generic response. An outside specialist was then brought in to do further discovery, but came to the same conclusion as the vendor. The result was a network interface card (NIC) driver upgrade. The night after the change, a successful backup was completed and the environment was stabilised a month after the initial failures.
This is an example of what happens when there are no controls in place and a lack of server history and testing procedures have the IT staff hesitant to implement potentially system altering changes. This time around, the firm was lucky there wasn't a collapse of the email environment during the period it was at risk. How much data would have been lost if there had been an emergency, and how much would it have cost to recoup?
The second example is of a large organization with mature processes and procedures governing system changes. In this case, a simple patch needed to be made to several databases to employ an agent-based backup method to replace daily data dumps. Each change required every application and server team associated with the database to be notified. Scheduling allowed two weeks between change creation and the approval deadline, but a majority of them resulted in failure due to one of the following scenarios:
- Every person contacted spawned a chain of emails asking questions and requesting conference calls. Hundreds of emails were generated, with no time for every team to conference. The change would end up being rejected by one or two people because their requests for a conference went unfulfilled.
- One person in the contact list would miss the email notification and not approve the change. Painstaking measures were taken to ensure that everyone on the approval list would approve on time. If one person on that list missed the approval deadline, the change would be rejected. Personal holidays became change killers.
This is an example of the overuse of change control management. Change notifications and logs were distributed to all of those involved to keep them informed but, due to company culture, it became a hindrance to getting work completed. In this case, the process did not allow significant disk storage savings to be realised.
A lack of change control can doom an IT organisation by creating a void where important system information is lost. But for change control management to actually manage change, people need to be trained in the processes and then actually put that training into practice. In practice, the system needs to be tailored to company culture so that it can be informative and not an excuse that prevents work from being finished. Change control management exists as a tool to help IT organisations track and inform, not to become a well-designed headache.
About the author: Brian Sakovitch, is a senior consultant at GlassHouse Technologies (UK). Prior to joining GlassHouse in 2005, Sakovitch held a role as a backup operations lead in a medium-sized networking operations centre that supported a major pilot gaming venture for Multimedia Games.