Emergency change: when something on a system, application, or physical device is changed immediately in order to prevent an incident. It can be the result of an incident or of failed change.
Take my word for it; I’ve seen it first hand. Emergency changes are risky.
They are quick, “on the fly” production changes that usually don’t contain back out plans. For this reason, and for many more I won’t dive into just now, it’s critical to have outlined processes for emergency changes.
Risky or not, many organizations often don’t take pen to paper to write out these necessary processes to approve emergency changes. In such instances, I’ve found that people push through emergency changes as new code and skip the approval process all together. Why? Because the approval process by which to submit and review those emergency changes virtually does not exist.
No red tape and no CAB review bureaucracy… sounds like CAB-utopia, right? Wrong. The consequences are far reaching, and they might just catch up to you.
So what’s the bottom line?
Use your CAB to review each and every emergency change that occurs using an after change review. The CAB should assess whether or not it can work to prevent similar emergency changes in the future. It should strive to discover the root cause through deep analysis and should likewise explore ways to eliminate those moving forward.
And, if a large number of emergency changes occur each week, raise those red flags and dig a little deeper.
Maybe you have no clear policy for Emergency Change.
Perhaps you have not identified the true root cause of an incident.
Or maybe you have yourself a trending Emergency Change that needs addressing.
Whatever the case, whatever the cause, emergency changes come up quite a bit in Change Advisory Board (CAB) meetings. Don’t let them slide by. Are you prepared?
(Original image by Perspecsys Photos.)