Why ‘Production Freeze’ Should Be a Four-Letter Word

It has long been standard practice for IT organizations to enact ‘production freezes’ during important business periods. Whether it’s tax season, HR open enrollment, the days leading up to the election, or any other business-specific “critical time,” this practice makes intuitive sense. If the system is stable, you don’t want to risk business operations by destabilizing it. Change introduces uncertainty, which generates real as well as potential instability.

This approach, however, suffers from two important misunderstandings. First, “the system is stable” really means “the system is stable, as far as we can tell, at this particular moment.” By definition, business-critical events stress systems. Freezing production means removing your ability to respond to unforeseen instability.

Second, production freezes presume that the system meets user expectations. Just as with stability, functional validity is a matter of “as far as we can tell, at this particular moment.” If you discover you were wrong about customer needs, you’ve removed your ability to respond. Neither can you respond if you observe customer needs changing during the freeze period.

Freezing production during business-critical times prevents you from repairing sources of customer dissatisfaction when you most need to be able to do so. Due to the timing, the intensity of that dissatisfaction will be at its greatest. Not only will you not be able to soothe their frustration quickly; by definition, you won’t be able to help them until AFTER their most critical need subsidies.

Typically, organizations address this conundrum with an intentionally draconian exception-review process. Only the most critical changes are allowed to thaw the freeze. The changes that do pass muster undergo an extraordinarily rigorous review and planning process. Unfortunately, this well-meaning (and again, intuitively correct) solution suffers from a fundamental misconception.

Why do you over-review changes during freeze periods? Because you lack confidence in your normal process. Why do you restrict changes to those that are the most critical? Because you lack confidence in your normal process. In other words, you have instituted an approach that has you making the most important changes, at the most important times, with the least possible confidence that they will work.

Imagine instead a scenario where you do away with production freezes. You use exactly the same process for making changes during business-critical times as you do during the off-season. This scenario has two implications: first, your ordinary change process must be robust enough to work at the most important times, with the most important changes. Second, your critical-time change process has been tested over and over again, throughout the year. As a result, you have maximum confidence in it.

Fear leads us to do the riskiest things as seldom as possible. If, however, the riskiest things are also the most important ones, we need to engineer the risk, and thus the fear, out of them. The best way to do that is to reverse our logic and do them as often as possible. If we can make election-day IT change risk-free, we can shift our attention from unintentionally minimizing customer satisfaction of out fear, to confidently and continuously maximizing it.

Post Comment