We often talk about tension caused competing bonus / success structures for development and operations teams in the build and release process. At Gartner IOM Cameron Haight framed this using the classic Prisoner’s Dilemma concept from game theory. I wanted to elaborate on that idea.
Two men are arrested, but the police do not possess enough information for a conviction. Following the separation of the two men, the police offer both a similar deal—if one testifies against his partner (defects/betrays), and the other remains silent (cooperates/assists), the betrayer goes free and the cooperator receives the full one-year sentence. If both remain silent, both are sentenced to only one month in jail for a minor charge. If each ‘rats out’ the other, each receives a three-month sentence. Each prisoner must choose either to betray or remain silent; the decision of each is kept quiet. What should they do?
Lets imagine a bonus or annual review structure in a typical IT shop. Development is being told to be more Agile and is being rewarded primarily based on how quickly they are delivering new capability. Operations is responsible for keeping production systems running. They get a bonus if they can reduce the frequency and duration of outages. Plus, their lives are better when the pager doesn’t go off in the middle of the night.
Development can “rat out” Operations by pushing a large number of unstable releases towards production, and using the business to apply pressure on Ops to release with inadequate instructions, rollback strategies, etc. Operations can betray Development by creating needlessly long process gates, having a small number of deployment windows, etc all designed to reduce outages by preventing releases.
People tend to betray their friend when the game is only played once. But if we repeatedly play the game with the same partner, the game theory analysis suggests that we should try being nice, and only betray in retaliation. The strategy also calls for eventual forgiveness to avoid a bitter cycle of retaliation.
Life in IT is closer to the repetition scenario. We release pretty regularly, and so we have constant opportunities to help the other silo or betray them. Betrayal has become so frequent in many organizations that the Dev / Ops relationship is simply unproductive and hurting both sides.
A reconciliation back to cooperation is needed to get out of this toxic cycle.
While a preference for being cooperative is ideal, we should really blow up the whole game and change the rules. When we truly embrace DevOps we are cooperating from the start and taking joint ownership of the product throughout its lifecycle. Ops folks join feature and architecture planning meetings and developers wear pagers.
In terms of the prisoner’s dilemma we are no longer in a separate room making separate decisions. We’re making the decision together. It’s much, much, much easier to cooperate when we’re not worried about what the other guy is doing behind closed doors.
Executives looking to push DevOps forward within their organizations should tweak their bonus plans / review scoring systems accordingly.
* Dilemma example from wikipedia.