One of the reasons I met up with him was because I’m planning a small surprise party for another close friend, and I wanted to benefit from his experience and advice. What made Michael’s parties so good? During our chat, we realized that it was it the people attending, and their conversations. Not only that, it was also the music. Of course, we couldn’t forget the food and drink.

In the end, I decided to ask Michael to help me organize the party. He knew exactly what needed to be done for a party to be a success, and I was convinced that, if we just did the same things he always did, my party would turn out great. So, what was the first thing he said?

“Joseph, if you want to have a great party, you have to invite all of my friends.”

I thought I didn’t hear right. I wanted to invite my own friends, but on the other hand, I knew that with his friends, the party would be a smash, like his parties. Sensing my disappointment, he tried to cheer me up by saying that he could save me a lot of money. I wouldn’t need a DJ, he said, because he noted all the songs his DJ played, and he could just give me a playlist for my iPod. I’d have to lock the iPod away, so that no one could manipulate the song order, but he was sure that with these specific songs, played exactly in this order, the party would be a success. Also, it seems he videotaped his last party, and noted what everyone ate and drank. He’d provide me with scripts for all the guests, with precise timings, so that everybody ate and drank the proper things at the proper time. Of course, all conversation and interaction would also be scripted.

So, just imagine you plan to have a party, and it must be as good as the last one you held. You invite the same people, and just to make sure you’ve covered everything, you tell them to wear the same things, and speak only to the same people as last time. You’ve got the same music playlist, the same food, drinks and layout. Will this be a good party? Does something seem wrong with this story?

The answers to these last questions seem logical, although you can’t easily explain why. There is a reason, though, and this reason is key to understanding the basic issues surrounding complex problems and agile methods.

It’s called retrospective coherence.

One thing that makes complex systems complex is their causality. In an ordered system, if you do something, you expect a specific result. Do it again, expect the same result. It’s that simple. In a complex system, though, the causality emerges as the system emerges. As the party goes on, the reasons for its success become established, not before. After it’s over, you can say that a party was a success, and that these people were there, this music was played, and this much beer was downed  – but you cannot say that the party was a success because these people were there, this music was played, and this much beer was downed!

In a complex system, causality emerges as the system itself emerges, so that at the end, you can say how you got to where you are, but you can’t guarantee that by doing exactly the same things, you’ll get to the same place again – and you probably won’t. In complex systems, we say the causality is retrospectively coherent.

So let’s take a closer look, and consider what it actually was that made Michael’s party so good. It wasn’t any one factor, but how they interacted. It wasn’t the components, but the emergent complex system, that was a success. An interesting thing to note here is that the things that people often interpret as being the causes of a success aren’t that, they’re the symptoms. For instance, was it the people that made the party a good one? The music that was played? His choice of fine food and drink? Well, yes and no. Michael played a significant part because he’s a great host. He’s constantly walking around, looking, questioning. Is anyone standing all alone? Introduce them to a group! Does everyone have a drink? If not, get some more beer! Are people dancing? Trying to talk? Put on upbeat or down-tempo music as appropriate! All the time, the good host is surveying the scene and making small adjustments.

As a good host, Michael drew some boundaries. His bedroom was off-limits. His single-malt collection locked away. And, he set up a few things that people would be attracted to: a wide-screen television, a keg of beer, some games maybe. Every party needs attractors. Did you ever wonder why so many people congregate in the kitchen at a party, even if it’s the smallest room?
One thing he didn’t do is to define success, prescribe or proscribe behaviour (could you imagine saying that a party would only be a success if someone danced on the table with a lampshade on their head?). He let the party evolve, observed which patterns emerged, gently supported the good patterns, disrupted the bad patterns as soon as he noticed them, but generally let the party run itself.

Now shift the focus to a project. We tend to make the mistakes above when it comes to project planning. What worked last time? Why? Well, it must have been the people, the methodology, the meeting schedule – let’s do it the exact same way again. Like the repeat party, a project planned on this basis is likely to be a flop.

Contrary to Einstein’s definition, in a socially complex system, insanity is doing the same thing over and over again and expecting the same result!

(n.b. Dave Snowden has a classic story on organizing a child’s birthday party, and his story served as an inspiration for my version. You can find the story here. Thanks, Dave!)

Level 5: Burning
(results in team burnout and death marching)
At the burning level, the heat is too high. The stress level results in chaos, aggression, and a regression to primitive behaviour patterns. This is a dangerous level, since the high pressure increases the friction between the team members, increasing the heat even more.

Characteristic behaviour patterns for this level
•    Countless overtime hours leading to no result
•    Stress
•    Disagreements, fights
•    Sickness
•    Lack of communication
•    Inefficiency
•    Hectic
•    Blame
•    Restlessness
•    Self-preservation
•    Paralysis
•    Panic
•    Fear of attack

Level 4: Cooking
(ideal temperature for continuous improvement)
This is the optimal temperature for mixing up and forming teams. At this level, the heat is high enough to force disruption of behaviour patterns, but not so high that team members regress into pre-conventional mode.

Characteristic behaviour patterns for this level
•    Differences are resolved constructively
•    Continuous improvement
•    Enthusiasm
•    Work is fun
•    High productivity
•    Constructive discussions
•    Consensus
•    Joy
•    “We” – feeling
•    Self-reflection
•    Open discussions
•    Progress
•    Results
•    Creativity
•    Fun
•    High performance
•    Organised chaos
•    Good communication

Level 3: Cooling (Stagnation)
(discipline is lost and bad behaviour begins to fester)
If not enough heat is applied, or when things cool down after a while (entropy), a team enters the cooling or stagnating stage. In the kitchen, this is where what was once a fine-tasting soup has become a substrate for fungus and bacteria growth. Most health departments even have laws requiring cooked foods to be cooled down within a certain short period of time, as to pass through this stage as quickly as possible.

Characteristic behaviour patterns for this level
•    Slacking off
•    Work becomes routine
•    Loss of discipline
•    Monotony
•    Indifference
•    9 to 5
•    Too much discussion
•    Minimal communication

Level 2: Congealing
(team gets too comfortable to achieve and bad habits become the norm)
Increasing pressure to adhere to “norms“ – “that’s the way we do things here”

Characteristic behaviour patterns for this level
•    Change becomes difficult
•    Many meetings
•    Multiple implementation of work
•    Problems become difficult to solve, because everything’s a mass/mess
•    Boredom
•    Much talk, no action
•    Consensus becomes difficult
•    Resignation
•    Splitting of the teams into cliques
•    Everyone works for themselves
•    Minimalism
•    Defensive stance
•    Hiding behind procedures and rules

Level 1: Solid/frozen
(control takes over and change is no longer possible)
Inflexible, rigid burocracy – “that’s the way things are done!”

Characteristic behaviour patterns for this level
•    No interest or excitement
•    No initiative
•    No exchange or communication
•    No motivation
•    Standing still
•    CYA
•    Feeling of Powerlessness
•    Retreating to and hiding behind rules
•    Avoiding work
•    Lack of motivation

A topological observation of the heat model

An interesting observation, and one which may help explain the entropy analogy, is the fact that the characteristic behaviour patterns seen when the system cools down to the solidifying level and similar to the ones seen at the burning level. The system seems to wrap around topologically (spiral?), similar to the bottom section of the Cynefin butterfly model.

In the “turning up the heat” model, we differentiate between 5 different levels of heat: burning, cooking, cooling, congealing, and solidifying. It takes effort and energy to maintain any level of heat. The dissipation of heat in teams can be thought of as ‘social entropy’.  The natural forces of entropy as described in the Second Law of Thermodynamics apply as much to social organizations and interpersonal relationships as they do to molecules or galaxies.

Why do teams, even ones that are experiencing problems, not want to change? My thesis is that most of the time people and teams are in stasis and so resistant to change. To make change occur we have to raise the heat just enough people no longer feel comfortable in their current environment. I believe that when we see teams that stop doing TDD or other practices that it’s often because the heat has been turned off. Working in a rigorous fashion I recommend that we only make one change at a time in response to problems we want to improve. This allows us time to observe the system for retrospective coherence and adapt accordingly. It also reflects the fact that changes happen with a delay, and helps us to avoid over-correction (see Dörner’s example, later in this chapter).

The use of heat has analogy in cooking – most people on turn heat on or off but as coaches we need to understand the five levels. To connect the analogy:

Burning – food tastes burnt, teams fall apart.
Cooking – flavours in food are well integrated, teams adapt to new ideas
Cooling – bacteria grows in food, teams stagnate and start to stop using tools
Congealing – teams are starting to lose their flexibility and lock in their habits
Solid/Frozen – bureaucracy has set in, there are forms to fill out and sign offs

A coach needs to recognise the characteristics of a team in each of the stages. Most often, a team will be moving between levels, unless they’ve got to the state of complete burnout or complete stasis.

In the next book excerpt, we’ll look at the individual levels in more detail. It’s challenging for me to explain them, and I find that they’re most easily defined by example.

Vegetable soup

My grandfather was a fantastic chef, and he was a great inspiration to me. My parents named me after him, and not only did I inherit his love of, and talent for, cooking, but unfortunately also his girth. One of the first things I learned from him, at a very early age, was how to make vegetable soup. OK, maybe he just wanted some cheap help for mise en place, but at that age, I didn’t care.

Vegetable soup is simple. It consists of carrots, celery, onions, leeks, some herbs, and lots of water. That’s all. He’d let me peel the carrots and onions, and then he’d chop up all the ingredients and throw them into a pot that was so big it reminded me of something the cannibals threw the missionaries into in an old movie. An hour or so later, magic had happened, and out of all these ingredients emerged a tasty soup.

One day, after I was old enough to be allowed to use a small knife myself, I decided to surprise my grandfather and cook vegetable soup myself. Early one morning, I snuck down into the kitchen, got the ingredients, prepped them, then threw everything into the pot and waited for my grandfather to come. When he entered the kitchen, I jumped up, ran over to him, told him I had a big surprise for him, and proudly pulled him over to the stove. He looked into the pot, and what he saw was – mush. Careful to not upset me, my grandfather said that I had forgotten something. “But I put in all the ingredients as you always do,” I said. “Yes,” he said, “but the most important thing in making soup is not an ingredient itself – it’s turning up the heat.”

The same thing happens to us every day. We get a group of great people together, expect they’ll make a great team, and then watch as they act like a bunch of idiots, without understanding why this is happening, or what we still need to do to get them to work well together. We’re forgetting the main ingredient!

The team is the fundamental unit of software development. A team of skilled and highly motivated individuals, creative, focused and productive, is the cornerstone of the Agile philosophy. It does not follow that a group of people, however carefully selected, will auto-magically form a team capable of producing excellent software, on time and within budget.

Any group of people will self organize, according to the social and psychological principles, or ‘human nature’ that have influenced human interactions since the days of cave dwellers. Self-organisation will happen eventually, then, but there are two reasons why leaving a group alone to self organise for software development will almost undoubtedly fail. First, few projects incorporate the time or resources necessary to allow a stable organization to emerge before any work is done. As the saying goes, ‘a drop of water may hollow out the hardest stone,’ but we can’t afford to wait that long. Second, the team that emerges will be likely to have developed according to individuals’ ingrained behaviour patterns, many of which will reflect our primate ancestry. All groups of primates self-organize. This mainly results in the establishment of a pecking order, a rank and power hierarchy. Self-organization ends up being a fight for alpha male dominance. This what probably not what you want – unless you’re the alpha.

So, for a group to become a team, you must turn up the heat high enough that the team members cannot maintain their ingrained behavioural patterns.

If you don’t increase the heat, people won’t need to change the way they normally behave, and so they won’t change. Any self-organization will revert to being the basic primate fight for alpha male dominance. Unfortunately, this isn’t always an easy thing to do. Another thing I learned from my grandfather: for most people who haven’t trained as cooks, heat is a Boolean. It’s either all the way on, or all the way off. A good cook, however, knows how to regulate heat, how to use it to his advantage. He also knows which kind of heat to use in which situation. Too much heat, and things burn. Too little, and they don’t cook, don’t blend well enough to become soup – or whatever else you’re cooking. No heat at all, and they stay in their primitive state – raw. Only when the heat is right does cooking happen, do flavours blend, whilst retaining nuances of individuality.

It’s the same thing with people. Too much heat, and they burn out. Too little, and they don’t feel the need to change and adapt. Only when the heat is right do they adapt their behaviour and meld together to form a team.

As the posts go on, we’ll look at the Heat model, its different temperatures, the thermometers (analysis tools) available, and the various stoves (interventions and techniques) available to turn up the heat on a team.

Another reason for posting book excerpts is to establish a bit of precendence. I’ve been researching and talking about why all this agile stuff works for years now, but since I never wrote much about it, and since a lot of others are starting to learn the same lessons, I guess I have to “stake my claim”.

I released a small excerpt back in October, which was posted on the Agile Journal website. It was intended to be the first part of a multi-part post. Unfortunately, after posting, 2 things happened: first, I crashed and burnt out, which put me out of work for a few months; and second, during the time I had to work on the book, the whole direction and structure of it changed. This meant that a continuation of the post would not have been a book excerpt, but a separate article, and I didn’t have the energy to do that. My apologies to all of you waiting for the follow-up.