• Mark Haddon, The Curious Incident of the Dog in the Night-time
• Ursula leGuin, The Dispossessed
• Dava Sobel, Longitude

In addition, it would be a good idea to view at least this excerpt from the Star Trek Next Generation episode Darmok.

Scientific Papers

If you’re attending CALM-Alpha, you’ve probably read the relevant papers already.  If not, this would be my personal list of papers to read to start out:

• Cynthia Kurtz and Dave Snowden, The new dynamics of strategy: Sense-making in a complex and complicated world
• Dave Snowden and Peter Stanbridge, The Landscape of Management
• Dave Snowden, Multi-ontology sense making; a new simplicity in decision making
• Joseph Pelrine, On Understanding Software Agility— A Social Complexity Point Of View

The Cognitive Edge site has many other papers worth reading.

Figure 1: Priority poker

A question popped up recently in one of my product owner courses which made me re-think a few premises about priority poker. Two groups came up with a different number of stories for an exercise we were doing, but I gave them both the same number of poker chips. Looking at the results of the group’s prioritisation exercise, I realised that there must be some connection between the number of stories and the number of chips. What would be the optimum relationship/proportion between chips and possible stories to ensure the best possible prioritisation? With too few chips, there’s not enough flexibility in choosing. With too many chips, things can become equally important, and one loses the nuances of such an exercise [2]. Is there a mathematical formula to calculate this? Empirical data? Rules of thumb?

I’m lucky to have many friends who are a lot smarter than I am, and who don’t mind me asking them questions like these. In this case, my victim was my friend Dr. Natasha Zharanova, who works in the finance department at eBay in Amsterdam, and who did her PhD in economics at Princeton. Natasha once attended a product owner training I did while working at eBay where we did priority poker, and she mentioned to me that she had done her master’s research on a similar topic, cumulative voting. Her answers to my questions caused me a thoughtful and sleepless night, tossing and turning in bed thinking about this question, until I finally gave up and started hacking Smalltalk at 3:30 AM to run some simulations.

Natasha pointed out that one of the goals of cumulative voting is to ensure adequate minority representation. Indeed, when we did the exercise together, we had given various stakeholders a few chips, and their strategic placement of chips influenced the final prioritisation in interesting and unexpected ways. (Note: you probably know cumulative voting as “dot voting”, where each member of a working group gets 3 dots to place on their e.g. favourite topics to be discussed).

Some points which Natasha pointed out to me:

- the total number of votes allotted to a voter is equal to the number of winning candidates.

- This works when all voters are seen as equal. When it comes to shareholders, usually the number of votes one gets is also proportional to his/her number of shares (I.e., a shareholder with 10\% of shares gets 9 times less than one with 90\% of shares), but the total sum is still related to the number of winning candidates.

- Of course, in most examples where this is used (I.e., board of directors elections), the number of open positions is fixed. In my example, though, the number of stories selected is not known in advance; but perhaps this is where another assumption / rule of thumb could be introduced? Something along the lines of “there is no way that more than 5 stories can be top priority simultaneously”? Then everybody gets 5 chips…

With a product owner (in a strict sense), the number of voters is 1. This simplifies things a bit, as it forces the focus onto the relationship between votes (chips) and options (user stories). Natasha’s question of “how many projects can be top priority simultaneously?” got me thinking, and I ended up reversing it, asking “how many projects or stories can we afford to ignore?”

I think the optimum number of votes would be the minimum number that ensures that each option can receive a unique, non-zero number of votes. For 1 item, it would be 1 chip. For 2 items, it would be 2+1, or 3, chips, for 3 items 3+2+1, or 6 chips, etc. The number of votes would be a number in the sequence 1, 3, 6, 10, 15, 21, 28 … or essentially, for n items:

or simply

This would be the smallest number of chips that would allow all options to be represented, while still forcing an increase in value of one option at the cost of another.

Looking at the first formula, the last (n – 1) becomes interesting. This is our lower bound, normally 1, but what it represents is the number of options we can afford to ignore, right? If we substitute this number with a higher bound, thus saying that we can afford to ignore more items, then the number of votes necessary to ensure prioritisation decreases. For me, this lower bound would be the number of stories that we can allow to have 0 chips.

I couldn’t get that number sequence 1, 3, 6, 10, 15, 21, 28 … out of my head. I was sure I’d seen it somewhere before, but couldn’t remember the name. Factorial? Sort of, but adding, not multiplying. Fibonacci? Sort of, but summing up all previous numbers, not just the last two.

In cases like these, I normally call on my math wizards, Tatiana Pastukhova and Wiggert Loonstra. They both gave me the answer: triangular numbers. Think billiard balls here.

Figure 2: Triangle numbers

A triangular number is the number of balls in an equilateral triangle completely filled with balls. Looking at the Wikipedia page on triangular numbers, I found the most efficient way to calculate them:

So, to figure out the number of chips you need for prioritising your backlog using priority poker, first figure out how many stories you can afford to do without (this is a good reality check in any case). Then let

and the number of chips needed will be

Simple, right?

Notes:

1. See Emil J. Posavac and Raymond G. Carey, Program Evaluation: Methods and Case Studies, Prentice-Hall, 1989; W. Edwards and J.R. Newman, “Multiattribute Evaluation”, in H.R. Arkes and K.R. Hammond (eds.), Judgment and Decision Making, Cambridge University Press, Cambridge, England, 1986, 13-37; and C. Kirkwood, Strategic Decision Making: Multiobjective Decision Analysis with Spreadsheets, Duxbury Press, 1997, 53-61.
2. See Barry Schwartz, The Paradox of Choice, Harper Collins, 2005.

The workshop started with Kurt explaining the motivation behind organising it, after which followed the round of what Mark McKergow called “creeping-death” introductions. Although I also dislike such introduction rounds, it was fascinating to hear what diversity and depth of experience the attendees brought with them, and it was comforting to hear that they understood the same thing that I do for terms such as complexity and self-organisation.

After the introductions, we enjoyed the first part of a talk by George Rzevski, professor emeritus for complexity science at the Open University. I won’t go into the talk in detail; the slides are available on the workshop website. One slide, though, was very thought- and discussion-provoking. There is still no generally agreed-on definition of complexity, and dozens of attempts exist. George presented his 7 criteria for complexity, which we found that (although there were differing points of view that could be discussed) seemed to all of us to be ok, good enough, a good first step towards a common definition.

George Rzevski’s Seven Criteria for Complexity

1. INTERACTION – A complex system consists of a large number of diverse components (Agents) engaged in rich interaction
2. AUTONOMY – Agents are largely autonomous but subject to certain laws, rules or norms; there is no central control but agent behaviour is not random
3. EMERGENCE – Global behaviour of a complex system “emerges” from the interaction of agents and is therefore unpredictable
4. FAR FROM EQUILIBRIUM – Complex systems are “far from equilibrium” because frequent occurrences of disruptive events do not allow the system to return to the equilibrium
5. NONLINEARITY – Nonlinearity occasionally causes an insignificant input to be amplified into an extreme event (butterfly effect)
6. SELF-ORGANISATION – Complex systems are capable of self-organisation in response to disruptive events
7. CO-EVOLUTION – Complex systems irreversibly co-evolve with their environments

After lunch, we held an extended poster session. Having never done a poster session before, I simply drew one using visualisation techniques (thanks to Martin Haussmann for teaching me!), and was surpised at the positive responses I got. My topic is the bi-directional insights of using the Cynefin method to understand why Agile works, and using Scrum as a framework for managing work in the complex domain. I’ll be presenting my paper on this later this morning, and am looking forward to the feedback and discussions that will come. During the poster session, I also had interesting talks with Mark McKergow on solution-focused therapy and complexity, and with Ed Olson on his CDE model and the Cynefin ABIDE model, which I use quite heavily in my work on self-organising teams.

When the poster session was done, there was still some time until the evening barbeque, and George Rzevski presented the second part of his talk. He’s done fascinating and world-class work, I highly recommend checking out his slides.

Dinner, drinks and conversation followed, and by the time I got to bed, I was suffering from information overload, and was too tired to write this blog post, which is why I’m doing it now.

More later.

I was pleased because this acceptance is recognition of the research and work I’ve been doing for almost 15 years on understanding why Agile processes work. Even more, I was happy because my session on this topic, as well as my session on coaching self-organising teams, were both rejected for the Agile 2010 conference, although both sessions had been presented there twice in the past (as well as at many other conferences), and had received excellent reviews each time.

I’m starting to think that the average Agile practitioner’s position with regards to complexity science etc. can best be described as “a little knowledge is a dangerous thing”. Terms such as complexity, self-organisation, emergence, and so on, are bandied about in a way that they’ve become as content-free as the term “Agile” itself. People also seem to be content to accept such idiotic statements as “Architecture self-organizes around working code”. I don’t know about you, but the last time I saw architecture self-organising was last Friday night, when a re-run of Terminator 2 came on late-night television.

In any case, that’s enough for this little rant. I’m going to get back to my research and writing, and hope that some of you will be as interested as I am in finding out why this Agile stuff really works. More (including the finished paper) later.

http://www.cio.com/article/447712/_Agile_Leadership_Lessons_for_the_Suits
http://www.agilitrix.com/2009/12/coaching-self-organizing-teams/
http://www.infoq.com/presentations/coaching-self-org-teams
http://www.infoq.com/news/2008/08/coaching_teams
http://agilepainrelief.com/notesfromatooluser/2008/08/coaching-self-organizing-teams.html
http://chcrudy.wordpress.com/2009/08/30/joseph-pelrine-self-organizing-teams-and-turning-up-the-heat/

Now that’s what I call customer service! Thanks, Captain Martin Simon!

One of the reasons I met up with him was because I’m planning a small surprise party for another close friend, and I wanted to benefit from his experience and advice. What made Michael’s parties so good? During our chat, we realized that it was it the people attending, and their conversations. Not only that, it was also the music. Of course, we couldn’t forget the food and drink.

In the end, I decided to ask Michael to help me organize the party. He knew exactly what needed to be done for a party to be a success, and I was convinced that, if we just did the same things he always did, my party would turn out great. So, what was the first thing he said?

“Joseph, if you want to have a great party, you have to invite all of my friends.”

I thought I didn’t hear right. I wanted to invite my own friends, but on the other hand, I knew that with his friends, the party would be a smash, like his parties. Sensing my disappointment, he tried to cheer me up by saying that he could save me a lot of money. I wouldn’t need a DJ, he said, because he noted all the songs his DJ played, and he could just give me a playlist for my iPod. I’d have to lock the iPod away, so that no one could manipulate the song order, but he was sure that with these specific songs, played exactly in this order, the party would be a success. Also, it seems he videotaped his last party, and noted what everyone ate and drank. He’d provide me with scripts for all the guests, with precise timings, so that everybody ate and drank the proper things at the proper time. Of course, all conversation and interaction would also be scripted.

So, just imagine you plan to have a party, and it must be as good as the last one you held. You invite the same people, and just to make sure you’ve covered everything, you tell them to wear the same things, and speak only to the same people as last time. You’ve got the same music playlist, the same food, drinks and layout. Will this be a good party? Does something seem wrong with this story?

The answers to these last questions seem logical, although you can’t easily explain why. There is a reason, though, and this reason is key to understanding the basic issues surrounding complex problems and agile methods.

It’s called retrospective coherence.

One thing that makes complex systems complex is their causality. In an ordered system, if you do something, you expect a specific result. Do it again, expect the same result. It’s that simple. In a complex system, though, the causality emerges as the system emerges. As the party goes on, the reasons for its success become established, not before. After it’s over, you can say that a party was a success, and that these people were there, this music was played, and this much beer was downed  – but you cannot say that the party was a success because these people were there, this music was played, and this much beer was downed!

In a complex system, causality emerges as the system itself emerges, so that at the end, you can say how you got to where you are, but you can’t guarantee that by doing exactly the same things, you’ll get to the same place again – and you probably won’t. In complex systems, we say the causality is retrospectively coherent.

So let’s take a closer look, and consider what it actually was that made Michael’s party so good. It wasn’t any one factor, but how they interacted. It wasn’t the components, but the emergent complex system, that was a success. An interesting thing to note here is that the things that people often interpret as being the causes of a success aren’t that, they’re the symptoms. For instance, was it the people that made the party a good one? The music that was played? His choice of fine food and drink? Well, yes and no. Michael played a significant part because he’s a great host. He’s constantly walking around, looking, questioning. Is anyone standing all alone? Introduce them to a group! Does everyone have a drink? If not, get some more beer! Are people dancing? Trying to talk? Put on upbeat or down-tempo music as appropriate! All the time, the good host is surveying the scene and making small adjustments.

As a good host, Michael drew some boundaries. His bedroom was off-limits. His single-malt collection locked away. And, he set up a few things that people would be attracted to: a wide-screen television, a keg of beer, some games maybe. Every party needs attractors. Did you ever wonder why so many people congregate in the kitchen at a party, even if it’s the smallest room?
One thing he didn’t do is to define success, prescribe or proscribe behaviour (could you imagine saying that a party would only be a success if someone danced on the table with a lampshade on their head?). He let the party evolve, observed which patterns emerged, gently supported the good patterns, disrupted the bad patterns as soon as he noticed them, but generally let the party run itself.

Now shift the focus to a project. We tend to make the mistakes above when it comes to project planning. What worked last time? Why? Well, it must have been the people, the methodology, the meeting schedule – let’s do it the exact same way again. Like the repeat party, a project planned on this basis is likely to be a flop.

Contrary to Einstein’s definition, in a socially complex system, insanity is doing the same thing over and over again and expecting the same result!

(n.b. Dave Snowden has a classic story on organizing a child’s birthday party, and his story served as an inspiration for my version. You can find the story here. Thanks, Dave!)

Level 5: Burning
(results in team burnout and death marching)
At the burning level, the heat is too high. The stress level results in chaos, aggression, and a regression to primitive behaviour patterns. This is a dangerous level, since the high pressure increases the friction between the team members, increasing the heat even more.

Characteristic behaviour patterns for this level
•    Countless overtime hours leading to no result
•    Stress
•    Disagreements, fights
•    Sickness
•    Lack of communication
•    Inefficiency
•    Hectic
•    Blame
•    Restlessness
•    Self-preservation
•    Paralysis
•    Panic
•    Fear of attack

Level 4: Cooking
(ideal temperature for continuous improvement)
This is the optimal temperature for mixing up and forming teams. At this level, the heat is high enough to force disruption of behaviour patterns, but not so high that team members regress into pre-conventional mode.

Characteristic behaviour patterns for this level
•    Differences are resolved constructively
•    Continuous improvement
•    Enthusiasm
•    Work is fun
•    High productivity
•    Constructive discussions
•    Consensus
•    Joy
•    “We” – feeling
•    Self-reflection
•    Open discussions
•    Progress
•    Results
•    Creativity
•    Fun
•    High performance
•    Organised chaos
•    Good communication

Level 3: Cooling (Stagnation)
(discipline is lost and bad behaviour begins to fester)
If not enough heat is applied, or when things cool down after a while (entropy), a team enters the cooling or stagnating stage. In the kitchen, this is where what was once a fine-tasting soup has become a substrate for fungus and bacteria growth. Most health departments even have laws requiring cooked foods to be cooled down within a certain short period of time, as to pass through this stage as quickly as possible.

Characteristic behaviour patterns for this level
•    Slacking off
•    Work becomes routine
•    Loss of discipline
•    Monotony
•    Indifference
•    9 to 5
•    Too much discussion
•    Minimal communication

Level 2: Congealing
(team gets too comfortable to achieve and bad habits become the norm)
Increasing pressure to adhere to “norms“ – “that’s the way we do things here”

Characteristic behaviour patterns for this level
•    Change becomes difficult
•    Many meetings
•    Multiple implementation of work
•    Problems become difficult to solve, because everything’s a mass/mess
•    Boredom
•    Much talk, no action
•    Consensus becomes difficult
•    Resignation
•    Splitting of the teams into cliques
•    Everyone works for themselves
•    Minimalism
•    Defensive stance
•    Hiding behind procedures and rules

Level 1: Solid/frozen
(control takes over and change is no longer possible)
Inflexible, rigid burocracy – “that’s the way things are done!”

Characteristic behaviour patterns for this level
•    No interest or excitement
•    No initiative
•    No exchange or communication
•    No motivation
•    Standing still
•    CYA
•    Feeling of Powerlessness
•    Retreating to and hiding behind rules
•    Avoiding work
•    Lack of motivation

A topological observation of the heat model

An interesting observation, and one which may help explain the entropy analogy, is the fact that the characteristic behaviour patterns seen when the system cools down to the solidifying level and similar to the ones seen at the burning level. The system seems to wrap around topologically (spiral?), similar to the bottom section of the Cynefin butterfly model.

In the “turning up the heat” model, we differentiate between 5 different levels of heat: burning, cooking, cooling, congealing, and solidifying. It takes effort and energy to maintain any level of heat. The dissipation of heat in teams can be thought of as ‘social entropy’.  The natural forces of entropy as described in the Second Law of Thermodynamics apply as much to social organizations and interpersonal relationships as they do to molecules or galaxies.

Why do teams, even ones that are experiencing problems, not want to change? My thesis is that most of the time people and teams are in stasis and so resistant to change. To make change occur we have to raise the heat just enough people no longer feel comfortable in their current environment. I believe that when we see teams that stop doing TDD or other practices that it’s often because the heat has been turned off. Working in a rigorous fashion I recommend that we only make one change at a time in response to problems we want to improve. This allows us time to observe the system for retrospective coherence and adapt accordingly. It also reflects the fact that changes happen with a delay, and helps us to avoid over-correction (see Dörner’s example, later in this chapter).

The use of heat has analogy in cooking – most people on turn heat on or off but as coaches we need to understand the five levels. To connect the analogy:

Burning – food tastes burnt, teams fall apart.
Cooking – flavours in food are well integrated, teams adapt to new ideas
Cooling – bacteria grows in food, teams stagnate and start to stop using tools
Congealing – teams are starting to lose their flexibility and lock in their habits
Solid/Frozen – bureaucracy has set in, there are forms to fill out and sign offs

A coach needs to recognise the characteristics of a team in each of the stages. Most often, a team will be moving between levels, unless they’ve got to the state of complete burnout or complete stasis.

In the next book excerpt, we’ll look at the individual levels in more detail. It’s challenging for me to explain them, and I find that they’re most easily defined by example.

Last week, I attended the first Swiss Lean/Agile/Scrum conference. As usual, the Swiss take a long time to catch up with new ideas and technology, but when there’s money to be had, they’re right up at the front of the line. Anyway, Ken Schwaber gave a very interesting presentation on the concept of “Done”. He showed the canonical burndown chart, and then took the individual vectors apart.

Seeing the variables separated like that got me to thinking, and I had the chance to run my thoughts past Karl Scotland and Keith Braithwaite, both of whom went out for a beer with me afterwards.

Back in the early days of XP, we defined a set of project variables:

The rule went, “Time, Resources, Quality, Scope – choose three”. Whichever three you chosse, the fourth variable would be a function of the other three. Which three were actually chosen was the customer’s decision. Some customers (and managers) don’t understand this rule, and try to grab control of all 4 variables. By doing that, the first variable that’s dropped is quality, followed by time.

So, how do these sets of variables map to each other? The first two are easy:

• time = time
• backlog = scope

The other 2 are a bit more difficult:

• V = f(R)

Velocity (burndown rate) is a function of the available resources. Regardless of what you have to do, having more resources will normally allow you to go faster. This function is non-simple and non-linear, unfortunately: as Fred Brooks rightly says, “adding manpower to a late software project makes it later”, but the simple case is a more or less linear function.

Quality is even more complicated:

• q = ∂V/∂t

As Dan Rawsthorne says, “Quality is the first derivative of the burndown”. The quality of a product is directly related to the development velocity/burndown rate.

So, so much for that thought. Keith spun the thought even further, proposing in his recent blog post that you can increase velocity by increasing quality. I agree with him, and I recommend you read the post.