From Agile Development to New Software Engineering

In 1997, long before agile project management became a buzzword, a paper was presented at a military software conference, entitled “Disciplines Delivering Success” [1]. I find it sobering to read that paper 10 years later and see how little our organizations have advanced over the decades. Brown presented six “project-saving disciplines ignored by management”:

• Good personnel practices


• Planning and tracking using activity networks and earned value


• Incremental release build plan


• Formal configuration management


• Test planning and project stability


• Metrics

It is sobering because those are very similar to the starting recommendations offered by agilists today. Clearly we agilists are not unique in requesting these items--good project managers have been requesting them (and getting ignored) for a long time. I show that list so that we don’t have to re-hash the past yet again. I assume you will take of those things.

More interesting than rehashing the past is deciding how to move forward. To this end I present how software engineering is being revised to take account of what has been learned in the agile movement. It indicates where we should put our attention in the next decade.

Leaving aside navel-gazing questions concerning the definition of the term “software engineering,” but getting straight to the point, the new variety of software engineering is built on three legs:

• Craft


• Cooperative gaming


• Lessons from lean manufacturing

Craft

Doing a bit of research [2], I found that the term “engineering” prior to the World War II-contained elements of craft, both in field practice and in academia. During the war, applied physicists amazed everyone with what they could do, with the result that engineering academia started emphasizing the mathematical aspects of engineering rather than the craft aspects.

We are seeing renewed attention to the craft aspects of our profession. Being in a craft profession brings with it certain expectations, such as lifelong learning, or “deepening” the proficiency in one’s craft. Programmers shouldn’t think that just because they once learned to program a computer, their programming skills are still sufficient. Craft professionals, programmers, project managers and others should always be looking to learn new tools and techniques, and get better in the ones they have.

In software development, we can immediately name seven craft specialties:

• Deciding what to build


• Modeling


• Managing the people and the project


• External design


• Large-scale design (architecture)


• Fine-scale design (programming)


• Verification and validation

In other disciplines, the technical craft set will vary, but managing the people and the project remain. For project management, it should mean that people are not simply assigned to project management positions from off the street or out of their programming cubicle, but rather they enter the position in recognition that they are at the starting stage of a new craft and skill set.

At the moment, what concerns us is that people are not interchangeable, nor are skill sets. Each involves life-long learning.

Cooperative Gaming

When people develop software, they are working to understand a problem that they don’t fully understand, and which keeps changing underneath them. They are inventing a solution that they don’t fully understand and which keeps changing underneath them. They are forced to use contrived languages that they don’t fully understand, and which keep changing underneath them. Each person is making decisions, each decision causes a ripple of economic consequences--and the project is economically constrained.

To rephrase that in a pithier way, software development is a cooperative game whose purpose is to deliver working software. The moves in that game are only to invent and to communicate to other people and to the computer.

Considering software development an economically constrained cooperative game of invention and communication helps us understand project management. It highlights that:

• Every game and every situation are potentially different--there is no formula for winning the game. Different, even opposite strategies may be needed at any instant.


• The quality of a move in the game is not absolute, but rather is only relative to how it improves the position of the team for its next move.


• The quality of community and communication among the team members matter enormously--they often make the difference between success and failure.

These lessons are immensely valuable for the project manager. Rather than pushing for “complete requirements” or a “complete design”, the project manager should be asking, “How many requirements, and at what quality level, do we need now so that we can get to a good position for our next move, considering the time and resource constraints we have on us at this point?”

The answer to that question varies by project and even by moment in a project. Sometimes, the answer is “relatively complete” (for fixed-price contracts). Sometimes it is “just a few, medium quality” (for certain internally funded projects); at other times it is anything in between. The same question applies to completeness and quality of the different elements of the design, of testing, of the project plan itself.

The cooperative game idea also highlights the importance of people, their talents, their skills and their interactions. Their interactions are important because people at odds with each other withhold information. Every piece of information not exchanged costs the project lost time.

In capsule form, the speed of the project is limited by the speed at which information flows between people. Everything that slows the movement of ideas between people slows the project!  Bad attitudes slow projects, distance slows projects and certain communication tools help more than others. This is why agile developers have their own preferred communication tools [3] (for example, information radiators [4]).

The cooperative gaming lexicon is rich, and I will draw more from it in the next column.

Lean Manufacturing

It is not immediately apparent that manufacturing has anything in common with software development. Manufacturing is all about making the same thing over and over, while software development is all about making something different each time.

If, however, we consider decisions as the thought-worker’s version of parts inventory, then suddenly we see that people hand other people decisions, people wait on each other for decisions, and some people have a bigger backlog of decisions than they can handle at the moment. Examining the dependency network of decisions in play in an organization, we see that this network is very similar to the dependency network of parts in a manufacturing plant. Quite surprisingly, the same mathematics applies to the situation, and many of the same strategies apply: just-in-time, pull, continuous flow and so on.

I will need one entire column to describe the application of lean manufacturing to project management. In the meantime, other people have written quite a lot on this topic [5, 6]. In the next columns in this mini-series, I will develop these ideas more, and show how they drive project management, whether agile, adaptive, neither or both.

[1] Brown, N., “Disciplines Delivering Success,”STC 1997.

[2] Cockburn, A., “The End of Software Engineering and the Start of Economic Cooperative Gaming”

[3]Cockburn, A., “What the Agile Toolbox Contains,” online at http://www.stsc.hill.af.mil/crossTalk/2004/11/0411Cockburn.html

[4] Cockburn, A., “Information Radiators”, online at http://alistair.cockburn.us/index.php/Information_radiator

[5] David Anderson’s site: http://www.agilemanagement.net/

[6] Mary and Tom Poppendieck’s site: http://www.poppendieck.com/publications.htm

Dr. Alistair Cockburn was named one of “The All-Time Top 150 i-Technology Heroes” in 2007. He is an internationally renowned project witchdoctor and IT strategist, best known for co-authoring the Manifesto for Agile Software Development and articulating how to write effective use cases. His specialties are organizational redesign and project management strategies using agile and lean principles. You might enjoy getting pleasantly lost among his many articles and talks posted at http://Alistair.Cockburn.us. http://www.gantthead.com/content/articles/

[gallery]