The ultimate sophistication

Simplicity is the ultimate sophistication” said Leonardo Da Vinci recognising how difficult it is to keep things simple. Simplicity is hard work and is not, as some might want to believe, an easy way out.

In Chapter 4 of their book Implementing Lean Software Development, Mary and Tom Poppendieck identify complexity as the root cause of all waste in software development.

If we were to look for the root cause of waste in software development, a really good candidate would be complexity. Complexity calcifies our code and causes it to turn brittle and break.

Implementing Lean Software Development

Complexity is the result of passive developers that allow the code to deteriorate over time. To achieve simplicity developers have to actively and consciously seek to improve the code. They have to be on their toes to identify the trouble spots and tackle them relentlessly through continuous incremental refactoring.

In Lean, work is divided into Minimum Useful Feature Sets (MUFS) and these sets are deployed one at a time. Each set would include a number of refactorings that keep the design clean and simple.

… the code is refactored and simplified with each new feature set. This helps minimize the complexity of the code base, thus minimizing its lifecycle cost.

Implementing Lean Software Development

The importance of refactoring can not be over-emphasised. It is an integral part of any agile development effort and needs to be recognised as such. It is too easy to fall in the ‘ain’t broke don’t fixt it’ trap and let the code rot to a slow and painful death. The only way to keep a lid on complexity is though the aggregation of small, frequent refactorings.

This in turn requires a good suite of tests to be able to refactor with confidence, CI builds to detect when tests are broken, frequent release of MUFS into production and, perhaps even harder to get, a general understanding that refactoring is an ongoing process that continues for the life of the product or project.


From Projects to Products in Enterprise Applications

The Project model commonly used in custom-built enterprise applications is fundamentally flawed. In Chapter 3 of their book Implementing Lean Software Development, Mary and Tom Poppendieck suggest that this type of software would benefit greatly from following a Product model instead.

Projects have a beginning and (apparently) an end. Products, on the other hand, have a beginning and (hopefully) no end. Software is much more like a product than a project, because most good software lives on and changes for a long time.

Projects are Waterfall

It is now common knowledge that waterfall software development doesn’t work. And yet we ignore the fact that the idea of software development as a series of steps with inputs and outputs came directly from traditional Project Management theory. In 1982 Dan McCracken and Michael Jackson published a note in SIGSFOT titled Life cycle concept considered harmful. They said:

Any form of life cycle is a project management structure imposed on system (software) development.

Software Development and Project Management are often confused but are clearly distinct, as shown in the table below from Leading Lean Software Development.

Let us not confuse

Software Development

Project Management

Success = accomplishing the system’s overall objective Success = achieving planned cost, schedule, and scope
Learning/feedback cycles Phases with exit criteria
Quality by construction Quality by inspection
Encapsulate complexity Manage scope
Skilled technical leaders Resource management
Technical disciplines Maturity levels
Historically Robust Historically Fragile

It is clear that projects are incompatible with Agile practices.

Projects waste money

One of the main reasons for projects being inherently waterfall is budgeting. Projects get all their money allocated at the beginning and their success is measured against that budget. This encourages Big Requirements Up Front to be able to come up with an estimate. We know we suck at estimating anything but we do it anyway.

The Standish Group’s “Chaos Report” found that only 31 percent of IT projects are reasonably on time and on budget and that on average projects cost 189 percent of what was originally estimated. These results belie the beliefs that fixed-price projects put a maximum on the financial outlay.

Is Fixed-Price Software Development Unethical?

An even bigger concern with budgets, particularly in the public sector, has to be the culture of ‘spend it or lose it’ that they foster.

One problem with cost budgets is that they are viewed as entitlements, so they act as a floor as well as a ceiling on spending. Similarly, the problem with plans is that Parkinson’s Law holds: Work expands to fit the time (budget) allotted.

Leading Lean Software Development

In contrast, Products are usually funded incrementally.

This incremental funding is a clear signal to all that the scope is expected to evolve as knowledge is gained. The success of product development is generally measured based on the market share and profitability that the product eventually achieves.

Implementing Lean Software Development

It is interesting to note the emergence of Beyond Budgeting as an alternative to traditional management practices, including budgeting, and how well their ideas align with Lean & Agile thinking.

Projects have no champions

People are moved (or sacked) between projects with no regard for the impact on the development process. Decisions are made not in the best interest of the customer but only to satisfy a project plan or a budget. Developers get accustomed to not finishing anything and don’t feel any responsibility for the code they write. They leave large amounts of unfinished code and ideas behind them.

Joel Spolsky puts it this way:

As soon as your program gets good enough, you have to stop working on it. Once the core functionality is there, the main problem is solved, there is absolutely no return-on-investment, no business reason to make the software any better. So all of these in house programs look like a dog’s breakfast: because it’s just not worth a penny to make them look nice. Forget any pride in workmanship or craftsmanship.

In this context it is impossible to have a Product Champion, Front-line Leaders or any kind of leader for that matter.

A product company on the other hand is forced to continuously improve its products to remain competitive. Developers are encouraged to find new ways to make the product better, easier to use, nicer, more robust. This keeps them honest and sharp.

All the innovation that results from competition in the marketplace ultimately benefits customers. None of this is present inside the enterprise walls but that is no excuse for sloppy software.

Just because software is being developed for a one-of-a-kind application does not excuse an organization from the obligation of creating value and delighting customers. The objective is still to create software that will deliver outstanding value to the organization that is paying for it.

Implementing Lean Software Development

In the public sector anything less than top-notch quality software is an irresponsible waste of tax-payers money. This level of quality will only be achieved when we start treating software as a product that needs to be sold to our business areas.

The Seven Principles of Lean Software Development

Chapter 2 of Implementing Lean Software Development From Concept to Cash deals with the seven principles of Lean Software Development.

Here are some quotes and personal thoughts about each principle.

  1. Eliminate Waste

    The biggest waste in software development is unnecessary features and unfinished work. They add complexity to the code and cause confusion. People make wrong assumptions based on code that should not even be there. Unnecessary features equate to over production in the manufacturing world.

  2. Build quality in

    Zero tolerance. Every defect that is identified is fixed immediately. Using TDD and CI your defect rate should be so low as to render a defect tracking system pointless. Testers still play an important role, but a tester finding a defect should be the exception not the rule.

  3. Create Knowledge

    Knowledge crunching. You will always know more tomorrow than you know today. Requirements and initial design are nothing but a forecast, an initial guess. Not only do you learn more about the business as you go along but you also learn about your own processes. Successful projects adjust their practices to rectify problems as they occur.

  4. Defer Commitment

    Leave your options open so that irreversible decisions are made as late as possible. This requires flexibility in the code areas more susceptible to change. There is a fine line between adding flexibility and producing waste.

    “In preparing for battle I have found that plans are useless, but planning is indispensable” – Dwight Eisenhower.

  5. Deliver Fast

    Deliver software so fast that our customers don’t have time to change their minds. This needs to be done with quality built in, otherwise it’s a hack.

  6. Respect People

    Self explanatory. Although it is worth noting that this has more to do with empowering people to make decisions and respecting their abilities and ambitions.

  7. Optimize the Whole

    A lean organization optimizes the whole value stream, from the time a feature is requested until it is deployed. This involves at least analysis, development and QA.  The separation between developers and testers is particularly harmful and common. They are often treated as two different teams with two different managers. Efforts to optimize each team separately are counter-productive.  Each team develops their own schedules and priorities which are not always aligned with the other teams.  The process as a whole suffers and the customer needs fall between the cracks.

Review: Leading Lean Software Development

Leading Lean Software Development: Results are not the point (2009) is the third book about Lean Software from Mary and Tom Poppendieck. Previously they wrote  Implementing Lean Software Development: From Concept to Cash (2006) and Lean Software Development: An Agile Toolkit (2003).

In a very clear progression they’ve gone from a toolkit where all the underlying principles are defined to implementation and then leadership. Too bad I only see this now after reading the last book first.

In Leading Lean Software Development they present 24 frames from which to view the software development process in terms of leadership. These frames represent manifestations of leadership across different aspects of the development process. Not one person is expected to fulfill all these leading roles. In fact the frames are grouped in fours and assigned to different leaders, with some overlap. Continue reading “Review: Leading Lean Software Development”