The ultimate sophistication

Simplicity is the ultimate sophistication” said Leonardo Da Vinci recognising how difficult it is to keep things simple. Simplicity is hard work and is not, as some might want to believe, an easy way out.

In Chapter 4 of their book Implementing Lean Software Development, Mary and Tom Poppendieck identify complexity as the root cause of all waste in software development.

If we were to look for the root cause of waste in software development, a really good candidate would be complexity. Complexity calcifies our code and causes it to turn brittle and break.

Implementing Lean Software Development

Complexity is the result of passive developers that allow the code to deteriorate over time. To achieve simplicity developers have to actively and consciously seek to improve the code. They have to be on their toes to identify the trouble spots and tackle them relentlessly through continuous incremental refactoring.

In Lean, work is divided into Minimum Useful Feature Sets (MUFS) and these sets are deployed one at a time. Each set would include a number of refactorings that keep the design clean and simple.

… the code is refactored and simplified with each new feature set. This helps minimize the complexity of the code base, thus minimizing its lifecycle cost.

Implementing Lean Software Development

The importance of refactoring can not be over-emphasised. It is an integral part of any agile development effort and needs to be recognised as such. It is too easy to fall in the ‘ain’t broke don’t fixt it’ trap and let the code rot to a slow and painful death. The only way to keep a lid on complexity is though the aggregation of small, frequent refactorings.

This in turn requires a good suite of tests to be able to refactor with confidence, CI builds to detect when tests are broken, frequent release of MUFS into production and, perhaps even harder to get, a general understanding that refactoring is an ongoing process that continues for the life of the product or project.

From Projects to Products in Enterprise Applications

The Project model commonly used in custom-built enterprise applications is fundamentally flawed. In Chapter 3 of their book Implementing Lean Software Development, Mary and Tom Poppendieck suggest that this type of software would benefit greatly from following a Product model instead.

Projects have a beginning and (apparently) an end. Products, on the other hand, have a beginning and (hopefully) no end. Software is much more like a product than a project, because most good software lives on and changes for a long time.

Projects are Waterfall

It is now common knowledge that waterfall software development doesn’t work. And yet we ignore the fact that the idea of software development as a series of steps with inputs and outputs came directly from traditional Project Management theory. In 1982 Dan McCracken and Michael Jackson published a note in SIGSFOT titled Life cycle concept considered harmful. They said:

Any form of life cycle is a project management structure imposed on system (software) development.

Software Development and Project Management are often confused but are clearly distinct, as shown in the table below from Leading Lean Software Development.

Let us not confuse

Software Development

Project Management

Success = accomplishing the system’s overall objective Success = achieving planned cost, schedule, and scope
Learning/feedback cycles Phases with exit criteria
Quality by construction Quality by inspection
Encapsulate complexity Manage scope
Skilled technical leaders Resource management
Technical disciplines Maturity levels
Historically Robust Historically Fragile

It is clear that projects are incompatible with Agile practices.

Projects waste money

One of the main reasons for projects being inherently waterfall is budgeting. Projects get all their money allocated at the beginning and their success is measured against that budget. This encourages Big Requirements Up Front to be able to come up with an estimate. We know we suck at estimating anything but we do it anyway.

The Standish Group’s “Chaos Report” found that only 31 percent of IT projects are reasonably on time and on budget and that on average projects cost 189 percent of what was originally estimated. These results belie the beliefs that fixed-price projects put a maximum on the financial outlay.

Is Fixed-Price Software Development Unethical?

An even bigger concern with budgets, particularly in the public sector, has to be the culture of ‘spend it or lose it’ that they foster.

One problem with cost budgets is that they are viewed as entitlements, so they act as a floor as well as a ceiling on spending. Similarly, the problem with plans is that Parkinson’s Law holds: Work expands to fit the time (budget) allotted.

Leading Lean Software Development

In contrast, Products are usually funded incrementally.

This incremental funding is a clear signal to all that the scope is expected to evolve as knowledge is gained. The success of product development is generally measured based on the market share and profitability that the product eventually achieves.

Implementing Lean Software Development

It is interesting to note the emergence of Beyond Budgeting as an alternative to traditional management practices, including budgeting, and how well their ideas align with Lean & Agile thinking.

Projects have no champions

People are moved (or sacked) between projects with no regard for the impact on the development process. Decisions are made not in the best interest of the customer but only to satisfy a project plan or a budget. Developers get accustomed to not finishing anything and don’t feel any responsibility for the code they write. They leave large amounts of unfinished code and ideas behind them.

Joel Spolsky puts it this way:

As soon as your program gets good enough, you have to stop working on it. Once the core functionality is there, the main problem is solved, there is absolutely no return-on-investment, no business reason to make the software any better. So all of these in house programs look like a dog’s breakfast: because it’s just not worth a penny to make them look nice. Forget any pride in workmanship or craftsmanship.

In this context it is impossible to have a Product Champion, Front-line Leaders or any kind of leader for that matter.

A product company on the other hand is forced to continuously improve its products to remain competitive. Developers are encouraged to find new ways to make the product better, easier to use, nicer, more robust. This keeps them honest and sharp.

All the innovation that results from competition in the marketplace ultimately benefits customers. None of this is present inside the enterprise walls but that is no excuse for sloppy software.

Just because software is being developed for a one-of-a-kind application does not excuse an organization from the obligation of creating value and delighting customers. The objective is still to create software that will deliver outstanding value to the organization that is paying for it.

Implementing Lean Software Development

In the public sector anything less than top-notch quality software is an irresponsible waste of tax-payers money. This level of quality will only be achieved when we start treating software as a product that needs to be sold to our business areas.

The Seven Principles of Lean Software Development

Chapter 2 of Implementing Lean Software Development From Concept to Cash deals with the seven principles of Lean Software Development.

Here are some quotes and personal thoughts about each principle.

  1. Eliminate Waste

    The biggest waste in software development is unnecessary features and unfinished work. They add complexity to the code and cause confusion. People make wrong assumptions based on code that should not even be there. Unnecessary features equate to over production in the manufacturing world.

  2. Build quality in

    Zero tolerance. Every defect that is identified is fixed immediately. Using TDD and CI your defect rate should be so low as to render a defect tracking system pointless. Testers still play an important role, but a tester finding a defect should be the exception not the rule.

  3. Create Knowledge

    Knowledge crunching. You will always know more tomorrow than you know today. Requirements and initial design are nothing but a forecast, an initial guess. Not only do you learn more about the business as you go along but you also learn about your own processes. Successful projects adjust their practices to rectify problems as they occur.

  4. Defer Commitment

    Leave your options open so that irreversible decisions are made as late as possible. This requires flexibility in the code areas more susceptible to change. There is a fine line between adding flexibility and producing waste.

    “In preparing for battle I have found that plans are useless, but planning is indispensable” – Dwight Eisenhower.

  5. Deliver Fast

    Deliver software so fast that our customers don’t have time to change their minds. This needs to be done with quality built in, otherwise it’s a hack.

  6. Respect People

    Self explanatory. Although it is worth noting that this has more to do with empowering people to make decisions and respecting their abilities and ambitions.

  7. Optimize the Whole

    A lean organization optimizes the whole value stream, from the time a feature is requested until it is deployed. This involves at least analysis, development and QA.  The separation between developers and testers is particularly harmful and common. They are often treated as two different teams with two different managers. Efforts to optimize each team separately are counter-productive.  Each team develops their own schedules and priorities which are not always aligned with the other teams.  The process as a whole suffers and the customer needs fall between the cracks.

Review: Leading Lean Software Development

Leading Lean Software Development: Results are not the point (2009) is the third book about Lean Software from Mary and Tom Poppendieck. Previously they wrote  Implementing Lean Software Development: From Concept to Cash (2006) and Lean Software Development: An Agile Toolkit (2003).

In a very clear progression they’ve gone from a toolkit where all the underlying principles are defined to implementation and then leadership. Too bad I only see this now after reading the last book first.

In Leading Lean Software Development they present 24 frames from which to view the software development process in terms of leadership. These frames represent manifestations of leadership across different aspects of the development process. Not one person is expected to fulfill all these leading roles. In fact the frames are grouped in fours and assigned to different leaders, with some overlap. Continue reading “Review: Leading Lean Software Development”

Back to Certification

I’m starting the process of upgrading my MCAD certification.

After some initial confusion I have settled on the following Upgrade path:

  • 70-558: Upgrade: MCAD Skills to MCTS Windows Applications by Using the Microsoft .NET Framework
    • 70-536: TS: Microsoft .NET Framework – Application Development Foundation
    • 70-526: TS: Microsoft .NET Framework 2.0 – Windows-Based Client Development
  • 70-528: TS: Microsoft .NET Framework 2.0 – Web-based Client Development
  • 70-529: TS: Microsoft .NET Framework 2.0 – Distributed Application Development
  • 70-549: PRO: Designing and Developing Enterprise Applications by Using the Microsoft .NET Framework

70-558 is essentially a bundle of the two exams it covers.

I have a Safari Books Online subscription which is going to be very handy for this.  I’ve added the trainning kits for 70-536 and 70-526 to my bookshelf.  I don’t particulary like the Microsoft Press books but they should be a good starting point to prepare for the exam. Good thing is I won’t have hard copies to throw away.

Satellite assemblies for custom cultures

Pre-defined Cultures

When dealing with pre-defined cultures Visual Studio does all the work for you. Add a resource file with the right culture name and build your project. A satellite assembly is created behind the scenes and placed in the output directory.

files

Notice how having the file Resources.es-VE.resx results in ResourceSpikeTest.resources.dll being created in the Debug folder, under the right sub-directory for the es-VE culture.

Also notice how the default resource file Resource.resx is not embedded in a separate assembly. This resource file (in a binary format) will be embedded in the project’s dll itself.

The project I’m working with here is a Test project since I want to write some unit tests around the satellite assembly. At this point the satellite assembly is correctly placed in the output directory, but the interesting thing about Test projects is that they don’t run from the target directory but from the TestResults folder instead.

We therefore have to deploy the satellite assembly so that it is available when the test runs.

[TestMethod]
[DeploymentItem( @"es-VE\ResourceSpikeTest.resources.dll", "es-VE")]
public void TestPredefinedCulture()
{
  Thread.CurrentThread.CurrentUICulture =
     new CultureInfo("es-VE");
  Assert.AreEqual("epale", Resource.msg);
}

The DeploymentItem attribute causes our satellite assembly to be copied from the target directory to an es-VE sub-folder inside TestResults (more specifically inside the ‘Out’ folder for each test run). With the satellite assembly correctly deployed the test will pass.

Custom Cultures

Custom cultures are a different beast. Visual Studio will not generate the satellite assembly for you. You have to do it by hand using resgen.exe an al.exe. There is plenty of information around on how to use these tools. Adding the following code to your project’s post-build events will do the trick:

mkdir "$(TargetDir)%cultureName%"

set path=%path%;"$(FrameworkSDKDir)"bin\
set baseName=Resource
set cultureName=x-en-ClientA
set namespace=ResourceSpikeTest

resgen $(ProjectDir)%baseName%.%cultureName%.resx  ^
$(ProjectDir)%namespace%.%baseName%.%cultureName%.resources

AL /t:lib ^
/out:"$(TargetDir)%cultureName%\%namespace%.resources.dll" ^
/c:%cultureName%  ^
/embed:"$(ProjectDir)%namespace%.%baseName%.%cultureName%.resources"

Now after building the project we have a satellite assembly for the x-en-ClientA culture in our target directory, just like we did for the pre-defined culture.

To be able to run a similar unit test as the previous one we must ensure that the custom culture has been properly registered in the machine where the test is running.

public UnitTest1()
{
  if (!CustomCultureIsNotRegistered("x-en-ClientA"))
  {
    var cib = new CultureAndRegionInfoBuilder("x-en-ClientA",
                    CultureAndRegionModifiers.None);

    var ci = new CultureInfo("en-AU");
    cib.LoadDataFromCultureInfo(ci);
    cib.Parent = ci;

    var ri = new RegionInfo("AU");
    cib.LoadDataFromRegionInfo(ri);
    cib.Register();
 }
}

private bool CustomCultureIsNotRegistered(string name)
{
  return CultureInfo.GetCultures(CultureTypes.UserCustomCulture)
     .ToList().Any(c => c.Name.Equals(name));
 }

And once again we need to deploy the satellite assembly using the DeploymentItem attribute in our test method.

[TestMethod]
[DeploymentItem(@"x-en-ClientA\ResourceSpikeTest.resources.dll", "x-en-ClientA")]
public void TestPredefinedCulture()
{
  Thread.CurrentThread.CurrentUICulture =
     new CultureInfo("x-en-ClientA");
  Assert.AreEqual("ClientA rocks!", Resource.msg);
}

It’s worth pointing out that when we want to reference a resource we use the strongly-typed default Resource file.  Any other resource file that we add to the project we can set to ‘No code generation’.  The strongly-typed resource file uses a ResourceManager to find the resource based on the current culture.

Updating an Entity internal to an Aggregate

Say we have an Aggregate with Customer as the root and Address as an internal Entity (Addresses need to be distinguishable within the Customer Aggregate).

A common way of updating a particular Address for a Customer is

Customer customer = customerRepository.Find(id);
customer.UpdateAddress(addressId, addressData);

I can’t help but think this is a waste. Instead of getting the customer from the persistent storage and using it to update the address I want to update the address directly.

If I can grab hold of my UoW (NHibernate’s Session) I could just call session.Update(addressData) and that would be that (provided addressData has the id in it).

But NHibernate is hidden behind the repositories and I only have a repository for customer since it is the aggregate root. I would have to have the following in my CustomerRepository.

customerRepository.UpdateAddress(address);

Internally this would call SaveOrUpdate on the Session.

The quetion is how many DDD rules am I breaking with this?