The Real Cost of Changing Designs

Duke Nukem Forever took over 14 years to develop due to a string of requirements changes, scope creep and unforeseen challenges. Originally announced in 1997 as the highly anticipated follow-up to the wildly successful Duke Nukem 3D, things didn’t go according to plan. The team were struggling to create the graphics they needed with their game engine and decided to swap game engines after over a year and a half of work. This design change required them to effectively start over with the game.

There are countless other examples of projects that have gone off the rails due to late changes and poor planning. The Standish Group’s Chaos Report found that 19% of projects are cancelled before they ever get completed. This is why project managers are so protective of their project scope and development leads are so resistant to change.

But at the same time we have examples like Amazon releasing 10,000 times a day and other companies working iteratively in short cycles. So what’s the difference? Why do some companies thrive on change while others are crushed by it?

The Fear of Rework in Software Projects

Predictability is the most important metric in many organisations. A budget has been allocated based on the business case and bonuses and promotions rely on the project being delivered on time and on budget. Change is the enemy of predictability.

The Waterfall methodology was based on how large scale construction projects were managed. In construction, changes to foundational elements require extensive demolition and rebuilding, so a linear and sequential approach makes sense. Each phase must be completed before the next one begins because it is costly to revisit previous stages.

However, software development is inherently different. You don't need to demolish a database to modify its schema. You don't need to tear down services to refactor their interfaces. Software components can be gradually evolved, refactored, and improved without disrupting the entire system.

So if software isn’t hard to change, why do long delays occur?

The Real Cause of Delays: Our Processes

Overly Complex Processes

Long documentation cycles, endless sign-offs, and hierarchical approvals can turn a minor design tweak into a weeks-long affair. Consider a simple API change in a typical enterprise environment. The actual code modification might take just a few hours, but the process often looks like this:

• Week 1: Draft and submit a change request

• Week 2: Wait for funding approval

• Week 3: Submit design for architectural approval

• Week 4: Revise based on feedback and re-submit

• Week 5: Make the change

The irony is that these processes, intended to make changes safer, often discourage teams from making necessary improvements. I know a developer who noticed a bug in their code when working on another feature and fixed it. But because that was the current behaviour in production they needed to file a change request to get the test suites updated. They ended up putting the bug back into the code.

Time-Consuming Regression Testing

Testing is crucial for any software product. If a project relies heavily on manual tests, every rework triggers a tedious round of checks to ensure that there were no unintended consequences and nothing else broke. For a large-scale system, these regression test suites can take an enormous amount of time.

Architectural Changes

Sometimes changes are more extreme. There are different technical solutions for different scales of problems. An architecture that works with a few hundred or thousand customers may not scale to a few million. This is the type of challenge that the Duke Nukem team faced when they decided to switch their game engine.

Fixing our Processes to Make Change Easy

From Approvals to Empowerment

The goal of having an approval process is to ensure that the limited resources of an organisation are spent wisely. The original scope for the project was signed off because of the alignment with the business goals, and any changes should be reviewed to ensure that they are still relevant. The fear for management is that a team will waste money over-engineering a solution or that stakeholders will push for changes to squeeze in their pet feature.

Approval processes achieve these aims but they delay the delivery of value and can even discourage teams from doing the right thing.

There is another way. In Turning the Ship Around, David Marquet describes how he turned around the worst-performing submarine in the US Navy by empowering his crew. Certify, Don't Brief was one of the key changes that David implemented. Instead of having the crew ask for permission to do something, they would tell the captain what they were going to do. The captain would then certify that it was safe and that it was in line with the overall mission.

But how will the team know which changes to pursue? Management need to make the business goals explicit. That way the team will be able to judge whether their ideas are worth pursuing or not.

From Manual to Automated Testing

Products are not static - they are constantly being updated and tweaked. Each new feature or modification introduces potential ripple effects throughout the system. Relying solely on manual testing creates an unsustainable overhead as the product grows.

This is where "shifting left" becomes crucial – by embedding automated testing early in the development process, we catch issues before they cascade into larger problems. Developers write unit tests as they code, integration tests verify component interactions, and end-to-end tests ensure complete user journeys work as expected. Instead of taking days or weeks, tests can be completed in minutes.

Making Architectural Changes Easy

Large architectural changes don't have to be daunting all-or-nothing endeavors. The key lies in incremental transformation patterns like the Strangler Pattern, where new functionality gradually envelops and replaces the old system.

Instead of attempting a "big bang" rewrite, teams can extract features one by one. While this approach might seem slower, it actually prevents the massive delays typically seen in big rewrite projects. The incremental approach, coupled with comprehensive tests that verify the business logic, keeps the system deployable at every step, maintains business continuity, and allows the team to learn and adjust their architectural decisions based on real production experience.

Conclusion

The real cost of change in software development isn't in the technical modifications – it's in our processes and mindset. By recognising software's inherently malleable nature and embracing modern development practices, teams can break free from the perception of changes as risky. The key is to build systems and processes that embrace change rather than resist it.

Organisations should examine their change management processes critically. Are they truly preventing problems, or just adding delay? Are approval processes adding value, or creating bottlenecks? By streamlining these processes while investing in automation and testing, teams can maintain system quality while becoming more adaptable to change.

Remember: software's flexibility is a feature, not a bug.