How to structure teams for scale

One team is easy. The challenge comes when you are trying to coordinate the work of multiple teams.

Our team structures need to ensure that the teams are both effective and efficient. Effectiveness comes from a deep understanding of the end users and their needs. Efficiency is measured in the lead time that it takes to go from idea to satisfied customers.

Project Teams

The most common way of scaling software development is the project structure. Teams are formed around specific projects or initiatives, typically comprising members from different disciplines. This provides flexibility for the organisation because it enables the allocation of specialists to projects based on the needs of the features they are building.

Effectiveness

But there are drawbacks. The project structure relies on a business case to fund the project team. And to create that business case you need to know what you are planning to build so you can put together a benefits case and cost assessment. Having a fixed scope and timeline forces teams to focus on delivery because the benefits have already been assumed. Unfortunately this leads to really poor outcomes because new products and features require users to change their behaviour and we can’t predict that ahead of time.

Efficiency

The incentive in a project is to complete the full scope, rather than achieve the desired outcome. When trying to deliver a scope it is more efficient to do a single release when all functionality has been completed. This leads to a very long lead time however. If we were guaranteed that the scope was what customers wanted this tradeoff would be acceptable, but 90% of the time we are wrong.

We need a new way of working that empowers teams to iterate on potential solutions with real users and iterate based on the feedback that they receive so that they can achieve the desired business goals.

Product Teams

Product Teams are long-lived teams, as opposed to being created just for the delivery of a specified feature.

When it comes to product teams though, there are a lot of choices on how to structure the teams. We can’t fit everyone who works on a product into a single team because it would be both an inefficient use of people as well as prohibitively slow to try to keep everyone aligned.

The most common ways of splitting up product teams are:

By Feature

This is a continuation of the project model, but instead of building a team for each feature, the team is long-lived and it picks up new features after each delivery. The benefit of this structure over projects is that the team know each other and are able to hit the ground running much quicker when starting on a new feature.

The challenge with this structure is that multiple teams may be working on the same part of the application at the same time. This can lead to conflicts at the code level resulting in merge hell as well as the need for teams to queue to release their work. If another team has touched the same code as you and they are trying to release, you will need to wait until they have released and then merge all of their changes into your code.

This does improve the efficiency over projects but there is still a lot further improvements that could be made.

By Technical Stack

Another way of structuring teams is by technical stack. This is easy to implement when coming from the project model as specialists are often already in functional silos such as front-end, back-end and data. The benefit of this approach is that the teams will take more ownership of their codebases and manage their tech debt better. In projects, because nobody really owns the code base, it becomes subject to the Tragedy of the Commons. This is where the code base degrades because the short term project deadline incentives outweigh the long term code base health. When a team owns the code base, the long-term incentives become more aligned.

The challenge however is that customers don’t think in discrete tech stack elements. This means that to deliver a useful feature there needs to be coordination across multiple teams. Features are only releasable when all teams have finished.

Again effectiveness and efficiency are not improved with this approach.

By User Journey

Instead of structuring based on what is easiest to manage internally, another option is to structure around how customers see our products. This involves creating cross-functional teams who own sections of the product, or value streams, based on the customer journey. The benefit is that a single team can deliver a meaningful feature to customers without relying on and having to coordinate with other teams and since the team is long lived they can get a deep understanding of the users in their associated area.

The challenge for structuring by customer journey though is that we still need to build these features in code, and how a customer thinks of our product does not align with the logical entities that we would need to code such as users, products, orders etc. This leads to the same challenge of having to queue to release features as well as the risk of code base degradation.

Scaling Leads to Diseconomies of Scale

There is no silver bullet - each structure has different tradeoffs.

The typical way to resolve the dependencies between teams using any of the approaches above is through project and program management that relies on scheduling the work to ensure high utilisation and limit the impacts of the path to production congestion.

The challenge is that we are building new products and features so unexpected events always arise. The best laid plans become invalidated almost the second they are published. And because companies are trying to release as much as possible, to spread the costs of staff over more features, this means that there is very little slack in the plan to absorb delays. The resulting cascade that requires all of the projects to replan is incredibly wasteful. A lot of management time is wasted negotiating which features are higher priority and which should take the brunt of the delays. And there is a lot of waste in the teams as priorities shift and teams are delayed.

Dunbar’s number highlights the exponential rise in interpersonal relationships as a group size increases. The overhead of managing interpersonal relationships is why it is often recommended to keep team sizes to 8 people or less.

We need a similar number for projects, or dependent teams working in parallel. The number of dependencies between projects increases exponentially with the number of projects just like the connections within a team. There is a tipping point where the overhead of trying to manage dependencies outweighs the benefits of trying to run more projects. In 2023, many companies, including Meta and Airbnb, reported that they had actually increased the pace of delivery after reducing headcount. I suspect the reduction in dependencies between teams was a large factor.

Removing Dependencies

We could run less projects and, for many companies, this will increase the rate of, and lower the cost of, delivery. But with ever increasing customer demands and increasing competition there is pressure to increase the total number of deliveries. To achieve this we need to take a different approach. Instead of trying to manage dependencies, we need to remove them.

The two different types of dependency that we have based on the structure above are coordination between teams to build a feature (tech stack aligned teams) or coordination between teams to release software (feature and customer journey aligned teams). We can’t make customers think of our products in terms of a tech stack but we can change how we architect our products. By intentionally isolating code and data to align with our value stream boundaries we can enable both feature and code autonomy; the best of both worlds.

This approach achieves both of our goals of effectiveness and efficiency.

There are obvious costs and overheads involved in isolating this code. We would need to redesign our systems and even duplicate some code and data storage. But, when operating at scale, this is more than offset by the ability for teams to operate independently - as long as the value stream boundaries are correctly identified.

Fortunately there are techniques like Event Storming and Domain Driven Design that can help teams identify the logical boundaries in their systems and design the communication patterns between teams.

You have to make the hard choices

Product development is complex and, unfortunately, you can’t remove that complexity. But you can choose where you place it.

Ultimately our goal is to build amazing products that customers love and deliver on the business goals. We know that locking in scope early leads to ineffective products so we need teams to take ownership of delivering outcomes instead of outputs. But unless we make teams autonomous, so they can release as quickly and as often as they need to, we can’t hold them accountable for results.

We need to get to zero blocking dependencies to empower teams to release as quickly and as often as they need to in order to iterate towards what customers really want. That is why aligning teams to the customer journey and isolating out their code bases is the best way to structure teams for scale.