How to measure the flow of value to improve delivery

In the 2019 Accelerate State of DevOps Report, DevOps Research and Assessment (DORA) suggested four metrics that form a great foundation for your DevOps initiatives. If you measure nothing else, then these are going to give you a good indication of the health of your value stream and how it’s evolving.

Deployment frequency – This measures how often you’re deploying to production and should be easy to assess. The more frequently you deploy, the more value you’re delivering to your customers, so it’s the clearest indicator of a successful DevOps organisation. While it shows you have a mature deployment process where you’re willing to push change through, it doesn’t show how valuable the software is.

Lead time for changes – This measures how long it takes from code commit to that code being live into production. Depending on the company, this might be a few minutes or a few months. It demonstrates whether teams are able to deploy changes without getting stuck in multiple bottlenecks. If your lead time for changes is long, that means there’s lots to improve through streamlining and automation.

Change failure rate – This measures how many of those changes fail. If you’re deploying frequently and your lead time for changes is fast, it’s important that those deployments don’t always need fixing. If you’re experiencing lots of failures, you’re having to roll changes back, which means customers are missing out. Likewise, if you’re not experiencing any, chances are you’re moving too slowly, so customers aren’t enjoying any real value either. Change failure rate is a good indication of how successful your testing coverage is.

Mean time to restore services – Lastly, this measures the amount of time it takes to bounce back when something does fail. It shows how quickly you’re able to respond and get a service back online, either by fixing bugs fast or rolling back a previous iteration. The quicker your recovery time is, the more likely your business will innovate and take risks, generating more revenue and profit, and ensuring you’re always one step ahead of the competition.

Once you’ve got your head around DevOps metrics, you can start to gain even more insight by finding out what’s happening in the value stream itself. That’s where flow metrics come in. According to the Flow Framework®, flow metrics measure the flow of business value through all the activities involved in producing it. The four to focus on are:

Velocity – This measures throughput over time to help gauge whether your value delivery is speeding up. It’s calculated by measuring the number of flow items (units of flow in your value stream) completed over a defined period of time.

Time – This measures the amount of time it takes for an item to go to market. It calculates the time elapsed from when the work is started to when it’s completed, including any periods of inactivity.

Efficiency – This measures the amount of waste in a given value stream. Your flow efficiency is the ratio of active time out of the total flow time calculated above. It helps you to identify where value is being lost from waiting around.

Load – This measures how value streams are being used by assessing the number of flow items that are in progress at a given time. The more work you take on, the more context switching is required. This helps you to keep track of the work in progress and how it relates to outcomes.

DevOps and flow metrics are essential to increasing efficiency and identifying waste. But you will also need to identify a core metric (or maybe a few) that are more closely tied to your organisation’s raison d’etre. We call these North Star metrics (NSMs).

NSMs are linked to the business’s core goals and act as a guiding light to help focus everyone on what they’re working towards. They also support decision-making – if actions aren’t aligned with an NSM, they’re probably not a priority for the business. They’re more focused on the big picture, which encourages your people to think about long-term growth rather than short-term gains. They also help to avoid any so-called vanity metrics that might make people feel good but don’t add any real substance to your value stream.

There’s no one correct NSM – it will depend on your business – but to find yours, think about how you can measure if your client is happy. Your NSM should be aligned with the core value you provide, easily understood by everyone in the organisation, and used to assess whether your actions and other metrics are in line with your business goals.