The Top 10 DevOps Metrics: How To Measure What Matters – Part Two

Jeff Keyes, VP of Product at Plutora, comes in with five more valuable metrics for DevOps teams

DevOps is focused on improving the way organisations benefit from software delivery, but in many cases, understanding the factors that determine success can be a real challenge. In Part One of this guide, we looked at five methods to help DevOps stakeholders focus on measuring what matters. In this second and final part, we examine five more ways to measure what happens across the development process, and how this insight can improve outcomes:

6. Monitor the Rate of Deployment

How quickly teams create and deploy new versions of your app is a clear indication of their competitiveness. However, it’s difficult to optimise and improve deployment rates without an effective system managing every other step that comes before. For example, imagine a situation where you have a thousand commits awaiting tests – if you’re not using the most effective testing methods (automated, AI-powered, etc), then even your best developers won’t deploy much.

Consider Amazon for a moment. It’s often pointed out that it deploys thousands of times daily but its site remains largely unchanged for months at a time. In reality, the Amazon site (and others, such as Netflix) is anything but simple, and behind the consistent, user-friendly interface resides a complex network of microservices. DevOps teams working on these have to make improvements daily, even if users don’t see obvious differences when they arrive.

At the other end of the scale, smaller enterprises making just a few regular deployments are demonstrating that they are committed to development and innovation. But just like Amazon, any reduction across this metric can help the ops side of DevOps improve processes that lead to deployments.

7. Look at Deployment Speed

When examining the deployment process, it’s important to optimise each instance, and using a ‘speed of deployment’ DevOps metric helps measure the time it takes for a single deployment to complete. In practice, some apps require a few minutes to build and package their executables, while some stand out because they regularly take an hour. This should raise questions about optimisation.

Naturally the deployment time for each app will vary based on the final size of the file. But, hosting applications in the cloud, for instance, can safely and effectively remove the problem of deployment speed from the devs.

8. Report on Deployment Rollbacks and Fails Frequency

Deployment rollbacks offer another useful metric in the pursuit of DevOps insight. They can be required for a number of reasons, such as undoing changes that miss the scope of an app’s requirements, as a result of executive directions, or to fix bugs that escaped testing. Whatever the motivation, the frequency of rollbacks provides qualitative information to help improve other parts of the process that lead to deployment.

In addition, continuous integration / continuous deployment (CI/CD) tools keep logs of successfully completed deployments as well as those that fail before completion. When these indicate that a lot of rollbacks are happening due to features going forward before proper testing, for example it’s sensible to put more manual checks into the process before builds.

Finally, be sure to monitor the fails frequency of your cycle. This should focus not only on deployment, but every other step along the way. Ultimately, understanding these particular metrics not only promotes better coding and commits, but also leads to fewer rollbacks after deployment. The ideal scenario is to have as few rollbacks as possible, not only because it’s efficient and cost-effective, but it also demonstrates that a smooth and well-managed pipeline is in place.

9. Assess Version Lead Time

Version lead time is arguably one of the most accurate indicators of DevOps process productivity. It’s akin to starting the clock at the very moment that a user submits a ticket, and then measuring how long it takes to resolve the issue. A problem with this approach is that it doesn’t necessarily focus on each step in the process specifically enough, but by doing so, it becomes easier to identify the rate-determining steps more quickly.

A pipeline’s lead time is also determined by the difficulty of each task, and it’s wise to include the relative weight of a change in any reports that focus on lead times. This should also include how many developers were involved in the change, because in general, the more resources that are applied to implementing a change, the less time it should take to complete.

10. Measure the Rate of Response to Tickets

Ultimately, improving the development pipeline needs to start at the beginning of the process with ticketing. Optimising processes and resources to improve speed depends on monitoring the rate of response to new tickets. This can help ensure that enough resources are allocated to each ticket, cutting the expected lead time, while increasing the number of commits.

Ensuring a strong ticket response rate depends on having an effective value stream management tool to connect information across all the applications in use by the departments along your pipeline. The best outcome from having one is increased visibility: every stakeholder is alerted early in the process, it produces more background information for developers working on fixes or new features, and down the line, ticket issues reach the deploy stage quicker and with a reduced likelihood of being rolled back.

Organisations that are committed to DevOps should be, by definition, focused on continual improvement. By applying as many of these 10 metrics as possible, they can put themselves in a much better position to measure what matters and ensure DevOps delivers to its full potential.