Quality Intelligence for DevOps

Quality in the context of Qentinel Pace is not just about bugs and conformance to requirements. Quality is the ability to create value in four dimensions:

  1. Product or technical quality, meeting the customer and regulatory requirements, is still very important but not the whole picture of quality.

  2. Benefits must emerge as the result of using the product.

  3. Feeling, the experience of using the product, must be good. If the users don't use the product, the desired benefits will not be realized.

  4. Some doing will change as the result when a new product or service gets deployed, and it must change in a good way so that the expected benefits will be achieved. This is particularly essential in digitalization initiatives.

Quality Intelligence is intelligence about value creation in all four dimensions above. Quality Intelligence is Qentinel's proprietary method for making value creation transparent, measurable, and actionable. The aim of Quality Intelligence is to enable better, well informed, and timely decisions. You apply Quality Intelligence to manage and assess IT investments, to lead R&D effort, to optimize operational processes, and to many other purposes.

Quality Intelligence for DevOps

You can model any problem using Quality Intelligence and find the right and actionable KPIs. Qentinel Pace standard offering comes with a Quality Intelligence for DevOps, the integrations however are not off the shelf yet. Below is a brief description on how Quality Intelligence can provide you with a holistic view of your DevOps and tell you about the quality of your software development.

The DevOps Value Creation Model below has been drawn using the DevOps cycle as the background to provide a clear context for what we measure. The model shows how each phase in the process creates value or imposes a risk for losing value. All nodes in the model are candidates for metrics. The circles with green, yellow and red colors indicate that they are actually measured, and the traffic light colors show the status vs. set targets. The causality chains can be used for identifying the leading indicators that have a positive or negative impact to your goals. Blue arrows in the picture denote positive (assumed) causality, so for instance when Production deployment frequency increases, so does the Pace of value deliveries. Respectively, a red arrow means that the variables move to the opposite direction. The higher is Technical debt, the lower is Sprint predictability.


The actual metrics are defined for the nodes of the Value Creation Model. Some nodes are measured by several metrics. In those cases, we can calculate an index score from 0-100 where 100 represents the intended target level. The actual measurement scorecard, some metrics trees opened, looks like this in Qentinel Pace:


There are some key indices on top to give an overview of current state. All of them seem to be below 100 so this team has room for improvement in all areas. DevOps Value Creation Index is the DevOps performance index calculated from all the metric trees i.e. from all four value paths and metrics under them. There are separate index scores for the production environment, production release and release candidate quality. The Value Paths for customer satisfaction to product/service and velocity of value creation have their own indices, too. Release Candidate quality index is only 62.17 so there must be some major quality issues and therefore this release candidate must not be deployed to production. A quick look at the Value Creation Model tells that there is at least a security issue and technical quality is on yellow so some metrics are not meeting the goals. With a few clicks in the metrics tree we can find out that CPU usage is too high and service response times are not within the acceptable control band. This kind of analyses are easy and fast to carry out now that the metric causalities have been modeled into the Value Creation Model and the metrics have been organized into hierarchic trees accordingly.