What are meaningful Testing KPIs?

KPI… Key Performance Indicator. Such a popular shortening, used in almost every organisation, talked in every 2nd meeting at and traced by millions of brains around the world. Be it IT, be it production factory, be it delivery company. We all are measuring our output and performance against our target goals.

Image for post
Image for post

So what KPIs to use?

I am not going to tell you about counting defects and test cases. It is straight forward exercise, usually leaving everyone in the meeting clueless about what it really means. I suggest trying to track the following information:

  • Technical debt increase/decrease per release. Is your product getting better after a few months or weeks of development? Will your users be happy? If you could be even more specific, which areas are affected by problem accumulation, even better!
  • How much time test case has to be executed and in which areas. None likes repeating test scripts, especially if this is done manually. However, information about test repeats could allow planning of the next testing phase, identify automatable areas or features, where tests fail (due to any reasons, including defects).
  • The ratio of automatable tests versus automated tests. Measure your progress against the plan, not just plane number.
  • Exploratory Session time pro release. I could write a whole new essay on how exploratory/session testing wins against scripted testing. Knowing that team of professional test specialists had a chance to inspect system without predefined scenarios, and thus identify new issues, questions or even requirements is the best quality assurance and security net you could get for any product.
  • End-to-End environment testability time versus planned testing time pro r release. Handy KPI to measure, if with new development methods (Agile, SAFe) the system is enough time under test. I know systems who could be E2E tested in a few hours, and I know systems which even with automated checks needs a week or two for one regression test set.
  • How buggy is your business flow. Do not track defects in total. Track business flows in colours, with defect number nearby. You get the attention of each stakeholder, who will see a red colour near her area.

Rules applying KPIs

I would allow myself to be laconic and name just a few:

  • Information should be available online every time with life data
  • Link to KPIs is sent to all involved stakeholders periodically, making everyone get used to seeing that numbers. Be sure you have an intro meeting and that your target group understands how to read the data. As the old saying says — there is nothing worse than the CEO with statistics.
  • KPIs measure processes, not people. And KPIs measure processes, that are important, not just because data are available.

Test manager, team lead, bookworm, writer and lifelong learner, traveler, passionate about new ideas and history. https://amzn.to/2YvxNEY or bit.ly/39GbErS

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store