What are meaningful Testing KPIs?

So what KPIs to use?

I am not going to tell you about counting defects and test cases. It is straight forward exercise, usually leaving everyone in the meeting clueless about what it really means. I suggest trying to track the following information:

  • Defect ration in each testing phase (ie Agile Sprint defects / Pre-prod regression defects). This KPI, and especially trend over time, would allow gathering information about early testing availability. Late defects in development cycle could mean non-testable features, lack of integrated test environment, bad quality of requirements, not stable code etc. In other words, used wisely, it could give valuable insights into the whole development chain.
  • Technical debt increase/decrease per release. Is your product getting better after a few months or weeks of development? Will your users be happy? If you could be even more specific, which areas are affected by problem accumulation, even better!
  • How much time test case has to be executed and in which areas. None likes repeating test scripts, especially if this is done manually. However, information about test repeats could allow planning of the next testing phase, identify automatable areas or features, where tests fail (due to any reasons, including defects).
  • The ratio of automatable tests versus automated tests. Measure your progress against the plan, not just plane number.
  • Exploratory Session time pro release. I could write a whole new essay on how exploratory/session testing wins against scripted testing. Knowing that team of professional test specialists had a chance to inspect system without predefined scenarios, and thus identify new issues, questions or even requirements is the best quality assurance and security net you could get for any product.
  • End-to-End environment testability time versus planned testing time pro r release. Handy KPI to measure, if with new development methods (Agile, SAFe) the system is enough time under test. I know systems who could be E2E tested in a few hours, and I know systems which even with automated checks needs a week or two for one regression test set.
  • How buggy is your business flow. Do not track defects in total. Track business flows in colours, with defect number nearby. You get the attention of each stakeholder, who will see a red colour near her area.

Rules applying KPIs

I would allow myself to be laconic and name just a few:

  • Tracking this information should not be done manually, but generated automatically using tools you have.
  • Information should be available online every time with life data
  • Link to KPIs is sent to all involved stakeholders periodically, making everyone get used to seeing that numbers. Be sure you have an intro meeting and that your target group understands how to read the data. As the old saying says — there is nothing worse than the CEO with statistics.
  • KPIs measure processes, not people. And KPIs measure processes, that are important, not just because data are available.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Evelina Rimkute

Evelina Rimkute

126 Followers

Test manager, team lead, bookworm, writer and lifelong learner, traveler, passionate about new ideas and history. https://amzn.to/2YvxNEY or bit.ly/39GbErS