1.
Bug arrival rate à
this gives an indication of load on developers.
a. Are
developers loaded towards the end of regression/functional cycle?
b. This
shoots up risk as developers need to change more code toward end of cycle if
the graph is moving up.
2.
Bugs logged out of QA cycle à this gives an
indication of the severity of QA misses on testing. This is DRE. . DRE = Defects found by QA / [defects found by QA +
all other defects found by teams outside of QA]. 100% is the best number.
a. Logged
by PS/PP – CAT testing
b. Logged
in production for client tickets
c. Any
other bugs logged by teams outside of primary QA team.
3.
Latent bugs logged during QA cycle à very important as this
gives multiple indicators
a. This
is back log load QA is loading on the developers. Ideally these should have got
sorted out in the respective release.
b. Latent
bugs increase risks in projects as it warrants code changes to areas where no
new functionality is implemented.
c. It
also should be added to DRE (Defect removal
efficiency) of QA as these are bugs missed uncovering in the release cycle.
4.
Bug quality vs bug arrival rate.
a. Very
important metric. Are Good quality bugs being uncovered early in cycle? Or is
it the click there click here bugs?
i.
Click there click here bugs indicate lack of dev
testing also. this may be because of tight developer timelines or
unproductivity. Good unit testing also helps. Technically its not possible for
developers to do all the combinations of usecase from the UI side. Its easier
to have a bug bash for a day and weed them out.
1. Ideally
the click here click there bugs should be tracked separately and weeded out in
the first two days post a release with a bug bash concept.
b. Percentage
of good functional bugs vs click there click here bugs shows indication of
value add of QA teams
5.
Automation efficiency à important metrics to really
know if money invested in automation has ROI.
a. Per
regression cycle / HC cycle -> reports have to published for time saved by
using automation scripts. If 5000 testcases out of 10000 are automated,
regression cycle should show 50% savings.
b. Very
very important metrics à
how many of regression testing bugs are uncovered by automation scripts vs
manually found and logged.
Finally goes the important saying: “QA should be given the
goal to increase quality, QA should not be given the Goal to increase bugs count”.
History has always shown that teams which are given the goal of increasing bug
counts miss on quality bugs. A tester who logs good functional bug in 2 days is
equally as good as a tester who logs 10 bugs in the category of click here
click there bugs.
Santhosh
Kartha
No comments:
Post a Comment