A major problem for directors, managers, product managers and quality assurance personnel is to decide when the software is 'reliable enough' to be released to the market. Such decisions are primarily made subjectively or by probablistic methods rather than using quantitative means and decision theory to objectively measure and make informed release time decisions.
Last week, I was in Edmonton at the CIPS ICE Conference giving a talk on this topic and was surprised to find out how many organizations struggle with this decision.
I presented a multi-dimensional defect prioritization and release model/methodology which sets priorities on defects based on project constraints, stakeholder opinions, risk of not fixing a defect and analysis of defect types, arrival patterns and collection mechanisms.
The methodology presents various release options based on target reliability levels, acceptable level of risk, effort spent on testing and time to market. I got a lot of feedback from the audience and a lot of questions about the case studies I presented.
One of the biggest challenges of implementing this teachnique in an organization is leadership perception and their buy-in. The technique also uses Rough Set Analysis to conduct root cause analysis and the audience was very interested in pattern generation.
I had especially struggled with this area when building this methodology. All the standard techniques for defect root cause analysis such as IBM's ODC, HP Defect Classification and IEEE Classification require complete and consistent data, but the data which I had to generate patterns from was incomplete and inconsistent. Rough Set Analysis is definitely best suited for such situations and led to eye-opening results.
Currently, I am working on a paper to get this work published, but will definitely push ahead aggressively to find situations where "when to release" decisions become tricky to make.
The methodology presents various release options based on target reliability levels, acceptable level of risk, effort spent on testing and time to market. I got a lot of feedback from the audience and a lot of questions about the case studies I presented.
One of the biggest challenges of implementing this teachnique in an organization is leadership perception and their buy-in. The technique also uses Rough Set Analysis to conduct root cause analysis and the audience was very interested in pattern generation.
I had especially struggled with this area when building this methodology. All the standard techniques for defect root cause analysis such as IBM's ODC, HP Defect Classification and IEEE Classification require complete and consistent data, but the data which I had to generate patterns from was incomplete and inconsistent. Rough Set Analysis is definitely best suited for such situations and led to eye-opening results.
Currently, I am working on a paper to get this work published, but will definitely push ahead aggressively to find situations where "when to release" decisions become tricky to make.
1 comment:
I am also trying to experiment with the so called Anticipatory Failure Determination (AFD) or Subversion Analysis - a TRIZ technique. I have a feeling the AFD approach can lead to a more robust methodology for making a robust software system
Post a Comment