Q&A - Software Fix Quality - SFQ

The Contact Us function at the top of every page on the tl9000.org website is the preferred means for asking questions and receiving answers from the subject matter experts of the TIA QuEST Forum. Over the last few years many questions have been answered through this means. The number of each question is the ticket number in the Contact Us tracking system.

These questions generally relate to the Software Fix Quality (SFQ) measurement.

Question 10267 — Counting rule clarification: If the organization delivers a disc (fix R4.2) to customer in July 2019 and a problem (critical or major) occurs in network, it is a potential to be counted as a (SFQ) defective fix. During root cause analysis it is determined that the actual cause was attributable to a fix (R4.1) delivered previously in Dec 2017 (more than 6 months ago) and was not part of the generic/baseline release of the software. Do we count it as SFQ for fix R4.2 or re-state R4.1 or do not count at all?

Answer — Counting rule 8.1.4 b) 9) says that ‘defective fixes are counted in the month they are found defective.’ It does not matter when the fix was installed or under what release it was installed. It is reported in the month it was found defective. This measure can be confusing since there is an obvious ‘disconnect’ between when the fix was installed and found defective. In an effort to minimize work on the organization it was decided to simplify the reporting of the measurement by simply counting the number of fixes installed in the last 12 months and the number of fixes (previously installed in any prior month) found defective. In this particular case, since the issue is with the R4.1 fix, it would not count since that fix is more than 6 months old.

Question 10731 — If you are only going for hardware certification do you also need to report software problems in your common metrics. In other words, we do have software but are only going for hardware certification why would we have to report software problem reports.

Answer — The measurements that must be reported are determined solely by the Registration Option (H, S, V, or combination) and the category.

If your registration's option is H only then you will not be asked to submit software measurements. In fact, the MRS will not allow you to submit software measurements.

You can determine which measurements are required by looking at the data submission templates for your category and registration option. The templates are available on the public web site at https://tl9000.org/alerts/data_submissions.html near the bottom of the page.

Question 10511 — Clarification: for SFQ, 8.1.4) b) counting rules, 6) "Fixes addressing similar defects across multiple releases are counted separately." Does this mean if I have 1 software patch that covers 5 different releases, that counts as 5?

Answer — If the exact same patch, with no differences in code, is applied, then it would count once. If there are changes that must be made to the code for each of the five releases, then there would be five fixes. The key wording here is "Fixes addressing..." as opposed to "A fix addressing...

Question 13268 — I have questions regarding to the SFQ counting rule. 8.1.4 b) 8) states –
A defective Fix meets one or more of the following criteria:

- Within the first 12 months of the fix release date:
   o the fix cannot be installed or
   o the fix does not correct the intended problem or
   o the fix is withdrawn because of a potential or actual problem with the intended fix.

- Within the first 6 months of fix release date, there is a critical or major problem that is a side effect found to be attributable to the fix.

Question 1: What does the 3rd bullet mean? Does it strictly mean that the load containing this fix has to be removed from service? Or only the fix is identified as with problems, it does not care whether the load stays in service. In what situation a fix is withdrawn? Looks that the word "withdrawn" brought me some trouble to well understand the statement.

Question 2: It looks to some my colleagues that the second clause is duplicated as the three elements of the first clause should have covered it. Is there any particular reason to put it here?

Answer — Bullet 3 represents a case where the fix release was installed, and it may have even corrected the problem. However, it may have caused a number of other problems, even minor, where the customer questions the stability, and feels it must be withdrawn. It could also be a case where the supplier recommends a withdrawal. In either of these cases, it would be a defective fix.

The second clause represents cases where a fix is installed successfully, it does correct the intended problem, and does not have to be withdrawn - yet during the first 6 months after the fix release, a critical or major problem occurs. The problem may impact other functionality that the fix was not intended to impact. This is an example of a case where in the process of fixing one problem, another may be created. If a minor problem report is generated (as opposed to critical or major) during the six month, this clause does not apply.