POSTULATE: Paul Reese / Perry King (August 5, 2009) The Bionetics Corporation

To verify that Test & Measuring Equipment complies with established specifications, it is compared to other laboratory standards during calibration. There exists a 95.13% End-Of-Period-Reliability (EOPR) threshold for the Unit Under Test (UUT), above which, it may not be necessary to evaluate the Test Uncertainty Ratio (TUR) to demonstrate that the Probability of False Accept (PFA or Risk) is less than 2 % for compliance with ANSI/NCSLI Z540.3.

The following data and graphs were obtained from this spreadsheet: \\met03\metrologist\NASA\ELECTRICAL\LAB3\Misc.%20Spreadsheets\Z540.3_Methods.xlsm Consider a relatively “full” family of TUR curves (very low to very high) on a graph of PFA vs. EOPR. The chart below is representative of such curves. A useful observation is to note that, as long as the TUR is greater than 4.7:1, the PFA is less than 2 % for ALL EOPR values. Thus, if the TUR is observed to be better than 4.7:1, it is not necessary to know (or to keep track of) the EOPR for any device, where compliance with the Z540.3 2 % False-Accept-Risk objective is to be met. If the TUR is >4.7:1, the Z5403 2 % PFA requirement has been automatically satisfied.

From inspecting the previous graph above, it becomes evident that there are many “cross over” points where the various TUR curves intersect one another (see below). The obvious conclusion is that there is no single TUR that results in a “worst case” PFA for all EPOR values. Rather, the “worst case” TUR (yielding the highest PFA) depends on what the EOPR is at that particular value.

So it is then possible to consider the “outer envelope” of all the above TUR curves (see below). That is, a “mother curve” can be generated which plots the PFA as a function of the EOPR, with the PFA always computed at the “worst case” TUR. This results in a graph of PFA that is a function only of the EOPR, with each instantaneous point computed at the TUR which maximizes the PFA. In other words, a “worst-case” TUR has been assumed at each and every point on the curve below. This curve represents the absolute worst possible PFA for any given EOPR, regardless of what the TUR is.

One particularly useful observation is the fact that the PFA can NEVER be >2 % for any item which has an EOPR of >95.13 %. This fact can be asserted for ANY possible TUR, even TUR’s much less than 1:1. This illustrates the dominating weight that the EOPR variable carries in the PFA computations. The useful conclusion of this observation is that, for compliance with the Z540.3 2 % False-Accept-Risk requirement, it may not be necessary to evaluate the Test Uncertainty Ratio for any item which exhibits an EOPR >95.13 %. There is no TUR in existence which would yield a PFA above 2 % for EOPR’s >95.13 %. Therefore, whatever the TUR is for items exhibiting EOPR >95.13 %, it is adequate for the job and the TUR need not be investigated to ensure that PFA is <2 %. A second curious observation is that the PFA can NEVER be > 13.6 % for ANY combination of TUR and EOPR. That is, ALL calibration processes result in a PFA of less than 13.6 %, regardless of how low the TUR is and how low the EOPR is. No calibration scenario can be proposed which will result in a PFA of >13.6 %. This maximum value of 13.6 % PFA results when the TUR is 0.29:1 and the EOPR is 40.9 %. Any change (higher or lower) for either the TUR or EOPR will result in a PFA lower than 13.6 %.

These same phenomena can be viewed via an alternate approach. For example, we can also consider the converse of the above graphs… that is, a relatively “full” family of EOPR curves (very low to very high) on a graph of PFA vs. TUR (see below). The chart below is representative of such curves. The same observations can be noted. I.e., as long as the EOPR is greater than 95.13 %, the PFA is less than 2 % for ALL TUR values. Thus, if the EOPR is observed to be better than 95.13 %, it is not necessary to know (or to keep track of) the TUR for any device, where compliance with the Z540.3 2 % False-Accept-Risk objective is to be met. If the EOPR is >95.13 %, the Z5403 2 % PFA requirement has been automatically satisfied.

From inspecting the previous graph above, it becomes evident that there are many “cross over” points where the various EOPR curves intersect one another (see below). The obvious conclusion is that there is no single EOPR that results in a “worst case” PFA for all TUR values. Rather, the “worst case” EOPR (yielding the highest PFA) depends on what the TUR is at that particular value.

So it is then possible to consider the “outer envelope” of all the above EOPR curves (see below). That is, a “mother curve” can be generated which plots the PFA as a function of the TUR, with the PFA always computed at the “worst case” EOPR. This results in a graph of PFA that is a function only of the TUR, with each instantaneous point computed at the EOPR which maximizes the PFA. In other words, a “worst-case” EOPR has been assumed at each and every point on the curve below. This curve represents the absolute worst possible PFA for any given TUR, regardless of what the EOPR is.

One particularly useful observation is the fact that the PFA can NEVER be >2 % for any item which has an TUR of >4.7:1. This fact can be asserted for ANY possible EOPR, even EOPR much less than 90 %. The useful conclusion of this observation is that, for compliance with the Z540.3 2 % False-Accept-Risk requirement, it may not be necessary to keep track of the End of Period Reliability for any item which exhibits a TUR of >4.7:1. There is no EOPR in existence which would yield a PFA above 2 % for TUR’s >4.7:1. Therefore, whatever the EOPR is for items exhibiting TUR >4.7:1, it is adequate and need not be investigated to ensure that PFA is <2 %. The same curious observation is observed on the graph below. That is, the PFA can NEVER be > 13.6 % for ANY combination of TUR and EOPR. ALL calibration processes result in a PFA of less than 13.6 %, regardless of how low the TUR is and how low the EOPR is. No calibration scenario can be proposed which will result in a PFA of >13.6 %. This maximum value of 13.6 % PFA results when the TUR is 0.29:1 and the EOPR is 40.9 %. Any change (higher or lower) for either the TUR or EOPR will result in a PFA lower than 13.6 %.

It is also possible to form a surface plot of the False Accept Risk as a function of TUR and EOPR. This combines both aspects affecting “Risk” into one visual representation that can further illustrate the relationship between these three variables. Two surface plots are provided below. It is easily observed that PFA will never exceed 2 % for any possible values of TUR where the True EOPR is greater than 95.13 %.

Note: The previous graph was generated using theoretically True EOPR. However, Observed EOPR is always lower or worse (higher spread or standard deviation) than True EOPR, due to real-world imperfect measurements being accompanied by uncertainty. True EOPR can be estimated from Observed EOPR by removing the effects of the measurement uncertainty from the “spread” of the Observed EOPR using a form of the variance addition rule (root difference of squares). It can then be asserted that for an Observed EOPR of 89.5 % or greater, the False Accept Risk can never be greater than 2 % under any circumstances (e.g. regardless of TUR, the False Accept Risk will always be <2 %). It should be noted that Observed EOPR cannot be adequately corrected to True EOPR.

One method of visualizing the 3D surface plot is to consider the family of TUR curves previously presented. If these curves were “stacked” in a third dimension (protruding out-of and recessed in-to this page) in descending order from high TUR to low TUR, they would form the 3D surface plot shown below. The surface

plot begins with a TUR of 4.7. This is the leading edge of the surface shown below in purple. An infinite number of TUR curves can be imagined and plotted along the TUR axis from front-to-back, in-to and out-of the page.

This visualization can also be performed by considering the family of EOPR curves previously presented. If these curves were “stacked” in a third dimension (protruding out-of and recessed in-to this page) in ascending order from low EOPR to high EOPR, they would form the 3D surface plot shown below. An infinite

number of EOPR curves can be imagined and plotted along the TUR axis from front-to-back, in-to and out-of the page. This maximum peak value for PFA of 13.6 % occurs in the red region when the TUR is 0.29:1 and the EOPR is 40.9 %.