Pass-Fail Testing: Statistical Requirements and Interpretations

In evaluating the effectiveness of equipment used to detect hidden contraband goods or hazardous materials, the equipment is usually tested according to the requirements specified in the agreement established by the national or international standards body, Performance is measured. The performance requirements of these standards include detection probability (PD) and false positive probability (PFA) at a specific statistical confidence level.

It is assumed that the detection system discussed in this paper behaves according to the binomial distribution. Independent trials where forbidden items are present, only two outcomes are considered. The detection system reports whether it correctly reports detection. In addition, the probability of detection must be kept constant during the test. Otherwise, a test based on a binomial model is performed and it is judged that the meaning of this number estimate may be small. Likewise, in the case of a test without forbidden items, the detection system either reports correctly that it could not be detected, or reports the existence of smuggled items by mistake. We assume that the false alarm probability is constant throughout the test

In the inspection system, PD or PFA can only be accurately determined by a sufficient number of tests. However, there is a number called a confidence level (CL) that can fully understand the results of a series of test results for a particular size.

CL is defined according to a binomial probability mass function also known as a binomial discrete density function b (m; n, p).

Where m = 0, 1, ..., n represents the number of successful tests or false positives in n independent tests. p = PD or p = PFA, 0 ≤ p ≤ 1 (see Johnson, Kotz and Kemp, 1992). If the success or failure probability succeeds and the probability of success is fixed per trial, the number of trials successful n times in a trial match matches this function.

Section 2 discusses the definition of CL and appropriate thresholds for detection problems. Section 3 provides a statistical explanation of these values ​​based on hypothesis tests and confidence intervals. The explanation ends with Sec. Some examples are included in 4

Statistical hypothesis tests may return values ​​called p or p values. This is the amount you can use to interpret or quantify the test results and to reject or reject the null hypothesis. This is done by comparing the p value with a preselected threshold called the significance level. You can think of a statistical test based on the dichotomy of rejecting and accepting the null hypothesis. The danger is that if you say "accept" the null hypothesis, the language indicates that the null hypothesis is correct. On the contrary, it is safer to say that you can not refuse the null hypothesis because there is not enough statistical evidence to reject it.

All statistical tests are done under the null hypothesis. It is not a hypothesis. According to the result of the statistical test, 1) it is not possible to reject the null hypothesis, or 2) the null hypothesis can not be rejected. Please never use "Accept Zero Assumption". When you say "reject the null hypothesis", this means that you can rationally judge that the null hypothesis is wrong. When you say that you are not rejecting the null hypothesis, it means that there is not enough evidence to argue that the null hypothesis is wrong.

Hypothesis testing is a general way of thinking in statistics. There is a series of data, hypotheses, statistical tests. The final result is a decision as to whether the null hypothesis can be rejected to support a particular choice. The null hypothesis is almost always the same condition (that is, things of interest are equal to others). Sometimes it is a range (ie, what is of interest is between the two values). If you do not reject the null hypothesis, you will notice that the null hypothesis is unknown, you can see that the particular choice tested for a particular dataset in a particular way did not reject the null hypothesis.