ANSI INCITS 409 3-2005 pdf download Biometric Performance Testing and Reporting – Part 3: Scenario Testing and Reporting
Scenario tests may compare performance of verification and identification systems in the same application, such as access control. A careful approach to test execution and reporting is necessary to ensure equitable presentation of findings. For example, the order of enrollment, genuine, and impostor trials may change depending on whether systems perform identification or verification. In scenario tests that incorporate verification and identification functionality, the experimenter shall address the manner in which design and reporting accounts for such usage.
The experimenter shall generate performance results based on the matching functionality to be incorporated in the test in accordance with the requirements in 4.2.
NOTE Scenario test design and execution can change considerably depending on the matching functionality the test incorporates. Scenario tests typically evaluate systems in which 1:1 transactions are executed based on a claimed identity. Large-scale identification system performance is not generally evaluated in scenario tests due to the difficulty in recruiting a sufficiently large test population as well as the limited utility of modeling online transactions in this environment. However, small-scale identification system performance such as that encountered in access control systems can be realistically evaluated in scenario tests, as such systems provide real-time feedback.
4.1.3 Test transactions Depending on the experimenter’s level of access to the test population, each test subject may be able to execute multiple transactions over the course of multiple visits. The number of transactions and visits shall be maximized to the degree possible, with the exception of tests in which attempts to measure test subject habituation dictate less frequent usage.