AS ISO/IEC 19795.2—2010 pdf – Information technology—Biometric performance testing and reporting Part 2: Testing methodologies for technology and scenario evaluation

AS ISO/IEC 19795.2—2010 pdf – Information technology—Biometric performance testing and reporting Part 2: Testing methodologies for technology and scenario evaluation.
4 Terms and definitions For the purposes of this document, the terms and definitions given in ISO/IEC 19795-1:2006 and the following apply. 4.1 Biometric data 4.1.1 biometric reference ?template, model?? user’s stored reference measure based on features extracted from enrolment samples 4.2 Components of a biometric system 4.2.1 feature extractor apparatus that extracts features from a sample 4.2.2 biometric reference generator apparatus that transforms a sample into a biometric reference 4.3 User interaction with a biometric system 4.3.1 acclimatization reduction, over the course of an evaluation, in a temporal condition of a biometric characteristic that may impact the ability of a sensor to process a sample 4.3.2 effort level number of presentations, attempts or transactions needed to successfully enrol or match in a biometric system 4.3.3 enrolment attempt submission of one or more biometric samples for a Test Subject for the purpose of enrolment in a biometric system NOTE 1 One or more enrolment attempts may be permitted or required to constitute an enrolment transaction. An enrolment attempt may comprise one or more enrolment presentations. NOTE 2 See Annex B for illustration of the relationship between presentation, attempt and transaction. 4.3.4 enrolment attempt limit maximum number of attempts, or the maximum duration, a Test Subject is permitted before an enrolment transaction is terminated 4.3.5 enrolment presentation submission of an instance of a biometric characteristic for a Test Subject for the purpose of enrolment NOTE One or more enrolment presentations may be permitted or required to constitute an enrolment attempt. An enrolment presentation may or may not result in an enrolment attempt. 4.3.6 enrolment presentation limit maximum number of presentations, or the maximum duration, a Test Subject is permitted before an enrolment attempt is terminated
4.3.7 guidance direction provided by an Administrator to a Test Subject in the course of enrolment or recognition NOTE Guidance is separate from feedback provided by a biometric system or device in the course of enrolment or recognition, such as audible or visual presentation queues. 4.3.8 habituation degree of familiarity a Test Subject has with a device NOTE A Test Subject having substantial familiarity with a biometric device, such as that gained in the course of employment, is referred to as a habituated Test Subject. 4.3.9 comparison attempt submission of one or more biometric samples for a Test Subject for the purpose of comparison in a biometric system 4.3.10 comparison attempt limit maximum number of attempts, or the maximum duration, a Test Subject is permitted before a comparison transaction is terminated 4.3.11 comparison presentation submission of an instance of a single biometric characteristic for a Test Subject for the purpose of comparison NOTE One or more comparison presentations may be permitted or required to constitute a comparison attempt. A comparison presentation may or may not result in a comparison attempt. 4.3.12 comparison presentation limit maximum number of presentations, or the maximum duration, a Test Subject is permitted before a comparison attempt is terminated 4.4 Performance measures 4.4.1 failure at source rate proportion of samples discarded from the corpus either manually or by use of an automated biometric system prior to use in a technology evaluation EXAMPLE A proportion of images collected in a face data collection effort may be discarded due to lack of a face in the image. 5 Overview of technology evaluations and scenario evaluations This standard addresses two types of evaluation methodologies: technology evaluations and scenario evaluations. A test report shall state whether it presents results from a technology evaluation, a scenario evaluation, or an evaluation that combines aspects of both technology and scenario evaluations.
6.1.2 Application realism If the test intends to evaluate performance within an application or concept of operations, the test shall be designed and executed so that it mimics the functional (input to output) and procedural (e.g. enrolment or verification processes) aspects of such an application or concept of operations. EXAMPLE If several images are typically gathered to constitute an enrolment transaction in a real-world enrolment attempt, technology test design should follow a similar process. For testing purposes, the implementations under test should, if possible, return the comparison score of each comparison attempt. 6.1.3 Determination of appropriate performance measures Experimenters shall determine which performance measures are applicable to their evaluation, in addition to those listed at clause 6.3. Test design shall ensure that all required metrics can be generated.