Recent Guidelines Related to the Implementation of Speaker - - PowerPoint PPT Presentation
Forensic Panel Odyssey 2016 23 June 2016, Bilbao, Spain Recent Guidelines Related to the Implementation of Speaker Recognition In Court in Europe Daniel Ramos ATVS Biometric Recognition Group Universidad Autonoma de Madrid
Forensic Panel Odyssey 2016 23 June 2016, Bilbao, Spain
Forensic Panel, OdysseBilbao, 23 June, 2016. 2
Forensic Panel, OdysseBilbao, 23 June, 2016. 3
Forensic Panel, OdysseBilbao, 23 June, 2016. 4
Forensic Panel, OdysseBilbao, 23 June, 2016. 5
http://www.enfsi.eu/documents/forensic-guidelines
Forensic Panel, OdysseBilbao, 23 June, 2016. 6
n Probabilistic expressions of conclusions
q Verbal Scales are recommended q Avoid absolute statements (identification/exclusion)!
Forensic Panel, OdysseBilbao, 23 June, 2016. 7
n Probabilistic expressions of conclusions
q Verbal Scales are recommended q Avoid absolute statements (identification/exclusion)!
n Hypotheses in a case always must consider at least one alternative
q Clear definition of the population of potential alternative authors
Forensic Panel, OdysseBilbao, 23 June, 2016. 8
n Probabilistic expressions of conclusions
q Verbal Scales are recommended q Avoid absolute statements (identification/exclusion)!
n Hypotheses in a case always must consider at least one alternative
q Clear definition of the population of potential alternative authors
n Findings must be evaluated given each of all hypotheses
q Likelihood ratios (LR)
Forensic Panel, OdysseBilbao, 23 June, 2016. 9
n Probabilistic expressions of conclusions
q Verbal Scales are recommended q Avoid absolute statements (identification/exclusion)!
n Hypotheses in a case always must consider at least one alternative
q Clear definition of the population of potential alternative authors
n Findings must be evaluated given each of all hypotheses
q Likelihood ratios (LR)
n Conclusions expressed in terms of support to the hypotheses (LR)
q Avoid probabilities of the hypotheses (posteriors)
Forensic Panel, OdysseBilbao, 23 June, 2016. 10
n Probabilistic expressions of conclusions
q Verbal Scales are recommended q Avoid absolute statements (identification/exclusion)!
n Hypotheses in a case always must consider at least one alternative
q Clear definition of the population of potential alternative authors
n Findings must be evaluated given each of all hypotheses
q Likelihood ratios (LR)
n Conclusions expressed in terms of support to the hypotheses (LR)
q Avoid probabilities of the hypotheses (posteriors)
n Towards data-driven testable and repeatable methodologies, but…
q Subjective judgements (probabilities, LRs) are allowed
Forensic Panel, OdysseBilbao, 23 June, 2016. 11
Forensic Panel, OdysseBilbao, 23 June, 2016. 12
Forensic Panel, OdysseBilbao, 23 June, 2016. 13
n Objective
q Determine if a LR method is valid to be used in casework q All the validation process should be documented for transparency
Forensic Panel, OdysseBilbao, 23 June, 2016. 14
n Objective
q Determine if a LR method is valid to be used in casework q All the validation process should be documented for transparency q Towards standardization of procedures (for biometrics)
Forensic Panel, OdysseBilbao, 23 June, 2016. 15
n Objective
q Determine if a LR method is valid to be used in casework q All the validation process should be documented for transparency q Towards standardization of procedures (for biometrics) q Proposed some relevant performance characteristics
Performance Characteristic Performance Metric Graphical Representation Validation criteria Experiment Data Analytical result Validation Decision Accuracy Cllr, EER ECE plot NBE plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Discriminating power Cllrmin ECE plot NBE plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Calibration Cllrcal Tippett plot ECE plot NBE plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Robustness Variation of Cllr, EER ECE plot DET plot Tippett plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Coherence Variation of Cllr, EER ECE plot DET plot Tippett plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Generalization Variation of Cllr, EER ECE plot DET plot Tippett plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail
Forensic Panel, OdysseBilbao, 23 June, 2016. 16
n Objective
q Determine if a LR method is valid to be used in casework q All the validation process should be documented for transparency q Towards standardization of procedures (for biometrics) q Proposed some relevant performance characteristics
Performance Characteristic Performance Metric Graphical Representation Validation criteria Experiment Data Analytical result Validation Decision Accuracy Cllr, EER ECE plot NBE plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Discriminating power Cllrmin ECE plot NBE plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Calibration Cllrcal Tippett plot ECE plot NBE plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Robustness Variation of Cllr, EER ECE plot DET plot Tippett plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Coherence Variation of Cllr, EER ECE plot DET plot Tippett plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Generalization Variation of Cllr, EER ECE plot DET plot Tippett plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail
!
Forensic Panel, OdysseBilbao, 23 June, 2016. 17
n Objective
q Determine if a LR method is valid to be used in casework q All the validation process should be documented for transparency q Towards standardization of procedures (for biometrics) q Proposed some relevant performance characteristics
Performance Characteristic Performance Metric Graphical Representation Validation criteria Experiment Data Analytical result Validation Decision Accuracy Cllr, EER ECE plot NBE plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Discriminating power Cllrmin ECE plot NBE plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Calibration Cllrcal Tippett plot ECE plot NBE plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Robustness Variation of Cllr, EER ECE plot DET plot Tippett plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Coherence Variation of Cllr, EER ECE plot DET plot Tippett plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail Generalization Variation of Cllr, EER ECE plot DET plot Tippett plot According to the definition Description Data used +/- [%] compared to the baseline Pass / fail
!
Forensic Panel, OdysseBilbao, 23 June, 2016. 18
http://www.enfsi.eu/documents/forensic-guidelines
Forensic Panel, OdysseBilbao, 23 June, 2016. 19
n Compatible with ENFSI Guideline for reporting
q Reporting likelihood ratios
http://www.enfsi.eu/documents/forensic-guidelines
Forensic Panel, OdysseBilbao, 23 June, 2016. 20
n Compatible with ENFSI Guideline for reporting
q Reporting likelihood ratios
n Compatible with Guideline for Validation
q Similar performance characteristics q Emphasis in forensically relevant data
http://www.enfsi.eu/documents/forensic-guidelines
Forensic Panel, OdysseBilbao, 23 June, 2016. 21
n Compatible with ENFSI Guideline for reporting
q Reporting likelihood ratios
n Compatible with Guideline for Validation
q Similar performance characteristics q Emphasis in forensically relevant data
n Addresses many other issues
q Most used technologies and methods q Most used features q Audio pre-processing q …
http://www.enfsi.eu/documents/forensic-guidelines
Forensic Panel Odyssey 2016 23 June 2016, Bilbao, Spain