A State-of-the-art Neural Network for Robust Face Verification - - PowerPoint PPT Presentation

.

A State-of-the-art Neural Network for Robust Face Verification

Sebastien Marcel and Samy Bengio

2

• Outline

➨ General Framework for Face Verification The proposed approach The XM2VTS database Results Future Work

3

• Face Verification (1/4)

Accepting or Rejecting a claimed identity

(Client vs Impostor)

Building a model for each client

Training Model 1 C 1 C <> 1 M 1 Training Model # C # C <> # M # Training Model N C N C <> N M N

4

• Face Verification (1/4)

Building a model for each client

(1) Reference images of client # (2) Reference images of non-client #

Generative vs Discriminant models

Generative if only (1) Discriminant if (1) and (2)

Training Model # Model #

Reference images

• f client #

Reference images of

• ther persons than #

5

• Face Verification (2/4)

Accepting a client claiming identity #

Model # Decision

I

C

X is claiming identity # X is accepted

6

• Face Verification (3/4)

Rejecting an impostor claiming identity #

Y is claiming identity # Y is rejected

Model # Decision

I

C

7

• Outline

✘ General Framework for Face Verification ➨ The proposed approach : MLP and Skin Color The XM2VTS database Results Future Work

8

• MLP using Face Image and Color

Model use: MLP (client patterns vs other clients) Features: 30x40 face template + skin color distribution

Subwindow extraction Downsizing Normalisation

MLP Decision

R A Skin feature vector

• f dimension 96 (3x32)

Final feature vector

• f dimension 1296

Face template vector

• f dimension 1200 (30x40)

Filtering skin color pixels Computing skin pixels distribution

9

• The XM2VTS* database

Using XM2VTS Database :

Fusion (Face verification + Speech verification), Face detection evaluation.

Content : 295 persons x 4 sessions x 2 shots

2 audio digit sequences + 1 image

000_1_2 000_3_1 369_1_1 369_4_2

* http://www.ee.surrey.ac.uk/Research/VSSP/xm2vtsdb/

10

• XM2VTS : Lausanne Protocol*

Clients / Impostors

200 clients, 25 impostors for evaluation, 70 impostors for test.

Protocols for training, evaluation and test :

Configuration I, Configuration II.

* ftp://ftp.idiap.ch/pub/reports/1998/com98-05.ps.gz

11

• XM2VTS : Lausanne Protocol

Configuration I :

Training : 200 C x 3 I (shot 1 of sessions 1,2,3) => 600 Evaluation Clients : 200 C x 3 I (shot 2 of sessions 1,2,3) => 600

Configuration II :

Training : 200 C x 4 I (shots of sessions 1,2) => 800 Evaluation Clients : 200 C x 2 I (shots of session 3) => 400

Common to both configurations :

Evaluation Impostors : 200 CM x 25 Imp x 8 I (shots of sessions 1-4) Test Clients : 200 C x 2 I (shots of session 4) => 400 Test Impostors : 200 CM x 70 Imp x 8 I (shots of sessions 1-4)

=> 40000 => 112000

12

• Evaluation using LP

Computing Errors for a given threshold

FAR = # False acceptance / # impostor accesses FRR = # False rejection / # client access

Estimate threshold reaching the EER i.e.

where FAR=FRR on the evaluation set

Compute FAR and FRR with the selected

threshold on the test set

The unique measure is HTER = (FAR + FRR) / 2

13

• Comparative results

HTER on the test set:

Method HTER LP1 HTER LP2 NC 61x57 3.15 1.50 MLP 30x40

• 2.807

MLP 30x40 + C

• 2.125

MLP 30x40 + C 1.87 1.85

14

• Future Work

Investigate new models:

GMM, HMM2

Investigate new features:

edges, gabor wavelets

Investigate full-automatic face verification:

integrate automatic face localisation, evaluate degradation of performances compared to perfect face

localisation.