[PPT] - E x p l a i n i n g M a c h i n e L e a r n i PowerPoint Presentation

SLIDE 1

1 / 4 5

E x p l a i n i n g M a c h i n e L e a r n i n g D e c i s i

n

s

G r é g

i

r e M

n

t a v

n

, T U B e r l i n

J

i

n t w

r

k w i t h : W

j

c i e c h S a m e k , K l a u s

R
b

e r t M ü l l e r , S e b a s t i a n L a p u s c h k i n , A l e x a n d e r B i n d e r

1 8 / 9 / 2 1 8 I n t l . W

r

k s h

p

M L & A I , T e l e c

m

P a r i s T e c h

SLIDE 2

2 / 4 5

F r

m

M L S u c c e s s e s t

A

p p l i c a t i

n

s

Autonomous Driving Medical Diagnosis Networks (smart grids, etc.)

Visual Reasoning AlphaGo beats Go human champ Deep Net outperforms humans in image classification

SLIDE 3

3 / 4 5

Can we interpret what a ML model has learned?

SLIDE 4

4 / 4 5

First, we need to define what we want from interpretable ML.

SLIDE 5

5 / 4 5

U n d e r s t a n d i n g D e e p N e t s : T w

V

i e w s

Understanding what mechanism the network uses to solve a problem or implement a function. Understanding how the network relates the input to the output variables.

SLIDE 6

6 / 4 5 interpreting predicted classes explaining individual decisions

SLIDE 7

I n t e r p r e t i n g P r e d i c t e d C l a s s e s

Image from Symonian’13

E x a m p l e :

“How does a goose typically look like according to the neural network?”

goose non-goose

SLIDE 8

8 / 4 5

E x p l a i n i n g I n d i v i d u a l D e c i s i

n

s

Images from Lapuschkin’16

E x a m p l e :

“Why is a given image classified as a sheep?”

sheep non-sheep

SLIDE 9

9 / 4 5

E x a m p l e : A u t

n
m
u

s D r i v i n g

[ B

j

a r s k i ’ 1 7 ]

I n p u t : D e c i s i

n

B

j

a r s k i e t a l . 2 1 7 “ E x p l a i n i n g H

w

a D e e p N e u r a l N e t w

r

k T r a i n e d w i t h E n d

t
E

n d L e a r n i n g S t e e r s a C a r ” P i l

t

N e t E x p l a n a t i

n

:

SLIDE 10

1 / 4 5

E x a m p l e : P a s c a l V O C C l a s s i fi c a t i

n

[ L a p u s c h k i n ’ 1 6 ]

C

m

p a r i n g P e r f

r

m a n c e

n

P a s c a l V O C 2 7 ( F i s h e r V e c t

r

C l a s s i fi e r v s . D e e p N e t p r e t r a i n e d

n

I m a g e N e t ) F i s h e r c l a s s i fi e r ( p r e t r a i n e d ) d e e p n e t

L a p u s c h k i n e t a l . 2 1 6 . A n a l y z i n g C l a s s i fi e r s : F i s h e r V e c t

r

s a n d D e e p N e u r a l N e t w

r

k s

SLIDE 11

1 1 / 4 5

E x a m p l e : P a s c a l V O C C l a s s i fi c a t i

n

[ L a p u s c h k i n ’ 1 6 ]

L a p u s c h k i n e t a l . 2 1 6 . A n a l y z i n g C l a s s i fi e r s : F i s h e r V e c t

r

s a n d D e e p N e u r a l N e t w

r

k s

SLIDE 12

1 2 / 4 5

E x a m p l e : M e d i c a l D i a g n

s

i s

[ B i n d e r ’ 1 8 ]

A : I n v a s i v e b r e a s t c a n c e r , H & E s t a i n ; B : N

r

m a l m a m m a r y g l a n d s a n d fi b r

u

s t i s s u e , H & E s t a i n ; C : D i f f u s e c a r c i n

m

a i n fi l t r a t e i n fi b r

u

s t i s s u e , H e m a t

x

y l i n s t a i n

B i n d e r e t a l . 2 1 8 “ T

w

a r d s c

m

p u t a t i

n

a l fl u

r

e s c e n c e m i c r

s

c

p

y : M a c h i n e l e a r n i n g

b

a s e d i n t e g r a t e d p r e d i c t i

n
f

m

r

p h

l
g

i c a l a n d m

l

e c u l a r t u m

r

p r

fi

l e s ”

SLIDE 13

1 3 / 4 5

E x a m p l e : Q u a n t u m C h e m i s t r y

[ S c h ü t t ’ 1 7 ]

DFT calculation of the stationary Schrödinger Equation D T N N , S c h ü t t ’ 1 7

m

l

e c u l a r s t r u c t u r e ( e . g . a t

m

s p

s

i t i

n

s ) m

l

e c u l a r e l e c t r

n

i c p r

p

e r t i e s ( e . g . a t

m

i z a t i

n

e n e r g y )

P B E , P e d r e w ’ 8 6

i n t e r p r e t a b l e i n s i g h t

S c h ü t t e t a l . 2 1 7 : Q u a n t u m

C

h e m i c a l I n s i g h t s f r

m

D e e p T e n s

r

N e u r a l N e t w

r

k s

SLIDE 14

1 4 / 4 5

E x a m p l e s

f

E x p l a n a t i

n

M e t h

d

s

SLIDE 15

1 5 / 4 5

E x p l a i n i n g b y D e c

m

p

s

i n g

I m p

r

t a n c e

f

a v a r i a b l e i s t h e s h a r e

f

t h e f u n c t i

n

s c

r

e t h a t c a n b e a t t r i b u t e d t

i

t . D e c

m

p

s

i t i

n

p r

p

e r t y : i n p u t

D N N

SLIDE 16

1 6 / 4 5

E x p l a n i n g L i n e a r M

d

e l s

A s i m p l e m e t h

d

:

SLIDE 17

1 7 / 4 5

E x p l a n i n g L i n e a r M

d

e l s

T a y l

r

d e c

m

p

s

i t i

n

a p p r

a

c h : I n s i g h t : e x p l a n a t i

n

d e p e n d s

n

t h e r

t

p

i

n t .

SLIDE 18

1 8 / 4 5

E x p l a i n i n g N

n

l i n e a r M

d

e l s

s e c

n

d

r

d e r t e r m s a r e h a r d t

i

n t e r p r e t a n d c a n b e v e r y l a r g e

SLIDE 19

1 9 / 4 5

E x p l a i n i n g N

n

l i n e a r M

d

e l s b y P r

p

a g a t i

n

L a y e r

W

i s e R e l e v a n c e P r

p

a g a t i

n

( L R P ) [ B a c h ’ 1 5 ]

SLIDE 20

2 / 4 5

E x p l a i n i n g N

n

l i n e a r M

d

e l s b y P r

p

a g a t i

n

Is there an underlying mathematical framework?

SLIDE 21

2 1 / 4 5

D e e p T a y l

r

D e c

m

p

s

i t i

n

( D T D )

[ M

n

t a v

n

’ 1 7 ] Question: Suppose that we have propagated LRP scores (“relevance”) until a given layer. How should it be propagated one layer further? Key idea: Let’s use Taylor expansions for this.

SLIDE 22

2 2 / 4 5

D T D S t e p 1 : T h e S t r u c t u r e

f

R e l e v a n c e

O b s e r v a t i

n

: R e l e v a n c e a t e a c h l a y e r i s a p r

d

u c t

f

t h e a c t i v a t i

n

a n d a n a p p r

x

i m a t e l y c

n

s t a n t t e r m .

SLIDE 23

2 3 / 4 5

D T D S t e p 1 : T h e S t u c t u r e

f

R e l e v a n c e

SLIDE 24

2 4 / 4 5

D T D S t e p 2 : T a y l

r

E x p a n s i

n

SLIDE 25

2 5 / 4 5

D T D S t e p 2 : T a y l

r

E x p a n s i

n

T a y l

r

e x p a n s i

n

a t r

t

p

i

n t : R e l e v a n c e c a n n

w

b e b a c k w a r d p r

p

a g a t e d

SLIDE 26

2 6 / 4 5

D T D S t e p 3 : C h

s

i n g t h e R

t

P

i

n t

(same as LRP-

α

1

β

)

( D e e p T a y l

r

g e n e r i c ) ✔

1 . n e a r e s t r

t

2 . r e s c a l e d e x c i t a t i

n

s C h

i

c e

f

r

t

p

i

n t

SLIDE 27

2 7 / 4 5

D T D : C h

s

i n g t h e R

t

P

i

n t

( D e e p T a y l

r

g e n e r i c ) P i x e l s d

ma

i n

C h

i

c e

f

r

t

p

i

n t

SLIDE 28

2 8 / 4 5

D T D : C h

s

i n g t h e R

t

P

i

n t

( D e e p T a y l

r

g e n e r i c )

C h

i

c e

f

r

t

p

i

n t

E mb e d d i n g :

i m a g e s

u

r c e : T e n s

r

fl

w

t u t

r

i a l

SLIDE 29

2 9 / 4 5

D T D : A p p l i c a t i

n

t

P
l

i n g L a y e r s

A s u m

p
l

i n g l a y e r

v

e r p

s

i t i v e a c t i v a t i

n

s i s e q u i v a l e n t t

a

R e L U l a y e r w i t h w e i g h t s 1 . A p

n
r

m p

l

i n g l a y e r c a n b e a p p r

x

i m a t e d a s a s u m

p
l

i n g l a y e r m u l t i p l i e d b y a r a t i

f

n

r

m s t h a t w e t r e a t a s c

n

s t a n t [ M

n

t a v

n

’ 1 7 ] .

→ T r e a t p

l

i n g l a y e r s a s R e L U d e t e c t i

n

l a y e r s

SLIDE 30

3 / 4 5

D e e p T a y l

r

D e c

m

p

s

i t i

n
n

C

n

v N e t s

L R P

α

1

β D T D f

r

p i x e l s L R P

α

1

β L R P

α

1

β L R P

α

1

β L R P

α

1

β L R P

α

1

β *

* F

r

t

p
l

a y e r s ,

t

h e r r u l e s m a y i m p r

v

e s e l e c t i v i t y f

r

w a r d p a s s b a c k w a r d p a s s

SLIDE 31

3 1 / 4 5

I m p l e m e n t i n g P r

p

a g a t i

n

R u l e s

S e q u e n c e

f

e l e m e n t

w

i s e c

m

p u t a t i

n

s S e q u e n c e

f

v e c t

r

c

m

p u t a t i

n

s E x a m p l e :

L R P

α

1

β

:

SLIDE 32

3 2 / 4 5

I m p l e m e n t i n g P r

p

a g a t i

n

R u l e s

E x a m p l e :

L R P

α

1

β

: C

d

e t h a t r e u s e s f

r

w a r d a n d g r a d i e n t c

m

p u t a t i

n

s :

SLIDE 33

3 3 / 4 5

H

w

D e e p T a y l

r

/ L R P S c a l e s

SLIDE 34

3 4 / 4 5

I m p l e m e n t a t i

n
n

L a r g e

S

c a l e M

d

e l s

[ A l b e r ’ 1 8 ] h t t p s : / / g i t h u b . c

m

/ a l b e r m a x / i n n v e s t i g a t e

SLIDE 35

3 5 / 4 5

D T D f

r

K e r n e l M

d

e l s

[ K a u f f m a n n ’ 1 8 ]

1 . B u i l d a n e u r a l n e t w

r

k e q u i v a l e n t

f

t h e O n e

C

l a s s S V M :

G a u s s i a n / L a p l a c e K e r n e l S t u d e n t K e r n e l

2 . C

mp

u t e s i t s d e e p T a y l

r

d e c

mp
s

i t i

n

O u t l i e r s c

r

e

SLIDE 36

3 6 / 4 5

D T D : C h

s

i n g t h e R

t

P

i

n t ( R e v i s i t e d )

✔ ✔

1 . n e a r e s t r

t

3 . r e s c a l e d a c t i v a t i

n

2 . r e s c a l e d e x c i t a t i

n

s C h

i

c e

f

r

t

p

i

n t

2 . L R P

α

1

β ( D e e p T a y l

r

g e n e r i c ) 3 . A n

t

h e r r u l e

SLIDE 37

3 7 / 4 5

S e l e c t i n g t h e E x p l a n a t i

n

T e c h n i q u e How to select the best root points?

SLIDE 38

3 8 / 4 5

S e l e c t i n g t h e E x p l a n a t i

n

T e c h n i q u e Which rule leads to the best explanation?

SLIDE 39

3 9 / 4 5

S e l e c t i n g t h e E x p l a n a t i

n

T e c h n i q u e What axioms should an explanation satisfy?

SLIDE 40

4 / 4 5

S e l e c t i

n

b a s e d

n

A x i

m

s

d i v i s i

n

b y z e r

→

s c

r

e s e x p l

d

e .

L R P

α

1

β

SLIDE 41

4 1 / 4 5

S e l e c t i

n

b a s e d

n

A x i

m

s

d i s c

n

t i n u

u

s s t e p f u n c t i

n

f

r

b

j

=

L R P

α

1

β

SLIDE 42

4 2 / 4 5

E x p l a i n a b l e M L : C h a l l e n g e s Validating Explanations

U n d e r l y i n g ma t h e ma t i c a l f r a me w

r

k A x i

ms
f

a n e x p l a n a t i

n

P e r t u r b a t i

n

a n a l y s i s [ S a m e k ’ 1 7 ] S i m i l a r i t y t

g

r

u

n d t r u t h H u ma n p e r c e p t i

n

SLIDE 43

4 3 / 4 5

E x p l a i n a b l e M L : O p p

r

t u n i t i e s Using Explanations

H u m a n i n t e r a c t i

n

D e s i g n i n g b e t t e r M L a l g

r

i t h m s ? E x t r a c t i n g n e w d

m

a i n k n

w

l e d g e D e t e c t i n g u n e x p e c t e d M L b e h a v i

r

F i n d i n g w e a k n e s s e s

f

a d a t a s e t

SLIDE 44

4 4 / 4 5

C h e c k

u

r w e b p a g e

w i t h i n t e r a c t i v e d e m

s

, s

f

t w a r e , t u t

r

i a l s , . . .

a n d

u

r t u t

r

i a l p a p e r : M

n

t a v

n

, G . , S a m e k , W . , M ü l l e r , K .

R

. M e t h

d

s f

r

I n t e r p r e t i n g a n d U n d e r s t a n d i n g D e e p N e u r a l N e t w

r

k s , D i g i t a l S i g n a l P r

c

e s s i n g , 2 1 8

SLIDE 45

4 5 / 4 5

R e f e r e n c e s

[

A l b e r ’ 1 8 ] A l b e r , M . L a p u s c h k i n , S . , S e e g e r e r , P . , H ä g e l e , M . , S c h ü t t , K . , M

n

t a v

n

, G . , S a m e k , W . , M ü l l e r , K .

R

. , D ä h n e , S . , K i n d e r m a n s . P .

J

. i N N v e s t i g a t e n e u r a l n e t w

r

k s . C

R

R a b s / 1 8 8 . 4 2 6 , 2 1 8

[

B a c h ’ 1 5 ] B a c h , S . , B i n d e r , A . , M

n

t a v

n

, G . , K l a u s c h e n , F . , M ü l l e r , K .

R

. , S a m e k , W . O n p i x e l

w

i s e e x p l a n a t i

n

s f

r

n

n

l i n e a r c l a s s i fi e r d e c i s i

n

s b y l a y e r

w

i s e r e l e v a n c e p r

p

a g a t i

n

. P L O S O N E 1 ( 7 ) , 2 1 5

[

B i n d e r ’ 1 8 ] B i n d e r , A . , B

c

k m a y r , M . , … , M ü l l e r , K .

R

. , K l a u s c h e n , F . T

w

a r d s c

m

p u t a t i

n

a l fl u

r

e s c e n c e m i c r

s

c

p

y : M a c h i n e l e a r n i n g

b

a s e d i n t e g r a t e d p r e d i c t i

n
f

m

r

p h

l
g

i c a l a n d m

l

e c u l a r t u m

r

p r

fi

l e s C

R

R a b s / 1 8 5 . 1 1 1 7 8 , 2 1 8

[

B

j

a r s k i ’ 1 7 ] B

j

a r s k i , M . , Y e r e s , P . , C h

r
m

a n s k a , A . , C h

r
m

a n s k i , K . , F i r n e r , B . , J a c k e l , L . D . , M u l l e r , U . E x p l a i n i n g h

w

a d e e p n e u r a l n e t w

r

k t r a i n e d w i t h e n d

t
e

n d l e a r n i n g s t e e r s a c a r . C

R

R a b s / 1 7 4 . 7 9 1 1 , 2 1 7

[

K a u f f ma n n ’ 1 8 ] K a u f f m a n n , J . M ü l l e r , K .

R

. , M

n

t a v

n

, G . , T

w

a r d s E x p l a i n i n g A n

m

a l i e s : A D e e p T a y l

r

D e c

m

p

s

i t i

n
f

O n e

C

l a s s M

d

e l s C

R

R a b s / 1 8 5 . 6 2 3 , 2 1 8

[

L a p u s c h k i n ’ 1 6 ] L a p u s c h k i n , S . , B i n d e r , A . , M

n

t a v

n

, G . , M ü l l e r , K .

R

. , S a m e k , W . A n a l y z i n g c l a s s i fi e r s : F i s h e r v e c t

r

s a n d d e e p n e u r a l n e t w

r

k s . C V P R 2 1 6

[

M

n

t a v

n

’ 1 7 ] M

n

t a v

n

, G . , L a p u s c h k i n , S . , B i n d e r , A . , S a m e k , W . , M ü l l e r , K .

R

. E x p l a i n i n g n

n

l i n e a r c l a s s i fi c a t i

n

d e c i s i

n

s w i t h d e e p T a y l

r

d e c

m

p

s

i t i

n

. P a t t e r n R e c

g

n i t i

n

6 5 , 2 1 1 – 2 2 2 , 2 1 7

[

S a me k ’ 1 7 ] S a m e k , W . , B i n d e r , A . , M

n

t a v

n

, G . , L a p u s c h k i n , S . , M ü l l e r , K .

R

. E v a l u a t i n g t h e v i s u a l i z a t i

n
f

w h a t a d e e p n e u r a l n e t w

r

k h a s l e a r n e d . I E E E T r a n s a c t i

n

s

n

N e u r a l N e t w

r

k s a n d L e a r n i n g S y s t e m s , 2 1 7

[

S c h ü t t ’ 1 7 ] S c h ü t t , K . T . , A r b a b z a d a h , F . , C h m i e l a , S . , M ü l l e r , K .

R

. , T k a t c h e n k

,

A . Q u a n t u m

C

h e m i c a l I n s i g h t s f r

m

D e e p T e n s

r

N e u r a l N e t w

r

k s , N a t u r e C

m

m u n i c a t i

n

s 8 , 1 3 8 9 , 2 1 7

[

S y mo n i a n ’ 1 3 ] S y m

n

i a n , K . V e d a l d i , A . Z i s s e r m a n , A . D e e p I n s i d e C

n

v

l

u t i

n

a l N e t w

r

k s : V i s u a l i s i n g I m a g e C l a s s i fi c a t i

n

M

d

e l s a n d S a l i e n c y M a p s . A r X i v 2 1 3