Rationalizing Medical Relation Prediction from Corpus-level - - PowerPoint PPT Presentation

▶
rationalizing medical relation prediction from corpus
SMART_READER_LITE
LIVE PREVIEW

Rationalizing Medical Relation Prediction from Corpus-level - - PowerPoint PPT Presentation

Oct 13, 2023 174 likes •708 views

ACL 2020 Rationalizing Medical Relation Prediction from Corpus-level Statistics Zhen Wang The Ohio State University In collaboration with Jennifer Lee (NCH), Simon Lin (NCH), Huan Sun (OSU) One-Minute Summary What is this paper about? Task :

slide-1
SLIDE 1

Rationalizing Medical Relation Prediction from Corpus-level Statistics

Zhen Wang The Ohio State University

In collaboration with Jennifer Lee (NCH), Simon Lin (NCH), Huan Sun (OSU)

ACL 2020

slide-2
SLIDE 2

1-Min Summary

One-Minute Summary

1

What is this paper about?

Task: Predicting relations between two given terms from a text corpus Goal: Make accurate prediction & Provide justifications for it (Rationalization)

521 74 122 2341 18 428 1356 38

macase matreat matreat

122 C-cceceLik (ihc) AciaiRecall AiRecgii

Aii

Caee Mae PaRee Fee

Headache

  • 3. Justify the prediction (Aspirin may

treat headache) by highlighting important associations and assumptions

  • 1. Draw inspirations from human

memory recall and recognition

  • 2. Recall global associations (blue)

from corpus-level statistics and recognize meaningful relational assumptions (red) between them

slide-3
SLIDE 3

Overview

Black-Box Relation Classifier

2 Medical Records

Aspirin Headache Input Pair Black-Box Classifier Class Distribution

slide-4
SLIDE 4

Overview

Black-Box Relation Classifier

3 Medical Records

Aspirin Headache Input Pair Black-Box Classifier Class Distribution

High Risk!

Medicine Finance Judiciary

slide-5
SLIDE 5

Overview

Can We Open the Black-Box?

4 Medical Records

Aspirin Input Pair Class Distribution Headache

slide-6
SLIDE 6

Overview

Can We Open the Black-Box?

5 Medical Records

Aspirin Input Pair Class Distribution

To solve the problem, we draw inspirations from human memory theories – Recall and Recognition!

Headache

slide-7
SLIDE 7

Overview

Memory Recall and Recognition

6

Retrieve association information from long-term memory

Recall Recognition

Identify previously learned information

slide-8
SLIDE 8

Overview

Memory Recall and Recognition

7

Example from http://webdesign-review.blogspot.com/2016/04/recognition-vs-recall-in-mobile-web.html

Recall Recognition

Retrieve association information from long-term memory Identify previously learned information

Recall my friend’s name Recognize my friend’s face

slide-9
SLIDE 9

Overview

Rationalizing Medical Relation Prediction

8

Entity Pair

Recall Memory

Associations

Recognition Memory

Assumptions

Pred.

CogStage-1 CogStage-2 CogStage-3

Rationalized By

slide-10
SLIDE 10

Overview

Rationalizing Medical Relation Prediction

9

Entity Pair

Aspirin Headache

slide-11
SLIDE 11

Overview

Rationalizing Medical Relation Prediction

10

Entity Pair

Recall Memory Aspirin Headache

CogStage-1

slide-12
SLIDE 12

Overview

Rationalizing Medical Relation Prediction

11

Entity Pair

Recall Memory Aspirin Headache

Caffeine

Associations

CogStage-1

Migraine Pain Relief Fever

slide-13
SLIDE 13

Overview

Rationalizing Medical Relation Prediction

12

83 463 84 123 146 385 130 353 428

Corpus-level Statistics

Entity Pair

Recall Memory Aspirin Headache

Caffeine

Associations

CogStage-1

Migraine Pain Relief Fever

slide-14
SLIDE 14

Overview

Rationalizing Medical Relation Prediction

13

83 463 84 123 146 385 130 353 428

Corpus-level Statistics

Entity Pair

Recall Memory Aspirin Headache

Caffeine

Associations

CogStage-1

Migraine Pain Relief Fever Medical Records

Aspirin

Headache

slide-15
SLIDE 15

Overview

Rationalizing Medical Relation Prediction

14

83 463 84 123 146 385 130 353 428

Corpus-level Statistics

Entity Pair

Recall Memory Aspirin Headache

Caffeine

Associations

CogStage-1

Migraine Pain Relief Fever

Association Strength

slide-16
SLIDE 16

Overview

Rationalizing Medical Relation Prediction

15

Entity Pair

Recall Memory

Associations

Recognition Memory Aspirin

Caffeine

Headache

CogStage-1 CogStage-2

Migraine Pain Relief Fever

slide-17
SLIDE 17

Overview

Rationalizing Medical Relation Prediction

16

Entity Pair

Recall Memory

Associations

Recognition Memory Aspirin

Caffeine

Headache

may_cause CogStage-1 CogStage-2

Migraine Pain Relief Fever

slide-18
SLIDE 18

Overview

Rationalizing Medical Relation Prediction

17

Entity Pair

Recall Memory

Associations

Recognition Memory Aspirin

Caffeine

Headache

may_cause CogStage-1 CogStage-2

Assumptions

Migraine Pain Relief Fever

slide-19
SLIDE 19

Overview

Rationalizing Medical Relation Prediction

18

Entity Pair

Recall Memory

Associations

Recognition Memory

Assumptions

Pred.

Aspirin

Caffeine Migraine

Headache

may_cause CogStage-1 CogStage-2 CogStage-3

Migraine Pain Relief Fever

slide-20
SLIDE 20

Overview

Rationalizing Medical Relation Prediction

19

Entity Pair

Recall Memory

Associations

Recognition Memory

Assumptions

Pred.

Aspirin

Caffeine Migraine Pain Relief Fever

Headache

may_cause CogStage-1 CogStage-2 CogStage-3

Rationalized By

slide-21
SLIDE 21

Overview

Rationalizing Medical Relation Prediction

20

Entity Pair

Recall Memory

Associations

Recognition Memory

Assumptions

Pred.

Aspirin

Caffeine Migraine Pain Relief Fever

Headache

may_treat may_treat may_cause CogStage-1 CogStage-2 CogStage-3

Rationalized By

slide-22
SLIDE 22

Model

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

Framework Overview

21

slide-23
SLIDE 23

Model

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

CogStage-1: Global Association Recall

22

slide-24
SLIDE 24

Model

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

CogStage-1: Global Association Recall

23

3 43 4 123 14

Empirical Context Distribution

slide-25
SLIDE 25

Model

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

CogStage-1: Global Association Recall

24 p (ej|ei) = exp ⇣ v0T

ej · vei

⌘ P|V|

k=1 exp

⇣ vT

e0

k · vei

⌘

3 43 4 123 14

Empirical Context Distribution

slide-26
SLIDE 26

Model

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

CogStage-1: Global Association Recall

25 p (ej|ei) = exp ⇣ v0T

ej · vei

⌘ P|V|

k=1 exp

⇣ vT

e0

k · vei

⌘

3 43 4 123 14

Empirical Context Distribution Estimated Context Distribution

slide-27
SLIDE 27

Model

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

CogStage-1: Global Association Recall

26

3 43 4 123 14

p (ej|ei) = exp ⇣ v0T

ej · vei

⌘ P|V|

k=1 exp

⇣ vT

e0

k · vei

⌘ Empirical Context Distribution Estimated Context Distribution

slide-28
SLIDE 28

Model

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

CogStage-2: Assumption Formation and Representation

27

Assumption: Recognize whether context 𝑏!

" and 𝑏# $ hold a relationship 𝑠 %?

slide-29
SLIDE 29

Model

CogStage-2: Assumption Formation and Representation

28

Closed World Assumption (CWA): Open World Assumption (OWA):

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

Assumption: Recognize whether context 𝑏!

" and 𝑏# $ hold a relationship 𝑠 %?

slide-30
SLIDE 30

Model

CogStage-2: Assumption Formation and Representation

29

Closed World Assumption (CWA): Only consider the facts that exist in KBs Open World Assumption (OWA): Consider all possible combinations and select the best ones.

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

Assumption: Recognize whether context 𝑏!

" and 𝑏# $ hold a relationship 𝑠 %?

slide-31
SLIDE 31

Model

CogStage-2: Assumption Formation and Representation

30

…

Head Vector Tail Vector

OWA Assumptions

Relation Embeddings

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

𝑺𝟐 𝑺𝟑 𝑺𝑳

slide-32
SLIDE 32

Model

CogStage-2: Assumption Formation and Representation

31

… …

Head Vector Tail Vector

Relation Embeddings sij

k = f

⇣ ai

h, rk, aj t

⌘ = −

  • vai

h + ξk − vaj t

  • 1

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

…

𝑺𝟐 𝑺𝟑 𝑺𝑳

OWA Assumptions

slide-33
SLIDE 33

Model

CogStage-2: Assumption Formation and Representation

32

+

Head Vector Tail Vector Relation Vector

Relation Embeddings

𝑏!"

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

… … …

𝑺𝟐 𝑺𝟑 𝑺𝑳

OWA Assumptions

slide-34
SLIDE 34

Model

CogStage-2: Assumption Formation and Representation

33

Head Vector Tail Vector Relation Vector

Assumption Representation

Relation Embeddings

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

+ 𝑏!" … … …

𝑺𝟐 𝑺𝟑 𝑺𝑳

OWA Assumptions

slide-35
SLIDE 35

Model

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

CogStage-3:Prediction Decision Making

34

Representations for all assumptions

…

slide-36
SLIDE 36

Model

CogStage-3:Prediction Decision Making

35

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

+ 𝑞!" …

Attention Weights Representations for all assumptions

slide-37
SLIDE 37

Model

CogStage-3:Prediction Decision Making

36

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

+ 𝑞!" …

Attention Weights Representations for all assumptions

slide-38
SLIDE 38

Model

Framework Training

37

Rela. Ped.

83 463 84 123 146 385 130 353 428

GlbalAciaiRecall AiFai&Reeeai DeciiMakig

Corpus-levelStatistics

OWA aiale

ea.ec. headec. aiec.

Ln = − X

(ei,ej)∈V

ˆ p (ej|ei) log (p (ej|ei)) Lr = − X

(h,r,t)∈P

log p(h|t, r) − X

(h,r,t)∈P

log p(t|h, r)

Lp = −

M

X

i=1

  • yi · log
  • p
  • r|ei

h, ei t

  • + (1 − yi) · log
  • 1 − p
  • r|ei

h, ei t

slide-39
SLIDE 39

Experiments

§ Datasets

  • Medical Term-Term Co-occurrence Graph
  • 20 Million Clinical Notes from Stanford Hospital and Clinics since 1995
  • 52,804 Nodes, 16,197,319 Edges

Experimental Setup

38

slide-40
SLIDE 40

Experiments

§ Datasets

  • Medical Term-Term Co-occurrence Graph
  • 20 Million Clinical Notes from Stanford Hospital and Clinics since 1995
  • 52,804 Nodes, 16,197,319 Edges
  • Five Popular Medical Relations

Experimental Setup

39

slide-41
SLIDE 41

Experiments

§ Datasets

  • Medical Term-Term Co-occurrence Graph
  • 20 Million Clinical Notes from Stanford Hospital and Clinics since 1995
  • 52,804 Nodes, 16,197,319 Edges
  • Five Popular Medical Relations

§ Baseline Methods

  • Entity Encoder + Relation Scoring

Experimental Setup

40

Word2vec DeepWalk LINE REPEL-D DistMult RESCAL NTN

slide-42
SLIDE 42

Experiments

Experiment Results: Predictive Performance

41

slide-43
SLIDE 43

Experiments

Experiment Results: Predictive Performance

42

Competitive predictive performance compared with a comprehensive list of baselines

slide-44
SLIDE 44

Experiments

Experiment Results: Predictive Performance

43

Best predictive performance when the training data is large

slide-45
SLIDE 45

Experiments

Experiment Results: Human Evaluation

44

Human Evaluation Interface

slide-46
SLIDE 46

Experiments

Experiment Results: Human Evaluation

45

Human Evaluation Interface

slide-47
SLIDE 47

Experiments

Experiment Results: Human Evaluation

46

Human Evaluation Score

slide-48
SLIDE 48

Experiments

Experiment Results: Human Evaluation

47

Case Study

slide-49
SLIDE 49

Experiments

Experiment Results: Human Evaluation

48

The rationales can help justify the correct prediction.

Case Study See more details in the paper.

slide-50
SLIDE 50

Experiments

§ We propose an interpretable framework to rationalize medical relation prediction based on corpus-level statistics

Conclusions

49

slide-51
SLIDE 51

Experiments

§ We propose an interpretable framework to rationalize medical relation prediction based on corpus-level statistics § Inspired by existing cognitive theories, the reasoning process can be easily understood by users and provides reasonable explanations to justify its prediction

Conclusions

50

slide-52
SLIDE 52

Experiments

§ We propose an interpretable framework to rationalize medical relation prediction based on corpus-level statistics § Inspired by existing cognitive theories, the reasoning process can be easily understood by users and provides reasonable explanations to justify its prediction § We demonstrate the model effectiveness by its predictive performance and human evaluation.

Conclusions

51

slide-53
SLIDE 53

Thank You!

Supported By:

Code available at:

https://github.com/zhenwang9102/X-MedRELA

Zhen Wang The Ohio State University wang.9125@osu.edu

52