Conversations Gone Awry Detecting Early Signs of Conversational - - PowerPoint PPT Presentation

▶
conversations gone awry
SMART_READER_LITE
LIVE PREVIEW

Conversations Gone Awry Detecting Early Signs of Conversational - - PowerPoint PPT Presentation

Jan 03, 2024 48 likes •802 views

Conversations Gone Awry Detecting Early Signs of Conversational Failure Justine Zhang, Jonathan P. Chang , Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Dario Taraborelli, and Nithum Thain To be presented at ACL 2018 (July 15-20,

slide-1
SLIDE 1

Conversations Gone Awry

Detecting Early Signs of Conversational Failure

Justine Zhang, Jonathan P. Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Dario Taraborelli, and Nithum Thain To be presented at ACL 2018 (July 15-20, Melbourne, Australia)

Paper, code, and data available at http://www.cs.cornell.edu/~cristian/Conversations_gone_awry.html

slide-2
SLIDE 2

Motivation

1999: “The Internet is becoming the town square for the global village of tomorrow” - Bill Gates

slide-3
SLIDE 3

Motivation

1999: “The Internet is becoming the town square for the global village of tomorrow” - Bill Gates Present Day:

slide-4
SLIDE 4

Motivation

1999: “The Internet is becoming the town square for the global village of tomorrow” - Bill Gates Present Day: What makes civil conversations turn awry?

slide-5
SLIDE 5

Conversations Going Awry: An Example

Conversation A

slide-6
SLIDE 6

Conversations Going Awry: An Example

Conversation A Conversation B

slide-7
SLIDE 7

Conversations Going Awry: An Example

Conversation A Conversation B Which one leads to: “Wow, you’re coming off as a total d**k...what the hell is wrong with you?”

slide-8
SLIDE 8

Conversations Going Awry: An Example

Conversation A Conversation B Which one leads to: “Wow, you’re coming off as a total d**k...what the hell is wrong with you?” More examples (quiz): http://awry.infosci.cornell.edu/

slide-9
SLIDE 9

Capturing Human Intuition

We seem to have some intuition for when things are going bad

  • Human accuracy is 72% - more on this later

We would like to reconstruct some of this intuition

  • Contrast with prior work: predict toxicity rather than detecting it after the

fact (Cheng et al., 2017; Wulczyn et al., 2017) Two high level challenges: 1. Find cases of conversations “going awry” 2. Encode intuitive signs in some concrete way

slide-10
SLIDE 10

Pitfalls to Avoid

Confounding toxicity with disagreement

  • Civil disagreement is healthy! (Coser, 1956; De Dreu and Weingart, 2003)

Getting too topic-specific

  • Political conversations are more likely to turn toxic ‒ but this doesn’t tell

us anything about the nature of conversation

  • Definitely don’t want to end up only flagging sensitive topics!
slide-11
SLIDE 11

Finding Conversations Gone Awry

slide-12
SLIDE 12

What Are We Looking For?

slide-13
SLIDE 13

What Are We Looking For?

Conversation

slide-14
SLIDE 14

Civil Start

What Are We Looking For?

Conversation

slide-15
SLIDE 15

Civil Start

What Are We Looking For?

Conversation

. . .

slide-16
SLIDE 16

Toxic End Civil Start

What Are We Looking For?

Conversation

. . .

slide-17
SLIDE 17

Toxic End Civil Start

What Are We Looking For?

Conversation

. . .

2 or more civil comments by different users

slide-18
SLIDE 18

Toxic End Civil Start

What Are We Looking For?

Conversation

. . .

Personal attack from within (Arazy et al, 2013) 2 or more civil comments by different users

slide-19
SLIDE 19

What Are We Looking For?

Conversation

. . .

~ 50 million conversations

Raw data

slide-20
SLIDE 20

What Are We Looking For?

Conversation

. . .

~ 50 million conversations

Raw data

~3,000 toxic candidates

Automated pre-filtering

slide-21
SLIDE 21

What Are We Looking For?

Talk Page

Conversation

. . .

~ 50 million conversations

Raw data

~3,000 toxic candidates

Automated pre-filtering

slide-22
SLIDE 22

What Are We Looking For?

Talk Page

Conversation

. . .

Conversation

. . .

~ 50 million conversations

Raw data

~3,000 toxic candidates

Automated pre-filtering

slide-23
SLIDE 23

What Are We Looking For?

Talk Page

Conversation

. . .

Conversation

. . .

~ 50 million conversations

Raw data

~3,000 toxic candidates

Automated pre-filtering

635 pairs

Human-validated set

slide-24
SLIDE 24

Recovering Human Intuition

slide-25
SLIDE 25

Back to our example...

Conversation A Conversation B

slide-26
SLIDE 26

Back to our example...

Conversation A Conversation B How did we decide?

slide-27
SLIDE 27

Back to our example...

Conversation A Conversation B

slide-28
SLIDE 28

Back to our example...

Conversation A Conversation B

slide-29
SLIDE 29

Back to our example...

Conversation A Conversation B Direct questioning

slide-30
SLIDE 30

Back to our example...

Conversation A Conversation B Direct questioning

slide-31
SLIDE 31

Back to our example...

Conversation A Conversation B Direct questioning

slide-32
SLIDE 32

Back to our example...

Conversation A Conversation B Direct questioning Hedging

slide-33
SLIDE 33

Back to our example...

Conversation A Conversation B Direct questioning Hedging

Politeness strategies

(Brown and Levinson, 1987)

slide-34
SLIDE 34

The Role of Politeness

Theory suggests role of politeness in determining conversation trajectory

  • Fraser, 1980: Politeness softens the perceived force of a message
  • Brown and Levinson, 1987: Politeness acts as a buffer between speakers’

conflicting goals

  • Goffman, 1955: Politeness is a face-saving tool

But, little empirical investigation so far

slide-35
SLIDE 35

Measuring Politeness

How can we detect uses of politeness strategies?

slide-36
SLIDE 36

Measuring Politeness

How can we detect uses of politeness strategies? Danescu-Niculescu-Mizil et al., 2013: pattern match on parsed sentences

  • Think regular expressions, but at level of sentence structure
  • Try it out: http://politeness.cornell.edu/

I [think|feel|believe] that ...

slide-37
SLIDE 37

Beyond Politeness: Other Rhetorical Devices

Politeness is a promising feature ‒ but it’s very general How do we account for domain-specific behavior patterns?

slide-38
SLIDE 38

The Example, Once Again

Conversation A Conversation B

slide-39
SLIDE 39

The Example, Once Again

Conversation A Conversation B

slide-40
SLIDE 40

The Example, Once Again

Conversation A Conversation B

“Plan (to)...”, “like (to)...”, “help…”, etc. - coordination

slide-41
SLIDE 41

Conversational Prompt Types

A “template” used to initiate conversations

slide-42
SLIDE 42

Conversational Prompt Types

A “template” used to initiate conversations Want to discover these automatically - no supervision

slide-43
SLIDE 43

Conversational Prompt Types

A “template” used to initiate conversations Want to discover these automatically - no supervision Solution: extend methodology for finding question types (Zhang et al., 2017)

  • Original intuition: similar questions trigger similar answers
  • Our extension: similar prompts trigger similar replies
slide-44
SLIDE 44

Conversational Prompt Types on Wikipedia

Prompt Type (names manually assigned) Example Factual Check The census is not talking about families here. Moderation He’s accused me of being a troll. Coordination I could do with your help. Casual Remark What’s with this flag image? Action Statement The page was deleted as self-promotion. Opinion I think it should be the other way around.

slide-45
SLIDE 45

Analysis

slide-46
SLIDE 46

Question of Interest

How well do the prompt types and politeness strategies features actually capture human intuition? Two ways to answer this question: 1. See if any features are significantly more likely to show up in awry-turning conversations 2. Use the features to create a machine learning classifier that plays the “guessing game” (like the example) and compare to human performance

slide-47
SLIDE 47

Feature Comparisons (First Comment Only)

slide-48
SLIDE 48

Feature Comparisons (First Comment Only)

More likely to turn awry

slide-49
SLIDE 49

Feature Comparisons (First Comment Only)

More likely to turn awry

slide-50
SLIDE 50

Feature Comparisons (First Comment Only)

More likely to turn awry

The census is not talking about families here.

slide-51
SLIDE 51

Feature Comparisons (First Comment Only)

More likely to turn awry

slide-52
SLIDE 52

Feature Comparisons (First Comment Only)

More likely to turn awry

slide-53
SLIDE 53

Feature Comparisons (First Comment Only)

More likely to turn awry

I think it should be the

  • ther way around.
slide-54
SLIDE 54

Feature Comparisons (First Comment + Reply)

More likely to turn awry

slide-55
SLIDE 55

“Guessing Game” Performance

slide-56
SLIDE 56

“Guessing Game” Performance

50% 100%

Accuracy

slide-57
SLIDE 57

“Guessing Game” Performance

50% 100%

Accuracy

Random Guessing

slide-58
SLIDE 58

“Guessing Game” Performance

50% 100%

Accuracy

72% Humans Random Guessing

slide-59
SLIDE 59

“Guessing Game” Performance

50% 100%

Accuracy

72% Humans 57% Random Guessing Bag of Words

slide-60
SLIDE 60

“Guessing Game” Performance

50% 100%

Accuracy

72% Humans 65% Our System Random Guessing 57% Bag of Words

slide-61
SLIDE 61

“Guessing Game” Performance

50% 100%

Accuracy

72% Humans 65% Our System Filling the gap? 57% Random Guessing Bag of Words

slide-62
SLIDE 62

Future Work: Closing the Gap

What parts of human intuition are missing from model? How do we find out? Idea: examine cases that humans get right, but model gets wrong

  • Model correctly guesses 80% of cases humans got right - what about the
  • ther 20%?
slide-63
SLIDE 63

Future Work: Beyond Conversation Starters

Currently limited to looking only at start of conversation

  • Ideal model would pick up signal from anywhere in conversation
  • Can imagine conversations escalating over time - want to model this
slide-64
SLIDE 64

Future Work: Overcoming Biases

What are sources of bias in the current model?

slide-65
SLIDE 65

Future Work: Overcoming Biases

What are sources of bias in the current model? ~ 50 million conversations

Raw data

~3,000 toxic candidates

Automated pre-filtering

635 pairs

Human-validated set

slide-66
SLIDE 66

Future Work: Overcoming Biases

What are sources of bias in the current model? ~ 50 million conversations

Raw data

~3,000 toxic candidates

Automated pre-filtering

635 pairs

Human-validated set

Pre-filtering bias: inherit biases of ML model used for pre-filtering

slide-67
SLIDE 67

Future Work: Overcoming Biases

What are sources of bias in the current model? ~ 50 million conversations

Raw data

~3,000 toxic candidates

Automated pre-filtering

635 pairs

Human-validated set

Labeling bias: crowdsourcing inherently captures biases of human annotators

slide-68
SLIDE 68

Future Work: Overcoming Biases

What are sources of bias in the current model? ~ 50 million conversations

Raw data

~3,000 toxic candidates

Automated pre-filtering

635 pairs

Human-validated set

Data source bias: model currently trained only on English Wikipedia

slide-69
SLIDE 69

Future Work: Overcoming Biases

What are sources of bias in the current model? What can we do about it?

  • Current direction: explore other ways of pre-filtering and/or labeling

~ 50 million conversations

Raw data

~3,000 toxic candidates

Automated pre-filtering

635 pairs

Human-validated set

slide-70
SLIDE 70

Future Work: Conversation Recovery

Conversation

. . .

slide-71
SLIDE 71

Future Work: Conversation Recovery

Conversation

. . . . . .

slide-72
SLIDE 72

Future Work: Conversation Recovery

Conversation

. . . . . .

What makes this happen?

slide-73
SLIDE 73

Conclusions

Forecasting future attacks in conversations is feasible Politeness strategies and prompt types capture some human intuition Experimental verification of politeness theories

slide-74
SLIDE 74

Acknowledgements

Everyone who worked on the Wikipedia conversation reconstruction project The Wikimedia Foundation anti-harassment program Crowdflower workers who annotated our data The volunteers who provided annotations for human performance estimate

slide-75
SLIDE 75

Questions?

More information at: http://www.cs.cornell.edu/~cristian/Conversations_gone_awry.html Data and code: http://convokit.infosci.cornell.edu Online guessing game: http://awry.infosci.cornell.edu/