Are You Going to Answer That? Measuring User Responses to Anti- - - PowerPoint PPT Presentation

▶
are you going to answer that measuring user responses to
SMART_READER_LITE
LIVE PREVIEW

Are You Going to Answer That? Measuring User Responses to Anti- - - PowerPoint PPT Presentation

Aug 14, 2023 143 likes •542 views

===== Are You Going to Answer That? Measuring User Responses to Anti- Robocall Application Indicators Im Imani N. N. Sher erman , Jasmine D. Bowers, Keith McNamara Jr., Juan Gilbert, Jaime Ruiz, Patrick Traynor 0 Florida institute for

slide-1
SLIDE 1

=====

Are You Going to Answer That? Measuring User Responses to Anti- Robocall Application Indicators

Im Imani N.

  • N. Sher

erman, Jasmine D. Bowers, Keith McNamara Jr., Juan Gilbert, Jaime Ruiz, Patrick Traynor

Florida institute for Cybersecurity Research

slide-2
SLIDE 2

=====

Robocalls Can Be Annoying and Costly

Florida institute for Cybersecurity Research 1

slide-3
SLIDE 3

=====

Robocalls Can Be Annoying and Costly

■ 4.7 billion robocalls, Jan 2020 ■ Scams ■ Tech Support ■ Callback ■ Social Security

Florida institute for Cybersecurity Research 2

Did she say my Social Security number expired?

slide-4
SLIDE 4

=====

How are Robocalls made?

Florida institute for Cybersecurity Research 3

Internet STIR/SHAKEN

slide-5
SLIDE 5

=====

Why look at warning designs?

  • Browsers
  • Influence Decision Making
  • User Independence

4 Florida institute for Cybersecurity Research

slide-6
SLIDE 6

=====

Goal

  • Identify trends
  • Determine user preference
  • Test and evaluate warning designs

Florida institute for Cybersecurity Research 5

This work is NOT only about declining spam calls… …but also about answering legitimate calls.

slide-7
SLIDE 7

=====

Overview Su Survey y Anti-Ro Robocall Applications Purpose: Collect current trends in robocall warning design User Exp xperience Collection Purpose: Understand what users desire in robocall warnings Wa Warning Design User Study Purpose: Show how users respond to currently used and user driven warning designs in best case scenario

6 Florida institute for Cybersecurity Research

slide-8
SLIDE 8

=====

Sur Survey o

  • f An

f Anti-Ro Robocall Ap Application

  • ns

Purpose - Collect current trends in robocall warning design

7

Survey of Anti-Robocall Apps

Florida institute for Cybersecurity Research

slide-9
SLIDE 9

=====

Methodology

  • 10 anti-robocall apps
  • Search term: “Spam call Blocker”
  • Free
  • 4-star rating
  • Not affiliated with a telephone carrier

8

Survey of Anti-robocall Apps

Florida institute for Cybersecurity Research

slide-10
SLIDE 10

=====

Ten Selected Apps for Review

9

Name: Call App (A1) Call Blocker (A2) Call Control (A3) Caller ID & Call Blocker (A4) Clever Dialer (A5) Stars: 4.6 4.6 4.4 4.6 4.6 Installs: 100M+ 10M+ 5M+ 5M+ 1M+ Name: hiya (A6) Mr.Number (A7) Should I Answer? (A8) Truecaller (A9) Who’s Calling (A10) Stars: 4.5 4.2 4.7 4.5 4.4 Installs: 100M+ 10M+ 1M+ 100M+ 10M+

Survey of Anti-Robocall Apps

Florida institute for Cybersecurity Research

100M+ 100M+ 100M+

slide-11
SLIDE 11

=====

Wolgalter’s Design Guidelines

  • Wording
  • Layout & Placement
  • Pictorial Symbols

10

Survey of Anti-Robocall Apps

Florida institute for Cybersecurity Research

  • Mr. Number
slide-12
SLIDE 12

=====

User Experience ce Collect ction

Purpose: Understand what users desire in robocall warnings through focus groups

11

User Experience Collection

Florida institute for Cybersecurity Research

slide-13
SLIDE 13

=====

Methodology

  • Conducted 6, 60-minute focus groups and 3, 60-minute

interviews

  • 18 participants
  • Participants discussed:
  • Robocall detection and response
  • Notification preferences
  • Desired Anti-robocall functionality

12

User Experience Collection

Florida institute for Cybersecurity Research

slide-14
SLIDE 14

=====

Not Notification

  • n Preference
  • Background Color
  • Icons
  • Authenticated Caller ID

13

User Experience Collection

Florida institute for Cybersecurity Research

slide-15
SLIDE 15

=====

Wa Warning Design User Study

Purpose: Show how users respond to currently used and user driven warning designs in best case scenario

14

Warning Design User Study

Florida institute for Cybersecurity Research

slide-16
SLIDE 16

=====

Survey

  • 34 participants
  • Age 20 to 32
  • None in the focus group
  • Survey Contents
  • 5 warning designs
  • 6 phone numbers

15

Warning Design User Study

Florida institute for Cybersecurity Research

slide-17
SLIDE 17

=====

Survey Warning Designs

16

Control

Warning Design User Study

Florida institute for Cybersecurity Research

Avail-CID Avail-Spam Focus-Spam Focus-AID

CALL AUTHENTICATED

slide-18
SLIDE 18

=====

Phone Numbers N1, N2: Two known numbers N3: Unknown number, contact name was random city/state N4: First 9 digits same as the participant’s first 9 digits N5: Same area code as the participant N6: Out of state loan company

17

Warning Design User Study

Florida institute for Cybersecurity Research

slide-19
SLIDE 19

=====

Results

  • Assessed how the following impacted participant Response:
  • Warning Design
  • Phone Number
  • Phone Number + Warning Design
  • Response: the average number of times a participant

answered a call.

18

Warning Design User Study

Florida institute for Cybersecurity Research

slide-20
SLIDE 20

=====

Results Do robocall warnings affect users’ response to incoming calls from unknown numbers? Yes

19

Unknown # Control 35% Focus-Spam 5% Avail-Spam 3%

% of Answered Calls

Warning Design User Study

Florida institute for Cybersecurity Research

Unknown # Control 35% Focus-AID 42% Avail-CID 34%

slide-21
SLIDE 21

=====

Results Do robocall warnings affect users’ response to incoming calls from known numbers? Yes

20

Known # Control 100% Focus-AID 100% Avail-CID 95%

% of Answered Calls

Warning Design User Study

Florida institute for Cybersecurity Research

Known # Control 100% Focus-Spam 65% Avail-Spam 34%

slide-22
SLIDE 22

=====

Results Will the Available and Focus design have significantly different effects on user response? Yes, for known numbers.

21

Known # Unknown # Focus-AID 100% 42% Avail-CID 95% 34% Focus-Spam 65% 5% Avail-Spam 34% 3%

% of Answered Calls

Florida institute for Cybersecurity Research

Warning Design User Study

slide-23
SLIDE 23

=====

So what did we learn?

22 Florida institute for Cybersecurity Research

slide-24
SLIDE 24

=====

Take-Away

  • Users were more likely to answer calls from unknown

numbers accompanied with Authenticated Caller ID.

  • Users were less likely to answer calls from known numbers

accompanied by a spam warning.

  • Warning designs work but are not perfect.

23 Florida institute for Cybersecurity Research

slide-25
SLIDE 25

=====

Thank you!

24 Florida institute for Cybersecurity Research

Keith McNamara Jr.

Patrick Traynor

Jaime Ruiz Juan Gilbert Jasmine D. Bowers Im Imani N.

  • N. Sher

erman shermani@ufl.edu @soulfulsherman

slide-26
SLIDE 26

=====

Cu Current So Solutions

  • Caller ID
  • Black and Whitelisting
  • Chatbots
  • Audio Analysis
  • Call Back Verification
  • Provider based solutions: SHAKEN/STIR
  • End-to-end solution: AuthentiCall
  • Mobile Applications (Caller ID + Black and Whitelisting)

25 Florida institute for Cybersecurity Research

slide-27
SLIDE 27

=====

Robocalls Can Be Annoying and Costly

■ 4.7 billion robocalls, Jan 2020 ■ “Tech Support” ■ One-Ring Scam ■ 50% of calls declined

Florida institute for Cybersecurity Research 26

CBS News, WWMT, West Michigan

slide-28
SLIDE 28

=====

Ten Selected Apps for Review

27

Survey of Anti-robocall Apps

slide-29
SLIDE 29

=====

Robocall Identification Method

  • All apps use blacklist
  • A3 uses its community and

FCC, FTC and IRS complaint data

  • A4 and A9 add customer

contacts to whitelist

28 A1 A6 A2 A5 A4 A10 A9 A7 A8 A3

Survey of Anti-robocall Apps

slide-30
SLIDE 30

=====

Wolgalter’s Warning Design Guidelines Wogalter, Michael S., Vincent C. Conzola, and Tonya L. Smith-

  • Jackson. “Research based guidelines for warning design and

evaluation." Applied ergonomics 33.3 (2002): 219-230.

  • Wording
  • Layout & Placement
  • Pictorial Symbols
  • Auditory Warning
  • Salience (Noticeability)
  • Personal Factors (Demographics)

Florida institute for Cybersecurity Research 29

slide-31
SLIDE 31

=====

Not Notification

  • n Preference
  • Background Color
  • Differ from normal call
  • Orange, Yellow
  • Red –mixed feelings
  • Icons
  • Lock is confusing
  • Emojis unprofessional
  • X-mark and Check-mark
  • Authenticated Caller ID

30

User Experience Collection

Florida institute for Cybersecurity Research

slide-32
SLIDE 32

=====

Stats Explained

  • 34 participants between the age 20 and 32
  • Survey: 5 warnings, 6 phone numbers, 30 combinations shown 6 times

to each participant randomly

  • RM ANOVA for reaction time
  • ANOVA for Response
  • No significant difference over rounds for time or response

31

Warning Design User Study

Florida institute for Cybersecurity Research

slide-33
SLIDE 33

=====

Reaction Time

Florida institute for Cybersecurity Research 32

slide-34
SLIDE 34

=====

Response Time

Florida institute for Cybersecurity Research 33

slide-35
SLIDE 35

=====

Comparison of % of Answered Calls

34

All Known # Unknown # Control 56.40% 100% 35% Focus-AID 61% 100% 42% Focus-Spam 25% 65% 5% Avail-CID 55% 95% 34% Avail-Spam 13% 34% 3%

slide-36
SLIDE 36

=====

% of Answered Calls by Number

35

slide-37
SLIDE 37

=====

Participant Reaction to Designs

Florida institute for Cybersecurity Research 36

slide-38
SLIDE 38

=====

Limitations

  • Participant Number
  • Lab Study
  • Lack of consequences

37

slide-39
SLIDE 39

=====

Goals :

1.How do robocall management applications warn users of robocalls now? 2.How do user handle robocalls? 3.What warning would they like to see? 4.How do users react to current warnings compared to the warnings they want to see?

38