Are Information Security professionals rational decision-makers? . - - PowerPoint PPT Presentation

Motivation Approach Design Findings Conclusions Future Work Thank you

Are Information Security professionals rational decision-makers?

. Konstantinos Mersinas Distance Learning Weekend Conference 2015 Royal Holloway, University of London . . contact: Konstantinos.Mersinas.2011@live.rhul.ac.uk

12-13 September 2015

Mersinas, K., Hartig, B., Martin, K. M., & Seltzer, A., Experimental Elicitation of Risk Behaviour amongst Information Security Professionals. Workshop on the Economics of Information Security (WEIS) 2015.

Are Information Security professionals rational decision-makers? – K. Mersinas 1/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Motivation

1 Motivation 2 Approach 3 Design 4 Findings 5 Conclusions 6 Future Work 7 Thank you

Are Information Security professionals rational decision-makers? – K. Mersinas 2/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Motivation

Information security professionals have to assess risk in order to make investment decisions on security measures Unbiased? Rationally? All quantitative risk assessment methodologies are subject to three significant limitations: Are based on many approximations These approximations are often biased by perception of risk Involved calculations can be easily manipulated

Are Information Security professionals rational decision-makers? – K. Mersinas 3/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Simplified Research Question

Are Information Security professionals rational decision-makers? – K. Mersinas 4/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Research Questions

Four Hypotheses: 1) Risk and Ambiguity Aversion 2) Worst-case thinking 3) Other-evaluation ambiguity aversion 4) Security vs Operability

Are Information Security professionals rational decision-makers? – K. Mersinas 5/29

Motivation Approach Design Findings Conclusions Future Work Thank you

We show that..

We show that: Security professionals exhibit distinct decision-making traits under risk and ambiguity Risk attitudes differ between professionals and the general population Professionals are not rational decision-makers Information security has distinctive aspects

Are Information Security professionals rational decision-makers? – K. Mersinas 6/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Approach

1 Motivation 2 Approach 3 Design 4 Findings 5 Conclusions 6 Future Work 7 Thank you

Are Information Security professionals rational decision-makers? – K. Mersinas 7/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Approach

Information Security context characteristics: Loss domain Evaluation by other parties Security and operability Background: Behavioural Economics Risk attitudes elicitation Survey

Are Information Security professionals rational decision-makers? – K. Mersinas 8/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Design

1 Motivation 2 Approach 3 Design 4 Findings 5 Conclusions 6 Future Work 7 Thank you

Are Information Security professionals rational decision-makers? – K. Mersinas 9/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Experiment & Survey

Online Performance-based payment Participants: 55 Professionals and 58 ‘Students’ Pool: distance learning MSc in Information Security (thank you!) and the Economics Lab of RHUL Please, join our next experiment!

Are Information Security professionals rational decision-makers? – K. Mersinas 10/29

Motivation Approach Design Findings Conclusions Future Work Thank you

WTP Lotteries

What is the maximum amount that you are willing to pay in order to avoid playing a lottery in which there is: .. a p% probability of losing $50 and losing nothing otherwise? .. a probability between p1% and p2% of losing $50? .. a p% probability of losing an amount between $20 and $80 and losing nothing otherwise? .. a probability between p1% and p2% of losing an amount between $20 and $80 and losing nothing otherwise?

Are Information Security professionals rational decision-makers? – K. Mersinas 11/29

Motivation Approach Design Findings Conclusions Future Work Thank you

WTP & Comparison Lotteries

How much are you willing to pay in order to avoid playing a lottery in which there is: a probability of 85% of losing $50 a probability of 8% of losing $170 a probability of 3.5% of losing $300 a probability of 2.5% of losing $400 a probability of 1% of losing $1000

Are Information Security professionals rational decision-makers? – K. Mersinas 12/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Other-evaluation Ambiguity Aversion

“Important note: Your choices and their corresponding possible outcomes in the following experiment will be further viewed and will go through an additional evaluation process, after the completion of the experiment.” Finding: There is no evidence that subjects change their risk behaviour when they are informed that they will be evaluated by other parties

Are Information Security professionals rational decision-makers? – K. Mersinas 13/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Security vs Operability

Scenario 1

Mechanism A Mechanism B Enhances Security of the system by 10% Enhances Operability of the system by 10%

Scenario 2

Choice A Mechanism B Choice C Remains at the cur- rent system state Reduces Security by x% Enhances Operability by 10% Indifferent be- tween A and B

Are Information Security professionals rational decision-makers? – K. Mersinas 14/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Findings

1 Motivation 2 Approach 3 Design 4 Findings 5 Conclusions 6 Future Work 7 Thank you

Are Information Security professionals rational decision-makers? – K. Mersinas 15/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Risk Aversion

Finding: Both professionals and students are risk averse for small probability losses, but become risk seeking for very likely losses

Mean Risk Averse (positive) and Risk Taking (negative) WTP of Students and Professionals per lottery. Bars represent (µ(WTP) − ExpectedValue).

Are Information Security professionals rational decision-makers? – K. Mersinas 16/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Ambiguity Aversion

Finding: Professionals reveal ambiguity aversion in all of their choices; such aversion is not consistently observed for the general population EV=-2.5 EV=-7.5 EV=-25 Students Professionals

Are Information Security professionals rational decision-makers? – K. Mersinas 17/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Expected Values

Finding: Professionals are better at estimating expected losses than the general population (WTP and Survey question)

Interaction of Pro or Student and variable H19 with General Risk as moderator

Are Information Security professionals rational decision-makers? – K. Mersinas 18/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Worst-case thinking

Salience Theory: disproportional focus on the most salient

• utcomes and quantification of decision weights

Finding: The majority of professionals have a distorted perception of probabilities; the general population reveals

• verall more consistent preferences than security professionals

0.2 0.4 0.6 0.8 1.0∆ 4 3 2 1 1 2 Sum 9

Distortion of probability perception for lotteries L9, L10: values of salience sum for L9 ≻ L10, δ ∈ (0, 1] (Students:47%, Professionals:58%)

Are Information Security professionals rational decision-makers? – K. Mersinas 19/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Security-Operability across Job Roles

Finding: Operability and Security preferences are significantly dependent on job role

Job Title / Role Senior executive Managerial IT & Security Compliance, Risk Other Enhance 5 3 7 8 Security (42%) Enhance 1 13 7 3 2 Operability (58%) χ2(4, N = 55) = 12.092, p = .017

different perspective based on job position

Are Information Security professionals rational decision-makers? – K. Mersinas 20/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Security and Operability Switching Points

Finding: Both groups weighted their favourite attribute twice as much as the attribute they did not choose

SWITCHPOINT_SEC 10.00 8.00 6.00 4.00 2.00 .00 Frequency 5 4 3 2 1 Mean = 5.26

• Std. Dev. =

3.018 N = 23

Security: (Sec(x%), Ops(10%))

SWITCHPOINT_OPS 10.00 8.00 6.00 4.00 2.00 .00 Frequency 5 4 3 2 1 Mean = 5.35

• Std. Dev. =

2.799 N = 26

Operability: (Sec(10%), Ops(x%))

Are Information Security professionals rational decision-makers? – K. Mersinas 21/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Security and Operability Loss Aversion

Finding: Professionals who prefer operability have a more balanced perception of the two attributes; relative loss aversion was more profound for security

LOSS_AV_SEC 10.00 8.00 6.00 4.00 2.00 .00 Frequency 12 10 8 6 4 2 Mean = 2.15

• Std. Dev. =

2.323 N = 20 LOSS_AV_OPS 10.00 8.00 6.00 4.00 2.00 .00 Frequency 12 10 8 6 4 2 Mean = 1.83

• Std. Dev. =

2.221 N = 29

Are Information Security professionals rational decision-makers? – K. Mersinas 22/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Procedural Variance

Finding: Security professionals exhibit preference inconsistencies between willingness-to-pay and choice tasks, similarly to the general population Lottery comparisons and willingness-to-pay inconsistencies Comparison Preference % of subjects that chose Li over Lj Li ≻ Lj and revealed choice inconsistency Students Professionals L9 vs L10 L9 ≻ L10 33% 47% L10 ≻ L9 58% 52% L10 vs L11 L10 ≻ L11 57% 43% L11 ≻ L10 13% 32%

Are Information Security professionals rational decision-makers? – K. Mersinas 23/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Conclusions

1 Motivation 2 Approach 3 Design 4 Findings 5 Conclusions 6 Future Work 7 Thank you

Are Information Security professionals rational decision-makers? – K. Mersinas 24/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Conclusions

Professionals react more to ambiguity than the general population Professionals are better at estimating expected losses Professionals shift attitude from risk averse to risk seeking when losses become more probable (FFP) Professionals have a distorted perception of probabilities Both samples reveal inconsistencies

Are Information Security professionals rational decision-makers? – K. Mersinas 25/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Future Work

1 Motivation 2 Approach 3 Design 4 Findings 5 Conclusions 6 Future Work 7 Thank you

Are Information Security professionals rational decision-makers? – K. Mersinas 26/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Future Work (more irrationality?)

Risk assessment methodologies (ISO 27005) Real-world decision data Perception of information security threats Preferences on risk treatment actions

Are Information Security professionals rational decision-makers? – K. Mersinas 27/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Thank you

1 Motivation 2 Approach 3 Design 4 Findings 5 Conclusions 6 Future Work 7 Thank you

Are Information Security professionals rational decision-makers? – K. Mersinas 28/29

Motivation Approach Design Findings Conclusions Future Work Thank you

Thank you

More irrational behaviour!

Are Information Security professionals rational decision-makers? – K. Mersinas 29/29