Software Security Economics: Theory, in Practice An Exploratory - - PowerPoint PPT Presentation

▶
software security economics theory in practice
SMART_READER_LITE
LIVE PREVIEW

Software Security Economics: Theory, in Practice An Exploratory - - PowerPoint PPT Presentation

Sep 22, 2023 179 likes •498 views

Software Security Economics: Theory, in Practice An Exploratory Analysis Stephan Neuhaus<neuhaust@tik.ee.ethz.ch> Bernhard Plattner <plattner@tik.ee.ethz.ch> Thursday, June 28, 12 Making the Best Use of Cybersecurity Economic

slide-1
SLIDE 1

Software Security Economics: Theory, in Practice

An Exploratory Analysis

Stephan Neuhaus<neuhaust@tik.ee.ethz.ch> Bernhard Plattner <plattner@tik.ee.ethz.ch>

Thursday, June 28, 12

slide-2
SLIDE 2

“Making the Best Use

  • f Cybersecurity

Economic Models”

  • Investing in software security always has

positive, but diminishing returns

  • Modeled by a “increasing convex function”,

which is “any increasing twice continuously differentiable function”

  • Statement without any qualification

Rachel Rue and Shari Lawrence Pfleeger. Making the best use of cybersecurity economic models. IEEE Security & Privacy, 7:52–60, 2009.

Thursday, June 28, 12

slide-3
SLIDE 3

∆I

Cumulative Investment Security

∆S1 ∆S2

∆S2 < ∆S1

Thursday, June 28, 12

slide-4
SLIDE 4

Security Functions

  • Increasing (Slope always positive): df/dI > 0
  • Diminishing returns (Later slopes smaller

than earlier ones): d2f/dI2 < 0

  • Hang on, that’s not convexity, that’s concavity
  • Not just any old twice continuously differen-

tiable function will do

  • Also, twice continuous differentiability not

needed, twice differentiable suffices

Thursday, June 28, 12

slide-5
SLIDE 5

What they say is not what they mean

Thursday, June 28, 12

slide-6
SLIDE 6

mean Is what they actually true?

Thursday, June 28, 12

slide-7
SLIDE 7

More Security Functions

  • Investment always yields positive returns?
  • Just throw money in the general direction of

security and it will never get any worse?

  • That would mean that it is impossible to choose a

wrong security mechanism

  • Security systems so badly implemented that no

security system would have been better! (TSA)

  • Just a plausibility argument, what does the data say?

Thursday, June 28, 12

slide-8
SLIDE 8

Consequences

  • Vulnerability fixing is security investment
  • Stretched over time
  • If same investment yields less improvement

tomorrow than it does today, then vulnerability fix rates should go down fix rate = #vulns fixed × #fixers unit time

Thursday, June 28, 12

slide-9
SLIDE 9

Data Sets

  • Mozilla (292 vulnerabilities)
  • Apache httpd (66 vulnerabilities)
  • Apache Tomcat (21 vulnerabilities)

Thursday, June 28, 12

slide-10
SLIDE 10

Mozilla

417 Advisories 382 with bug ID 292 with CVS commit message

Image source: Mozilla foundation

Thursday, June 28, 12

slide-11
SLIDE 11

Apache httpd

  • Security information published through CVE
  • No helpful links to bug reports
  • Manual mapping of vulnerabilities to fixes
  • Leaving out CVEs we can’t assign
  • Out of 100 CVEs, 66 remain

Image source: Apache foundation

Thursday, June 28, 12

slide-12
SLIDE 12

Apache Tomcat

  • From 2008 on, reports have revision

number of fixing checkin :-)

  • Before 2008, there is no link at all :-(
  • Out of the 89 CVEs, only 21 remain
  • Attribution absolutely certain

Image source: Apache foundation

Thursday, June 28, 12

slide-13
SLIDE 13

Vulnerability Fix Checkins (Mozilla)

Checkins

5 10 15 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

Thursday, June 28, 12

slide-14
SLIDE 14

Moving Average

  • Acts as a low-pass filter, removing high-

frequency jiggling

  • Smoothes out strong day-to-day variations
  • Leaves overall trend (if any)
  • Trend will appear with a lag
  • We chose window size 365
  • Even in leap years

Thursday, June 28, 12

slide-15
SLIDE 15

Average Vulnerability Fix Checkins (Combined)

Average Checkins per Day

0.003 0.01 0.03 0.1 0.3 1 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Product Mozilla Httpd Tomcat

Thursday, June 28, 12

slide-16
SLIDE 16

Average Vulnerability Fix Checkins (Combined)

Average Checkins per Day

0.003 0.01 0.03 0.1 0.3 1 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Product Mozilla Httpd Tomcat

Move to different repository, way fewer checkins, higher percentage

  • f backported vuln fixes

Thursday, June 28, 12

slide-17
SLIDE 17

Average Vulnerability Fix Checkins (Combined)

Average Checkins per Day

0.003 0.01 0.03 0.1 0.3 1 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Product Mozilla Httpd Tomcat

Thursday, June 28, 12

slide-18
SLIDE 18

Average Vulnerability Fix Checkins (Combined)

Average Checkins per Day

0.003 0.01 0.03 0.1 0.3 1 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Product Mozilla Httpd Tomcat

Thursday, June 28, 12

slide-19
SLIDE 19

Average Vulnerability Fix Checkins (Combined)

Average Checkins per Day

0.003 0.01 0.03 0.1 0.3 1 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Product Mozilla Httpd Tomcat

Thursday, June 28, 12

slide-20
SLIDE 20

Number of Committers (Combined)

Average Checkins per Day

1 3 10 30 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Product Mozilla Httpd Tomcat

Thursday, June 28, 12

slide-21
SLIDE 21

Why does the Standard Model not Fit?

  • Standard model is for static situation
  • Software development is dynamic, has phases
  • Next week is a feature freeze
  • In two months, writing that new feature
  • Supply of easy vulnerabilities has not run out
  • Why not? It’s superficially plausible that they should!

Thursday, June 28, 12

slide-22
SLIDE 22

Reservoirs

time Development Bug fixing average length μin average length μout

Thursday, June 28, 12

slide-23
SLIDE 23

Reservoirs

  • There is a reservoir of vulnerabilities
  • Phases of average length μin in which vulnerabilities

are put into the reservoir (development)

  • Phases of average length μout in which vulnerabilities

are taken out of the reservoir (bug fixing)

  • These phases alternate
  • Not perfect, but more plausible than standard model!

Thursday, June 28, 12

slide-24
SLIDE 24

Consequences

  • “The number of [vulnerabilities] V will be

heavy tailed with P(V > x) ∼ cx−(α−1)L(x), where c and α are constants and L a slowly varying function.”

  • Number of vulnerabilities unknown!

Thursday, June 28, 12

slide-25
SLIDE 25

More Consequences

  • Assume that in any given 365-day period, a

constant fraction of the available vulnerabilities will be fixed, it follows that the number of days with a given number of checkins will also be heavy tailed

  • #vulnerabilities fixed#vulnerabilities present
  • Not a priori clear, but let’s see what follows!

Thursday, June 28, 12

slide-26
SLIDE 26

Checkins per Day Days

100 101 102 103

  • 1

2 3 4 5 6 7 8 9 10 11 13 15 1 2 3 5 1 2 Product

  • Httpd

Mozilla Tomcat

Thursday, June 28, 12

slide-27
SLIDE 27

Other Things in the Paper

  • Is software security an arms race, where

attackers and defenders have to work very hard just to maintain the status quo?

  • Would lead to distribution between successive

days with fixes obeying a power law

  • No (or, if yes, lost in noise of other activity)

Neil Johnson, Spencer Carran, Joel Botner, Kyle Fontaine, Nathan Laxague, Philip Nuetzel, Jessica Turnley, and Brian Tivnan. Pattern in escalations in insurgent and terrorist

  • activity. Science, 333(6038):81–84, July 2011.

Thursday, June 28, 12

slide-28
SLIDE 28

You Should not Believe Me!

  • Make your own experiments on my data!
  • Find bugs in my scripts!
  • ftp://ftp.tik.ee.ethz.ch/pub/publications/

WEIS2012/fixrates.tar.gz

  • (Don’t write this down, it’s in the paper)

Thursday, June 28, 12

slide-29
SLIDE 29

Personal Remarks

  • Share your data and scripts!
  • Many thanks to reviewer who pointed out

problems with my “power law” analysis

  • This work is not about surprises
  • A scientific paper is not a news story

Thursday, June 28, 12

slide-30
SLIDE 30

Conclusion

  • Standard model not applicable for software

development

  • Hence perhaps not so standard
  • Reservoir model makes predictions that

actually fit the data

  • Hence perhaps a better model
  • Ultimately, phenomenon not understood

Thursday, June 28, 12