LECTURE 28: METRICS! CSE 442 Software Engineering Putting the E - - PowerPoint PPT Presentation

LECTURE 28: METRICS!

CSE 442 – Software Engineering

¨ Have mostly focused on creating software

Putting the E in SE

¨ Have mostly focused on creating software

¤ Team Dynamics

Putting the E in SE

¨ Have mostly focused on creating software

¤ Team Dynamics ¤ Management

Putting the E in SE

¨ Have mostly focused on creating software

¤ Team Dynamics ¤ Management ¤ Software Lifecycle

Putting the E in SE

¨ Have mostly focused on creating software

¤ Team Dynamics ¤ Management ¤ Software Lifecycle ¤ Requirements

Putting the E in SE

You want what?

¨ Have mostly focused on creating software

¤ Team Dynamics ¤ Management ¤ Software Lifecycle ¤ Requirements ¤ Design

Putting the E in SE

You want what?

¨ Have mostly focused on creating software

¤ Team Dynamics ¤ Management ¤ Software Lifecycle ¤ Requirements ¤ Design ¤ Testing

Putting the E in SE

¨ Have mostly focused on creating software

¤ Team Dynamics ¤ Management ¤ Software Lifecycle ¤ Requirements ¤ Design ¤ Testing ¤ Implementation

Putting the E in SE

¨ Demands attention to details, rules, precision…

Engineering Requirements

¨ Demands attention to details, rules, precision…

… and numbers

Engineering Requirements

¨ Demands attention to details, rules, precision…

… and numbers

¨ Results used to answer many types of questions:

¤ How much … ? ¤ How often … ? ¤ How many… ? ¤ Why?

Engineering Requirements

¨ Provide data which drives process of engineering

¤ Allow us to make changes and evaluate results ¤ Apply scientific process to improve process & results

Metrics

¨ Stop making all our decisions using intuition

Why We Use Metrics

¨ Stop making all our decisions using intuition

to process relying on facts & evidence

Why We Use Metrics

¨ Need solid basis by which number generated ¨ Number should accurately reflect reality

Numeric ≠ Useful

Metric Problems

There are 3 kinds of lies:

• 1. Lies
• 2. Damned Lies
• 3. Statistics

¨ Misused metrics no better than random guess

¤ Much, much worse in most situations ¤ People’s faith & trust replaced by doubt

¨ Correlation is not causation – cannot forget this

Metric Problems

¨ Fact: People with larger shoe sizes better spellers

Statistics Abuse

¨ Fact: People with larger shoe sizes better spellers

Statistics Abuse

¨ Fact: People with larger shoe sizes better spellers

Statistics Abuse

Adults good at spelling; Also wear grown-up shoes

¨ Fact: People with larger shoe sizes better spellers

Statistics Abuse

Adults good at spelling; Also wear grown-up shoes Kids bad at spelling; Small feet use kids' sizes

Why He Deserves a Raise

I wrote more code last year than any other employee

And The Boss Replies…

And The Boss Replies…

Measuring programming progress by lines of code is like measuring aircraft building progress by weight

¨ Of these lines, which each equal to 1 LoC?

/* Some of these lines are really ugly */ int countMatches(E[] arr, E ignore) { int sum = (2 * 4) – (int)Math.sqrt(64); for (int i = 0; i < arr.length; i++) { E val = arr[i]; if (val.equals(ignore)) ; else { sum++; } } return sum; }

Lines Of Code

What Are You Measuring?

¨ Are you sure?

What Are You Measuring?

¨ Cyclomatic complexity quick & easy to measure

¤ Equal to number of simple decisions + 1 ¤ || and && count also simple decisions

¨ Complexity is 4 for this example

¤ Written as V(G) = 4

¨ Provides deeper insight

¤ Counts different ways to screw up 2 1

Good Metric Example

3

Good Metric Example

¨ Cyclomatic complexity quick & easy to measure

¤ Equal to number of simple decisions + 1 ¤ || and && count also simple decisions

¨ Complexity is 4 for this example

¤ Written as V(G) = 4

¨ Provides deeper insight

¤ Counts different ways to screw up 2 1 3

Good Metric Example

¨ Cyclomatic complexity quick & easy to measure

¤ Equal to number of simple decisions + 1 ¤ || and && count also simple decisions

¨ Complexity is 4 for this example

¤ Written as V(G) = 4

¨ Provides deeper insight

¤ Counts different ways to screw up 2 1 3

Good Metric Example

¨ Cyclomatic complexity quick & easy to measure

¤ Equal to number of simple decisions + 1 ¤ || and && count also simple decisions

¨ Complexity is 4 for this example

¤ Written as V(G) = 4

¨ Provides deeper insight

¤ Counts different ways to screw up 2 1 3

What Makes it Good?

¨ Studies found V(G) correlated to error probability

¤ Opportunities for bugs much, much larger

¨ Inside complex methods find far more errors

V(G) # of Modules

20% of modules, but… 80% of bugs

Is It Enough?

¨ Probability of bugs jumps once V(G) gets too large

¤ Often suggest rewrite methods w/ complexity > 7 ¤ V(G) > 9 demand rewrite or split into pieces

¨ Possible strategy: test complex methods only?

Ego Check

¨ Who is the best programmer in the class?

Ego Check

¨ Who is the best programmer in the class?

¤ How can you prove it?

Real-World Questions

¨ Have 11 employees; but can only promote 3

Real-World Questions

¨ Have 11 employees; but can only promote 3

¤ Whom do you promote?

Real-World Questions

¨ Have 11 employees; but can only promote 3

¤ Whom do you promote? ¤ How will you explain your choice in court?

Real-World Questions

¨ Changed the application lifecycle used

Real-World Questions

¨ Changed the application lifecycle used

¤ 1st project sucked: client was an

Real-World Questions

¨ Changed the application lifecycle used

¤ 1st project sucked: client was an ¤ How do you justify your decision to your boss?

Singular of Data: Anecdote

¨ Single project cannot provide usable metrics

¤ Too many variables: must isolate cause-and-effect

¨ Results should converge given enough data

¤ If no convergence, results may be random ¤ May take time before convergence occurs

Singular of Data: Anecdote

¨ Important to track metrics for large data set

¤ Many coders on single project ¤ Track group over long period of time

¨ Be careful evaluating metrics

¤ Limit evaluation to trends ¤ Smooth results by using moving averages

Measurement Principles

¨ Define objectives first, collect data second

¤ Will always find patterns in data ¤ Simple coincidences should not be relied upon

Measurement Principles

¨ Select metrics using theory valid for domain

¤ Analyze designs with design concepts not coding ¤ Verify positive results would be found

Measurement Principles

¨ Tailor uses & analysis to answer your question

¤ If metric serves no purpose, do not collect that data ¤ Generate results you care about & not any others ¤ Check & double-check this does not bias results

Once Analysis Is Complete

¨ Interpretation ¤ Look for insights about process ¤ Check how well/if metric represents the results ¤ Secondary effects of the process should be evaluated

¨ Answer the question: What do these results mean?

Once Analysis Is Complete

¨ Interpretation ¤ Look for insights about process ¤ Check how well/if metric represents the results ¤ Secondary effects of the process should be evaluated

¨ Answer the question: What do these results mean?

Most Important Step

¨ Feedback ¤ Recommend changes based upon the results ¤ Clarify confusing results by proposing additional experiments ¤ Verify work as good as possible and collect the big money

Most Important Step

¨ Feedback ¤ Recommend changes based upon the results ¤ Clarify confusing results by proposing additional experiments ¤ Verify work as good as possible and collect the big money

Most Important Step

¨ Feedback ¤ Recommend changes based upon the results ¤ Clarify confusing results by proposing additional experiments ¤ Verify work as good as possible and collect the big money

How Much To Collect?

X

Class-Oriented Metrics

¨ Size of each class ¨ Average method size ¨ Method complexity ¨ Weighted methods per class ¨ Average number of parameters per method

Class-Oriented Metrics

¨ Number of subclasses ¨ Depth of the inheritance tree ¨ Number of overridden operations ¨ Number of added operations in subclass ¨ Specialization index