Testing/Debugging CS 1111 OCTOBER 21, 2019 Warm-up problem Write a - - PowerPoint PPT Presentation

Testing/Debugging

CS 1111 – OCTOBER 21, 2019

Warm-up problem

 Write a function that can take in a nested list of ints and return the

sum of all numbers in the nested list.

 Example:

[[1, 2, 3], [4, 5], [6], [7, 8, 9 10]] -> 55

Upcoming deadlines

 PA due dates

 PA 12: This morning (now “late”)  PA 13: Due Friday  PA 14: Due next Monday  PA 15: Due next Wednesday

 Exam 2: November 6  Lab this Wednesday in class

Consider the max function to the right

 Is there anything wrong with this code?

Consider the max function to the right

 Is there anything wrong with this code?  Yes!!!!

What about this max function?

What about this max function?

 This one works fine!

The point:

 It’s impossible to, at a glance, debug most code  With debugging, we are trying to fix logical errors

 This is, find the problem with our logic

 What strategy can we use to help us find bugs to fix them?

 TESTING

Testing

 Testing is the process of searching for bugs

 We test to FIND bugs, NOT to prove our software is bug free:

 Design tests as the following

 Input  Expected output

 What SHOULD return

 Actual output

 What DOES return

 Test PASSES if Expected == actual, FAILS if it does not

The Hidden Cost of Not Testing

 While most PAs so far have been small, they are getting larger  In CS 2110, you will write a single program across multiple files and

hundreds of lines of code

 In later classes and industry, you will write thousands, tens of

thousands, and more lines of code

 Bugs are costly:

NATS Air Traffic Failure

 Caused by one bug in one line of code

 “buried” among 4 million of lines of code

 Fault had existed, but not manifested, since

at least the 1990s, possibly earlier

Designing a test case

 Consider the following test case for get_max_3:  Input

 3, 2, 1

 Expected output (calculate by hand)

 3

 Actual output (run the code)

Designing a test case

 Consider the following test case for get_max_3:  Test Case 1

 Input

 3, 2, 1

 Expected output (calculate by hand)

 3

 Actual output (run the code)

 3

 Result: PASS

 wait, didn’t we have a bug

Multiple Tests

 We almost always want multiple tests  This is especially true in functions with conditionals and loops  One test passing may have no bearing on whether or not another

test passes

 Example Test Case 2:

 Input: 1, 2, 3  Expected: 3  Actual: 1  Result: Fail

Why do tests pass

 Test Case 1:

 Input: 3,2,1

Expected: 3 Actual: 3 Result: PASS

 Test Case 2:

 Input: 1,2,3 Expected: 3 Actual: 1 Result: FAIL

 Why does the first test fail, and the other pass?

Why do tests pass

 Test Case 1:

 Input: 3,2,1

Expected: 3 Actual: 3 Result: PASS

 Test Case 2:

 Input: 1,2,3 Expected: 3 Actual: 1 Result: FAIL

 Why does the first test fail, and the other pass?

 Because test case 2 executes the “fault” that

causes the bug

 Test case 1 does not execute the fault

True or False

 If a test executes a bug, that test will fail?

 Thus, we only need to look at lines of code

the failing test executes to find a bug, right?

True or False

 If a test executes a bug, that test will fail?

 Thus, we only need to look at lines of code

the failing test executes to find a bug, right?

 False!!!  Consider this test case (Test Case 3):

 Input: 3, 3, 3  Expected Output: 3  Actual Output: 3  Result: PASS

 But wait, that executes the fault!

Sound Tests

 We want to make sure our tests are sound

 That is, they are correct with the specification

 Example: Test Case 4

 Input: 4, 5, 6  Expected Output: 4 – this is erroneous  Actual Output: 4  Result: PASS – False Negative

 Example: Test Case 5

 Input: 9, 8, 7  Expected Output: 7– this is erroneous  Actual Output: 9  Result: FAIL – False Positive

Importance of Sound Testing

 We want all our tests to be sound  Unsound tests will mislead us to think

 There are no bugs when there are  There are bugs where there are none

 Sounds Tests required a detailed, well-understood specification  Example:

 Mars Climate Orbiter  One system expected Metric  Another system produced English

Debugging: Tracing

 If we have a failing, sound test, we now want to trace  To assist in tracing, we can print values as we trace to ensure they

are correctly updated

Debugging Example:

 Consider the code below:

 Test case 0 and 1 work fine

 Test Case 0: “python”, “o” Expected: 4

Test Case 1: “python”, “z” Expected: -1

 Test case 2 fails

 Test Case 2: “python”, “p” Expected: 1

def get_index_of(string, letter): index = -1 for i in range(1, len(string)): # check if we get the right character if string[i] == letter: index = i return index

Adding Print Statements to Help

 This is a temporary solution: We don’t want to do this long term to

leave this print statements in, but they can help us debug. Try this:

 These print statements can help us find the

reason Test case 2 fails.

We’re not done

 There’s still one more bug we haven’t found.  With a partner:  Design test case(s) to look for bugs  When you have a failing test, use print statements to find why the

bug occurs

 Fix it, and remove the print statements

Parting Thoughts

 Designing test cases is not easy

 There are a number of “testing strategies”, but none will ever be perfect

 Print statements are awkward

 Adding and removing print statements may not be the most efficient

way to debug code

 Debug Mode, Automatic Testing  Print statements are a starting point while you are still learning the basics

Next Time: Dictionaries

 For a given key, find a given value  Example: uh…well…dictionaries

 For a given word, find a definition

 Phone book:

 For a given name, find a phone number

 Define multiple fields of a single entity  willDictionary

 FirstName: Paul  LastName: McBurney  Age: 31  Weight: More than I’m comfortable with  Hair: None;

 willDictionary[“Age”] -> 31