Program Analysis, Python and Logarithms Tyler Moore CSE 3353, SMU, - - PDF document

SLIDE 1

Program Analysis, Python and Logarithms

Tyler Moore

CSE 3353, SMU, Dallas, TX

January 29, 2013

These slides have been adapted from the slides written by Prof. Steven Skiena at SUNY Stony Brook, author of Algorithm Design Manual. For more information see http://www.cs.sunysb.edu/~skiena/

Problem of the Day

Find two functions f (n) and g(n) that satisfy the following relationship. If no such f and g exist, write “None”.

1 f (n) = o(g(n)) and f (n) = Θ(g(n)) 2 f (n) = Θ(g(n)) and f (n) = o(g(n)) 3 f (n) = Θ(g(n)) and f (n) = O(g(n)) 4 f (n) = Ω(g(n)) and f (n) = O(g(n))

Hint: what does “little oh” notation mean?

f (n) = o(g(n)) ⇐ ⇒ g(n) dominates f (n) So n2 = o(n3) since n3 dominates n2.

2 / 20

You should come to accept the dominance ranking of the basic functions: n! ≫ 2n ≫ n3 ≫ n2 ≫ n log n ≫ n ≫ √n ≫ log n ≫ 1

3 / 20

Reasoning About Efficiency

Grossly reasoning about the running time of an algorithm is usually easy given a precise-enough written description of the algorithm. When you really understand an algorithm, this analysis can be done in your head. However, recognize there is always implicitly a written algorithm/program we are reasoning about.

4 / 20

Notes Notes Notes Notes

SLIDE 2

Selection sort (C)

1

s e l e c t i o n s o r t ( int s [ ] , int n)

2

{

3

int i , j ;

4

int min ;

5

for ( i =0; i <n ; i++) {

6

min=i ;

7

for ( j=i +1; j<n ; j++)

8

i f ( s [ j ] < s [ min ] ) min=j ;

9

swap(&s [ i ] ,& s [ min ] ) ;

10

}

Outer loop goes around n times Inner loop goes around at most n − 1 times for each iteration of the

• uter loop

Thus selection sort takes at most n × (n − 1) → O(n2) time in the worst case.

5 / 20

But what if we were more careful about the analysis?

An exact count of the number of times the if statement is executed is given by: S(n) =

n−1

• i=0

n−1

• j=i+1

1 =

n−1

• i=0

n − i − 1 S(n) = (n − 1) + (n − 2) + · · · + 2 + 1 + 0 = (n − 1)(n − 2) 2 Thus the worst-case running-time is Θ(n2).

6 / 20

Using upper and lower bounds to establish running time

First let’s find an upper bound

We loop through at most n terms For each term, the nested loop iterates at most n − 1 times Hence, S(n) ≤ n(n − 1) = O(n2)

Next let’s find a lower bound

For at least n/2 terms, the nested loop iterates at least n/2 times Hence, S(n) ≥ n/2 × n/2 = 1

4n2 = Ω(n2)

Since S(n) = O(n2) and S(n) = Ω(n2), S(n) = Θ(n2)

7 / 20

Efficiency Analysis of Insertion Sort (C)

1

i n s e r t i o n s o r t ( int s [ ] , int n)

2

{

3

int i , j ;

4

int min ;

5

for ( i =1; i <n ; i++) {

6

j = i ;

7

while (( j >0) &&

8

( s [ j ] < s [ j −1])) {

9

swap(&s [ j ] ,& s [ j −1]);

10

j = j −1;

11

}

12

}

13

}

Outer loop goes around n − 1 times What about the inner while loop? Depends on the input, so we can assume the worst case i times Since i < n, we can assume i = n and so O(n2)

8 / 20

Notes Notes Notes Notes

SLIDE 3

We already know Java and C++. Why learn Python?

Python has far less overhead than Java/C++ for the programmer. Python is closer to psuedo-code on the English-pseudocode-code spectrum than Java or C/C++, but it actually executes! Python is handy for data manipulation and transformation, and anything ”quick and dirty.” Python is very powerful, thanks to all those extension modules.

9 / 20

Python Resources

Download from python.org for all major platforms I’ve written an online tutorial: http://lyle.smu.edu/~tylerm/courses/cse3353/python.html Beginning Python, Magnus Lie Hetland: http://link.springer. com/book/10.1007/978-1-4302-0634-7/page/1

10 / 20

Back to the Selection Sort (C)

1

s e l e c t i o n s o r t ( int s [ ] , int n)

2

{

3

int i , j ;

4

int min ;

5

for ( i =0; i <n ; i++) {

6

min=i ;

7

for ( j=i +1; j<n ; j++)

8

i f ( s [ j ] < s [ min ] ) min=j ;

9

swap(&s [ i ] ,& s [ min ] ) ;

10

}

11 / 20

Selection Sort, now in Python

1 def

s e l e c t i o n s o r t ( s ) :

2

”””

3

Input : l i s t s to be s o r t e d

4

Output : s o r t e d l i s t

5

”””

6

for i in range ( l e n ( s ) ) :

7

#don ’ t name min s i n c e r e s e r v e d word

8

minidx=i

9

for j in range ( i +1, l e n ( s ) ) :

10

i f s [ j ]<s [ minidx ] :

11

minidx=j

12

s [ i ] , s [ minidx ]= s [ minidx ] , s [ i ]

13

return s

Whitespace matters! Dynamic, implicit typing Iterators make looping easy Has built-in lists, dictionaries Multiple variable assignment per line

12 / 20

Notes Notes Notes Notes

SLIDE 4

Problem of the Day due Thursday at beginning of class

1 Install Python on your computer 2 Write Python code to implement an insertion sort. You may refer to

the C code on p. 4 of the ADM You may also refer to the Python code implementing selection sort at http://lyle.smu.edu/~tylerm/courses/cse3353/code/l3.zip Submit the code via Blackboard (link made available later today)

13 / 20

Logarithms

It is important to understand deep in your bones what logarithms are and where they come from. A logarithm is simply an inverse exponential function. Saying bx = y is equivalent to saying that x = logb y. Logarithms reflect how many times we can double something until we get to n, or halve something until we get to 1.

14 / 20

Binary Search and Logarithms

In binary search we throw away half the possible number of keys after each comparison. Thus twenty comparisons suffice to find any name in the million-name Manhattan phone book! Question: how many times can we halve n before getting to 1?

15 / 20

Logarithms and Binary Trees

How tall a binary tree do we need until we have n leaves? → The number of potential leaves doubles with each level. How many times can we double 1 until we get to n?

16 / 20

Notes Notes Notes Notes

SLIDE 5

Logarithms and Bits

How many bits do you need to represent the numbers from 0 to 2i − 1?

17 / 20

Logarithms and Multiplication

Recall that loga(xy) = loga(x) + loga(y) This is how people used to multiply before calculators, and remains useful for analysis. What if x = a?

18 / 20

The Base is not Asymptotically Important

Recall the definition, clogc x = x and that logb a = logc a logc b So for a = 2 and c = 100: log2 n = log100 n log100 2 Since

1 log100 2 = 6.643 is a constant, we can ignore it when calculating

Big Oh

19 / 20

Federal Sentencing Guidelines

F1.1. Fraud and Deceit; Forgery; Offenses Involving Altered or Counterfeit Instruments other than Counterfeit Bearer Obligations of the United States. (a) Base offense Level: 6 (b) Specific offense Characteristics (1) If the loss exceeded $2,000, increase the offense level as follows:

Loss(Apply the Greatest) Increase in Level (A) $2,000 or less no increase (B) More than $2,000 add 1 (C) More than $5,000 add 2 (D) More than $10,000 add 3 (E) More than $20,000 add 4 (F) More than $40,000 add 5 (G) More than $70,000 add 6 (H) More than $120,000 add 7 (I) More than $200,000 add 8 (J) More than $350,000 add 9 (K) More than $500,000 add 10 (L) More than $800,000 add 11 (M) More than $1,500,000 add 12 (N) More than $2,500,000 add 13 (O) More than $5,000,000 add 14 (P) More than $10,000,000 add 15 (Q) More than $20,000,000 add 16 (R) More than $40,000,000 add 17 (Q) More than $80,000,000 add 18

The increase in punishment level grows logarithmically in the amount of money stolen. Thus it pays to commit one big crime rather than many small crimes totaling the same amount. In other words, Make the Crime Worth the Time

20 / 20

Notes Notes Notes Notes