CPSC 320: Intermediate Algorithm Design and Analysis August 6, 2014 - - PowerPoint PPT Presentation

1

CPSC 320: Intermediate Algorithm Design and Analysis

August 6, 2014

2

Schedule

• Monday: BC Day, no classes
• Wednesday:
• Quiz 5 (NP and NP completeness)
• Review: Amortized Analysis
• Friday:
• Survey (bonus marks)
• Review: probably Graph Theory and Dynamic Programming

3

Amortized Analysis

4

Amortized cost

• Although the worst case for one call has a large bound, we are interested in the

total cost of a sequence of calls

• In other words, 𝑙 calls to a 𝑃 𝑔 𝑜

function may be better than 𝑃 𝑙 ⋅ 𝑔 𝑜

• The amortized cost of the function takes the total amount in consideration

5

Aggregate Method

• Evaluate a sequence of calls, and calculate the total cost
• Usually changes in data structure are considered instead of operations

6

Potential Method

• Each operation receives a potential cost, which if larger than actual cost can be

“saved” towards future operations

• Operations that take longer can use “saved” value
• Total cost of sequence of operations is the sum of potential costs
• Properties:
• Assume 𝐸𝑗 is state of algorithm after 𝑗 iterations
• Φ 𝐸𝑗 ≥ 0 (you cannot use more than what you saved)
• Φ 𝐸0 = 0 (you start with no saved cost – usually)
• 𝑑𝑝𝑡𝑢𝑏𝑛 𝑝𝑞𝑗 = Φ 𝐸𝑗 − Φ 𝐸𝑗−1 + 𝑑𝑝𝑡𝑢𝑠𝑓𝑏𝑚(𝑝𝑞𝑗)
• Alternative notation:
• 𝑑𝑗 for real cost,

𝑑𝑗 for amortized cost

7

Garbage Collection

• Assume a memory management process where:
• allocating memory takes a constant time 𝑏 in general case
• there is no freeing process, but memory reference is taken into account
• if there is no memory left (or memory runs over a limit), unreferenced

memory space is freed

• each freed element takes constant time 𝑔
• For simplicity, assume there is trivial way to retrieve unreferenced memory space

8

Garbage Collection

• Real cost of allocate:
• If no garbage collection is needed: 𝑏
• If garbage collection is needed: 𝑏 + 𝑔𝑠 (𝑠 is number of freeable objects)
• Amortized cost of allocate:
• Assuming initially no elements are allocated
• Each freed element in garbage collection requires a previous allocation, and

every allocation has a single free

• Sequence of 𝑙 calls to allocate costs less than 𝑏𝑙 + 𝑔𝑙
• Each allocate can be assumed to have amortized cost 𝑏 + 𝑔 ∈ Θ(1)

9

Garbage Collection – Potential Method

• What information in the data structure determines an expensive operation?
• Allocate is costly if number of elements is large
• In worst-case, allocate takes time linear to size of memory
• Potential function: number of allocated words × 𝑔
• If no GC is needed: 𝑑𝑝𝑡𝑢𝑏𝑛 𝑏𝑚𝑚𝑝𝑑 = 𝑑𝑝𝑡𝑢𝑠𝑓𝑏𝑚 𝑏𝑚𝑚𝑝𝑑 + Δ Φ = 𝑏 + 𝑔
• If GC is needed: 𝑑𝑝𝑡𝑢𝑏𝑛 𝑏𝑚𝑚𝑝𝑑 = 𝑑𝑝𝑡𝑢𝑠𝑓𝑏𝑚 𝑏𝑚𝑚𝑝𝑑 + Δ Φ = 𝑏 + 𝑔𝑠 + 𝑔 − 𝑔𝑠 = 𝑏 + 𝑔

10

More examples

• Consider an operation that, when called the 𝑜th time, costs 𝑜 if 𝑜 is a power of 2,

and costs 1 otherwise

• Aggregate analysis:
• A sequence of 𝑜 operations will cost 1 every time except lg 𝑜 + 1 times

(20, 21, … 2 lg 𝑜 )

• 𝑜 operations will cost:

𝐷 ≤ 𝑜 +

𝑗=0 lg 𝑜

2𝑗 ≤ 𝑜 + 2lg 𝑜+1 − 1 2 − 1 = 𝑜 + 2𝑜 − 1 ≤ 3𝑜 ∈ 𝑃 𝑜

• Each operation has an amortized constant cost

11

More examples

• Consider an operation that, when called the 𝑜th time, costs 𝑜2 if 𝑜 is a power of 2,

and costs 1 otherwise

• Aggregate analysis:
• A sequence of 𝑜 operations will cost 1 every time except lg 𝑜 + 1 times

(20, 21, … 2 lg 𝑜 )

• 𝑜 operations will cost:

𝐷 ≤ 𝑜 +

𝑗=0 lg 𝑜

22𝑗 ≤ 𝑜 + 4lg 𝑜+1 − 1 4 − 1 = 𝑜 + 4𝑜2 − 1 3 ∈ 𝑃 𝑜2

• Each operation has an amortized linear cost

12

Exercise

Georgia Street has many tall buildings, but only some of them have a clear view of Stanley Park. Suppose we are given an array 𝐵 1. . 𝑜 that stores the height of 𝑜 buildings on a city block, indexed from South to North. Building 𝑗 has a good view of Stanley Park if and only if every building to the North of 𝑗 is shorter than 𝑗. Here is an algorithm that computes which buildings have a good view of Stanley

• Park. What is the running time of this algorithm?

Algorithm Goodview(A[1 .. n]): Initialize a stack S For i ← 1 to n Do While (S not empty and A[i] > A[Top(S)]) Do Pop(S) Push(S, i) Return S

13

Good view – Aggregate Analysis

• Consider each different building
• How many times is it pushed?
• How many times is it popped?
• How many times is Top called?
• For each building:
• Push is called exactly once
• Pop is called no more than once
• Top is called once for every pop and once for every push
• So, potentially, each building triggers up to 4 queue operations
• Since there are 𝑜 buildings, this algorithm takes 𝑃(𝑜) total time

14

Good view – Potential Method

• Potential method corresponds to twice the size of the stack
• Push:

𝑑𝑝𝑡𝑢𝑏𝑛 𝑞𝑣𝑡ℎ = 𝑑𝑝𝑡𝑢𝑠𝑓𝑏𝑚 𝑞𝑣𝑡ℎ + Δ Φ = 1 + 2 = 3

• Pop:

𝑑𝑝𝑡𝑢𝑏𝑛 𝑞𝑝𝑞 = 𝑑𝑝𝑡𝑢𝑠𝑓𝑏𝑚 𝑞𝑝𝑞 + Δ Φ = 1 − 2 = −1

• Top:

𝑑𝑝𝑡𝑢𝑏𝑛 𝑢𝑝𝑞 = 𝑑𝑝𝑡𝑢𝑠𝑓𝑏𝑚 𝑢𝑝𝑞 + Δ Φ = 1 + 0 = 1

• In each iteration of the for loop, if there are 𝑙 calls to Pop, there are 𝑙 + 1 calls to Top
• Cost for each iteration: 𝑙 ⋅ −1 + 𝑙 + 1 ⋅ 1 + 3 = 4
• Since the loop iterates 𝑜 times, we have a total cost of 4𝑜 ∈ 𝑃 𝑜