CPSC 320: Intermediate Algorithm Design and Analysis July 25, 2014 - - PowerPoint PPT Presentation

1

CPSC 320: Intermediate Algorithm Design and Analysis

July 25, 2014

2

Course Outline

• Introduction and basic concepts
• Asymptotic notation
• Greedy algorithms
• Graph theory
• Amortized analysis
• Recursion
• Divide-and-conquer algorithms
• Randomized algorithms
• Dynamic programming algorithms
• NP-completeness

3

Dynamic Programming

4

Dynamic Programming

• Greedy algorithm: combines a choice with result of taking the choice
• What if right choice can’t be found easily?

5

Coin Change Problem

• Assume a set of possible coins, and an amount, find the smallest number of coins

that add up to the exact amount

• Example: CoinChange({1,5,10,25}, 47)=[25, 10,10,1,1]
• Is a greedy solution always optimal?
• Example: GreedyCoinChange({1,5,12,25}, 29)=[25, 1,1,1,1], optimal is [12,12,5]

6

Coin Change Problem

• What is a subproblem that can be solved
• If an optimal solution 𝑡 for 𝑂 contains 𝑑, then an optimal solution for 𝑂 − 𝑑 is

𝑡 − 𝑑

• So the subproblem is solving for 𝑂 − 𝑑
• Let’s define the problem recursively:
• For each coin 𝑑, calculate 𝑡𝑑 = CoinChange(𝐷, 𝑂 − 𝑑)
• Select coin 𝑑 with smallest number of coins in subproblem
• Attach coin 𝑑 to 𝑡𝑑 and return

7

Coin Change Recursive

Algorithm CoinChangeRecursive(𝐷, 𝑂) – 𝐷 is set of coins, 𝑂 is amount If 𝑂 = 0 Then Return ∅ 𝑐 ← +∞, 𝑡 ← ∅ For Each 𝑑 ∈ 𝐷 Do If 𝑑 ≤ 𝑂 Then 𝑡𝑑 ← CoinChangeRecursive(𝐷, 𝑂 − 𝑑) If 𝑡𝑑 + 1 < 𝑐 Then 𝑐 ← 𝑡𝑑 + 1 𝑡 ← 𝑡𝑑 ∪ {𝑑} Return 𝑡

8

Recursive

• Is this efficient?
• Value for subproblems is calculated repeatedly
• Example: CoinChange( 1,3,4 ,90), value for 86 is calculated on:
• 4 +CoinChange(86)
• 3 + 1 +CoinChange(86)
• 1 + 3 +CoinChange(86)
• 1 + 1 + 1 + 1 +CoinChange(86)
• Can we save the value (“memoize”) and reuse it?
• Can we calculate from the bottom-up?

9

Dynamic Programming Approach

Algorithm CoinChangeDP(𝐷, 𝑂) – 𝐷 is set of coins, 𝑂 is amount 𝑇 0 ← ∅ For 𝑗 ← 1 To 𝑂 Do 𝑇 𝑗 ← ∅ (length infinity) For Each 𝑑 ∈ 𝐷 Do If 𝑑 ≤ 𝑗 And 𝑇 𝑗 − 𝑑 + 1 < 𝑇 𝑗 Then 𝑇[𝑗] ← 𝑇 𝑗 − 𝑑 ∪ {𝑑} Return 𝑇[𝑗]

10

Dynamic Programming Approach - Faster

Algorithm CoinChangeDP2(𝐷, 𝑂) – 𝐷 is set of coins, 𝑂 is amount 𝑚[0] ← ∅, 𝑐 0 ← 0 For 𝑗 ← 1 To 𝑂 Do 𝑚 𝑗 ← ∅, 𝑐 𝑗 ← +∞ For Each 𝑑 ∈ 𝐷 Do If 𝑑 ≤ 𝑗 And 𝑐 𝑗 − 𝑑 + 1 < 𝑐[𝑗] Then 𝑚 𝑗 ← 𝑑, 𝑐[𝑗] ← 𝑐[𝑗 − 𝑑] + 1 𝑡 ← ∅ While 𝑂 > 0 Do 𝑡 ← 𝑡 ∪ 𝑚 𝑂 , 𝑂 ← 𝑂 − 𝑚[𝑂] Return 𝑡

11

Rod Cutting Problem

• Problem: given a rod of length 𝑜 and a table of prices 𝑞𝑗 for rods of length 𝑗 ≤ 𝑜,

determine the maximum revenue 𝑠

𝑜 obtainable by cutting it in pieces

• Example: 𝑜 = 5, 𝑞1 = 1, 𝑞2 = 5, 𝑞3 = 8, 𝑞4 = 9, 𝑞5 = 10
• Full rod: revenue is 10
• Cut in 5 pieces of 1: revenue is 5
• Cut in pieces of 2 and 3: revenue is 13
• Problem can be solved with dynamic programming
• Choose each possible size, then solve for remainder

12

Rod Cutting Problem

Algorithm RodCutting(𝑜, 𝑞) – 𝑜 is size, 𝑞 is price array 𝑠 0 = 0 For 𝑗 ← 1 To 𝑜 Do 𝑠 𝑗 ← 𝑞[𝑗], 𝑚 𝑗 ← 𝑗 For 𝑘 ≥ 1 To 𝑗 − 1 Do If 𝑠 𝑗 < 𝑠 𝑗 − 𝑘 + 𝑞[𝑘] Then 𝑠 𝑗 ← 𝑠 𝑗 − 𝑘 + 𝑞[𝑘], 𝑚 𝑗 ← 𝑘 𝑡 ← ∅ While 𝑜 > 0 Do 𝑡 ← 𝑡 ∪ 𝑚 𝑜 , 𝑜 ← 𝑜 − 𝑚[𝑜] Return 𝑡

13

Weighted Interval Scheduling

• Problem: same as interval scheduling, but with preferences
• Given a set of intervals 𝑇, and a weight function 𝑥, return a compatible subset
• f 𝑇 with maximum total weight
• Greedy approach doesn’t work anymore
• Define the subproblem
• In optimal solution, if we remove last interval 𝑗, remainder is optimal ending

before 𝑡𝑗

14

Weighted Interval Scheduling

Algorithm WeightedIntervalSched(𝐽, 𝑥) – 𝐽 is set of intervals, 𝑥 is weight function Sort 𝐽 by increasing finishing time 𝑋 0 ← 0 For 𝑘 ← 1 To 𝐽 Do 𝑞 ← last event that ends before 𝐽[𝑘] starts (0 if none) If 𝑋 𝑘 − 1 > 𝑥 𝐽 𝑘 + 𝑋 𝑞 Then 𝑋 𝑘 ← 𝑋 𝑘 − 1 , 𝐷 𝑘 ← false Else 𝑋 𝑘 ← 𝑥 𝐽 𝑘 + 𝑋 𝑞 , 𝐷 𝑘 ← true …

15

Weighted Interval Scheduling (cont.)

… 𝑇 ← ∅ For 𝑘 ← |𝐽| DownTo 1 Do If 𝐷[𝑘] Then Add 𝐽[𝑘] to 𝑇 𝑘 ← last event that ends before 𝐽[𝑘] starts (0 if none) Return 𝑇

16

Subset Sum Problem

• Problem: given a set of weights, find the largest sum of weights below a limit
• Applications: get a bag as full as possible, fill a server with limited resource

time as much as possible

• Extension: each weight has a value associated to it (knapsack problem)
• Now the subproblem is slightly different
• If an optimal solution contains weight 𝑥 for limit 𝑀, then the remainder is
• ptimal for limit 𝑀 − 𝑥

17

Subset Sum Problem

Algorithm SubsetSum(𝑥, 𝑁) – 𝑥 is array of weights, 𝑁 is limit 𝑠 0, 𝑛 ← 0, 𝑚 0, 𝑛 ← false for 𝑛 = 0,1,2, … , 𝑁 For 𝑗 ← 1 To 𝑥 Do For 𝑛 ← 1 To 𝑁 Do If 𝑥 𝑗 > 𝑛 Or 𝑠 𝑗 − 1, 𝑛 > 𝑠 𝑗 − 1, 𝑛 − 𝑥 𝑗 + 𝑥[𝑗] Then 𝑠 𝑗, 𝑛 ← 𝑠 𝑗 − 1, 𝑛 , 𝑚 𝑗, 𝑛 ← 𝑚[𝑗 − 1, 𝑛] Else 𝑠 𝑗, 𝑛 ← 𝑠 𝑗 − 1, 𝑛 − 𝑥 𝑗 + 𝑥[𝑗], 𝑚 𝑗, 𝑛 ← 𝑗 𝑡 ← ∅, 𝑦 ← 𝑥 While 𝑁 > 0 And 𝑚[𝑦, 𝑁] is not false Do 𝑦 ← 𝑚[𝑦, 𝑁], 𝑡 ← 𝑡 ∪ 𝑦, 𝑁 ← 𝑁 − 𝑥 𝑦 , 𝑦 ← 𝑦 − 1 Return 𝑡

18

Knapsack Problem

Algorithm Knapsack(𝑥, 𝑞, 𝑁) – 𝑥 is array of weights, 𝑞 is array of values, 𝑁 is limit 𝑠 0, 𝑛 ← 0, 𝑚 0, 𝑛 ← false for 𝑛 = 0,1,2, … , 𝑁 For 𝑗 ← 1 To 𝑥 Do For 𝑛 ← 1 To 𝑁 Do If 𝑥 𝑗 > 𝑛 Or 𝑠 𝑗 − 1, 𝑛 > 𝑠 𝑗 − 1, 𝑛 − 𝑥 𝑗 + 𝑞[𝑗] Then 𝑠 𝑗, 𝑛 ← 𝑠 𝑗 − 1, 𝑛 , 𝑚 𝑗, 𝑛 ← 𝑚[𝑗 − 1, 𝑛] Else 𝑠 𝑗, 𝑛 ← 𝑠 𝑗 − 1, 𝑛 − 𝑥 𝑗 + 𝑞[𝑗], 𝑚 𝑗, 𝑛 ← 𝑗 𝑡 ← ∅, 𝑦 ← 𝑥 While 𝑁 > 0 And 𝑚[𝑦, 𝑁] is not false Do 𝑦 ← 𝑚[𝑦, 𝑁], 𝑡 ← 𝑡 ∪ 𝑦, 𝑁 ← 𝑁 − 𝑥 𝑦 , 𝑦 ← 𝑦 − 1 Return 𝑡