CPSC 490: Problem Solving in Computer Science Presentations are - - PowerPoint PPT Presentation

CPSC 490: Problem Solving in Computer Science

Lecture 10: Tree DP, Smaller-to-larger optimization, DP on graphs

Henry Xia, Brandon Zhang

based on CPSC 490 slides from 2014-2018

2019-02-05

University of British Columbia

Announcements

• Presentation topic choices are due Wednesday.
• Presentations are next week (Feb 12 and 14).
• (Part of) assignment 4 is released (due Mar 1).

1

You already know how to do tree DP!

Recall that Dynamic Programming = break down problem recursively into similar sub-problems and then combining their answers When we need to do something on a tree...

• What are the sub-problems?
• What are the base cases?
• How do we do our computation?

Tree DP example: compute the size of the tree

• Leaf (i.e. base case): f u

1

• Non-leaf (i.e. recurrence): f u

1

vchild of u f v

• Answer: f root
• Time complexity: O N

2

You already know how to do tree DP!

Recall that Dynamic Programming = break down problem recursively into similar sub-problems and then combining their answers When we need to do something on a tree...

• What are the sub-problems?
• What are the base cases?
• How do we do our computation?

Tree DP example: compute the size of the tree

• Leaf (i.e. base case): f u

1

• Non-leaf (i.e. recurrence): f u

1

vchild of u f v

• Answer: f root
• Time complexity: O N

2

You already know how to do tree DP!

Recall that Dynamic Programming = break down problem recursively into similar sub-problems and then combining their answers When we need to do something on a tree...

• What are the sub-problems? Subtrees!
• What are the base cases?
• How do we do our computation?

Tree DP example: compute the size of the tree

• Leaf (i.e. base case): f u

1

• Non-leaf (i.e. recurrence): f u

1

vchild of u f v

• Answer: f root
• Time complexity: O N

2

You already know how to do tree DP!

Recall that Dynamic Programming = break down problem recursively into similar sub-problems and then combining their answers When we need to do something on a tree...

• What are the sub-problems? Subtrees!
• What are the base cases? Leaf nodes!
• How do we do our computation?

Tree DP example: compute the size of the tree

• Leaf (i.e. base case): f u

1

• Non-leaf (i.e. recurrence): f u

1

vchild of u f v

• Answer: f root
• Time complexity: O N

2

You already know how to do tree DP!

Recall that Dynamic Programming = break down problem recursively into similar sub-problems and then combining their answers When we need to do something on a tree...

• What are the sub-problems? Subtrees!
• What are the base cases? Leaf nodes!
• How do we do our computation? DFS!

Tree DP example: compute the size of the tree

• Leaf (i.e. base case): f u

1

• Non-leaf (i.e. recurrence): f u

1

vchild of u f v

• Answer: f root
• Time complexity: O N

2

You already know how to do tree DP!

Recall that Dynamic Programming = break down problem recursively into similar sub-problems and then combining their answers When we need to do something on a tree...

• What are the sub-problems? Subtrees!
• What are the base cases? Leaf nodes!
• How do we do our computation? DFS!

Tree DP example: compute the size of the tree

• Leaf (i.e. base case): f(u) = 1
• Non-leaf (i.e. recurrence): f(u) = 1 + ∑

vchild of u f(v)

• Answer: f(root)
• Time complexity: O(N)

2

Problem 1 – Order statistics

Modify your favourite balanced binary search tree (e.g. AVL, red-black, splay) to support finding the kth largest element in O(log n) time.

3

Problem 1 – Solution

• In every node, keep track of the size of subtree
• Update operation (e.g. rotations): update child if necessary, then recompute size of

current node

• Find k-th largest element:

1

def find(T, k):

2

if T.right.size == k - 1:

3

return T.value

4

if T.right.size >= k:

5

return find(T.right, k)

6

else:

7

return find(T.left, k - T.right.size - 1)

• All operations (insert, delete, find, get k-th largest) are still O(log n)

4

Problem 2 – Bookkeeping

Input: a rooted tree with N ≤ 105 nodes, with each node containing an integer value. Output: the following statistics for every subtree and for the entire tree:

• Size
• Height
• Diameter (length of longest simple path)
• Average of values
• Median of values
• k-th largest value

5

Problem 2 – Solution (Part 1)

Size of each subtree

• f(leaf) = 1, f(node) = 1 + ∑ f(child)

Height of each subtree

• f leaf

1, f node 1 f child Diameter of the tree

• Idea: keep two numbers per node
• f u

length of the diameter of the subtree rooted at u

• g u

length of longest path that starts in subtree and ends at u

• f leaf

g leaf

• f node

f child 2 two largest values of g child Time complexity: O N

6

Problem 2 – Solution (Part 1)

Size of each subtree

• f(leaf) = 1, f(node) = 1 + ∑ f(child)

Height of each subtree

• f(leaf) = 1, f(node) = 1 + max f(child)

Diameter of the tree

• Idea: keep two numbers per node
• f u

length of the diameter of the subtree rooted at u

• g u

length of longest path that starts in subtree and ends at u

• f leaf

g leaf

• f node

f child 2 two largest values of g child Time complexity: O N

6

Problem 2 – Solution (Part 1)

Size of each subtree

• f(leaf) = 1, f(node) = 1 + ∑ f(child)

Height of each subtree

• f(leaf) = 1, f(node) = 1 + max f(child)

Diameter of the tree

• Idea: keep two numbers per node
• f(u) = length of the diameter of the subtree rooted at u
• g(u) = length of longest path that starts in subtree and ends at u
• f(leaf) = g(leaf) = 0
• f(node) = max{f(child), 2 + two largest values of g(child)}

Time complexity: O(N)

6

Problem 2 – Solution (Part 2)

Average

• f(node) = val(node) + ∑ f(child)
• Avg(node) = f(node)/size(node)

Time complexity: O(N) Median, k-th largest value

• A v

sorted list of values in v’s subtree

• Answer for v’s subtree

the kth element in the list

• To compute A v recursively, we need to merge all the lists of v’s children
• How do we merge?
• What data structure do we use?

7

Problem 2 – Solution (Part 2)

Average

• f(node) = val(node) + ∑ f(child)
• Avg(node) = f(node)/size(node)

Time complexity: O(N) Median, k-th largest value

• A(v) = sorted list of values in v’s subtree
• Answer for v’s subtree = the kth element in the list
• To compute A(v) recursively, we need to merge all the lists of v’s children
• How do we merge?
• What data structure do we use?

7

Merging sets

Naive approach:

• Fill up an array, then call sort()
• Time complexity: up to O(N log N) per sort, so possibly O(N2 log N) in a deep tree
• The problem is that we keep moving items we’ve already merged

Merging sets with BSTs

• Let’s store each set using... a set!
• While there is more than 1 set: pick any arbitrary pair, add all elements from the

smaller set to the larger set Time complexity

• Each time an element moves, it gets into a set twice as big
• Each element is moved at most O

N times

• O N

2 N in total 8

Merging sets

Naive approach:

• Fill up an array, then call sort()
• Time complexity: up to O(N log N) per sort, so possibly O(N2 log N) in a deep tree
• The problem is that we keep moving items we’ve already merged

Merging sets with BSTs

• Let’s store each set using... a set!
• While there is more than 1 set: pick any arbitrary pair, add all elements from the

smaller set to the larger set Time complexity

• Each time an element moves, it gets into a set twice as big
• Each element is moved at most O

N times

• O N

2 N in total 8

Merging sets

Naive approach:

• Fill up an array, then call sort()
• Time complexity: up to O(N log N) per sort, so possibly O(N2 log N) in a deep tree
• The problem is that we keep moving items we’ve already merged

Merging sets with BSTs

• Let’s store each set using... a set!
• While there is more than 1 set: pick any arbitrary pair, add all elements from the

smaller set to the larger set Time complexity

• Each time an element moves, it gets into a set twice as big
• ⇒ Each element is moved at most O(log N) times
• ⇒ O(N log2 N) in total

8

Problem 2 – Solution (Part 3)

How do we use this to compute our kth largest value statistic?

• Start with a set at every leaf, storing the value at that leaf.
• For the vertex v, merge all of v’s children and add the value at v.
• We can query for the kth largest element in O(log n) time to get the answer.

9

Problem 3 – Adding states

You are the head of the a large organization of 10000 members. You are planning to hold a retreat involving some of the members. The organization can be represented (in modern terminology) as a tree of managers. You want to select a subset of people that are connected in this tree.

• How many ways can you form the group if you are part of it?
• What if you are not part of the group?
• What if you want to limit the group size to K? Assume K ≤ 100.

10

Problem 3 – Solution (Part 1)

Let f(u) = number of connected subgraphs of subtree rooted at u that contain u

• For each child v of u, we either choose to include it (f(v) ways), or we exclude it

(exactly one way).

• f(u) = ∏

child v(1 + f(v))

f(root) gives the number of ways to select a group that contains you! Time complexity: O(N)

11

Problem 3 – Solution (Part 2)

Let f(u) = number of connected subgraphs of subtree rooted at u that contain u (we compute this as usual) Let g(u) = number of connected subgraphs of subtree rooted at u that do NOT contain u

• For each child v of u, we just add all possible ways of putting the subgraph in that

child – since we can’t use more than one child without touching the node.

• g(u) = ∑

child v(f(v) + g(v))

g(root) gives the number of ways to select a group without you! Time complexity: O(N)

12

Problem 3 – Solution (Part 3)

We care about size now, so let’s add another state to track size. Let f u k number of connected subgraphs of size k containing u in subtree of u Let g u k number of connected subgraphs of size k NOT containing u in subtree of u

• At each node, we use the same idea as before, but we replace the simple product by

a knapsack-like DP

• h i j

number of connected subgraphs of size j containing u in subtree rooted at u, given that we only use children 1 i

• h 0 1

1, h 0 j 1 0, h i j

j k 1 h i

1 j k f i k

• f u k

h #child k

• g u k

child v f v k

g v k

K k 1 f root k and/or K k 1 g root k now give the answer!

Time complexity: O NK2

13

Problem 3 – Solution (Part 3)

We care about size now, so let’s add another state to track size. Let f(u, k) = number of connected subgraphs of size k containing u in subtree of u Let g(u, k) = number of connected subgraphs of size k NOT containing u in subtree of u

• At each node, we use the same idea as before, but we replace the simple product by

a knapsack-like DP

• h i j

number of connected subgraphs of size j containing u in subtree rooted at u, given that we only use children 1 i

• h 0 1

1, h 0 j 1 0, h i j

j k 1 h i

1 j k f i k

• f u k

h #child k

• g u k

child v f v k

g v k

K k 1 f root k and/or K k 1 g root k now give the answer!

Time complexity: O NK2

13

Problem 3 – Solution (Part 3)

We care about size now, so let’s add another state to track size. Let f(u, k) = number of connected subgraphs of size k containing u in subtree of u Let g(u, k) = number of connected subgraphs of size k NOT containing u in subtree of u

• At each node, we use the same idea as before, but we replace the simple product by

a knapsack-like DP

• h i j

number of connected subgraphs of size j containing u in subtree rooted at u, given that we only use children 1 i

• h 0 1

1, h 0 j 1 0, h i j

j k 1 h i

1 j k f i k

• f u k

h #child k

• g u k

child v f v k

g v k

K k 1 f root k and/or K k 1 g root k now give the answer!

Time complexity: O NK2

13

Problem 3 – Solution (Part 3)

We care about size now, so let’s add another state to track size. Let f(u, k) = number of connected subgraphs of size k containing u in subtree of u Let g(u, k) = number of connected subgraphs of size k NOT containing u in subtree of u

• At each node, we use the same idea as before, but we replace the simple product by

a knapsack-like DP

• h(i, j) = number of connected subgraphs of size j containing u in subtree rooted at u,

given that we only use children 1, . . . , i

• h(0, 1) = 1, h(0, j > 1) = 0, h(i, j) = ∑j

k=1 h(i − 1, j − k) · f(i, k)

• f u k

h #child k

• g u k

child v f v k

g v k

K k 1 f root k and/or K k 1 g root k now give the answer!

Time complexity: O NK2

13

Problem 3 – Solution (Part 3)

We care about size now, so let’s add another state to track size. Let f(u, k) = number of connected subgraphs of size k containing u in subtree of u Let g(u, k) = number of connected subgraphs of size k NOT containing u in subtree of u

• At each node, we use the same idea as before, but we replace the simple product by

a knapsack-like DP

• h(i, j) = number of connected subgraphs of size j containing u in subtree rooted at u,

given that we only use children 1, . . . , i

• h(0, 1) = 1, h(0, j > 1) = 0, h(i, j) = ∑j

k=1 h(i − 1, j − k) · f(i, k)

• f(u, k) = h(#child, k)
• g(u, k) = ∑

child v(f(v, k) + g(v, k)) K k 1 f root k and/or K k 1 g root k now give the answer!

Time complexity: O NK2

13

Problem 3 – Solution (Part 3)

We care about size now, so let’s add another state to track size. Let f(u, k) = number of connected subgraphs of size k containing u in subtree of u Let g(u, k) = number of connected subgraphs of size k NOT containing u in subtree of u

• At each node, we use the same idea as before, but we replace the simple product by

a knapsack-like DP

• h(i, j) = number of connected subgraphs of size j containing u in subtree rooted at u,

given that we only use children 1, . . . , i

• h(0, 1) = 1, h(0, j > 1) = 0, h(i, j) = ∑j

k=1 h(i − 1, j − k) · f(i, k)

• f(u, k) = h(#child, k)
• g(u, k) = ∑

child v(f(v, k) + g(v, k))

∑K

k=1 f(root, k) and/or ∑K k=1 g(root, k) now give the answer!

Time complexity: O(NK2)

13

Problem 4 – Tree growing

Input: a binary tree with n ≤ 100 nodes, and a number x ≤ 1000 which represents the amount of “growth powder” you have. You are running water from the root of the tree to the leaves. Initially, each edge can transport water at a rate of 1 unit per second. If you apply k units of growth powder to an edge, it can transport (1 + k)2 units of water per second. Output: the maximum amount of water that can flow from the root to the leaves.

14

Problem 4 – Solution

f(v, k) = the most water that can flow from v to the leaves of v’s subtree, given that we have k units of growth powder to apply to the subtree and the edge to v’s parent f leaf k 1 k 2 To compute f v k , iterate over all ways to pass down growth powder to v’s children. The amount we have leħt is applied to the parent edge. f v k

i j k

f v left i f v right j 1 k i j

2

Special case: if v root, then we don’t need to put any powder on the parent edge. Time complexity: O nx2

15

Problem 4 – Solution

f(v, k) = the most water that can flow from v to the leaves of v’s subtree, given that we have k units of growth powder to apply to the subtree and the edge to v’s parent f(leaf, k) = (1 + k)2 To compute f(v, k), iterate over all ways to pass down growth powder to v’s children. The amount we have leħt is applied to the parent edge. f(v, k) = max

i+j≤k min(f(v.left, i) + f(v.right, j), (1 + (k − (i + j)))2)

Special case: if v = root, then we don’t need to put any powder on the parent edge. Time complexity: O nx2

15

Problem 4 – Solution

f(v, k) = the most water that can flow from v to the leaves of v’s subtree, given that we have k units of growth powder to apply to the subtree and the edge to v’s parent f(leaf, k) = (1 + k)2 To compute f(v, k), iterate over all ways to pass down growth powder to v’s children. The amount we have leħt is applied to the parent edge. f(v, k) = max

i+j≤k min(f(v.left, i) + f(v.right, j), (1 + (k − (i + j)))2)

Special case: if v = root, then we don’t need to put any powder on the parent edge. Time complexity: O(nx2)

15

Can we DP on a general graph?

16

Longest increasing subsequence, revisited

We had the following DP to find the length of an LIS: f(k) = length of LIS ending at A[k] f(1) = 1, f(k) = max

j<k,A[j]<A[k] f(j) + 1

Let’s make a graph where the nodes are elements of the array, and there is an edge j k if j k and A j A k . This is a DAG! Finding the LIS is equivalent to finding the longest path in this DAG! DP for longest path in a DAG: f v

u v f u

d u v

17

Longest increasing subsequence, revisited

We had the following DP to find the length of an LIS: f(k) = length of LIS ending at A[k] f(1) = 1, f(k) = max

j<k,A[j]<A[k] f(j) + 1

Let’s make a graph where the nodes are elements of the array, and there is an edge j → k if j < k and A[j] < A[k]. This is a DAG! Finding the LIS is equivalent to finding the longest path in this DAG! DP for longest path in a DAG: f v

u v f u

d u v

17

Longest increasing subsequence, revisited

We had the following DP to find the length of an LIS: f(k) = length of LIS ending at A[k] f(1) = 1, f(k) = max

j<k,A[j]<A[k] f(j) + 1

Let’s make a graph where the nodes are elements of the array, and there is an edge j → k if j < k and A[j] < A[k]. This is a DAG! Finding the LIS is equivalent to finding the longest path in this DAG! DP for longest path in a DAG: f(v) = maxu→v f(u) + d(u, v)

17

DPs and DAGs

In general, if we represent our DP states as nodes and transitions as edges, we should get a DAG (since the recurrence isn’t cyclic). This suggests we should be able to run DPs on DAGs...

18

Problem 6 – Path counting

Input: a DAG with N ≤ 105 nodes, and two nodes s, t. Output: the number of difgerent paths from s to t.

19

Problem 6 – Solution

f(v) = # of difgerent paths from s to v. f s 1, f v

u v f u .

How can we compute f? We can do it recursively as usual, or we can do it bottom-up with topological sort! Time complexity: O V E

20

Problem 6 – Solution

f(v) = # of difgerent paths from s to v. f(s) = 1, f(v) = ∑

u→v f(u).

How can we compute f? We can do it recursively as usual, or we can do it bottom-up with topological sort! Time complexity: O(|V| + |E|)

20

Problem 7 – Path counting (harder)

Input: a DAG with N ≤ 105 nodes, and two nodes s, t. Output: the number of shortest paths from s to t.

21

Problem 7 – Solution

First, we’ll compute the length of the shortest path from s to every other node. d(v) = length of the shortest path from s to v. d(s) = 0, d(v) = minu→v d(u) + c(u, v). How does this help?

22

Problem 7 – Solution

We can use this to figure out which edges are on a shortest path! An edge u → v is on a shortest path ⇔ d(v) = d(u) + c(u, v). f(v) = number of shortest paths from s to v. f(s) = 1, f(v) = ∑

u→v d(v)=d(u)+c(u,v)

f(u). Time complexity: O(|V| + |E|)

23