[PPT] - Computer Science 331 Computation of Minimum-Cost Spanning Trees PowerPoint Presentation

SLIDE 1

Computer Science 331

Computation of Minimum-Cost Spanning Trees — Prim’s Algorithm Mike Jacobson

Department of Computer Science University of Calgary

Lecture #35

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 1 / 20

SLIDE 2

Outline

1

Introduction

2

Min-Cost Spanning Trees

3

Algorithm General Construction Problem and Algorithm

4

Example

5

Termination and Efficiency

6

Additional Comments and References

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 2 / 20

SLIDE 3

Introduction

Computation of Min-Cost Spanning Trees

Motivation: Given a set of sites (represented by vertices of a graph), connect these all as cheaply as possible (using connections represented by the edges of a weighted graph). Goals for Today: presentation of the definitions needed to formally define a problem motivated by the above presentation of an algorithm (Prim’s Algorithm) for solving the problem

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 3 / 20

SLIDE 4

Min-Cost Spanning Trees

Costs of Spanning Trees in Weighted Graphs

Recall that if G = (V , E) is a connected, undirected graph, then a spanning tree of G is a subgraph G = ( V , E) such that

V = V (so

G includes all the vertices in G)

G is a tree

Suppose now that G = (V , E) is a connected weighted graph with weight function w : E → N, and that G1 = (V1, E1) is a spanning tree of G The cost of G1, w(G1), is the sum of the weights of the edges in G1, that is, w(G1) =

e∈E1

w(e).

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 4 / 20

SLIDE 5

Min-Cost Spanning Trees

Example

Suppose G is a weighted graph with weights as shown below.

a c b d e f g 2 5 1 2 4 1 1 3 3 2 2 1

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 5 / 20

SLIDE 6

Min-Cost Spanning Trees

Example

The cost of the following spanning tree, G1 = (V1, E1), is 8.

a c b d e f g 1 1 1 2 2 1

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 6 / 20

SLIDE 7

Min-Cost Spanning Trees

Example

The cost of the following spanning tree, G2 = (V2, E2), is 16.

a c b d e f g 2 5 2 3 2 2

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 7 / 20

SLIDE 8

Min-Cost Spanning Trees

Minimum-Cost Spanning Trees

Suppose (G, w) is a weighted graph. A subgraph G1 of G is a minimum-cost spanning tree of (G, w) if the following properties are satisfied.

1 G1 is a spanning tree of G. 2 w(G1) ≤ w(G2) for every spanning tree G2 of G.

Example: In the previous example, G2 is clearly not a minimum-cost spanning tree, because G1 is a spanning tree of G such that w(G2) > w(G1). It can be shown that G1 is a minimum-cost spanning tree of (G, w).

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 8 / 20

SLIDE 9

Algorithm General Construction

Building a Minimum-Cost Spanning Tree

To construct a minimum-cost spanning tree of G = (V , E):

1 Start with

G = ( V , E), where V ⊆ V and E = ∅. Note: G is a subgraph of some minimum-cost spanning tree of (G, w).

2 Repeatedly add vertices (if necessary) and edges — ensuring that

G is still a subgraph of a minimum-cost spanning tree as you do so. Continue doing this until V = V and | E| = |V | − 1 (so that G is a spanning tree of G).

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 9 / 20

SLIDE 10

Algorithm General Construction

Building a Minimum-Cost Spanning Tree

Additional Notes: This can be done in several different ways, and there are at least two different algorithms that use this approach to solve this problem. The algorithm to be presented here begins with V = {s} for some vertex s ∈ V , and makes sure that G is always a tree. As a result, this algorithm is structurally very similar to Dijkstra’s Algorithm to compute minimum-cost paths (which we have already discussed in class).

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 10 / 20

SLIDE 11

Algorithm Problem and Algorithm

Specification of Requirements

Pre-Condition G = (V , E) is a connected graph with weight function w Post-Condition: π is a function π : V → V ∪ {NIL} If

E = {(π(v), v) | v ∈ V and π(v) = NIL}

then (V , E) is a minimum-cost spanning tree for G The graph G = (V , E) and its weight function have not been changed

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 11 / 20

SLIDE 12

Algorithm Problem and Algorithm

Data Structures

The algorithm (to be presented next) will use a priority queue to store information about weights of edges that are being considered for inclusion The priority queue will be a MinHeap: the entry with the smallest priority will be at the top of the heap Each node in the priority queue will store a vertex in G and the weight of an edge incident to this vertex The weight will be used as the vertex’s priority An array-based representation of the priority queue will be used A second array will be used to locate each entry of the priority queue for a given node in constant time Note: The data structures will, therefore, look very much like the data structures used by Dijkstra’s algorithm.

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 12 / 20

SLIDE 13

Algorithm Problem and Algorithm

Pseudocode

MST-Prim(G, w, s) for v ∈ V do colour[v] = white d[v] = +∞ π[v] = NIL end for Initialize an empty priority queue Q colour[s] = grey d[s] = 0 add s with priority 0 to Q

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 13 / 20

SLIDE 14

Algorithm Problem and Algorithm

Pseudocode, Continued

while (Q is not empty) do (u, c) = extract-min(Q) {Note: c = d[u]} for each v ∈ Adj[u] do if (colour[v] == white) then d[v] = w((u, v)) colour[v] = grey; π[v] = u add v with priority d[v] to Q else if (colour[v] == grey) then Update information about v (Shown on next slide) end if end for colour[u] = black end while return π

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 14 / 20

SLIDE 15

Algorithm Problem and Algorithm

Pseudocode, Concluded

Updating Information About v if (w((u, v)) < d[v]) then

ld = d[v]

d[v] = w((u, v)) π[v] = u Use Decrease-Priority to replace (v, old) in Q with (v, d[v]) end if

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 15 / 20

SLIDE 16

Example

Consider the execution of MST-Prim(G):

a c b d e f g 2 5 1 2 4 1 1 3 3 2 2 1

a b c d e f g d π

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 17

Example

a c b d e f g 2 5 1 2 4 1 1 3 3 2 2 1

Step 0: initialization a b c d e f g d

π
Q: (empty)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 18

Example

a c b d e f g 2 5 1 2 4 1 1 3 3 2 2 1

Step 0: initialization enqueue(a, 0) a b c d e f g d

π
Q:

(a,0)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 19

Example

a c b d e f g 2 5 1 2 4 1 1 3 3 2 2 1

Step 1: Extract-Min (returns (a, 0)) a b c d e f g d

π
Q: (empty)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 20

Example

c b d e f g 2 5 1 2 4 1 1 3 3 2 2 1 a

Step 1: Extract-Min (returns (a, 0)) add b a b c d e f g d 2

π
a
Q:

(b,2)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 21

Example

c b d e f g 2 5 1 2 4 1 1 3 3 2 2 1 a

Step 1: Extract-Min (returns (a, 0)) add b add c a b c d e f g d 2 1

π
a

a

Q:

(b,2) (c,1)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 22

Example

c d e f g 2 5 1 2 4 1 1 3 3 2 2 1 b a

Step 1: Extract-Min (returns (a, 0)) add b add c color a black a b c d e f g d 2 1

π
a

a

Q:

(b,2) (c,1)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 23

Example

c d e f g 2 5 1 2 4 1 1 3 3 2 2 1 b a

Step 2: Extract-Min (returns (c, 1)) a b c d e f g d 2 1

π
a

a

Q:

(b,2)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 24

Example

c d e f g 2 5 1 2 4 1 1 3 3 2 2 1 b a

Step 2: Extract-Min (returns (c, 1)) update b a b c d e f g d 1 1

π
c

a

Q:

(b,1)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 25

Example

c d e f g 2 5 1 2 4 1 1 3 3 2 2 1 b a

Step 2: Extract-Min (returns (c, 1)) update b add d a b c d e f g d 1 1 2

π
c

a c

Q:

(b,1) (d,2)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 26

Example

c d e f g 2 5 1 2 4 1 1 3 3 2 2 1 b a

Step 2: Extract-Min (returns (c, 1)) update b add d add f a b c d e f g d 1 1 2

4
π
c

a c

c
Q:

(b,1) (d,2) (f,4)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 27

Example

d e f g 2 5 1 2 4 1 1 3 3 2 2 1 b a c

Step 2: Extract-Min (returns (c, 1)) update b add d add f color c black a b c d e f g d 1 1 2

4
π
c

a c

c
Q:

(b,1) (d,2) (f,4)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 28

Example

d e f g 2 5 1 2 4 1 1 3 3 2 2 1 b a c

Step 3: Extract-Min (returns (b, 1)) a b c d e f g d 1 1 2

4
π
c

a c

c
Q:

(d,2) (f,4)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 29

Example

d e f g 2 5 1 2 4 1 1 3 3 2 2 1 b a c

Step 3: Extract-Min (returns (b, 1)) update d (no change) a b c d e f g d 1 1 2

4
π
c

a c

c
Q:

(d,2) (f,4)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 30

Example

d e f g 2 5 1 2 4 1 1 3 3 2 2 1 b a c

Step 3: Extract-Min (returns (b, 1)) update d (no change) add e a b c d e f g d 1 1 2 5 4

π
c

a c b c

Q:

(d,2) (f,4) (e,5)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 31

Example

d e f g 2 5 1 2 4 1 1 3 3 2 2 1 a c b

Step 3: Extract-Min (returns (b, 1)) update d (no change) add e color b black a b c d e f g d 1 1 2 5 4

π
c

a c b c

Q:

(d,2) (f,4) (e,5)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 32

Example

d e f g 2 5 1 2 4 1 1 3 3 2 2 1 a c b

Step 4: Extract-Min (returns (d, 2)) a b c d e f g d 1 1 2 5 4

π
c

a c b c

Q:

(f,4) (e,5)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 33

Example

d e f g 2 5 1 2 4 1 1 3 3 2 2 1 a c b

Step 4: Extract-Min (returns (d, 2)) update e a b c d e f g d 1 1 2 2 4

π
c

a c d c

Q:

(e,2) (f,4)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 34

Example

d e f g 2 5 1 2 4 1 1 3 3 2 2 1 a c b

Step 4: Extract-Min (returns (d, 2)) update e update f a b c d e f g d 1 1 2 2 1

π
c

a c d d

Q:

(f,1) (e,2)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 35

Example

e f g 2 5 1 2 4 1 1 3 3 2 2 1 a c b d

Step 4: Extract-Min (returns (d, 2)) update e update f color d black a b c d e f g d 1 1 2 2 1

π
c

a c d d

Q:

(f,1) (e,2)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 36

Example

e f g 2 5 1 2 4 1 1 3 3 2 2 1 a c b d

Step 5: Extract-Min (returns (f , 1)) a b c d e f g d 1 1 2 2 1

π
c

a c d d

Q:

(e,2)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 37

Example

e f g 2 5 1 2 4 1 1 3 3 2 2 1 a c b d

Step 5: Extract-Min (returns (f , 1)) update e (no change) a b c d e f g d 1 1 2 2 1

π
c

a c d d

Q:

(e,2)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 38

Example

e f g 2 5 1 2 4 1 1 3 3 2 2 1 a c b d

Step 5: Extract-Min (returns (f , 1)) update e (no change) add g a b c d e f g d 1 1 2 2 1 2 π

c

a c d d f Q:

(e,2) (g,2)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 39

Example

e g 2 5 1 2 4 1 1 3 3 2 2 1 a c b d f

Step 5: Extract-Min (returns (f , 1)) update e (no change) add g color f black a b c d e f g d 1 1 2 2 1 2 π

c

a c d d f Q:

(e,2) (g,2)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 40

Example

e g 2 5 1 2 4 1 1 3 3 2 2 1 a c b d f

Step 6: Extract-Min (returns (e, 2)) a b c d e f g d 1 1 2 2 1 2 π

c

a c d d f Q:

(g,2)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 41

Example

e g 2 5 1 2 4 1 1 3 3 2 2 1 a c b d f

Step 6: Extract-Min (returns (e, 2)) update g a b c d e f g d 1 1 2 2 1 1 π

c

a c d d e Q:

(g,1)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 42

Example

g 2 5 1 2 4 1 1 3 3 2 2 1 a c b d f e

Step 6: Extract-Min (returns (e, 2)) update g color e black a b c d e f g d 1 1 2 2 1 1 π

c

a c d d e Q:

(g,1)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 43

Example

g 2 5 1 2 4 1 1 3 3 2 2 1 a c b d f e

Step 7: Extract-Min (returns (g, 1)) a b c d e f g d 1 1 2 2 1 1 π

c

a c d d e Q: (empty)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 44

Example

2 5 1 2 4 1 1 3 3 2 2 1 a c b d f e g

Step 7: Extract-Min (returns (g, 1)) color g black — done! a b c d e f g d 1 1 2 2 1 1 π

c

a c d d e Q: (empty)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 45

Example

1 1 1 2 2 1 a c b d f e g

Step 7: Extract-Min (returns (g, 1)) color g black — done! a b c d e f g d 1 1 2 2 1 1 π

c

a c d d e Q: (empty)

Eg. one MST (total cost is 8):

{(π(b), b), (π(c), c), (π(d), d), (π(e), e), (π(f ), f ), (π(g), g)} = {(c, b), (a, c), (c, d), (d, e), (d, f ), (e, g)}

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 16 / 20

SLIDE 46

Termination and Efficiency

Claim:

If MST-Prim is executed on a weighted undirected graph G = (V , E) then the algorithm terminates after performing O((|V | + |E|) log |V |) steps in the worst case.

Proof.

This is virtually identical to the proof of the corresponding result for Dijkstra’s algorithm (to compute minimum-cost paths). The number of operations on the priority queue, and the number of

perations that do not involve this data structure, are each

in Θ(|V | + |E|) in the worst case (by the argument that has been applied to the last three algorithms considered).

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 17 / 20

SLIDE 47

Termination and Efficiency

Termination and Efficiency (cont.)

Proof (continued).

Since the size of the priority queue never exceeds |V | and since the

nly operations on the priority queue used are insertions, decreases of

key values, and extractions of the minimum (top priority) element, the cost of each operation on the data structure is in O(log |V |). It follows immediately that the total number of steps is in O((|V | + |E|) log |V |), as claimed. O(|V | log |V | + |E|) using a Fibonacci heap (amortized)

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 18 / 20

SLIDE 48

Additional Comments and References

Additional Comments

On Greedy Algorithms Prim’s algorithm is an example of a greedy algorithm: A “global”

ptimization problem (finding a minimum-cost spanning tree) is

solved by making a sequence of “local” greedy choices (by extending a tree with edges whose weights are as small as possible). Proving correctness of greedy algorithms is often challenging. Indeed, greedy heuristics are often incorrect. On the other hand, when they are correct, greedy algorithms are frequently simpler and more efficient than other algorithms for the same computation. See CPSC 413 for more about greedy algorithms!

Mike Jacobson (University of Calgary) Computer Science 331 Lecture #35 19 / 20

SLIDE 49

Additional Comments and References