Efficient Algorithms and Problem Complexity Divide and Conquer - - PowerPoint PPT Presentation

Efficient Algorithms and Problem Complexity – Divide and Conquer –

Frank Drewes Department of Computing Science Ume˚ a University

Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 1 / 14

Outline

Today’s Menu

1

What is a Divide-and-Conquer Algorithm?

2

Example 1: Mergesort

3

Example 2: Matrix Multiplication

Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 2 / 14

What is a Divide-and-Conquer Algorithm?

Divide et impera (divide and rule)

Historically A quote attributed to Julius Caesar describing the political (social, . . . ) strategy to divide those you want to keep under control into competing groups having roughly the same power, so that none of them can become a leader with the power to threaten you. In computer science The problem solving strategy that consists in dividing an input into smaller

• nes, solving them recursively, and combining the solutions to a solution

for the original input.

Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 3 / 14

What is a Divide-and-Conquer Algorithm?

General pattern (1)

For a problem instance (input) I is of size n . . . divide I into a smaller instances I1, . . . , Ia, solve I1, . . . , Ia, yielding results R1, . . . , Ra, and combine R1, . . . , Ra to a result R for I. Typically,. . . a is a constant ≥ 2 the size of each Ij is ≤ ⌈n/b⌉ for a constant b > 1, and “divide” and “combine” take O(nk) steps for a constant k.

Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 4 / 14

What is a Divide-and-Conquer Algorithm?

General pattern (2)

Typically,. . . a is a constant ≥ 2 the size of each Ij is ≤ ⌈n/b⌉ for a constant b > 1, and “divide” and “combine” take O(nk) steps for a constant k. ⇒ The running time is bounded by the recurrence T(n) ≤ aT(n/b) + O(nk). ⇒ We can apply the Main Recurrence Theorem to bound T(n) e.g., a = b = k = 2 yields the bound O(n2).

Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 5 / 14

Example 1: Mergesort

Recalling Mergesort (1)

Mergesort sorts an array of items according to their keys. We assume that items only consist of keys, that are integers. Sorting an array a of size n > 1 works by

1 recursively sorting a[1, . . . , ⌈n/2⌉] and a[⌈n/2⌉ + 1, . . . , n] and 2 merging the two (now sorted) sub-arrays into the final result. Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 6 / 14

Example 1: Mergesort

Recalling Mergesort (2)

The pseudocode: Mergesort(a[1, . . . , n], i, j) where initially i = 1, j = n if i < j then k ← ⌊(i + j)/2⌋ Mergesort(a, i, k) Mergesort(a, k + 1, j) Merge(a, i, k + 1, j) The obvious implementation of Merge(a, i, k, j) runs in time Θ(i − j).

Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 7 / 14

Example 1: Mergesort

Overall time required by Mergesort

For array size n two recursive calls are executed (a = 2), each with a problem size ≤ ⌈n/2⌉ (b = 2), and the time used by the non-recursive part is Θ(n) (k = 1). ⇒ the resulting recurrence relation is T(n) ≤ 2T(n/2) + O(n). ⇒ Mergesort runs in time Θ(n log n) (by the Main Recurrence Theorem, and since a = 2 = 21 = bk).

Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 8 / 14

Example 2: Matrix Multiplication

Multiplying two n × n matrices

Given: A = (ai j) =    a1 1 · · · a1 n . . . ... . . . an 1 · · · an n    and B = (bi j) =    b1 1 · · · b1 n . . . ... . . . bn 1 · · · bn n   . Task Compute the product C = AB, i.e., C = (ci j) with ci j = n

k=1 ai kbk j.

The obvious algorithm computes the entries ci j one by one. The computation of each ci j requires 2n − 1 arithmetic operations. ⇒ in total, Θ(n3) arithmetic operations are used.

Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 9 / 14

Example 2: Matrix Multiplication

Matrices of matrices . . .

How can we do this by divide-and-conquer? Suppose for simplicity that n is a power of 2. ⇒ We can write A, B, and C as 2 × 2 matrices of n/2 × n/2 matrices: A = A1 1 A1 2 A2 1 A2 2

• B =

B1 1 B1 2 B2 1 B2 2

• C =

C1 1 C1 2 C2 1 C2 2

• Then, Ci j = Ai 1B1 j + Ai 2B2 j (just the ordinary matrix multiplication,

but now with matrices as entries). [Verify!] ⇒ we get a recursive algorithm for matrix multiplication.

Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 10 / 14

Example 2: Matrix Multiplication

A first recursive algorithm

RecMMult(A, B, C) where A, B, C are n×n matrices, n = 2m if n = 1 then return C = (a1 1b1 1) else for all (i, j) ∈ {1, 2}2 do Ci j = RecMMult(Ai 1, B1 j) + RecMMult(Ai 2, B2,j) return C = C1 1 C1 2 C2 1 C2 2

• Resulting recurrence relation: T(n) ≤ 8T(n/2) + O(n2).

⇒ running time O(nlg 8) = O(n3). :( The problem is the factor 8!

Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 11 / 14

Example 2: Matrix Multiplication

Strassen’s recursive algorithm

How can we reduce the number of recursive calls? Strassen’s observation: If we let                    D1 = (A1 1 + A2 2) · (B1 1 + B2 2) D2 = (A2 1 + A2 2) · B1 1 D3 = A1 1 · (B1 2 − B2 2) D4 = A2 2 · (B2 1 − B1 1) D5 = (A1 1 + A1 2) · B2 2 D6 = (A2 1 − A1 1) · (B1 1 + B1 2) D7 = (A1 2 − A2 2) · (B2 1 + B2 2)                    then C = D1 + D4 − D5 + D7 D3 + D5 D2 + D4 D1 − D2 + D3 + D6

• .

Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 12 / 14

Example 2: Matrix Multiplication

Running time of Strassen’s algorithm

What do we loose/gain? The non-recursive part still requires O(n2) operations. (It’s a fixed number of matrix additions – but notice that the factor is 18 instead of 4!) We need only 7 recursive calls. The size of matrices in recursive calls is he same as before. ⇒ the recurrence relation turns into T(n) ≤ 7T(n/2) + O(n2). ⇒ the time required is O(nlg 7) = O(n2.81...).

Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 13 / 14

Example 2: Matrix Multiplication

Concluding notes

Notes Strassen’s algorithm beats the naive one only for very large matrices ⇒ in practice, we need to stop the recursion long before n = 1. The assumption that n is a power of 2 must be removed (e.g., by filling up with zeroes). The best known algorithm [Coppersmith/Winograd 1987] solves the problem in time Θ(n2.376) (though with a huge constant factor). This is surprisingly near the trivial lower bound Ω(n2). Read Chapter 5 of the textbook, in particular Section 5.3.

Frank Drewes (Ume˚ a University) Efficient Algorithms and Problem Complexity Lecture 2 14 / 14