NP-Completeness : Concepts Why Studying NP-Completeness ? - - PDF document

1

NP-Completeness : Concepts

• Why Studying NP-Completeness ?

♣ Pursuing your Ph.D. ♣ Keeping your job Before studying NP-completeness :

“I can’t find an efficient algorithm, I guess I’m just too dumb.”

2

After studying NP-completeness :

“I can’t find an efficient algorithm, because no such algorithm is possible !” “I can’t find an efficient algorithm, but neither can all these famous people.”

3

• Measure to Time Complexity

l : measure to the time complexity of an algorithm The discussion of NP-completeness considers l the input size, i.e., the total length of all inputs to the algorithm. Two assumptions : (1) all inputs are integers (a rational number can be represented by a pair of integers); (2) each integer has a binary representation.

• Ex. Sorting a1, a1, …, an.

l = (

)

n i i

a

=

+

• 2

1

log 1 .

4

• Ex. Consider the following procedure.

input(n); s ← ← ← ← 0; for i ← ← ← ← 1 to n do s ← ← ← ← s + i;

• utput(s).

l = log2 n + 1. The procedure takes O(n) = O(2l) time. an exponential-time algorithm !

5

• Polynomial-Time Algorithms vs.

Exponential-Time Algorithms

Suppose that your computer takes 1 second to perform 106 operations. The following is the time requirement for your computer to perform f(n) operations, where f(n) = n, n2, n3, n5, 2n, 3n

and n = 10, 20, 30, 40,

50, 60.

6

The following shows the largest value of n such that f(n) operations can be performed in 1 hour

• n a faster computer.

7

An algorithm is referred to as a polynomial-time algorithm, if its time complexity can be bounded above by a polynomial function of input size. An algorithm is referred to as an exponential-time algorithm, if its time complexity cannot be thus bounded (even if the function is not normally regarded as an exponential one, like nlog n). Usually, a problem is referred to as tractable if it can be solved with a polynomial-time algorithm, and intractable otherwise. The two tables above give us a reason why polynomial-time algorithms are much more desirable than exponential-time algorithms. They also motive us to study the theory of NP-completeness.

8

• Maximal vs. Maximum

Ex. maximal cliques : {1, 2, 3}, {2, 3, 4, 5}, {4, 6} maximum cliques : {2, 3, 4, 5}

9

• Decision Problems vs. Optimization

Problems

A decision problem asks a solution of “yes” or “no”. An optimization problem asks a solution of an

• ptimal value (a maximum or a minimum).
• Ex. The maximum clique problem can be expressed

as a decision problem as follows. Instance : An undirected graph G = (V, E) and a positive integer k ≤ ≤ ≤ ≤ |V|. Question : Does G contain a clique of size ≥ ≥ ≥ ≥ k ? It can be also expressed as an optimization problem as follows. Instance : An undirected graph G = (V, E). Question : What is the size of a maximum clique

• f G ?

10

• Ex. The traveling salesman problem can be expressed

as a decision problem as follows. Instance : A set C of m cities, distances di,j > 0 for all pairs of cities i, j ∈ ∈ ∈ ∈ C, and a positive integer k. Question : Is there a tour of length ≤ ≤ ≤ ≤ k that starts at any city, visits each of the other m − − − − 1 cities exactly once, and returns to the initial city ? It can be also expressed as an optimization problem as follows. Instance : A set C of m cities and distances di,j > 0 for all pairs of cities i, j ∈ ∈ ∈ ∈ C. Question : What is the length of a shortest tour that starts at any city, visits each of the other m − − − − 1 cities exactly once, and returns to the initial city ?

11

• Ex. The problem of sorting a1, a1, …, an can be

expressed as a decision problem as follows. Instance : Given a1, a2, …, an and a positive integer k. Question : Is there a permutation of a1, a2, …, an, denoted by a’1, a’2, …, a’n, such that |a’2 − − − − a’1| + |a’3 − − − − a’2| + … + |a’n − − − − a’n−

− − −1| ≤

≤ ≤ ≤ k ? An optimization problem is “harder” than its corresponding decision problem. Since the NP-completeness concerns whether or not a problem can be solved in polynomial time, the discussion of NP-completeness considers only decision problems. (If a decision problem is not polynomial-time solvable, then its corresponding optimization problem is not polynomial-time solvable either.)

12

• Problem Reduction

A problem P1 reduces to another problem P2, denoted by P1 ∝ ∝ ∝ ∝ P2, if any instance of P1 can be transformed into an instance of P2 such that the solution for P1 can be obtained from the solution for P2. T∝

∝ ∝ ∝ : the reduction time.

T : the time required to obtain the solution for P1 from the solution for P2. Since the NP-completeness concerns whether or not a problem can be solved in polynomial time, we consider only the reductions with both T∝

∝ ∝ ∝ and

T polynomial. (Thus, P2 ∈ ∈ ∈ ∈ P

• P1 ∈

∈ ∈ ∈ P or P1 ∉ ∉ ∉ ∉ P

• P2 ∉

∉ ∉ ∉ P.) If P1 ∝ ∝ ∝ ∝ P2 and P2 ∝ ∝ ∝ ∝ P3, then P1 ∝ ∝ ∝ ∝ P3.

13

• P, NP, and NP-Complete

Three classes of decision problems : P, NP, and NP-complete. P : the set of decision problems that can be solved in polynomial time by deterministic algorithms. NP : the set of decision problems that can be solved in polynomial time by non-deterministic algorithms. Any non-deterministic algorithm consists of two phases : guessing and checking.

14

For the maximum clique problem, the guessing phase will return a clique, and the checking phase will decide whether or not the clique size is greater than or equal to k. For the traveling salesman problem, the guessing phase will return a tour, and the checking phase will decide whether or not the tour length is greater than or equal to k. A decision problem has an AFFIRMATIVE answer. ⇔ ⇔ ⇔ ⇔ The guessing is SUCCESSFUL. Notice that non-deterministic algorithms are

• imaginary. A more detailed description of non-

deterministic algorithms and more illustrative examples can be found in Ref. (2).

15

Every decision problem in P is also in NP, i.e., P ⊆ ⊆ ⊆ ⊆ NP. An NP problem is NP-complete if every NP problem can reduce to it in polynomial time.

• If any NP-complete problem can be solved in

polynomial time, then every NP problem can be solved in polynomial time (i.e., P = NP). (Intuitively, NP-complete problems are the “hardest” problems in NP.) It is one of the most famous open problems in computer science whether P ≠ ≠ ≠ ≠ NP or P = NP.

16

When P ≠ ≠ ≠ ≠ NP,

P NP NP-Complete

(There exist problems in NP that are neither in P, nor in NP-complete (see Chap. 7 in Ref. (1).) When P = NP,

P = NP = NP-Complete

Almost all people believe P ≠ ≠ ≠ ≠ NP.

17

A problem is NP-hard if an NP-complete problem can be reduced to it in polynomial time. (Equivalently, a problem is NP-hard if every NP problem can be reduced to it in polynomial time.)

• If any NP-hard problem can be solved in

polynomial time, then all NP-complete problems can be solved in polynomial time. (Intuitively, NP-hard problems are “harder” than NP-complete problems.)

NP NP-hard NP-complete

The class of NP-hard problems contains both decision problems and optimization problems.

18

If an NP-hard problem is in NP, then it is an NP-complete problem. (Intuitively, NP-complete problems are an “easier” subclass of NP-hard problems.) The corresponding optimization problems of NP-complete problems are NP-hard. The well-known halting problem (a decision problem), which is to determine whether or not an algorithm will terminate with a given input, is NP-hard, but not NP-complete.

19

• Pseudo-Polynomial Time Algorithms
• Ex. Given a set S = {a1, a1, …, an} of integers and an

integer M > 0, the sum-of-subset problem is to determine whether or not there exists a subset

• f S whose sum is equal to M.

This problem can be solved in O(nM) time by dynamic programming as follows. Let t(i, j) = true, if there exists a subset of {a1, a2, …, ai} whose sum is equal to j, and false else. Then, t(i, j) = t(i − − − − 1, j) + t(i − − − − 1, j − − − − ai), where i > 1. Initially, t(1, j) = true, if j = 0 or j = a1, and false else. The answer is t(n, M).

20

Although the time complexity is exponential with respect to M, the problem is considered polynomial-time solvable, if M is bounded. An algorithm like this is usually referred to as a pseudo-polynomial time algorithm. An NP-complete problem is in the strong sense if and only if there exists no pseudo-polynomial time algorithm for solving it (unless P = NP). Intuitively, NP-complete problems in the strong sense are “harder” NP-complete problems (refer to Ref. (1)).

21

• The Satisfiability Problem and Cook’s

Theorem

The satisfiability problem, which is the first NP-complete problem, is defined as follows. Instance : A set U of Boolean variables and a collection C of clauses over U. Question : Is there an assignment of U that can satisfy C ?

• Ex. When U = {x1, x2, x3} and C = {x1 ∨

∨ ∨ ∨ x2 ∨ ∨ ∨ ∨ x3, x1, x2 }, the assignment of U : x1 ← ← ← ← F, x2 ← ← ← ← F and x3 ← ← ← ← T, can satisfy C (i.e., (x1 ∨ ∨ ∨ ∨ x2 ∨ ∨ ∨ ∨ x3) ∧ ∧ ∧ ∧ ( x1) ∧ ∧ ∧ ∧ ( x2 ) = T).

22

• Ex. When U = {x1, x2} and C = {x1 ∨

∨ ∨ ∨ x2, x1 ∨ ∨ ∨ ∨ x2 , x1

∨

∨ ∨ ∨ x2, x1

∨

∨ ∨ ∨ x2 }, no assignment of U can satisfy C. Cook’s Theorem : The satisfiability problem is NP-complete. The proof of Cook’s Theorem, which is rather lengthy and complex, can be found in Ref. (1) and Ref. (2). There is an informal proof of Cook’s Theorem in the textbook.

23

• Six Basic NP-Complete Problems

(P1) 3-Satisfiability Instance : A set U of variables and a collection C = {c1, c2, …, cm} of clauses over U, where each clause of C contains three literals. Question : Is there a satisfying truth assignment for C ?

• Ex. When U = {x1, x2, x3} and C = {x1 ∨

∨ ∨ ∨ x2 ∨ ∨ ∨ ∨ x3, x1 ∨ ∨ ∨ ∨ x2 ∨ ∨ ∨ ∨ x3}, the assignment of U : x1 ← ← ← ← T, x2 ← ← ← ← F and x3 ← ← ← ← F, can satisfy C.

• Ex. When U = {x1, x2, x3} and C = {x1 ∨

∨ ∨ ∨ x2 ∨ ∨ ∨ ∨ x3, x1 ∨ ∨ ∨ ∨ x2 ∨ ∨ ∨ ∨ x3, x1 ∨ ∨ ∨ ∨ x2 ∨ ∨ ∨ ∨ x3, x1 ∨ ∨ ∨ ∨ x2 ∨ ∨ ∨ ∨ x3 , x1 ∨ ∨ ∨ ∨ x2 ∨ ∨ ∨ ∨ x3, x1 ∨ ∨ ∨ ∨ x2 ∨ ∨ ∨ ∨ x3 , x1 ∨ ∨ ∨ ∨ x2 ∨ ∨ ∨ ∨ x3 , x1 ∨ ∨ ∨ ∨ x2 ∨ ∨ ∨ ∨ x3 }, no assignment of U can satisfy C.

24

(P2) Vertex Cover Instance : An undirected graph G = (V, E) and a positive integer k ≤ ≤ ≤ ≤ |V|. Question : Does G contain a vertex cover of size at most k, i.e., a subset V’ ⊆ ⊆ ⊆ ⊆ V such that |V’| ≤ ≤ ≤ ≤ k and for each (u, v) ∈ ∈ ∈ ∈ E, at least

• ne of u and v belongs to V’ ?

Ex. |V’| = 4, 5

• V’ is a vertex cover;

|V’| = 3 : {1, 2, 3}, {1, 3, 4}, {1, 3, 5}, {2, 3, 4}, and {2, 3, 5} are vertex covers; |V’| < 3

• V’ is not a vertex cover.

25

(P3) 3-Dimensional Matching Instance : A set M ⊆ ⊆ ⊆ ⊆ W × × × × X × × × × Y, where W, X and Y are three disjoint sets, each having q elements. Question : Does M contain a matching, i.e., a subset M’ ⊆ ⊆ ⊆ ⊆ M such that each element

• f W, X and Y appears in M’ exactly
• nce (|M’| = q) ?
• Ex. Suppose W = {a, b}, X = {c, d}, and Y = {e, f}.

If M = {(a, c, f), (b, d, e), (a, d, f)}, then M contains a matching M’ = {(a, c, f), (b, d, e)}. If M = {(a, c, f), (b, c, e), (b, d, f)}, then M does not contain a matching.

26

(P4) Clique Instance : An undirected graph G = (V, E) and a positive integer k ≤ ≤ ≤ ≤ |V|. Question : Does G contain a clique of size at least k, i.e., a subset V’ ⊆ ⊆ ⊆ ⊆ V such that |V’| ≥ ≥ ≥ ≥ k and every two vertices

• f V’ are adjacent in G ?

Ex. |V’| = 4, 5

• V’ is not a clique;

|V’| = 3 : {1, 2, 3} is a clique; |V’| = 2 : {1, 2}, {1, 3}, {2, 3}, {3, 4} and {3, 5} are cliques.

27

(P5) Hamiltonian Cycle Instance : An undirected graph G = (V, E). Question : Does G contain a Hamiltonian cycle, i.e., an ordering (v1, v2, …, v|V|) of the vertices of G such that (v1, v|V|) ∈ ∈ ∈ ∈ E and (vi, vi+1) ∈ ∈ ∈ ∈ E for all 1 ≤ ≤ ≤ ≤ i < |V| ? Ex. The left graph has a Hamiltonian cycle, but the right graph does not.

28

(P6) Partition Instance : A multiset A = {a1, a2, …, a|A|} of positive integers. Question : Does there exist A’ ⊆ ⊆ ⊆ ⊆ A such that

i

i

a A'

a

• ∈

∈ ∈ ∈

=

j

j

a A A'

a

−

• ∈

∈ ∈ ∈

?

• Ex. The multiset {2, 2, 4, 4, 8} can be divided into

{2, 4, 4} and {2, 8} whose sums are equal. On the other hand, {2, 2, 4, 4, 7} cannot be divided similarly.

29

The six NP-complete problems above were shown in Ref. (1) in the following way, where each “→ → → →” represents a reduction “∝ ∝ ∝ ∝” (for example, Vertex Cover ∝ ∝ ∝ ∝ Clique).

Satisfiability 3-Satisfiability 3-Dimensional Matching Partition Vertex Cover Clique Hamiltonian Cycle

It is still possible to show these NP-complete problems (and others) in a different way, i.e., using different known NP-complete problems. A list of NP-complete problems can be found in Appendix of Ref. (1).

30

• Two-Sided Analysis of Problems

If some restrictions are imposed on a problem Π Π Π Π, then a restricted subproblem Π Π Π Π’ of Π Π Π Π results. Suppose Π Π Π Π, Π Π Π Π’ ∈ ∈ ∈ ∈ NP and P ≠ ≠ ≠ ≠ NP. Π Π Π Π’ is NP-complete

• Π

Π Π Π is NP-complete. Π Π Π Π is NP-complete

• Π

Π Π Π’ is in P or NP-complete

• r neither.

Π Π Π Π Π Π Π Π’

31

(“→ → → →” means “a subproblem of”) The frontier is narrowed down, if some open problems are shown to be in P or NP-complete.

• Ex. Let d be the maximal vertex degree in G.

Both Vertex Cover and Hamiltonian Cycle are in P if d ≤ ≤ ≤ ≤ 2, and NP-complete if d ≥ ≥ ≥ ≥ 3.

32

• Ex. Graph 3-Colorability

Instance : An undirected graph G = (V, E). Question : Is G 3-colorable, i.e., does there exist a function f : V → → → → {1, 2, 3} such that f(u) ≠ ≠ ≠ ≠ f(v) for all edges (u, v) ∈ ∈ ∈ ∈ E ? Graph 3-Colorability is in P if d ≤ ≤ ≤ ≤ 3, and NP-complete if d ≥ ≥ ≥ ≥ 4 or G is planar. Ex.

33

Ex. Precedence Constrained Scheduling Instance : A set T of “tasks”, each of “length” 1, a partial order

• n T, a “deadline” d, and

m “processors”. Question : Is there a “schedule” f : T → → → → {0, 1, …, d} such that f(t) < f(t’) if t t’, and for each i ∈ ∈ ∈ ∈ {0, 1, …, d}, |{t ∈ ∈ ∈ ∈ T : f(t) = i}| ≤ ≤ ≤ ≤ m ?

34

• Coping with NP-Hard Problems
• ptimal

polynomial solution ? time ? greedy (heuristic) not yes algorithms guaranteed dynamic yes experimentally programming & efficient branch-and-bound algorithms genetic algorithms & not experimentally ant algorithms guaranteed efficient approximation a guaranteed yes algorithms error bound (exclusive of approximation schemes) randomized a high probability yes algorithms

• r

yes a high probability average polynomial yes in average case time algorithm