Selected Topics of Theoretical Computer Science (456-335/1) Petr - - PowerPoint PPT Presentation

Selected Topics of Theoretical Computer Science (456-335/1)

Petr Janˇ car Dept of Computer Science Technical University Ostrava (FEI Vˇ SB-TU) www.cs.vsb.cz/jancar TU Ostrava, winter semester 2005/2006

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 1 / 24

A randomized communication protocol

From

• J. Hromkoviˇ

c: Theoretical Computer Science, Springer 2004

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 2 / 24

Number theoretic algorithms

(From Cormen, Leiserson, Rivest: Introduction to algorithms; the MIT Press, 1990) Size of inputs and cost of arithmetic operations number a: size β = log a bit operations multiplication in O(β2), similarly dividing (and mod ) (for practise sufficient) by divide-conquer one can get for multiplication O(βlog2 3), the fastest known is O(β log β log log β) Exerc.: binary-to-decimal representation (log overhead; O(β2 log β)

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 3 / 24

Elementary number theoretic notions

divisibility, primes, a = ((a ÷ n) · n) + (a mod n) definition of a ≡ b( mod n) Zn, Z6 = {0, 1, 2, 3, 4, 5} common divisors, gcd Theorem: a, b not both 0, gcd(a, b) is the smallest positive in {ax + by | x, y ∈ Z} relatively prime integers unique factorization Exercises: p prime, 0 < k < p; then p|(p

k ); so (a + b)p ≡ ap + bp( mod p)

Show a polyn. alg. to decide if a given n is a nontrivial power (n = ak for some k > 1)

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 4 / 24

Greatest common divisor

gcd(a, b) = gcd(b, a mod b) Euclid’s algorithm Fibonacci numbers 1, 1, 2, 3, 5, 8, 13, . . . are the worst-case for Euclid’s algorithm Since Fk is approximately φk/ √ 5 where φ is the golden ratio (1.618...) the number of recursive calls is O(β) (altogether the running time O(β3)). (By more detailed analysis O(β2).) (Golden ratio (divine proportion): A − − − B − −C : AC/AB = AB/BC = φ removing the maximal square from (golden) rectangle 1 : φ we get a smaller golden rectangle ...) Extended Euclid (gives d = gcd(a, b) = ax + by): (for a, b gives (d, x, y): if b = 0 then return (a, 1, 0) else (recursively) d = bx′ + (a mod b)y ′(= bx′ + (a − (a ÷ b)b)y ′ = ay ′ + b(x′ − (a ÷ b)y ′); return (d, y ′, x′ − (a ÷ b)y ′

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 5 / 24

Modular arithmetic

Two groups from Zn: (Zn, +n), (Z∗

n, ∗n)

In Z∗

n the relative primes with n

E.g. Z15 = {1, 2, 4, 7, 8, 11, 13, 14} Size of Z∗

n: Euler’s phi function

φ(n) = n

• p|n

(1 − 1/p) φ(45) = 45(1 − 1 3)(1 − 1 5) (n is prime iff φ(n) < n − 1)

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 6 / 24

Subgroups

Th.: A nonempty closed subset of a finite group is a subgroup.

• Th. (Lagrange): S finite group and S′ a subgroup. Then |S′| divides |S|.

(So if S′ is proper then |S′| ≤ |S|/2.)

• Proof. (More generally.) Let H be a subgroup of (even nonfinite) G.

Consider {gH | g ∈ G}. This is a partition of (the set) G. (ah1 = bh2 implies ah = ah1h−1

1 h = bh2h−1 1 h ∈ bH.)

|{gH | g ∈ G}| is called index of subgroup H in group G (denoted [G : H]). Note that for each g: |H| = |gH| (gh1 = gh2 implies h1 = h2). So |G| = [G : H] · |H|. Subgroups generated by an element (a, a + a, a + a + a, . . . ), or a, aa, aaa, . . . in the multiplicative notation the order of a (the least t s.t. at = e); equal to the size of the generated subgroup Due to Langrange th.: a|S| = e

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 7 / 24

Solving modular linear equations

ax ≡ b( mod n) denote d = gcd(a, n); the subgroup gen by a is {0, d, 2d, 3d, . . . , ((n/d) − 1)d} (size n/d). So: either d distinct solutions (if d|b) or no solution. Solution by EXT-Euclid: d = gcd(a, n) = ax′ + ny ′; x0 = x′(b/d); xi = x0 + i(n/d) if gcd(a, n) = 1 then ax ≡ b( mod n) has unique solution Multiplicative inverse: if gcd(a, n) = 1 then ax ≡ 1( mod n) has unique solution (otherwise no solution) (EXT-Euclid gives the inverse x: 1 = gcd(a, n) = ax + ny; Exerc: a polynomial f (x) mod p of degree t with coef. from Zp, p prime. If a ∈ Zp is zero (f (a) = 0) then f (x) ≡ (x − a)g(x)( mod p) for a pol. g of degree t − 1. At most t distinct zeros modulo p.

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 8 / 24

The Chinese remainder theorem

Let n1, n2, . . . , nk are pairwise relatively prime; n = n1n2 . . . nk. Then there is a natural one-to-one correspondence between Zn and Zn1 × Zn2 × . . . Znk (with addition and multiplication componentwise) a ↔ (a1, a2, . . . , ak) (ai ≡ a( mod ni)) a ≡ (a1c1 + a2c2 + . . . akck)( mod n) where ci ↔ (0, . . . , 0, 1, 0, . . . , 0) (ci uniquely determined) E.g. n = 4 · 5 · 9 = 180; c2 is the number s.t. c2 mod 4 · 9 = 0 (c2 ∈ {0, 36, 72, 108, 144}) and c2 mod 5 = 1, i.e. c2 = 36.

• C1. Corollary. If n1, n2, . . . , nk are pairwise relatively prime and

n = n1n2 . . . nk then the set of equations x ≡ a1( mod n1), x ≡ a2( mod n2), . . . , x ≡ ak( mod nk) has a unique solution modulo n.

• C2. Corollary. If n1, n2, . . . , nk are pairwise relatively prime and

n = n1n2 . . . nk then for all x, a we have x ≡ a( mod n1), x ≡ a( mod n2), . . . , x ≡ a( mod nk) iff x ≡ a( mod n) .

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 9 / 24

Powers of an element

Instead of considering 0a, 1a, 2a, 3a, . . . in Zn, we now consider a0, a1, a2, . . . in Z∗

n.

By a corollary of Langrange’s theorem we get Euler’s theorem aφ(n) ≡ 1( mod n) for all a ∈ Z∗

n

Fermat’s theorem for p prime, ap−1 ≡ 1( mod p) for all a ∈ Z∗

p = {1, 2, . . . , p − 1}

(Remark: also in the other direction: it is impossible that an−1 = 1 + kn when gcd(a, n) = d ≥ 2) If Z∗

n has a generator (also called a primitive root), it is called cyclic.

(A theorem says that Z∗

n (n > 1 is cyclic precisely for the values 2,4,pe,2pe

for odd prime p and any positive integer e.)

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 10 / 24

Discrete logarithm

indn,g(a) is z s.t. gz ≡ a( mod n) where g is generator of Z∗

n.

Discrete logarithm theorem: gx ≡ gy( mod n) iff x ≡ y( mod φ(n)) Theorem: if p is an odd prime and e ≥ 1, then the equation x2 ≡ 1( mod pe) has only two solutions, namely 1 and −1.

• Proof. Denote n = pe. (gindn,g (x))2 ≡ gindn,g (1)( mod n)

so 2 · indn,g(x) ≡ 0( mod φ(n)) (φ(n) = pe(1 − 1/p) = pe − pe−1 = pe−1(p − 1)) so gcd(2, φ(n)) = 2, so indn,g(x) can have exactly two values, so also x2 ≡ 1( mod pe) has exactly two solutions, namely +1 and −1. A number x is a nontrivial square root of 1 modulo n iff x2 ≡ 1( mod n) and x ≡ 1( mod n), x ≡ −1( mod n). (E.g. 62 ≡ 1( mod 35).) Corollary (useful for Miller-Rabin primality testing): If there is a nontrivial square root of 1 modulo n then n is composite.

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 11 / 24

Raising to powers with repeated squaring

To compute ab mod n; b in binary bkbk−1 · · ·1 b0 (invariant d = ac mod n; variable c used only for this invariant) c := 0; d := 1 for i := k downto 0 do c := 2c; d := d2 mod n; if bi = 1 then c := c + 1; d := d · a mod n (O(β) arithmetic operations, number of bit operations in O(β3).) Exerc.: Knowing φ(n), compute a−1 mod n (a ∈ Z∗

n) using

Modular-Exponentiation

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 12 / 24

Primality testing

Prime number theorem lim

n→∞

π(n) n/ ln n = 1 ln 10100 ≈ 230 (to approx. how quickly we can find a 100-digit random prime) Recall Fermat’s theorem, and consider (pseudoprimality testing): if Mod − Exp(2, n − 1) ≡ 1( mod n) then COMPOSITE (definitely) else PRIME (we hope) Surprisingly good (at random) but not for each number. Can we do with just adding Mod − Exp(3, n − 1) or so ? Or a random a ?

• No. E.g., Carmichael numbers (561, 1105, 1729, ...) satisfy an−1 ≡ 1(

mod n) for all a ∈ Z∗

n.

E.g. 561 = 3 · 11 · 17 φ(561) = 561(1 − 1

3)(1 − 1 11(1 − 1 17) = 320

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 13 / 24

The Miller-Rabin randomized primality test

• It tries several randomnly chosen base values a (not just a = 2)
• while computing modular exponentiation, it notices if a nontrivial square

root of 1 modulo n is discovered (then n is definitely composite) n − 1 in binary bkbk−1 · · ·1 b0 procedure Witness(a, n) (of compositeness of n) d := 1 for i := k downto 0 do x := d; d := d2 mod n; if d = 1 and x = 1 and x = n − 1 then return TRUE if bi = 1 then d := d · a mod n —– if d = 1 then return TRUE else return FALSE

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 14 / 24

Miller-Rabin

Miller − Rabin(n, s) for j := 1 to s do a := Random(1, n − 1); if Witness(a, n) then return COMPOSITE (surely) —- return PRIME (‘almost’ surely) O(sβ3) operations Theorem: If n is an odd composite number then the number of witnesses (a) is at least (n − 1)/2. Proof: we show that the number of nonwitnesses is at most (n − 1)/2. Every nonwitness a must be a member of Z∗

n (gcd(a, n) = 1). We show

that all nonwitnesses are contained in a proper subgroup B of Z∗

n; so (by

Langrange’s theorem) their number is less than |Z∗

n|/2, i.e., less than

(n − 1)/2. Case 1: ∃x ∈ Z∗

n s.t. xn−1 ≡ 1( mod n). Since B = {b ∈ Z∗ n | bn−1 ≡ 1(

mod n)} is closed under multiplication, it is a proper subgroup containing all nonwitnesses.

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 15 / 24

Miller-Rabin

Case 2 (Carmichael numbers) ∀x ∈ Z∗

n : xn−1 ≡ 1( mod n).

It can not be n = pe for e > 1: otherwise Z∗

n contains generator g s.t.

• rdn(g) = |Z∗

n| = φ(n) = (p − 1)pe−1. But then the discrete logarithm

theorem implies n − 1 ≡ 0( mod φ(n)), or (p − 1)pe−1 | pe − 1 (which is impossible for e > 1). So we can find n = n1n2 so that n1, n2 are relatively prime (and recall Chinese remainder theorem). Let n − 1 = 2tu where u is odd (n − 1 in binary finishes with t zeros). For any a (1, 2, . . . , n − 1) consider the last t + 1 steps of Mod − Exp(a, n − 1) ... au, a2u, a22u, . . . , a2tu. Note that (−1)20u ≡ −1( mod n); let j be the maximal ∈ {0, 1, . . . , t} so that there is v s.t. v 2ju ≡ −1( mod n); we fix such v. Let B = {x ∈ Z∗

n | x2ju ≡ +

− 1( mod n)}

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 16 / 24

Miller-Rabin

Case 2 (Carmichael numbers) - cont Every nonwitness must be a member of B, so it is sufficient to show that there is w ∈ Z∗

n − B. (Use of Chinese ...)

Since v 2ju ≡ −1( mod n), we have v 2ju ≡ −1( mod n1). There is w satisfying w ≡ v( mod n1) w ≡ 1( mod n2) Therefore w2ju ≡ −1( mod n1) w2ju ≡ 1( mod n2) This means that we can not have w2ju ≡ 1( mod n) neither w2ju ≡ −1( mod n). (by Corollary C2 of Chinese ...) So w ∈ B. Since v ∈ Z∗

n (gcd(v, n) = 1), we have gcd(v, n1) = 1, so gcd(w, n1) = 1,

so w ∈ Z∗

n.

——— (In fact, the number of nonwitnesses is at most (n − 1)/4, which happens for some numbers.)

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 17 / 24

RSA

To prove (xe)d ≡ x( mod n) for n = pq where ed ≡ 1( mod Φ(n)) (Φ(n) = (p − 1)(q − 1)) (p − 1) divides ed − 1, so (xed−1) · x = (xk(p−1)) · x Due to Fermat etc. (x(p−1))k · x ≡ x( mod p) Similarly we get (xe)d ≡ x( mod q) (and Chinese remainder theorem) Alternatively: via cyclic groups and discrete logarithm

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 18 / 24

Approximation algorithms - Vertex Cover Problem

(CLR, Hromkovic) Vertex cover problem: to find a minimal set of vertices such that all edges are incident with them (Remark: A is a clique in G = (V , E) iff V − A is a vertex cover in G = (V , E).) NP-hard (more precisely: the respective decision problem is NP-complete) 2-approximation algorithm for VCP: just construct a matching (a set of pairwise disjoint edges) (the nodes of included edges create the cover; during the process we eliminate covered edges)

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 19 / 24

Travelling Salesperson Problem

If P=NP then there is no polynomial d-approximation algorithm for TSP. (A reduction from Hamiltonian cycle problem) There is 2-approximation algorithm for ∆-TSP (triangle inequality): minimal spanning tree and depth first search

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 20 / 24

The set covering problem (in CLR)

E.g.: a set of skills to be covered; a set of people, each of which has some

• skills. Find a minimal ‘committee’ covering all skills. (Generalization of

vertex cover problem but the ‘matching’ approximation algorithm does not work.) A set X (of skills), a set F of subsets of X (of people) such that union of all members of F is X. A greedy heuristics gives a logarithmic ratio bound: Repeat: select a set from F (person) covering most of the sofar uncovered elements of X (skills)

• Approx. ratio bounded by H(max{|S|; S ∈ F})

where H(n) = n

i=1 1 i is the n-th harmonic number.

(Recall H(n) ≈ ln n [since 1

x dx = ln x ])

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 21 / 24

Proof of the approx. ratio

Algorithm gives the sets S1, S2, · · · = C; each new set adding cost 1 to the solution (whose cost is |C|). We spread this cost on the newly covered elements: each element x which is first covered by Si has cost cx = 1/| Si − (S1 ∪ S2 ∪ · · · ∪ Si−1) |. So |C| =

x∈X cx; this is ≤ S∈C∗

• x∈S cx (where C∗ is an optimal

solution). So it suffices to show that ∀S ∈ F :

x∈S cx ≤ H(|S|)

(Then |C| ≤ |C∗| · H(max{|S|; S ∈ F})) So fix a S ∈ F and let ui = | S − (S1 ∪ S2 ∪ · · · ∪ Si) |. Due to the greedeness: ui−1 ≤ | Si − (S1 ∪ S2 ∪ · · · ∪ Si−1) |. Let k be the smallest s.t. uk = 0

• x∈S cx = k

i=1(ui−1 − ui) · 1 | Si−(S1∪S2∪···∪Si−1) | ≤ k i=1(ui−1 − ui) · 1 ui−1

Since (b − a) · 1

b ≤ b i=a+1 1 i = H(b) − H(a), we have

• x∈S cx ≤ k

i=1(H(ui−1) − H(ui))

Telescoping sum ... = H(u0) − H(0) = H(u0) = H(|S|))

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 22 / 24

Subset sum problem

A fully polynomial-time approximation scheme

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 23 / 24

Petr Janˇ car (TU Ostrava) Selected Topics of TCS 2005/2006 24 / 24