Probabilistic Computation CSC5802, Computational Complexity Derek - - PowerPoint PPT Presentation

Probabilistic Computation

CSC5802, Computational Complexity Derek Kern December 11th, 2010

What will we be doing?

 We will be exploring the field of probabilistic computation  We'll see:

−

Examples

−

Turing machines



Deterministic



Nondeterministic



Probabilistic

−

Probabilistic Complexity Classes



RP/co-RP



ZPP



BPP



PP

−

Probabilistic Class Relationships

PRIME(x)

 The decision problem PRIME(x): Given an integer x, is x in

the set of prime integers?

 The Sieve of Eratostenes – First known solution

− Given an integer x whose primality is in question, write down all of

the integers i, 1 < i ≤ x. Starting with the smallest integer in the list q (at the beginning, this will always be 2), remove from the list all multiples of 2. Next, remove from the list multiples of the next remaining integer q in the list; repeat this step until q' > x or until x is removed from the list. If at any point, x is removed from the list, then x is composite; otherwise, it is prime.

− Unfortunately, Eratostenes' Sieve is exponential in the length of

its input

 For each q whose multiples must be removed, this q is a

prime number. So, for each prime q less than n, the leftover list entries must be traversed; the number of these entries is on the order of n.

PRIME(x)

 Yet, PRIME(x) is known to be in P [AKS2004]

− PRIME(x) is O(n log6 n)

 Prior to 2004, the best known algorithms for solving PRIME

were probabilistic

 These probabilistic algorithms have been used for prime

testing in a variety of applications (e.g. encryption)

− Many encryption algorithms need prime numbers to

• perate. Thus, in order to generate a prime number,

they choose a number within a range and then test its primality

Solovay/Strassen

 Let's look at the Solovay/Strassen probabilistic algorithm for

PRIME(x) (actually, for COMPOSITE)

 Quick note

− Euler's Criterion (EC): Given odd prime p and base

integer a, a(p-1)2 ≡ (a | p)(mod p)

− This congruence holds for all odd primes p over all

• bases. It also holds for some odd composites over

some bases a; these bases are called 'Euler liars'

− For any odd composite c and bases a < c, half of these

bases are liars and half are not

 The algorithm – Give an integer to test q

− Randomly choose base a where 2 < a < q − If EC does not hold for a and q, then return

“COMPOSITE”

− If EC holds, return “PROBABLY PRIME”

Solovay/Strassen

 Given a q and a, imagine that we received the answer

“PROBABLY PRIME”

− What do we do?

 Since only ½ the bases less than q and greater than 2 are

liars, we repeat by randomly choosing another base

− Each time we iterate, our confidence in the answer

“PROBABLY PRIME” grows.

− In fact, if we repeat the process k times (receiving

“PROBABLY PRIME” each time), then the chance that q isn't prime is 2-k

− Somewhere around k ≈ 20, our confidence in the

primality of q would be bounded by our confidence in the functioning of the computer

Solovay/Strassen

 Given that an iteration of SS runs in O(n), iterating SS

constant number of times provides us with a practical method for testing the primality of a given integer to a specific tolerance in polynomial-time

 The Solovay/Strassen algorithm is the complexity class RP

(a.k.a. Randomized Polynomial-time). We'll see RP later...

Turing Machinesrevisited

 Reminder: A deterministic TM (DTM) M is defined by the

quadruple (K, Σ, δ, s) [CP1994]

− K is a finite set of states − Σ is a finite alphabet (typically {1, 0, #, b}) − δ is a transition function. It maps K x Σ to K x Σ x {m:

tape head movement = m}

− s is a member of K. It is the initial state

 Reminder: A nondeterministic TM (NDTM) is defined by the

quadruple (K, Σ, δ, s) [CP1994]

− Essentially, a NDTM is the same as a DTM, except that

δ is not a transition function but a relation

− Since a relation allows many different outcomes given

a K x Σ combination, an oracle is used to guide computation

Turing Machinesrevisited

 So, what about Probabilistic Turing Machines? How are they

defined?

 A probabilistic TM (PTM) M is defined by the quintuple (K, Σ,

δ1, δ2, s)

− Note that a PTM has two transition functions: δ1 and δ2 − At each state, a PTM randomly chooses whether to

use δ1 or δ2

− Taken together, δ1 and δ2 amount to a relation where

the path choice is determined randomly

− For an actual PTM, δ1 and δ2 will often have most of

their transitions in common, which means that random choice will only affect a few transitions

Turing Machinesrevisited

 How do PTMs relate to DTMs?

− All DTMs are PTMs since we can simply let δ1 = δ2 − PTMs can also be defined in deterministic terms by

giving them an extra-tape whose cell values have been randomized

 How do PTMs relate to NDTMs?

− The class relationships between PTMs and NDTMs are

no more settled than those between DTMs and NDTMs

− As is evidenced by P ?= NP, L ?= NL and EXPTIME ?=

NEXPTIME, we do not understand whether non- determinism truly allows us to overcome intractability Thus, this issue will likely remain unsettled until we know the status of P ?= NP

Gamble anyone?

 Wisdom from Papadimitriou [CS1994]

– “In some very real sense, computation is inherently randomized. It can be argued that the probability that a computer will be destroyed by a meteorite during any given microsecond its operation is at least 2-100.”

Monte Carlo vs Las Vegas

 There are two general types of probabilistic algorithms

– Monte Carlo Algorithms [ER2009]

• These algorithms always run efficiently
• However, they can, at times, return an incorrect

answer, i.e. false positives or false negatives

• The probability of an incorrect answer is known

– Las Vegas Algorithms

• These algorithms always return correct answers;

no false positives or false negatives

• They may not run efficiently
• The probability of ineffecient operation is known

 Solovay/Strassen was a Monte Carlo algorithm. It always

runs efficiently but may return an incorrect answer

 Las Vegas algorithms were intially used for graph

isomorphism testing

Background: Error Reduction

 Question for the class: Which is of more practical use?

− Choice #1: An algorithm that returns YES or NO such

that there is a 0% chance of a false positive and a 51% chance of a false negative. This algorithm is O(n).

− Choice #2: An algorithm that returns YES or NO such

that there is a 0% chance of a false positive and a 8% chance of a false negative. This algorithm is O(n2)

 Let k be the number of iterations, the probability of error over

k-iterations is: (1 – ϵ)k [CP1994]. So, error diminishes very quickly as we iterate (and receive confirming answers)

 If we iterate Choice #1 for k = 4 iterations, then the chance of

false positive is ~6%. Furthermore, since k is a constant, Choice #1 is still in O(n)

 Thus, Choice #1 is better

Background: Error Reduction

 So, the lesson is: Even though many of the probabilistic

complexity classes include error bounds in their definitions, don't get too focused on the precise error bounds [CS1994] – Given a randomized algorithm A with a chance of incorrectness of 1 – ϵ, we can always create an A' that simple iterates A so that the error bounds of A' fits within prescribed error bounds

RP

 Also known as Randomized Polynomial  Essentially, RP is a class of languages accepted by Monte

Carlo algorithms

 Definition of RP

– Given a language L and PTM M

• If x is in L, then the probability that M accepts x is

greater than or equal to ½

• If x is not in L, then the probability that M accepts

x is zero – In other words, we can get false negatives when determining membership in an RP language – RP contains all languages L for which there is a PTM M that behaves as described above

 Remember Solovay/Strassen? It shows that PRIME(x) is in

RP

co-RP

 Complement of RP  Like RP, co-RP is a class of languages accepted by Monte

Carlo algorithms

 Definition of co-RP

– Given a language L and PTM M

• If x is in L, then the probability that M accepts x is
• ne
• If x is not in L, then the probability that M accepts

x is less than ½ – In other words, we can get false positives when determining membership in an co-RP language – co-RP contains all languages L for which there is a PTM M that behaves as described above

ZPP

 Also known as Zero-error Probabilistic Polynomial Time  Essentially, ZPP is the class of languages accepted by Las

Vegas algorithms

 Definition of ZPP [ER2009]

– Given a language L and PTM M

• If x is in L, then M accepts with probability one
• If x is not in L, then M rejects with probability one
• There is some polynomial function p(n) such that

for all inputs w and |w| = n the expected running time is less than p(n). On some random

• ccasions, M may run longer than p(n); on these
• ccasions M halts and returns 'UNKNOWN'

 So, for algorithms that accept languages in ZPP, YES/NO

answers are always accurate. However, there is a chance that we won't get an answer

ZPP

 ZPP = RP ∩ co-RP ← ZPP is sometimes defined in this way

– This definition is equivalent to the previous definition – Proof

• L in ZPP → L in RP ∩ co-RP

– Let MZPP, MRP and Mco-RP be PTMs – Given input w, MRP runs MZPP – If MZPP returns YES/NO, then MRP returns YES/NO – If MZPP returns UNKNOWN, then MRP rejects – Given that probability that MRP accepts is > ½ and that the running time is bounded by p(n), L must be in RP – Equivalent logic holds for Mco-RP

• L in RP ∩ co-RP → L in ZPP

ZPP

 ZPP = RP ∩ co-RP

– Proof (cont'd)

• L in RP ∩ co-RP → L in ZPP

– Since L is in RP and co-RP, there are PTMs MRP and Mco-RP that accept L – Let MZPP be a PTM – MZPP executes MRP. If MRP returns YES, then MZPP halts and returns YES – MZPP executes Mco-RP. If Mco-RP returns NO, then MZPP halts and returns NO – If Mco-RP returns YES, then MZPP trys again with MRP/Mco-RP – If after p(n) iterations, MZPP has not halted then it will return UNKNOWN

• Since L in ZPP → L in RP ∩ co-RP and L in RP ∩ co-RP

→ L in ZPP, ZPP = RP ∩ co-RP

BPP

 Also known as Bounded-error Probabilistic Polynomial  Definition of BPP

– Given a language L and PTM M

• If x is in L, then the probability that M accepts x is

greater than ½

• If x is not in L, then the probability that M rejects x

is greater than ½ – BPP contains all languages L for which there is a PTM M that behaves as described above

PP

 Also known as Probabilistic Polynomial  Definition of PP

– Given language L and PTM M

• A string x is in L iff more than half of the

computations of M are accepting computations

• Thus, x is in L iff a majority of computations,

however slim the majority, end up accepting

 Consider the language MAJSAT [CS1994]

– A wff is in MAJSAT iff a majority of its 2n truth assignments are satisfying – It is hard to imagine that this problem is in NP given that 2n-1 + 1 assignments must be satisfying. How could a certificate for this be verified in polynomial-time?

 MAJSAT is PP-complete

PP

 Think back to reducing error. We could improve our error

bounds by simply iterating

 Would this work for PP?  Another way to think of PP is: [WikiPP2010]

– Given that n = |w| – For a YES, YES is true with probability ½ + ½n

 So, the error probability depends, to some extent, on the size

• f the input. For RP, BPP, etc, the error probability depended

upon constant factors

 Thus, if n = 10, for example, then the error probability would

be something like 0.500000000233, which would require a huge number of iterations to improve. In fact, in general, to improve the error bounds of PP problems requires a number

• f iterations that is exponential in the size of input

Background Syntactic vs. Semantic Classes

 Question: Given an arbitrary language L in NP, how could

we determine whether it belongs in P?

 Answer: It belongs in P if there is a TM M that accepts L in

polynomial-time

 Question: Given an arbitrary language L in PP, how could

we determine whether it belongs in RP?

 Answer: It belongs in RP if there is a PTM M that rejects

non-members of L with no error and accepts members of L with error less than ½

 Question: Since L is arbitrary, M is arbitrary. How would we

know that M behaves as we would expect a machine that accepts a language in RP?

Background Syntactic vs. Semantic Classes

 Answer: We cannot. This question is undecidable.

– Given an evil adversary and an infinite set whose membership is to be determined, there is no point where we could stop and say, “We know that probability that M will accept the rest of this infinite set will match the its acceptance distribution thus far.” Our evil adversary could have been trying to fool us thus far.

 In general, there is no way for us to know the output

behavior of an arbitrary TM over all its possible inputs

 RP, co-RP and BPP are semantic classes. P, NP and PP

are syntactic classes

 For syntactic classes, we can superficially examine TMs to

determine whether they define contained languages

 For semantic classes, no such superficial check can be

performed [CS1994]

Background Syntactic vs. Semantic Classes

 Some complexity classes do not have complete problems

[CS1994]

 Semantic classes do not have complete problems

– Why? – Because, unlike syntatic classes, like NP, there is no way to speak about arbitrary members of the class

 So, in short, there are no, for example, BPP-complete

problems

 I like to think of the issue this way

– We defined these languages in terms of members of languages (x in L or x not in L) and chances that a PTM accepts these members as such – But wait, aren't TMs supposed to settle this for us in the first place? RP, BPP, etc have this detail reversed – There is a fact of the matter *and* there is the degree to which these PTMs get these facts correct

Class Relationships/Properties

 P is a subset of RP

− This is trivial given that PTMs can compute any

language that can be computed by DTMs

 P is subset of co-RP

− Same reasoning as above

 RP is a subset of BPP

− Given a PTM M that decides language L in RP, we can

simply iterate M until the probability of false negatives is less than the BPP error bound. Note, for any language in RP, the probability of false positives is zero

 co-RP is a subset of BPP

− Mutatis mutandis, see the reasoning for RP's

containment in BPP

Class Relationships/Properties

 P is subset of BPP

– P is a subset of RP and RP is a subset of BPP

 P is a subset of ZPP

– Remember ZPP = RP ∩ co-RP – Proof by contradiction

• [1] Assume P is not a subset of ZPP
• [2] P is a subset of RP and P is a subset of co-RP
• [3] By [1], there must be an x in P that is not in ZPP
• [4] By [3] and the definition of ZPP, there must be an x in

P that is not in RP ∩ co-RP

• [5] By [4], x is not in RP or x is not in co-RP
• [6] By [3] & [5], x is in P and ( x is not in RP or x is not in

co-RP )

• [7] By [2] & [6], we have a contradiction. P is subset of

RP and co-RP so x must be in both RP and co-RP; but, by [6], x cannot be in both RP and co-RP.

• Thus, P must be a subset of ZPP

Class Relationships/Properties

 BPP is a subset of PP

− This is trivial given that the error bounds for BPP fit

easily within those for PP

 BPP is a subset of EXPTIME [AB2007]

− Given a language L in BPP and PTM M that accepts L,

we can enumerate all random choices of M in time 2p(n) and compute the probability that M(w) = 1

 RP is a subset of NP [AB2007]

− We know this since an accepting computation for a

language L in RP is also a polynomial-time-verifiable qualifying certificate for L being in NP

 co-RP is a subset of co-NP

− We know this since a rejecting computation for a

language L in co-RP is also a polynomial-time- verifiable disqualifying certificate for L being in co-NP

Class Relationships/Properties

 PP is a subset of PSPACE

− Consider MAJSAT, which as we've seen is in PP − In fact, MAJSAT is PP-complete − We know that polynomially-bounded TMs can only use

polynomial amount of space

− Given a PTM M that accepts language L in PP, we

know that each computation of M is bounded- polynomially in both space and time

− For each computation, we can continue to reuse the

same space and accumulate the count of accepting computations

− Thus, the space to accept MAJSAT is polynomially-

bounded

− Therefore, if L is in PP, then L is in PSPACE

Class Relationships/Properties NP is a subset of PP

 Proof [CS1994]

− Let language L be in NP and be decided by NDTM MNP.

Let MPP be a PTM. MPP is the same as MNP except it has an initial state that chooses (randomly and evenly) between two paths.

− On the first path, MPP simulates MNP. On the second

path, it simply accepts

− So, half the time, MPP will accept by the second path. − If MPP accepts at least once via the first path, i.e. if MNP

accepts, then a majority of its computations will have been accepting.

− Thus, x is in L if x is in MPP(x) − Therefore, L is in NP if L is in PP and NP is a subset of

PP

Class Relationships/Properties PP closed under complementation

 Proof [WikiPP2010]

− Let L be a language in PP and M be the PTM that

accepts it

− Let ~L be the complement of L − Let ~M be a PTM that simulates M, but rejects

whenever M accepts and vice versa

− This seems to capture everything but does it? Does

~M as currently defined accept all of ~L?

− No, even splits are also a part of ~L. What do we do

with these? We introduce a very slight asymmetry

− Well, remember that languages in PP accept with

probability very slightly greater than ½

Class Relationships/Properties PP closed under complementation

 Proof

− We revise the acceptance criteria for ~M − Let this machine be called ~M' − ~M' rejects when ~M rejects − ~M' accepts when ~M accepts contingent upon a p(n)

number of coin flips. If all of these coin flips land on tails, then ~M' rejects; otherwise it accepts.

− In a nutshell, what does ~M' do?

 The coin flip is meant to ensure no even split

between acceptance/rejection by making the possibility of accepting an x that is not in L strictly less than ½

 The number of coin flips must be large enough such

that the probability of acceptance remains greater than ½

 Thus, both L and ~L are in PP

Class Relationships/Properties Open Questions

 BPP is a subset of NP?  NP is a subset of BPP?  We know that BPP is a subset of both EXPTIME and, thus,

• NEXPTIME. However, we do not know whether BPP is a

strict subset of either. In other words, for all that we know BPP = NEXPTIME

 Is NP a strict subset of PP?

− We think so given that a polynomial-time verifiable

certificate for MAJSAT is hard to imagine

Questions

?

References

 [CS1994] C. Papadimitiou, 1994. Computational Complexity.

Addison-Wesley.

 [AB2007] S. Arora, B. Barak, Computational Complexity: A Modern

• Approach. Draft.

 [AKS2004] M. Agrawal, N. Kayal, N. Saxena, "PRIMES is in P",

Annals of Mathematics 160 (2004), no. 2, pp. 781–793.

 [ER2009] E. Rich, 2009. Automata, Computability, and Complexity

Theory and Applications. Pearson Education, Inc.

 [WikiSS2010] Solovay-Strassen Primality Test.

http://en.wikipedia.org/w/index.php?title=Solovay %E2%80%93Strassen_primality_test&oldid=399157509

 [WikiPP2010] PP (complexity).

http://en.wikipedia.org/wiki/PP_(complexity)