for Planted Clique Part I Lecture Outline Part I: Planted Clique - - PowerPoint PPT Presentation

Lecture 12: SOS Lower Bounds for Planted Clique Part I

Lecture Outline

• Part I: Planted Clique and the Meka-Wigderson

Moments

• Part II: MPW Analysis Preprocessing
• Part III: MPW Analysis with Graph Matrices
• Part IV: The Pessimist Strikes Back

Part I: Planted Clique and the Meka-Wigderson Moments

Review: Planted Clique

• Recall the planted clique problem: Given a

random graph 𝐻 where a clique of size 𝑙 has been planted, can we find this planted clique?

• Variant we’ll analyze: Can we use SOS to prove

that a random 𝐻 𝑜, 1

2 graph has no clique of

size 𝑙 where 𝑙 ≫ 2𝑚𝑝𝑕𝑜 (the expected size of the largest clique in a random graph)?

Review: Planted Clique Equations

• Variable 𝑦𝑗 for each vertex i in G.
• Want 𝑦𝑗 = 1 if i is in the clique.
• Want 𝑦𝑗 = 0 if i is not in the clique.
• Equations:

𝑦𝑗

2 = 𝑦𝑗 for all i.

𝑦𝑗𝑦𝑘 = 0 if 𝑗, 𝑘 ∉ 𝐹(𝐻) σ𝑗 𝑦𝑗 ≥ 𝑙

• Theorem [MPW15]: ∃𝐷 > 0 such that whenever

𝑙 ≤ 𝐷𝑒

𝑜 𝑚𝑝𝑕𝑜 2

1 𝑒, with high probability degree

𝑒 SOS cannot prove the 𝑙-clique equations are infeasible.

First SOS Lower Bound

• To prove an SOS lower bound:
• 1. Come up with pseudo-expectation values ෨

𝐹 which

• bey the required linear equations
• 2. Show that the moment matrix 𝑁 is PSD

Review: SOS Lower Bound Strategy

• Idea: Give each 𝑒-clique the same weight
• Define 𝑦𝐽 = ς𝑗∈𝐽 𝑦𝑗
• Define 𝑂𝑒(𝐽) to be the number of 𝑒-cliques

containing 𝐽.

• MW moments: take ෨

𝐹 𝑦𝐽 =

𝑙 |𝐽| 𝑒 |𝐽|

⋅

𝑂𝑒(𝐽) 𝑂𝑒(∅)

MW Moments

• MW moments: take ෨

𝐹 𝑦𝐽 =

𝑙 |𝐽| 𝑒 |𝐽|

⋅

𝑂𝑒(𝐽) 𝑂𝑒(∅)

• MW moments obey the equation σ𝑗 𝑦𝑗 = 𝑙
• Proof: σ𝑗∉𝐽 𝑂𝑒(𝐽 ∪ 𝑗) = 𝑒 − 𝐽 𝑂𝑒(𝐽) as each

d-clique containing 𝐽 contains 𝑒 − |𝐽| of the 𝑗 ∉ 𝐽

• 𝑙

𝐽 +1 𝑒 𝐽 +1

= 𝑙−|𝐽|

𝑒−|𝐽| ⋅

𝑙 |𝐽| 𝑒 |𝐽|

• σ𝑗 ෨

𝐹 𝑦𝐽∪𝑗 = 𝐽 ෨ 𝐹 𝑦𝐽 + 𝑙 − 𝐽 ෨ 𝐹 𝑦𝐽 = 𝑙 ෨ 𝐹 𝑦𝐽

Checking σ𝑗 𝑦𝑗 = 𝑙

Part II: MPW Analysis Preprocessing

Analysis Outline

• For the MPW analysis, we do the following:
• 1. Preprocess the moment matrix 𝑁 to make it

easier to analyze. More specifically, we find a matrix 𝑁′ which is easier to analyze such that if 𝜇min 𝑁′ ≥

𝑙

𝑒 2

4𝑜

𝑒 2

then 𝑁 ≽ 0 with high probability

• 2. Decompose 𝑁′ = 𝐹 𝑁′ + 𝑆 and show that

𝐹 𝑁′ ≽

𝑙

𝑒 2

2𝑜

𝑒 2

𝐽𝑒 and w.h.p., 𝑆 ≤

𝑙

𝑒 2

4𝑜

𝑒 2

Restriction to Multilinear, Degree

𝑒 2

• Preprocessing Step #1: As we’ve seen from the

3XOR and knapsack lower bounds, since we have the constraints that 𝑦𝑗

2 = 𝑦𝑗 for all 𝑗 and

σ𝑗 𝑦𝑗 = 𝑙, it is sufficient to consider the submatrix of 𝑁 with multilinear, degree 𝑒

2

indices

Approximating ෨ 𝐹[𝑦𝐽]

• Preprocessing Step #2: Approximate ෨

𝐹[𝑦𝐽]

• Intuition: One view of ෨

𝐹[𝑦𝐽] is that ෨ 𝐹[𝑦𝐽] is the expected value of 𝑦𝐽 given what we can compute.

• Remark: This is connected to pseudo-

calibration/moment matching which we’ll see next lecture.

Approximating ෨ 𝐹[𝑦𝐽] Continued

• A priori, if we choose a clique of size 𝑙 at

random, |𝐽| is part of the clique with probability

𝑙 |𝐽| 𝑜 |𝐽|

≈

𝑙|𝐽| 𝑜|𝐽|

• If 𝐽 is not a clique, ෨

𝐹 𝑦𝐽 = 0. If 𝐽 is a clique, 𝐽 is 2

|𝐽| 2 times more likely to be part of the clique.

Thus, ෨ 𝐹 𝑦𝐽 ≈ 2

|𝐽| 2

𝑙|𝐽| 𝑜|𝐽| if 𝐽 is a clique and is 0

• therwise.
• See appendix for calculations confirming this.

Approximation Error

• Let 𝑁𝑏𝑞𝑞𝑠𝑝𝑦 be the matrix where

𝑁𝑏𝑞𝑞𝑠𝑝𝑦 𝐽𝐾 = 2

|𝐽∪𝐾| 2

𝑙|𝐽∪𝐾| 𝑜|𝐽∪𝐾| if 𝐽 ∪ 𝐾 is a clique

and 𝑁𝑏𝑞𝑞𝑠𝑝𝑦 𝐽𝐾 = 0 otherwise.

• Can show that the difference Δ = M − M𝑏𝑞𝑞𝑠𝑝𝑦

is small (see [MPW15] for details).

The matrix 𝑁′

• Preprocessing Step #3: Fill in zero rows and

columns of 𝑁𝑏𝑞𝑞𝑠𝑝𝑦

• If 𝐽 or 𝐾 is not a clique then (𝑁𝑏𝑞𝑞𝑠𝑝𝑦)𝐽𝐾= 0.
• These zero rows and columns make 𝑁𝑏𝑞𝑞𝑠𝑝𝑦

harder to analyze.

• Definition: Take 𝑁′ to be the matrix such that

𝑁′𝐽𝐾 = 2

|𝐽∪𝐾| 2

𝑙|𝐽∪𝐾| 𝑜|𝐽∪𝐾| if all edges are present

between 𝐽 ∖ 𝐾 and 𝐾 ∖ 𝐽 and 𝑁′𝐽𝐾 = 0 otherwise

𝑁′ ≽ 0 ⇒ 𝑁𝑏𝑞𝑞𝑠𝑝𝑦 ≽ 0

• Can view 𝑁𝑏𝑞𝑞𝑠𝑝𝑦 as a submatrix of 𝑁′.
• This immediately implies that if 𝑁′ ≽ 0 then

𝑁𝑏𝑞𝑞𝑠𝑝𝑦 ≽ 0

• Because of the error matrix Δ = M − M𝑏𝑞𝑞𝑠𝑝𝑦

we need the stronger statement that with high probability, 𝜇min 𝑁′ is significantly bigger than 0.

Summary

• We want to show that w.h.p. 𝑁′ ≽

𝑙

𝑒 2

4𝑜

𝑒 2

where 𝑁′ is the matrix such that 𝑁′𝐽𝐾 = 2

|𝐽∪𝐾| 2

𝑙|𝐽∪𝐾| 𝑜|𝐽∪𝐾| if

all edges are present between 𝐽 ∖ 𝐾 and 𝐾 ∖ 𝐽 and 𝑁′𝐽𝐾 = 0 otherwise

𝑁′ Picture for d = 4

12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 12 13 14 15 16 23 24 25 26 34 35 36 45 46 56

𝑁′ 𝑗,𝑘 {𝑗,𝑙} =

8𝑙3 𝑜3 if

j ∼ 𝑙 and 0

• therwise

𝑁′ 𝑗,𝑘 {𝑗,𝑘} =

2𝑙2 𝑜2

𝑁′ 𝑗,𝑘 {𝑙,𝑚} =

64𝑙4 𝑜4 if

𝑗 ∼ 𝑘, 𝑗 ∼ 𝑙, 𝑘 ∼ 𝑙, 𝑘 ∼ 𝑚 and is 0

• therwise

Part III: MPW Analysis with Graph Matrices

Recall Definition of 𝑆𝐼

• Definition: Definition: If 𝑊 𝐼 = 𝑉 ∪ 𝑊 then

define 𝑆𝐼 𝐵, 𝐶 = 𝜓𝜏(𝐹(𝐼)) where 𝜏: V H → 𝑊(𝐻) is the injective map satisfying 𝜏 𝑉 = 𝐵, 𝜏 𝑊 = 𝐶 and preserving the ordering of 𝑉, 𝑊.

• Last lecture: Did not require 𝐵, 𝐶 to be in

ascending order.

• This lecture: Will require 𝐵, 𝐶 to be in

ascending order.

• Note: This only reduces our norms, so the

probabilistic norm bounds still hold.

Review: Rough Norm Bound

• Theorem [MP16]: If 𝐼 has no isolated

vertices then with high probability, 𝑆𝐼 is ෨ 𝑃 𝑜( 𝑊 𝐼 −𝑡𝐼)/2 where 𝑡𝐼 is the minimal size of a vertex separator between 𝑉 and 𝑊 (S is a vertex separator of U and V if every path from U to V intersects S)

• Note: The ෨

𝑃 contains polylog factors and constants related to the size of 𝐼.

Decomposition of 𝑁𝑏𝑞𝑞𝑠𝑝𝑦 and 𝑁′

• Claim: 𝑁𝑏𝑞𝑞𝑠𝑝𝑦 = σ𝐼

𝑙|𝑉∪𝑊| 𝑜|𝑉∪𝑊| 𝑆𝐼 where we

sum over 𝐼 which have no middle vertices.

• Claim: 𝑁′ = σ𝐼 2

|𝑉| 2

+ |𝑊|

2

− |𝑉∩𝑊|

2

𝑙|𝑉∪𝑊| 𝑜|𝑉∪𝑊| 𝑆𝐼

where we sum over 𝐼 which have no middle vertices and which have no edges within 𝑉 or within 𝑊.

• Idea: Each of the 2

|𝑉| 2

+ |𝑊|

2

− |𝑉∩𝑊|

2

edges within 𝑉 or 𝑊 are given for free.

Entries of E[𝑁′]

• 𝑁′ = σ𝐼 2

|𝑉| 2

+ |𝑊|

2

− |𝑉∩𝑊|

2

𝑙|𝑉∪𝑊| 𝑜|𝑉∪𝑊| 𝑆𝐼 where

we sum over 𝐼 which have no middle vertices and which have no edges within 𝑉

• r within 𝑊.
• Claim: E 𝑁′ 𝐽𝐾 = 2

|𝐽| 2 + |𝐾| 2 − |𝐽∩𝐾| 2

𝑙|𝐽∪𝐾| 𝑜|𝐽∪𝐾|

• Idea: For any 𝐼 which has an edge,

𝐹 𝑆𝐼 = 0. Otherwise, 𝐹 𝑆𝐼 = 𝑆𝐼

𝐹[𝑁′] Picture for d = 4

12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 12 13 14 15 16 23 24 25 26 34 35 36 45 46 56

𝐹[𝑁′] 𝑗,𝑘 {𝑗,𝑙} =

4𝑙3 𝑜3

𝐹[𝑁′] 𝑗,𝑘 {𝑗,𝑘} =

2𝑙2 𝑜2

𝐹[𝑁′] 𝑗,𝑘 {𝑙,𝑚} =

4𝑙4 𝑜4

Analysis of 𝐹[𝑁′]

• 𝐹[𝑁′] belongs to the Johnson Scheme of

matrices 𝐵 whose entries 𝐵𝐽𝐾 only depend on |𝐽 ∩ 𝐾| (See Lecture 9 on SOS Lower Bounds for Knapsack)

• Can decompose 𝐹 𝑁′ as a sum of PSD

matrices, one of which is the identity matrix which has coefficient ≥

𝑙

𝑒 2

2𝑜

𝑒 2

𝐽𝑒.

One Piece of 𝑁′ − 𝐹[𝑁′] (𝑒 = 4)

12 13 14 15 16 23 24 25 26 34 35 36 45 46 56 12 13 14 15 16 23 24 25 26 34 35 36 45 46 56

60𝑙4 𝑜4 if all edges

between 𝐽 and 𝐾 are present. −

4𝑙4 𝑜4 otherwise

Piece of 𝑁′ − 𝐹[𝑁′] Decomposition

• This piece has coefficient

4𝑙4 𝑜4 in 𝑆𝐼 for all 𝐼

which have the following form (and 0 for all

• ther 𝑆𝐼):

𝑉 𝑣1 𝑣2 𝑊 𝑤1 𝑤2

Where 𝐹(𝐼) is non-empty and is a subset of the dashed lines

Piece of 𝑁′ − 𝐹[𝑁′] Analysis

• All 𝐼 here have minimum separator size 𝑡𝐼 at

least 1.

• This gives a norm bound of ෨

𝑃

𝑙4 𝑜4 ⋅ 𝑜

4−1 2

= ෨ 𝑃

𝑙2 𝑜 ⋅ 𝑙2 𝑜2

• This is much less than

𝑙2 4𝑜2 when 𝑙 ≪ 𝑜

1 4.

General Analysis of 𝑆 = 𝑁′ − 𝐹[𝑁′]

• Define 𝑆 = 𝑁′ − 𝐹[𝑁′]
• Claim: 𝑆 = σ𝐼 2

|𝑉| 2

+ |𝑊|

2

− |𝑉∩𝑊|

2

𝑙|𝑉∪𝑊| 𝑜|𝑉∪𝑊| 𝑆𝐼

where we sum over 𝐼 which have no middle vertices, which have no edges within 𝑉 or within 𝑊, and which have at least one edge.

• 𝑆 = σ𝐼 2

|𝑉| 2

+ |𝑊|

2

− |𝑉∩𝑊|

2

𝑙|𝑉∪𝑊| 𝑜|𝑉∪𝑊| 𝑆𝐼 where we

sum over 𝐼 which have no middle vertices, which have no edges within 𝑉 or within 𝑊, and which have at least one edge

• Norm bound: For any such 𝑆𝐼, w.h.p. 𝑆𝐼

is ෨ 𝑃(𝑜

𝑉∪𝑊 − 𝑉∩𝑊 −1 2

) as the minimal separator size 𝑡𝐼 between 𝑉 and 𝑊 is at least 𝑉 ∩ 𝑊 + 1

• Corollary: w.h.p. 𝑙|𝑉∪𝑊|

𝑜|𝑉∪𝑊| 𝑆𝐼 is ෨

𝑃

𝑙 𝑉∪𝑊 𝑜 𝑉∪𝑊 + 𝑉∩𝑊 +1

General Analysis of 𝑆 = 𝑁′ − 𝐹[𝑁′]

• 𝑆 is a sum of terms which w.h.p. have norm

෨ 𝑃

𝑙 𝑉∪𝑊 𝑜 𝑉∪𝑊 + 𝑉∩𝑊 +1

• 𝑉 ∪ 𝑊 ≤ 𝑒 and 𝑉 ∪ 𝑊 + 𝑉 ∩ 𝑊 = 𝑒, so

w.h.p. 𝑆 is ෨ 𝑃

𝑙

𝑒 2

𝑜

𝑒 2

⋅

𝑙

𝑒 2

𝑜 . This is much less than 𝑙

𝑒 2

4𝑜

𝑒 2

as long as 𝑙 ≪ 𝑜

1 𝑒

General Analysis of 𝑆 = 𝑁′ − 𝐹[𝑁′]

Part IV: The Pessimist Strikes Back

Limitations of MW moments

• Can we prove a stronger lower bound with the

MW moments?

• With a more careful analysis, a slightly stronger

lower bound can be shown. For 𝑒 = 4, [DM15] proved an ෩ Ω(𝑜

1 3) lower bound. [HKPRS16]

generalized this to ෩ Ω(𝑜

2 𝑒+2)

• By an argument of Jonathan Kelner, this is tight!

Pessimist’s Query

• Kelner’s argument: Pessimist can query the

following polynomial:

• Take 𝑞 = 𝐷𝑦𝑗 − σ

𝐾: 𝐾 =𝑒

2,𝑗∉𝐾 −1

𝐾∖𝑂 𝐽 𝑦𝐾 where

𝑂(𝐽) is the set of neighbors of 𝐽

• What is ෨

𝐹 𝑞2 ?

• Key idea: Cross terms will all be negative, but

there will be cancellation in the square terms.

Pessimist’s Query Analysis

• 𝑞 = 𝐷𝑦𝑗 − σ

𝐾: 𝐾 =𝑒

2,𝑗∉𝐾 −1

𝐾∖𝑂 𝑗 𝑦𝐾 where

𝑂(𝑗) is the set of neighbors of 𝐽 𝑞2 = 𝐷2𝑦𝑗 − 2𝐷 σ𝐾:𝐾∪{𝑗} 𝑗𝑡 𝑏 𝑑𝑚𝑗𝑟𝑣𝑓 𝑦𝐾∪{𝑗} + σ𝐾,𝐾′ −1

(𝐾Δ𝐾′)∖𝑂 𝐽 𝑦𝐾∪𝐾′

• We expect ෨

𝐹[𝐷2𝑦𝑗] to be Θ

𝐷2𝑙 𝑜

• We expect ෨

𝐹 2𝐷 σ𝐾:𝐾∪{𝑗} 𝑗𝑡 𝑏 𝑑𝑚𝑗𝑟𝑣𝑓 𝑦𝐾∪{𝑗} to be Θ

𝐷𝑙(𝑒/2)+1 𝑜

Pessimist’s Query Analysis Continued

• 𝑞2 = 𝐷2𝑦𝑗 − 2𝐷 σ𝐾:𝐾∪{𝑗} 𝑗𝑡 𝑏 𝑑𝑚𝑗𝑟𝑣𝑓 𝑦𝐾∪{𝑗} +

σ𝐾,𝐾′ −1

(𝐾Δ𝐾′)∖𝑂 𝐽 𝑦𝐾∪𝐾′

• All terms of σ𝐾,𝐾′ ෨

𝐹 −1

𝐾Δ𝐾′ ∖𝑂 𝐽 𝑦𝐾∪𝐾′ have

expected value ≈ 0 except for the ones where 𝐾′ = 𝐾.

• These terms contribute Θ(𝑙𝑒/2) and it turns
• ut that w.h.p. these terms are dominant

Pessimist’s Query Analysis Continued

• We expect ෨

𝐹[𝑞2] to be Θ

𝐷2𝑙 𝑜

− Θ

𝐷𝑙

𝑒 2 +1

𝑜

+ Θ(𝑙𝑒/2)

• Taking 𝐷 = 𝑙

𝑒 4−1 2 𝑜, this is

Θ(𝑙𝑒/2) − Θ

𝑙

3𝑒 4 +1 2

𝑜

= 𝑙𝑒/2Θ 1 −

𝑙

𝑒+2 4

𝑜

which is negative if 𝑙 ≫ 𝑜

2 𝑒+2

• Pessimist has disproven our (Optimist’s) first

attempt at bluffing, but perhaps we can come up with a better bluff.

• Let’s see what went wrong.

Back to the Drawing Board

Graphical Picture

• Can represent the polynomial Pessimist is

querying as follows:

𝑦𝑗 𝑦𝑘1 𝑦𝑘2 𝑦𝑘𝑠 𝑦𝑗

times its transpose 𝐷 −

Graphical Picture

• Multiplying graph matrices is tricky (more on

that next lecture!). Some terms that appear are:

𝑦𝑗 𝑦𝑘1 𝑦𝑘2 𝑦𝑘𝑠 𝑦𝑗

𝐷2 −

𝑦𝑗 𝑦𝑘1 𝑦𝑘2 𝑦𝑘𝑠 𝑦𝑘1

′

𝑦𝑘2

′

𝑦𝑘𝑠

′

𝐷

𝑦𝑗 𝑦𝑘1

′

𝑦𝑘2

′

𝑦𝑘𝑠

′

+ −𝐷

Potential Fix

• What if we add an appropriate multiple of

𝑦𝑗 𝑦𝑘1 𝑦𝑘2 𝑦𝑘𝑠 𝑦𝑘1

′

𝑦𝑘2

′

𝑦𝑘𝑠

′

to our moment matrix?

Potential Fix Analysis

• This fix does work for 𝑒 = 4 [HKPRS16]
• However, it seems rather ad-hoc.
• Remark: It is related to giving more weight to

cliques which have more common neighbors, but that’s not quite what it does…

• Can we find a more principled general fix? Yes,

see next lecture!

References

• [BHK+16] B. Barak, S. B. Hopkins, J. A. Kelner, P. Kothari, A. Moitra, and A. Potechin,

A nearly tight sum-of-squares lower bound for the planted clique problem, FOCS p.428–437, 2016.

• [DM15] Y. Deshpande and A. Montanari, Improved sum-of-squares lower bounds

for hidden clique and hidden submatrix problems, COLT, JMLR Workshop and Conference Proceedings, vol.40, JMLR.org, p.523–562,2015.

• [HKPRS16] S. Hopkins, P. Kothari, A. Potechin, P. Raghavendra, T. Schramm. Tight

Lower Bounds for Planted Clique in the Degree-4 SOS Program. SODA 2016

• [MP16] D. Medarametla, A. Potechin. Bounds on the Norms of Uniform Low Degree

Graph Matrices. RANDOM 2016. https://arxiv.org/abs/1604.03423

• [MPW15] R. Meka, Aaron Potechin, and Avi Wigderson, Sum-of-squares lower

bounds for planted clique. STOC p.87–96, 2015

Appendix

Approximating ෨ 𝐹[𝑦𝐽] Calculation

• ෨

𝐹 𝑦𝐽 =

𝑙 |𝐽| 𝑒 |𝐽|

⋅

𝑂𝑒(𝐽) 𝑂𝑒(∅)

• If 𝐽 is a clique then 𝑂𝑒 𝐽 ≈ 2

|𝐽| 2 − 𝑒 2

𝑜−|𝐽| 𝑒−|𝐽|

• As a special case, 𝑂𝑒 ∅ ≈ 2− 𝑒

2

𝑜 𝑒

• If 𝐽 is a clique then

෨ 𝐹 𝑦𝐽 ≈

𝑙 𝐽 2 𝐽 2 − 𝑒 2 𝑜− 𝐽 𝑒− 𝐽 𝑒 𝐽 2− 𝑒 2 𝑜 𝑒

= 2

𝐽 2 𝑙 𝐽 𝑜 𝐽

≈ 2

𝐽 2

𝑙|𝐽| 𝑜|𝐽|