for Planted Clique Part II Lecture Outline Part I: Relaxed k-clique - - PowerPoint PPT Presentation
for Planted Clique Part II Lecture Outline Part I: Relaxed k-clique - - PowerPoint PPT Presentation
Lecture 13: SOS Lower Bounds for Planted Clique Part II Lecture Outline Part I: Relaxed k-clique Equations and Theorem Statement Part II: Pseudo-Calibration/Moment Matching Part III: Decomposition of Graph Matrices via Minimum Vertex
Lecture Outline
- Part I: Relaxed k-clique Equations and Theorem
Statement
- Part II: Pseudo-Calibration/Moment Matching
- Part III: Decomposition of Graph Matrices via
Minimum Vertex Separators
- Part IV: Attempt #1: Bounding with Square Terms
- Part V: Approximate PSD Decomposition
- Part VI: Further Work and Open Problems
Part I: Relaxed k-clique Equations and Theorem Statement
Relaxed Planted Clique Equations
- Flaw in the current analysis: Need to relax the
π-clique equations slightly to make the combinatorics easier to analyze
- Relaxed π-clique Equations:
π¦π
2 = π¦π for all i.
π¦ππ¦π = 0 if π, π β πΉ(π») 1 β π π β€ Οπ π¦π β€ (1 + π)π
Planted Clique SOS Lower Bound
- Theorem 1.1 of [BHK+16]: βπ > 0 such that if
π β€ π
1 2βπ π ππππ, with high probability degree π
SOS cannot prove that the relaxed π-clique equations are infeasible.
- Note: For π = 4 there is a lower bound of
ΰ·© Ξ© π for the original π-clique equations.
High Level Idea
- High level idea: Show that it is hard to
distinguish between the random distribution π» π,
1 2 and the planted distribution where we
put each vertex in the planted clique with probability
π π.
- Remark: We take this planted distribution to
make the combinatorics easier. If we could analyze the planted distribution where the clique has size exactly π, we would satisfy the constraint Οπ π¦π = π exactly.
Part II: Pseudo-Calibration/Moment Matching
Choosing Pseudo-Expectation Values
- Last lecture, Pessimist disproved our first
attempt for pseudo-expectation values, the MW moments.
- How can we come up with better pseudo-
expectation values?
Pseudo-Calibration/Moment Matching
- Setup: We are trying to distinguish between a
random distribution (π» π,
1 2 ) and a planted
distribution (π» π,
1 2 + planted clique)
- Pseudo-calibration/moment matching: The
pseudo-expectation values over the random distribution should match the actual expected values over the planted distribution in expectation for all low degree tests.
Review: Discrete Fourier Analysis
- Requirements for discrete Fourier analysis
- 1. An inner product
- 2. An orthonormal basis of Fourier characters
- This gives us Fourier decompositions and
Parsevalβs Theorem
Fourier Analysis over the Hypercube
- Example: Fourier analysis on {β1,1}π
- Inner product: π β π = 1
2π Οπ¦ π π¦ π(π¦)
- Fourier characters: ππ΅(π¦) = Οπβπ΅ π¦π
- Fourier decomposition: π = Οπ α
π
π΅ ππ΅ where
α π
π΅ = π β ππ΅
- Parsevalβs Theorem: Οπ΅ α
π
π΅ 2 = π β π =
π 2
Fourier Analysis over π» π,
1 2
- Inner product: π β π = πΉπ»βΌπ» π,1
2
π π» π(π»)
- Fourier characters: ππΉ(π») = β1 |πΉ\E π» |
Pseudo-Calibration Equation
- Pseudo-Calibration Equation:
πΉπ»βΌπ» π,1
2
[ ΰ·¨ πΉ[π¦π] β ππΉ] = πΉπ»βΌπππππ’ππ πππ‘π’ [π¦π β ππΉ]
- We want this equation to hold for all small π
and πΉ
Pseudo-Calibration Calculation
- To calculate πΉπ»βΌπππππ’ππ πππ‘π’ π¦π β ππΉ , first
choose the planted clique and then choose the rest of the graph
- π¦π = 0 if any π β π is not in the planted clique
- πΉ[ππΉ(π»)] = 0 whenever πΉ is not fully
contained in the planted clique
- Def: Define π πΉ = endpoints of edges in πΉ
- If π βͺ π πΉ β πππππ’ππ πππππ£π then π¦πππΉ = 1
- πΉπ»βΌπππππ’ππ πππ‘π’ π¦π β ππΉ =
π π |πβͺπ(πΉ)|
Calculation Picture
- If all the vertices are in the planted clique then
π¦πππΉ(π») = 1 . Otherwise, either π¦π = 0 (because an π β π) is missing or πΉ ππΉ = 0 because each edge outside the clique is present with probability 1
2
π πΉ
Fourier Coefficients of ΰ·¨ πΉ[π¦π]
- From the pseudo-calibration calculation,
ΰ·£ ΰ·¨ πΉ[π¦π]πΉ = πΉπ»βΌπ» π,1
2
ΰ·¨ πΉ[π¦π] β ππΉ =
π π |πβͺπ(πΉ)|
- We take ΰ·¨
πΉ[π¦π] = ΟπΉ: πβͺπ πΉ β€πΈ
π π |πβͺπ(πΉ)|
where πΈ is a truncation parameter and then normalize so that ΰ·¨ πΉ[π¦β ] = ΰ·¨ πΉ[1] = 1
- Good exercise: What happens if we donβt
truncate at all?
Graph Matrix Decomposition
- Ignoring the normalization, π = ΟπΌ
π π |π(πΌ)|
ππΌ where we sum over ALL πΌ with at most πΈ vertices which have no isolated vertices outside
- f π and π.
Part III: Decomposition of Graph Matrices via Minimum Vertex Separators
Proof Sketch
- How can we show π β½ 0 with high probability?
- High level idea:
- 1. Find an approximate PSD decomposition πππππ’ of
π
- 2. Handle the error πππππ’ β π. Unfortunately, this
error is not small enough to ignore, so we carefully show that πππππ’ β π βΌ πππππ’ with high
- probability. We briefly sketch the ideas for this in
Appendix I. For the full details, see [BHK+16]
Technical Minefield
- Warning: This analysis is a technical minefield
Mine handled correctly Not quite correct, see Appendix II
Decomposition via Separators
- How can we handle all of the different ππΌ?
- Key idea: Decompose each πΌ into three parts
π, π, πβ²π based on the leftmost and rightmost minimum vertex separators π and π of πΌ U V S T H π π πβ²π
Separator Definitions
- Definition: Given a graph πΌ with
distinguished sets of vertices π and π, a vertex separator π is a set of vertices such that any path from π to π must intersect π.
- Definition: A leftmost minimum vertex
separator π is a set of vertices such that for any vertex sepator πβ² of minimum size, any path from π to πβ² intersects π.
- A rightmost minimum vertex separator is
defined analogously.
Existence of Minimum Separators
- Lemma 6.3 of [BHK+16]: Leftmost and
rightmost minimum vertex separators always exist and are unique.
Left, Middle, and Right Parts
- Let π, π be the leftmost and rightmost
minimum vertex separators of πΌ
- Definition: We take the left part π of πΌ to be
the part of πΌ between π and π, we take the middle part π of πΌ to be the part of πΌ between π and π, and we take the right part πβ²π of πΌ to be the part of πΌ between π and π
Conditions on Parts
- π, π, πβ²π satisfy the following:
- The unique minimum vertex separator of π is
π
π = π (where π π is the right side of π)
- The leftmost and rightmost minimum vertex
separators of π are ππ = π and π
π = π (where
ππ and π
π are the left and right sides of π)
- The unique minimum vertex separator of πβ²π is
ππβ²π = π (where ππβ²π is the left side of πβ²π)
Approximate Decomposition
- Claim: If π is the size of the minimum vertex
separator of πΌ, ππΌ β ππππππβ²π
- Idea: There is a bijection between injective
mappings π: π πΌ β π(π») and injective mappings π1: π π β π(π»), π2: π π β π(π»), and π3: π(πβ²π) β π(π») such that
1. π1, π2 agree on π and π2, π3 agree on π 2. Collectively, π1, π2, π3 donβt map two different vertices of πΌ to the same vertex of π»
Approximate Decomposition
- Claim: If π is the size of the minimum vertex
separator of πΌ, ππΌ β ππππππβ²π
- Corollary:
π π |π(πΌ)|
ππΌ β
π π π πΌ βπ
2 ππ
π π π πΌ βπ
ππ
π π π πΌ βπ
2 ππβ²π
U V S T ππΌ U S S T V T β Γ Γ ππ ππ ππβ²π
Intersection Terms
- Warning! There will be terms where π1, π2, π3
map multiple vertices to the same vertex. We call these intersection terms.
- We sketch how to handle intersection terms in
Appendix I. For now, we sweep this under the rug.
Part IV: Attempt #1: Bounding With Square Terms
Bounding With Square Terms
- How can we handle all of the ππππππβ²π terms?
- One idea: Can bound ππππππβ²π + ππππππβ²π
π
as follows.
- πππ β ππ
πβ²π π ππ π
πππ β ππ
πβ²π π ππ π π
β½ 0
Bounding With Square Terms
- πππ β ππ
πβ²π π ππ π
πππ β ππ
πβ²π π ππ π π
β½ 0
- Rearranging, ab ππππππβ²π + ππππππβ²π
π
βΌ π2ππππ
π + π2π πβ²π π ππ πππππβ²π βΌ π2ππππ π +
π2 ππ
πππ π πβ²π π ππβ²π
Example
- What square terms would the following ππΌ be
bounded by (ignoring intersection terms)? U V S T ππΌ
Example Answer
- Answer: Take the left part and its mirror
image and take the right part and its mirror image π ππ πβ² πβ²π
Bounding With Square Terms Failure
- Unfortunately, the coefficients on the square
terms arenβt high enough for this idea to work.
- We need a more sophisticated analysis.
Part V: Approximate PSD Decomposition
ππ ππ factorization
- Definition: Define ππ = Οπ:|π
π|=π
π π π π βπ
2 ππ
and define π π = Οπ:|ππ|=|π
π|=π
π π π π βπ
ππ where we require that π
π is the unique
minimum vertex separator of π and ππ, π
π are
the leftmost and rightmost minimum vertex separators of π. Define πππππ’ = Οπ =0
π 2
ππ π π ππ
π
- Claim: π β πππππ’ = Οπ =0
π 2
ππ π π ππ
π
Claim Justification
- Claim: π β πππππ’ = Οπ =0
π 2
ππ π π ππ
π
- This follows from the decomposition of each
πΌ into left, middle, and right parts π, π, πβ²π and the claim that up to intersection terms,
π π |π(πΌ)|
ππΌ =
π π π πΌ βπ
2 ππ
π π π πΌ βπ
ππ
π π π πΌ βπ
2 ππβ²π
Analysis of π π
- π π = Οπ:|ππ|=|π
π|=π
π π π π βπ
ππ
- Probabilistic norm bounds: With high
probability, ππ is ΰ·¨ π(π
π π βπ 2
) because π is the size of the minimum vertex separator of πΌ
- Corollary: If π β€ π
1 2βπ then with high
probability, π π β½
1 2 π½π as the identity is the
dominant term of π π
Summary
- If π β€ π
1 2βπ then with high probability,
πππππ’ = Οπ =0
π 2
ππ π π ππ
π β½ 1 2 Οπ =0
π 2
ππ ππ
π
- The 1
2 Οπ =0
π 2
ππ ππ
π allows us to deal with the
error πππππ’ β π.
Part VI: Further Work and Open Problems
Further Work
- The techniques used for planted clique can be
generalized to other planted problems where we are trying to distinguish a planted distribution from a random distribution [HKP+17]
Open Problems
- Can we prove the full lower bound for planted
clique with the exact constraint that Οπ=1
π
π¦π = π?
- How close to
π can we make the lower bound?
- It turns out that the current machinery
doesnβt work as well for random sparse
- graphs. What bounds can we prove for
problems such as densest k-subgraph and independent set on sparse graphs?
References
- [BHK+16] B. Barak, S. B. Hopkins, J. A. Kelner, P. Kothari, A. Moitra, and A. Potechin,
A nearly tight sum-of-squares lower bound for the planted clique problem, FOCS p.428β437, 2016.
- [HKP+17] S. Hopkins, P. Kothari, A. Potechin, P. Raghavendra, T. Schramm, D. Steurer.
The power of sum-of-squares for detecting hidden structures. FOCS 2017
Appendix I: Handling Intersection Terms
High Level Idea
- If there are intersections between the left,
middle, and right parts, this creates a new graph πΌ2.
- We can decompose πΌ2 into new left, middle,
and right parts!
π πΌ π = π π = π πΌ2 π π2 πβ² π2 π2 π2
β²π
Choosing New Separators
- How do we choose the new separators πβ² and
πβ²?
- We take πβ² to be the leftmost minimum vertex
separator between π and {intersected vertices} βͺ π.
- Similarly, we take πβ² to be the rightmost
minimum vertex separator between {intersected vertices} βͺ π and π.
Key Idea
- This decomposition works the same
regardless of what π2 and π2
β²π look like (see
Claim 6.11 of [BHK+16])!
- Thus, we get a new approximate
decomposition of the form Οπ β²=0
π 2
ππ β²π π β²
β² ππ β²
- This can be bounded by
1 2 Οπ =0
π 2
ππ ππ
π as long as
we always have that π π β²
β²
βͺ 1
Bounding New Middle Parts
- We need to show that the new middle parts
donβt have norms which are too high.
- This is done with the intersection tradeoff
lemma (Lemma 7.12 of [BHK+16])
Appendix II: Technical Mines
Approximate Decomposition Mine
- Claim: If π is the size of the minimum vertex
separator of πΌ, ππΌ β ππππππβ²π
- There are subtle issues related to the ordering
- f π and π, the leftmost and rightmost
minimum vertex separators of πΌ
- How these issues should be handled depends
- n whether we require matrix indices to be in
ascending order.
Approximate Decomposition Mine
- If we require matrix indices to be in ascending
- rder, what we actually have is ππΌ β
Οπ,π,πβ²π:πΌ=πβͺπβͺπβ²π ππππππβ²π where π βͺ π βͺ πβ²π is the graph formed by gluing π, π, πβ²π together.
- In fact, this equation is precisely what is
needed for the approximate PSD decomposition π β πππππ’ = Οπ =0
π 2
ππ π π ππ
π.
Approximate Decomposition Mine
- Remark: [BHK+16] navigates this issue by
keeping everything in terms of the individual ribbons (Fourier characters for a given matrix entry) until it is time to use the matrix norm bounds (see Definition 6.1 and subsection 6.4
- f [BHK+16])
Approximate Decomposition Mine
- If we do not require matrix indices to be in
ascending order, we actually have the following two equations
1. ππΌ β π΅π£π’ π, π, πβ²π ππππππβ²πwhere π΅π£π’ π, π, πβ²π is the number of different ways to decompose πΌ into π, π, πβ²π. 2. ππΌ β
1 π‘πΌ !2 Οπ,π,πβ²π:πΌ=πβͺπβͺπβ²π ππππππβ²π where
π βͺ π βͺ πβ²π is the graph formed by gluing π, π, πβ²π together.
Truncation Mine
- Definition: Define ππ = Οπ:|π
π|=π
π π π π βπ
2 ππ
and define π π = Οπ:|ππ|=|π
π|=π
π π π π βπ
ππ where we require that π
π is the unique
minimum vertex separator of π and ππ, π
π are
the leftmost and rightmost minimum vertex separators of π. Define πππππ’ = Οπ =0
π 2
ππ π π ππ
π
- Actually, we need to truncate ππ and ππ by only
taking π, π with at most πΈ vertices
Truncation Mine
- Warning: There is a mismatch between πΌ
which have at most πΈ vertices and triples π, π, πβ²π which each have at most πΈ vertices.
- This truncation error turns out to have very