Spikes & Sines Joel A. Tropp This month: The University of - PowerPoint PPT Presentation

Spikes & Sines ❦ Joel A. Tropp This month: The University of Michigan (Math) jtropp@umich.edu Next month: California Institute of Technology (ACM) jtropp@acm.caltech.edu Research supported in part by NSF and DARPA 1

Spikes & Sines ❧ Work in C n ❧ Define spike basis { e j : j = 1 , 2 , . . . , n } � 1 , t = j e j ( t ) = t = 1 , 2 , . . . , n 0 , t � = j ❧ Define sine basis { f j : j = 1 , 2 , . . . , n } 1 √ n e 2 π i jt/n f j ( t ) = t = 1 , 2 , . . . , n Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 2

In Captivity... 1 1/ √ d Spikes Sines Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 3

The DFT matrix ❧ Define the unitary DFT matrix f ∗   1 f ∗   2 F = . .   .   f ∗ n ❧ Suppose T and Ω are subsets of { 1 , 2 , . . . , n } ❧ Write F Ω T for the submatrix of F with rows in Ω and columns in T ❧ Note that � F Ω T � ≤ 1 Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 4

Linear Independence and the DFT ❧ Consider a collection of spikes and sines: X ( T, Ω) = { e j : j ∈ T } ∪ { f j : j ∈ Ω } ⊂ C n ❧ The Gram matrix of this collection is � � I | Ω | F Ω T G = ( F Ω T ) ∗ I | T | ❧ X ( T, Ω) is linearly independent if and only if G is nonsingular ❧ The extreme eigenvalues of G are 1 ± � F Ω T � ❧ Thus X ( T, Ω) is linearly independent if and only if � F Ω T � < 1 Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 5

The Donoho–Stark Bound Theorem 1. [Donoho–Stark 1989] If | T | · | Ω | < n then � F Ω T � < 1 . Proof. The matrix F Ω T has | Ω | rows and | T | columns; its entries have magnitude n − 1 / 2 . Thus � F Ω T � 2 ≤ � F Ω T � 2 F = � F Ω T , F Ω T � ≤ � F Ω T � 1 , 1 � F Ω T � ∞ , ∞ = | Ω | √ n · | T | √ n < 1 . If | T | + | Ω | < 2 √ n then � F Ω T � < 1 . Corollary 2. Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 6

The Donoho–Stark Uncertainty Principle ❧ Define supp( α ) = { j : α j � = 0 } and � α � 0 = | supp( α ) | Let x be a vector in C n . Consider its Corollary 3. [Discrete UP] representations in the spike and sine bases: � n � n x = j =1 α j e j x = j =1 β j f j . and Then � α � 0 · � β � 0 ≥ n . Proof. Set T = supp( α ) and Ω = supp( β ) . Note that � � j ∈ T α j e j − j ∈ Ω β j f j = 0 . Therefore, X ( T, Ω) is linearly dependent and | T | | Ω | ≥ n . Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 7

The Dirac Comb ❧ Let n be a square number ❧ Set T = Ω = {√ n, 2 √ n, 3 √ n, . . . , n } ❧ The Poisson summation formula gives � � j ∈ T e j = j ∈ Ω f j . ❧ Thus the Donoho–Stark results are all sharp ❧ Reason: Z / Z n has nontrivial subgroups for composite n Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 8

Tao Uncertainty Principle Idea: Counterexamples don’t exist for prime n Theorem 4. [Tao 2004] Suppose n is prime. ❧ If | T | + | Ω | ≤ n , then � F Ω T � < 1 . ❧ If | T | + | Ω | ≥ n + 1 , then � F Ω T � = 1 . ❧ Proof uses algebraic methods ❧ Some submatrices are very badly conditioned Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 9

Analytic Principle of the Large Sieve Idea: Counterexamples have rigid structure Define the spread of a set: ∆(Ω) = min {| j − k mod n | : j, k ∈ Ω , j � = k } Theorem 5. [Large Sieve Inequality] Suppose T has the form T = { m + 1 , m + 2 , . . . , m + | T |} for an integer m . If | T | + n/ ∆(Ω) < n + 1 , then � F Ω T � < 1 . References: [Donoho–Logan 1992, Jameson 2006] Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 10

Random Sets Idea: Generic collections of spikes and sines are linearly independent ❧ For an integer m , define class of index sets with cardinality m : S m = { S : S ⊂ { 1 , 2 , . . . , n } and | S | = m } ❧ Let Ω be a uniformly random element of S m , i.e., Prob { Ω = S } = | S m | − 1 for each S ∈ S m . ❧ Say “ Ω is a random set with cardinality | Ω | ” Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 11

The Cand` es–Romberg Bound Theorem 6. [Cand` es–Romberg 2006] Suppose that c n √ log n. | T | + | Ω | ≤ If T is an arbitrary set with cardinality | T | and Ω is a random set with cardinality | Ω | , then � F Ω T � 2 ≥ 0 . 5 � � ≤ n − 1 . Prob ❧ Proof uses the moment method and heavy-duty combinatorics Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 12

Sets are not Created Equal Theorem 7. [T 2006] Suppose that | T | log n + | Ω | ≤ c n. If T is an arbitrary set with cardinality | T | and Ω is a random set with cardinality | Ω | , then � F Ω T � 2 ≥ 0 . 5 � � ≤ n − 1 . Prob ❧ Proof uses Rudelson’s selection lemma Reference: [ Random Subdictionaries ] Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 13

Restricted Isometry Consequences Except with probability n − 1 , a random set Ω with cardinality | Ω | satisfies | Ω | 2 n ≤ � F Ω T � 2 ≤ 3 | Ω | c n for all T where | T | ≤ log 5 n. 2 n Except with probability n − 1 , a random set Ω has Corollary 8. [T 2007] the following property. For each set T whose cardinality c n | T | ≤ log 5 n, it holds that � F Ω T � 2 ≤ 0 . 5 . References: [Cand` es–Tao 2006, Rudelson–Vershynin 2006, Spikes & Sines ] Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 14

When Both Sets are Random Theorem 9. [T 2007] Fix ε > 0 . Suppose that n ≥ N ( ε ) and that | T | + | Ω | ≤ c ( ε ) · n. Let T and Ω be random sets with cardinalities | T | and | Ω | . Then � F Ω T � 2 ≥ 0 . 5 � � ≤ 4 exp {− n 1 / 2 − ε } . Prob Reference: [ Spikes & Sines ] Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 15

Near Optimality ❧ Assume it were possible to obtain � F Ω T � 2 ≥ 0 . 5 � � ≤ exp {− n 1 / 2+ ε } Prob ❧ Consider case where n is a square number and | T | + | Ω | = 2 √ n ❧ Only about exp { n 1 / 2 log n } ways to pick the sets ❧ Union bound ⇒ no pair of sets yields � F Ω T � = 1 ❧ Contradiction: The Dirac comb has � F Ω T � = 1 Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 16

Proof Techniques ❧ Reduction to square case with independent coordinate model ❧ Rudelson–Vershynin theorem on spectral norm of random submatrix ❧ Rudelson’s selection lemma ❧ Noncommutative Khintchine inequality ❧ Classical Khintchine inequality for (1 , 2) norm of random submatrix ❧ Bourgain and Tzafriri’s extrapolation ❧ Minimax property of Chebyshev polynomials Reference: [T 2006, Random Paving ] Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 17

Normalized Random Submatrices ❧ If | Ω | = δn then columns of F Ω T have ℓ 2 norm δ 1 / 2 ❧ Should normalize matrix by δ − 1 / 2 Theorem 10. [T 2007] Fix δ ∈ (0 , 1) . Suppose that n ≥ N ( δ ) and that | Ω | = | T | = δn. Let T and Ω be random sets with cardinalities | T | and | Ω | . Then � 1 � ≤ n − 1 . √ Prob � F Ω T � ≥ 10 δ Reference: [ Spikes & Sines ] Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 18

Numerics I Norm of random square submatrix drawn from n × n DFT 1 0.9 0.8 0.7 0.6 Expected norm 0.5 0.4 0.3 Conjectured limit 0.2 n = 1024 n = 128 0.1 n = 40 0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Proportion of rows/cols ( δ ) Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 19

Numerics II Scaled norm of random square submatrix drawn from n × n DFT 2 1.9 1.8 1.7 Expected norm * δ − 1/2 1.6 1.5 1.4 Conjectured limit 1.3 n = 1024 n = 256 1.2 n = 128 n = 80 1.1 n = 40 1 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Proportion of rows/cols ( δ ) Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 20

Numerics III Norm of random rectangular submatrix drawn from 128 × 128 DFT 1.2 1 0.8 Expected norm 0.6 0.4 0.2 0 1 0.8 0.6 0.4 1 0.9 0.8 0.7 0.2 0.6 0.5 0.4 Proportion of rows ( δ Ω ) 0.3 0.2 0 0.1 0 Proportion of columns ( δ T ) Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 21

Open Questions Conjecture 11. [Quartercircle Law] Suppose that δ = | T | + | Ω | ≤ 1 2 . 2 n If T and Ω are random sets with cardinalities | T | and | Ω | , then � E � F Ω T � ≤ 2 δ (1 − δ ) . The inequality becomes an equality as n → ∞ . ❧ What is the correct tail behavior? ❧ Study behavior of σ min ( F Ω T ) for random T , Ω Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 22

To learn more... E-mail: jtropp@umich.edu Web: http://www.umich.edu/~jtropp Papers: ❧ “Random subdictionaries of general dictionaries,” 2006 ❧ “The random paving property for uniformly bounded matrices,” 2006 ❧ “On the linear independence of spikes and sines,” 2007 Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 23