Near Optimal Compressed Sensing without Priors: Parametric SURE - - PowerPoint PPT Presentation

Near Optimal Compressed Sensing without Priors: Parametric SURE Approximate Message Passing

Chunli Guo, University College London Mike E. Davies, University of Edinburgh

1

Talk Outline

• Motivation for Parametric SURE-AMP

 What is approximate message passing (AMP) algorithm ?  Iterative Gaussian denoising nature of AMP

• Parametric SURE-AMP Algorithm

 SURE based denoiser design  Parameterization & optimization of denoisers

• Numerical Reconstruction Examples

What is AMP ?

• The CS reconstruction problem with ,
• The Generic AMP algorithm for i.i.d Gaussian [Donoho 09]
• Initialized with ,

For t = 0, 1….

y x  

m n 



m n 



ˆ x 

z y 

ˆ

t t T t

r x z  

1

ˆ ( )

t t t

x r 

 

1 1 '

ˆ ( )

t t t t t

z y x n z r m 

 

     

( )

t

 

Where is the non-linear function applied element-wise to the vector t

r

Onsager reaction term

Iterative Gaussian denoising nature of AMP

t=10 t=20 t=40 Quantile-Quantile Plot for against Gaussian distribution

t

r x 

t t

r x w c  

(0,1) w N

is the effective noise variance at each AMP iteration

t

c

AMP variants:

• L1-AMP: being the soft-thresholding function
• Bayesian optimal AMP: being the MMSE estimator

Where

( )

t

  ( )

t

 

Motivation for parametric SURE-AMP

• L1-AMP treats the signal denoising as a 1-d problem while the true

signal pdf is visible in the noisy estimate in the large system limit.

• Reconstruction goal: achieve recovery with minimum MSE (BAMP

reconstruction) without the prior

• Solution:
• Fitting the prior with finite number of Gaussians iteratively

EM-GAMP algorithm [Vila et al. 2013] – indirect way to minimize MSE

• Optimize the parametric denoiser iteratively

Parametric SURE-AMP – direct way to minimize MSE

t

r

( ) p x

Parametric SURE-AMP algorithm

Initialized with , , ˆ x 

z y 

2

c z  

For t = 0,1,….

1 ' 1 1 1 2 1 1 1

( , ) ˆ ( , ˆ ( , | ) ˆ | )

t t t t t t t t t t T t t t t t t t t t t t t t

r x z f r c n z y x z m H r c x f c c z r     

      

          

parameter selection function

parametric denoiser

SURE: Unbiased estimate of MSE

• Ideally we would like a denoiser with the mimimum MSE.

Calculating MSE requires , thus we need to find a surrogate for MSE

• Let be the noisy observation of with

The denoised signal is obtained via

x

r x w c  

x

(0,1) w N

ˆ ( , | ) ( , | ) x f r c r g r c     

Theorem [Stein 1981] SURE is defined as the expected value over the noisy data alone and is the unbiased estimate of the MSE

 

 

 

 

2 2 ˆ, ,

ˆ ( , | )

x x x

x x f r c x



     

 

2 '

( , | ) 2 ( , | )

r

c g r c cg r c     

Parameter Selection Function

The denoiser parameters are iteratively selected according to

• The parameters optimization relies purely on the noisy

data and the effective noise variance.

• If all MMSE estimators are included in the parametric

family, the parametric SURE-AMP achieves the BAMP performance without prior.

( , )

t t t t

H r c  

 

2 '

, | 2 ( , | )

t t t t t

g r c c g r c argmin



   

Practical Parametric Denoiser

1

( , | ) ( , | ) ( | ( ))

k i i i i

f c r g c f r c     



   



• The denoiser is parameterized as the weighted sum of

kernel functions

• The non-linear parameters of the kernels are tied up with

the effective noise variance where is fixed for all iterations.

• The linear weight for the kernels are optimized by solving

( )

i i

c c   

i



2 ' '

( , | ) 2 ( , | ) 2 ( , | ) ( , | ) ( , | )

i i i

c g r c cg r c d d d g r c g r c c g r c d d d                

Kernel Function Examples

1



2



2

 

1

 

Piecewise Linear Kernel [Donoho et al. 2012] Exponential Kernel [Luisier et al. 2007]

2 2

2 1 2

( ) , ( | T)

T

f f e



   



  6 T c 

MMSE estimator V.S. Kernel Based Denoiser

(x) 0.1N(0,1) 0.9 (x) p   

Reconstruction Comparison

(x) 0.1N(0,1) 0.9 (x) p   

Reconstruction Comparison

(x) 0.1N(0,1) 0.9 (0,0.01) p N  

Runtime Comparison

20 times faster than the EM-GM-GAMP algorithm for Bernoulli-Gaussian

Natural Images Reconstruction

Natural Images Reconstruction

Conclusion

• The parametric SURE-AMP directly minimizes the MSE
• f the reconstructed signal at each iteration.
• With proper design of the parametric family, the

parametric SURE-AMP algorithm achieves the BAMP performance without the signal prior.

• The parametric SURE is cheap in terms of the

computational cost.

• Further research involves considering more sophisticated

kernel families and the rigorous proof for the state evolution dynamics.