New Parameter Choice Rules for Regularization with Mixed Gaussian - PowerPoint PPT Presentation

Regularization Frequentist Approaches to Parameter Selection Numerical Results (preliminary) New Parameter Choice Rules for Regularization with Mixed Gaussian and Poissonian Noise Elias Helou (joint work with ´ Alvaro De Pierro) ICMC - USP elias@icmc.usp.br 1 de agosto de 2013 Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Frequentist Approaches to Parameter Selection Numerical Results (preliminary) Sum´ ario Regularization Conditioning and Regularization A Deterministic Approach A Frequentist Approach Frequentist Approaches to Parameter Selection Existing Methods The New Technique Numerical Results (preliminary) Tomography The (Still Preliminary) Results Concluding Remarks Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System x A x = b Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System x x ǫ A x ǫ = b + ǫ =: b ǫ Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System x x ǫ � x ǫ − x � = 25 . 00 � ǫ � Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System x x ǫ κ ( A ) = 50 . 02 Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System ◮ Ill-conditioned linear systems appear frequently in applications; Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System ◮ Ill-conditioned linear systems appear frequently in applications; ◮ Condition number will be much higher than example; Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Ill-Conditioned Linear System ◮ Ill-conditioned linear systems appear frequently in applications; ◮ Condition number will be much higher than example; ◮ Under the presence of noise, severe loss of precision. Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Regularization ◮ Replace the original problem by a stable perturbed version from a family { P γ ( b ǫ ) } γ ∈ Γ ; Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Regularization ◮ Replace the original problem by a stable perturbed version from a family { P γ ( b ǫ ) } γ ∈ Γ ; ◮ Example: Tikhonov Regularization (Γ = R + ) x tik � A x − b ǫ � 2 2 + γ � x � 2 γ ( b ǫ ) := argmin 2 x ∈ R n Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Regularization ◮ Replace the original problem by a stable perturbed version from a family { P γ ( b ǫ ) } γ ∈ Γ ; ◮ Example: Tikhonov Regularization (Γ = R + ) x tik � A x − b ǫ � 2 2 + γ � x � 2 γ ( b ǫ ) := argmin 2 x ∈ R n = ( A T A + γI ) − 1 A T b ǫ ; Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Regularization ◮ Replace the original problem by a stable perturbed version from a family { P γ ( b ǫ ) } γ ∈ Γ ; ◮ Example: Tikhonov Regularization (Γ = R + ) x tik � A x − b ǫ � 2 2 + γ � x � 2 γ ( b ǫ ) := argmin 2 x ∈ R n = ( A T A + γI ) − 1 A T b ǫ ; ◮ How to choose the regularization parameter γ ? Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Deterministic Regularization Parameter Choice ◮ If � ǫ k � → 0, then the parameter selection function ℓ ( b ǫ k , � ǫ k � ) must satisfy: x ℓ ( b ǫ k , � ǫ k � ) ( b ǫ k ) → x ; Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Deterministic Regularization Parameter Choice ◮ If � ǫ k � → 0, then the parameter selection function ℓ ( b ǫ k , � ǫ k � ) must satisfy: x ℓ ( b ǫ k , � ǫ k � ) ( b ǫ k ) → x ; ◮ Drawback: Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Deterministic Regularization Parameter Choice ◮ If � ǫ k � → 0, then the parameter selection function ℓ ( b ǫ k , � ǫ k � ) must satisfy: x ℓ ( b ǫ k , � ǫ k � ) ( b ǫ k ) → x ; ◮ Drawback: ◮ Impossible if � ǫ � is not available (is it ever available?); Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Deterministic Regularization Parameter Choice ◮ If � ǫ k � → 0, then the parameter selection function ℓ ( b ǫ k , � ǫ k � ) must satisfy: x ℓ ( b ǫ k , � ǫ k � ) ( b ǫ k ) → x ; ◮ Drawback: ◮ Impossible if � ǫ � is not available (is it ever available?); ◮ Says nothing when ǫ is not small. Regularization Parameter Choice – 29 ◦ CBM Elias Helou

Regularization Conditioning and Regularization Frequentist Approaches to Parameter Selection A Deterministic Approach Numerical Results (preliminary) A Frequentist Approach Frequentist Regularization Parameter Choice ◮ A stochastic model for the problem may be available if it is possible to estimate E ǫǫ T and we assume E ǫ = 0 ; Regularization Parameter Choice – 29 ◦ CBM Elias Helou