SLIDE 1
Variational Bayesian Optimal Experimental Design
Adam Foster† Martin Jankowiak‡ Eli Bingham‡ Paul Horsfall‡ Yee Whye Teh† Tom Rainforth† Noah D. Goodman‡§
†Oxford Statistics, ‡Uber AI, §Stanford
Spotlight, NeurIPS 2019
Variational Bayesian Optimal Experimental Design Adam Foster - - PowerPoint PPT Presentation
Variational Bayesian Optimal Experimental Design Adam Foster - - PowerPoint PPT Presentation
Variational Bayesian Optimal Experimental Design Adam Foster Martin Jankowiak Eli Bingham Paul Horsfall Yee Whye Teh Tom Rainforth Noah D. Goodman Oxford Statistics, Uber AI, Stanford Spotlight, NeurIPS
SLIDE 2
SLIDE 3
Adaptive experimentation
Design
Experimental setup Controlled by experimenter
Inference
Data analyzed Model fitted
Observation
Data generated Response sampled
θ d y
SLIDE 4
Which would you prefer?
What makes a good experiment?
SLIDE 5
Which would you prefer?
What makes a good experiment?
Which would you prefer?
SLIDE 6
Bayesian experimental design
y d 𝜄
prior likelihood posterior
𝜄 : latent variable of interest d : design y : data
SLIDE 7
Design Observation Inference
Low information gain High information gain
Which would you prefer? Which would you prefer?
SLIDE 8
Expected information gain (EIG)
Expected reduction in entropy from the prior to the posterior
prior entropy posterior entropy (Lindley, 1956)
SLIDE 9
Estimating the EIG is difficult!
simulate samples prior posterior
“Doubly intractable”
SLIDE 10
Our contribution: Variational estimators of the EIG
- Bound EIG to turn estimation into optimization
- This removes double intractability
approximate marginal density
SLIDE 11
Implicit? Variational estimator Consistent?
Marginal Posterior Variational NMC Marginal + likelihood
SLIDE 12
Much faster convergence rates!
Variational rate Nested Monte Carlo rate
T = computational cost
SLIDE 13
Intuition: amortization
- Approximate the functional form rather than computing
independent point estimates
NMC = Nested Monte Carlo
SLIDE 14
Experiments: EIG estimation accuracy
Ours Baseline
n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a Marginal + likelihood VNMC Marginal Posterior NMC Laplace LFIRE DV
SLIDE 15
Experiments: End-to-end adaptive experimentation
Which would you prefer?
Parameter recovery (RMSE)
SLIDE 16
Thank you
Implementation in Pyro Full paper
docs.pyro.ai/en/stable/contrib.oed.html papers.nips.cc/paper/9553-variational-bayesian-optimal-experimental-design.pdf