General and Robust Error Estimation and Reconstruction for Monte - - PowerPoint PPT Presentation

▶
general and robust error estimation and reconstruction
SMART_READER_LITE
LIVE PREVIEW

General and Robust Error Estimation and Reconstruction for Monte - - PowerPoint PPT Presentation

Oct 16, 2022 215 likes •688 views

General and Robust Error Estimation and Reconstruction for Monte Carlo Rendering Pablo Bauszat 1 , Martin Eisemann 1,2 , Elmar Eisemann 2 , Marcus Magnor 1 1 Computer Graphics Lab, TU Braunschweig, Germany 2 Delft University of Technology,

slide-1
SLIDE 1

General and Robust Error Estimation and Reconstruction for Monte Carlo Rendering

Pablo Bauszat1, Martin Eisemann1,2, Elmar Eisemann2, Marcus Magnor1

1 Computer Graphics Lab, TU Braunschweig, Germany 2 Delft University of Technology, Netherlands

slide-2
SLIDE 2

Monte Carlo Rendering

  • Today‘s industry standard
  • General and unbiased
  • Covers variety of natural phenomena
  • Requires extensive sampling
  • Pixel (2D integral)
  • Camera lens (2D integral)
  • Time (1D integral)
  • Global illumination (2D integral per bounce)
  • … and more …
2
slide-3
SLIDE 3

Noise

3
slide-4
SLIDE 4

Filtering

4

Noisy Reference Uniform filter (small) Uniform filter (large) Adaptive filtering

slide-5
SLIDE 5

Adaptive Reconstruction

  • Filter bank
  • Set of filters with different properties
  • Select best filter on a per-pixel level
5

Filter bank

slide-6
SLIDE 6

Problem statement

How to choose the best filter from the set for a pixel?

6
slide-7
SLIDE 7

Previous work

7

Li et al. 2012 Overbeck et al. 2009 Rousselle et al. 2011/2012/2013 Kalantari et al. 2013 Moon et al. 2014

slide-8
SLIDE 8

Limitations of previous work

  • Filter selection based on noisy image
  • Often tailored for specific filters
  • Switching filters may cause seams
8

Local selection

slide-9
SLIDE 9

Our method

9
slide-10
SLIDE 10

Insights

Our method is based on three key insights:

  • 1. Filter selection is often more crucial than sampling rate
  • 2. Filter error is locally smooth for most image regions
  • 3. Often multiple filters are close-to-optimal choices
10
slide-11
SLIDE 11
  • 1. Filter selection is often more crucial than sampling rate
11

Filter bank of 4 Gaussian and 4 Joint Bilateral filters

32 spp SURE 12.3 MSE-3 32 spp Best choice 1.6 MSE-3 (x 7.7) 16 spp Best choice 2.3 MSE-3 (x 5.3)

Recently employed by [Li2012] and [Rousselle2013]

slide-12
SLIDE 12
  • 1. Filter selection is often more crucial than sampling rate
12

Scene

SURE 32 spp Best choice 32 spp Best choice 16 spp Conference 12.327 1.605 (x 7.7) 2.344 (x 5.3) Sibenik 0.758 0.157 (x 4.8) 0.258 (x 2.9) Toasters 0.187 0.096 (x 1.9) 0.156 (x 1.2) San Miguel 16.880 6.419 (x 2.6) 9.831 (x 1.7)

Mean squared error (MSE) * 10-3 – Same filter bank

slide-13
SLIDE 13

Insights

Our method is based on three key insights:

  • 1. Filter selection is often more crucial than sampling rate
  • 2. Filter error is locally smooth for most image regions
  • 3. Often multiple filters are close-to-optimal choices
13
slide-14
SLIDE 14
  • 2. Error smoothness – Gaussian filters
14

Gaussian σ=7 Gaussian σ=11 Gaussian σ=13

slide-15
SLIDE 15
  • 2. Error smoothness – Guided Image Filtering [He2010]
15

Guided radius=4 Guided radius=8 Guided radius=16

slide-16
SLIDE 16

Insights

Our method is based on three key insights:

  • 1. Filter selection is often more crucial than sampling rate
  • 2. Filter error is locally smooth for most image regions
  • 3. Often multiple filters are close-to-optimal choices
16
slide-17
SLIDE 17
  • 3. Often multiple filters are close-to-optimal choices
17

Reference Filter A Filter B Filter C

slide-18
SLIDE 18
  • 3. Often multiple filters are close-to-optimal choices
18

Opti timal l se selec lectio ion via via ground tr truth

  • MSE down to 8.0

.0% from noisy image Regula lariz ized se sele lectio ion

  • MSE down to 8.4

.4% from noisy image

  • Variations in selection are penalized

Regula lariz ized se sele lectio ion

  • MSE down to 9.1

.1% from noisy image

  • Variations in selection are penalized
slide-19
SLIDE 19

What do we learn from the insights?

  • Filter selection is crucial
  • Filter error is piece-wise smooth
  • Non-optimal filter selection does not imply large error
19
slide-20
SLIDE 20

Our Method

20

Filter bank generation 1 Sparse reference pixels 2 Dense error interpolation 4 Filter compositing 5 Sparse error computation 3

slide-21
SLIDE 21
  • 1. Filter bank generation
21

16 spp Sample Budget Filter bank Filter 1 Filter 2 Filter n … … … … 32 spp 16 spp

slide-22
SLIDE 22
  • 2. Sparse reference pixels
22

16 spp Sample Budget Filter bank Filter 1 Filter 2 Filter n … … … … 16 spp 128 spp per reference pixel

slide-23
SLIDE 23
  • 3. Sparse error computation
23

16 spp Sample Budget Filter bank Filter 1 Filter 2 Filter n … … … … 128 spp per filter cache

  • Serves as reference
  • Used to estimate filter error
  • Low-variance estimator
slide-24
SLIDE 24
  • 4. Dense error interpolation
  • Interpolation of sparse error estimate (per filter)
24

Sparse error (zoom-in) Interpolated error (zoom-in) Filter error using reference (zoom-in)

slide-25
SLIDE 25
  • 4. Dense error interpolation
25
  • Best selection from interpolated error leads to seams

Optimal selection (per-pixel) Seams (closeup)

slide-26
SLIDE 26
  • 5. Filter compositing

Globally optimize filter selection (seek labeling 𝑀)

argmin

𝑀

𝐹 𝑀

Globally optimize filter selection (seek labeling 𝑀)

argmin

𝑀

𝐹 𝑀 = 𝐹𝐸𝑏𝑢𝑏 𝑀

26

Data term Local errormaps Regularization term Solution image gradients Minimize MSE Avoid seams

Globally optimize filter selection (seek labeling 𝑀)

argmin

𝑀

𝐹 𝑀 = 𝐹𝐸𝑏𝑢𝑏 𝑀 + 𝜇 ∙ 𝐹𝑠𝑓𝑕𝑣𝑚𝑏𝑠𝑗𝑨𝑓𝑠 𝑀

slide-27
SLIDE 27
  • 5. Filter compositing
27
  • Solve by graph-cuts

„Fast approximate energy minimization via graph cuts”, Boykov et al. 2001

Globally optimized label map Filter 1 Filter 2 Filter n … … … … Cut

slide-28
SLIDE 28
  • 5. Filter compositing
28
  • Solve by graph-cuts

„Fast approximate energy minimization via graph cuts”, Boykov et al. 2001

Local selection Global selection

slide-29
SLIDE 29

Our Method

29

Filter bank generation 1 Sparse reference pixels 2 Dense error interpolation 4 Filter compositing 5 Sparse error computation 3

slide-30
SLIDE 30

Bells & Whistles

  • Choice of regularization in filter compositing
  • Integration of high-quality radiance values (not included the filter bank)
  • Select „best“ pixels for sparse error estimate
30
slide-31
SLIDE 31

Adaptive placement of sparse estimates

  • Required for highly variant error regions
  • Reduces residual variance in radiance estimate
31

Filter bank variance Monte Carlo variance Poisson sampling Importance sampling

slide-32
SLIDE 32

Results

32
slide-33
SLIDE 33

Results – San Miguel

Global ill illumination

33

MC 4096 spp 15,449 sec Our result 32 spp 146 + 13 sec MC 32 spp 146 sec

slide-34
SLIDE 34

Results - Chess

Depth-of

  • f-field
34

MC 4096 spp 1,492 sec Our result 8 spp 9 + 29 sec MC 8 spp 9 sec

slide-35
SLIDE 35

Results - Poolball

Motion blu lur

35

MC 4096 spp 10,989 sec Our result 8 spp 25 + 25 sec MC 8 spp 25 sec

slide-36
SLIDE 36

Results - Teapot

Glossy materials

36

MC 4096 spp 3,619 sec Our result 16 spp 14 + 8 sec MC 16 spp 14 sec

slide-37
SLIDE 37

Results - Dragon

Participating media

37

MC 4096 spp 12,464 sec Our result 32 spp 95 + 12 sec MC 32 spp 95 sec

slide-38
SLIDE 38

Results - Timings

38

Intel Core i7-2600, 3.40 GHz, 16 GB RAM, NVIDIA GeForce 780 GTX, Windows 7 64-bit Rendered with PBRT 2 path tracing.

8 filter 4 filter 8 filter 4 filter 8 filter 4 filter 8 filter 4 filter 8 filter 4 filter Rendering Filtering Error estimation Filter composite

slide-39
SLIDE 39

Error analysis

39

Two error sources

Residual variance in radiance Interpolation error

slide-40
SLIDE 40

Results – GID

(„Removing the Noise in Monte Carlo Rendering with General Image Denoising Algorithms”, Kalantari et al. 2013)

40

GID (8 spp) Ours (8 spp) GID (32 spp) Ours (32 spp) Reference

Chess scene

MSE=2.6491 SSIM=0.9516 MSE=1.38179 SSIM=0.9874 MSE=2.4006 SSIM=0.9558 MSE=0.8962 SSIM=0.9948

slide-41
SLIDE 41

Results – RD

(„Robust Denoising using Feature and Color Information”, Rousselle et al. 2013)

41

RD (16 spp) Ours (16 spp) RD (32 spp) Ours (32 spp) Reference

Dragon scene

MSE=13.6693 SSIM=0.9654 MSE=10.1914 SSIM=0.9599 MSE=9.3887 SSIM=0.9781 MSE=7.8838 SSIM=0.9768

slide-42
SLIDE 42

Error sparsity

  • Sparsity of error maps in transform domain (CDF 9/7 wavelets)
  • Redundant information
42

Gaussian σ=7 Gaussian σ=11 Gaussian σ=13 86.46% 88.58% 89.86% Guided radius=4 Guided radius=8 Guided radius=16 81.34% 87.07% 89.43% NLM BM3D BLS-GSM 60.06% 67.35% 73.62%

slide-43
SLIDE 43

Results – SURE [Stein1981]

43

Noisy SURE Our approach Reference

Sibenik scene

MSE=6.0644 SSIM=0.9066 MSE=0.7681 SSIM=0.9643 MSE=0.3556 SSIM=0.9829

slide-44
SLIDE 44

Conclusion

  • Summary
  • Redistributing samples can improve filter selection
  • Global filter selection removes image seams
  • Benefits
  • Works with arbitrary filters
  • No assumptions regarding scene and image content
  • Easy integration into existing rendering frameworks
44
slide-45
SLIDE 45

Outlook

  • Investigate other interpolation schemes
  • Adaptive sampling feedback loop
  • Temporal coherence
45
slide-46
SLIDE 46

Thank you for your attention!

46

Pablo Bauszat Elmar Eisemann Martin Eisemann Marcus Magnor

graphics.tudelft.nl graphics.tu-bs.de