Gaussian Light Field: Estimation of Viewpoint-Dependent Blur for - - PowerPoint PPT Presentation

▶
gaussian light field estimation of viewpoint dependent
SMART_READER_LITE
LIVE PREVIEW

Gaussian Light Field: Estimation of Viewpoint-Dependent Blur for - - PowerPoint PPT Presentation

Dec 22, 2023 197 likes •723 views

Gaussian Light Field: Estimation of Viewpoint-Dependent Blur for Optical See-Through Head-Mounted Displays Yuta Itoh, Toshiyuki Amano, Daisuke Iwai, and Gudrun Klinker IEEE ISMAR 2016, Tue. 20th Sep. 2016 Gaussian Light Field: Estimation of

slide-1
SLIDE 1

Yuta Itoh, Toshiyuki Amano, Daisuke Iwai, and Gudrun Klinker

Gaussian Light Field: Estimation of Viewpoint-Dependent Blur for Optical See-Through Head-Mounted Displays

IEEE ISMAR 2016, Tue. 20th Sep. 2016

slide-2
SLIDE 2

IEEE ISMAR 2016, Tue. 20th Sep. 2016 Yuta Itoh, Toshiyuki Amano, Daisuke Iwai, and Gudrun Klinker

Gaussian Light Field: Estimation of Viewpoint-Dependent Blur for Optical See-Through Head-Mounted Displays

slide-3
SLIDE 3

Blur artifacts in HMDs

slide-4
SLIDE 4

Head-Mounted Displays (HMDs)

4/28

Non Optical See-Through Optical See-Through (OST-HMD)

slide-5
SLIDE 5

We want make AR with HMD real

Magic Leap, 2016

slide-6
SLIDE 6

Realism in AR (with OST HMDs) Visual

[Kiyokawa ’01] [Liu ’08] [Lee ’09]

Spatial

[Azuma ’95] [Tuceryan ’00] [Genc ’02] [Grubert ‘08]

Temporal

[Zheng ’14] [Lincoln ’16]

slide-7
SLIDE 7

Realism in AR (with OST HMDs) Visual

[Kiyokawa ’01] [Liu ’08] [Lee ’09]

Spatial

[Azuma ’95] [Tuceryan ’00] [Genc ’02] [Grubert ‘08]

Temporal

[Zheng ’14] [Lincoln ’16]

slide-8
SLIDE 8

Image blur in (OST-)HMDs

Ideal Output

slide-9
SLIDE 9

Image blur in (OST-)HMDs

Actual observation

slide-10
SLIDE 10

Image blur in (OST-)HMDs

Ideal Output

slide-11
SLIDE 11

Image blur in (OST-)HMDs

Actual observation

slide-12
SLIDE 12

Image blur  Impulse Responses

Impulse responses

slide-13
SLIDE 13

Image blur  Impulse Responses

Impulse responses

slide-14
SLIDE 14

Why do we care about ?

slide-15
SLIDE 15

Related Work: SharpView

SharpView: Improved Clarity of Defocused Content on Optical See- Through Head-Mounted Displays, Oshima et al. 2015

slide-16
SLIDE 16

Related Work: Pinlight Displays

“Pinlight Displays: Wide Field of View Augmented Reality Eyeglasses using Defocused Point Light Sources” Maimone et al., TOG’14

slide-17
SLIDE 17

Related Work: Pinlight Displays

“Pinlight Displays: Wide Field of View Augmented Reality Eyeglasses using Defocused Point Light Sources” Maimone et al., TOG’14

slide-18
SLIDE 18

Related Work: Light field displays

The Light Field Stereoscope, Huan et al., SIGGRPAH 2015

slide-19
SLIDE 19

Optics Designs of OST-HMDs

(Free-form) prism Point source Waveguide Laser source Half-mirror Eye

Prism-based Waveguide-based Half-mirror

Based on: Hainich and Bimber, Displays Fundamentals and Applications, 2011

Retinal Scanning

slide-20
SLIDE 20

 View-dependent

Image blur in HMDs

 Non-linear (and RGB-wise)

slide-21
SLIDE 21

Image Undistortion for OST-HMDs

Image Undistortion for OST-HMDs

Itoh et al. IEEE TVCG ‘15 (VR ’15) & IEEE ISMAR ‘15

slide-22
SLIDE 22

How to model?

4D Light Field model + Point Spread Function (PSF)

Itoh et al. IEEE TVCG ‘15 (VR ’15) & IEEE ISMAR ‘15

slide-23
SLIDE 23
slide-24
SLIDE 24
  • rigin
slide-25
SLIDE 25
  • rigin
slide-26
SLIDE 26
  • rigin
slide-27
SLIDE 27
  • rigin
  • PSF
slide-28
SLIDE 28
  • rigin
slide-29
SLIDE 29
  • rigin
slide-30
SLIDE 30
  • rigin
slide-31
SLIDE 31
  • rigin
slide-32
SLIDE 32

1/

  • u

Gaussian

  • rigin
slide-33
SLIDE 33

NVIS ST60 Lumus DK32

slide-34
SLIDE 34

A cubic prism Holographic gratings NVIS ST60 Lumus DK32

slide-35
SLIDE 35

NVIS ST60 Lumus DK32

slide-36
SLIDE 36

Data collection

Raw image Filtered (Raw PSF) Training PSF Input

slide-37
SLIDE 37

Data collection

Ideal view Observed PSF (filtered) Estimated PSF Baseline PSF

Raw Image Raw PSF Training PSF Baseline (Avg. PSF)

slide-38
SLIDE 38

Learning

Training data  Regression 

slide-39
SLIDE 39

ST60 Better Better Better

Testing

slide-40
SLIDE 40

We failed to get enough #

  • f data for the red channel

DK32 Better Better

Testing

slide-41
SLIDE 41

e.g., Jones et al., IEEE 3DUI 2016

Future work

 Best PSF model?  Eye orientation+model  Real-time processing

? … … …

slide-42
SLIDE 42

 Spatio-Visual Realism in AR

Image blur in HMDs

 View-dep. blur model

 4D light field

+ Point Spread Functions

 Evaluation on two HMDs

 Cubic prism & Waveguides

Summary

Contact: itoh@in.tum.de

slide-43
SLIDE 43
slide-44
SLIDE 44

Appendix

slide-45
SLIDE 45

Related Work: Multi-focus HMDs

Liu et al. 2008 Konrad et al. 2016

slide-46
SLIDE 46

O M M’ C

slide-47
SLIDE 47

Observed view ฀ Simulated view with Observed PSF ฀ Simulated view with Estimated PSF ฀ Simulated view with Baseline PSF

Testing

No difference??? Raw PSF Estimated PSF Baseline (Avg. PSF)

slide-48
SLIDE 48
slide-49
SLIDE 49

Local v.s. Global sampling

slide-50
SLIDE 50
slide-51
SLIDE 51
  • Global: 80,000 points

+ Local: <150 points/grid