3D RECONSTRUCTION Reconstruction method Reconstruction from images - - PowerPoint PPT Presentation

▶
3d reconstruction reconstruction method
SMART_READER_LITE
LIVE PREVIEW

3D RECONSTRUCTION Reconstruction method Reconstruction from images - - PowerPoint PPT Presentation

Apr 12, 2023 155 likes •514 views

3D RECONSTRUCTION Reconstruction method Reconstruction from images Reconstruction from video Using Kinect Raw Depth Image Infrared laser projector Monochrome CMOS sensor Demo Kinect Raw data Real-time Reconstruction Pipeline Measurement

slide-1
SLIDE 1

3D RECONSTRUCTION

slide-2
SLIDE 2

Reconstruction from images Reconstruction from video

Reconstruction method

slide-3
SLIDE 3

Raw Depth Image

Infrared laser projector Monochrome CMOS sensor

Using Kinect

slide-4
SLIDE 4

Demo Kinect Raw data

slide-5
SLIDE 5

Real-time Reconstruction

slide-6
SLIDE 6

Raw Depth Image Noise Reduction Bilateral Filtering Measurement Compute Surface Vertex and Normal Map Pose Estimation ICP Update Reconstruction TSDF Surface Prediction Ray-cast

rk Tgk Vk, Nk Rk Sk Vk, Nk

Pipeline

Input: 20 frames * 640 * 480 * 12 = 614.8 MB/s

slide-7
SLIDE 7

Demo

Bilateral Filtering

slide-8
SLIDE 8

Raw Depth Image Noise Reduction Bilateral Filtering Measurement Compute Surface Vertex and Normal Map Pose Estimation ICP Update Reconstruction TSDF Surface Prediction Ray-cast

rk Tgk Vk, Nk Rk Sk Vk, Nk

Pipeline

slide-9
SLIDE 9

SVD

ICP 3D shape alignment

slide-10
SLIDE 10

Demo

ICP 3D shape alignment

slide-11
SLIDE 11

Raw Depth Image Noise Reduction Bilateral Filtering Measurement Compute Surface Vertex and Normal Map Pose Estimation ICP Update Reconstruction TSDF Surface Prediction Ray-cast

rk Tgk Vk, Nk Rk Sk Vk, Nk

Pipeline

slide-12
SLIDE 12

TSDF

Signed Distance Function The value in the cube corresponds to the signed distance to the closest zero crossing( surface).

slide-13
SLIDE 13

Truncated Signed Distance Function Signed Distance Function

TSDF

slide-14
SLIDE 14

Truncated Signed Distance Function Signed Distance Function Integrate the cubes from different position.

TSDF

slide-15
SLIDE 15
  • 1

Depth Map from Kinect

TSDF

slide-16
SLIDE 16
  • 1
  • 0.2

Depth Map from Kinect

TSDF

slide-17
SLIDE 17
  • 1
  • 0.2

0.05 Depth Map from Kinect

TSDF

slide-18
SLIDE 18
  • 1
  • 0.2

0.2 0.05 Depth Map from Kinect

TSDF

slide-19
SLIDE 19
  • 1

1

  • 0.2

0.2 0.05 Depth Map from Kinect

TSDF

slide-20
SLIDE 20
  • 1

1 1

  • 0.2

0.2 0.05 Depth Map from Kinect

TSDF

slide-21
SLIDE 21
  • 1

1 1

  • 0.2

0.5 0.05

  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 0.8
  • 0.8
  • 0.5
  • 0.5
  • 0.5
  • 0.05
  • 0.1
  • 0.03

0.05 0.1 0.3 0.3 0.5 1 1 1 1 1 1 1 1 1 1 Integration? or update? We have depth maps from different camera positions, how can we integrate them together ? What makes integration possible ? Weighted? or add up?

TSDF

slide-22
SLIDE 22
  • 1

1 1

  • 0.2

0.5 0.05

  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 0.8
  • 0.8
  • 0.5
  • 0.5
  • 0.5
  • 0.05
  • 0.1
  • 0.03

0.05 0.1 0.3 0.3 0.5 1 1 1 1 1 1 1 1 1 1 Integration? or update? We have depth maps from different camera positions, how can we integrate them together ? What makes integration possible ? Weighted? or add up?

TSDF

slide-23
SLIDE 23
  • 1

1 1

  • 0.2

0.5 0.05

  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 0.8
  • 0.8
  • 0.5
  • 0.5
  • 0.5
  • 0.05
  • 0.1
  • 0.03

0.05 0.1 0.3 0.3 0.5 1 1 1 1 1 1 1 1 1 1 Integration? or update? We have depth maps from different camera positions, how can we integrate them together ? What makes integration possible ? Weighted? or add up?

TSDF

slide-24
SLIDE 24
  • 1

1 1

  • 0.2

0.5 0.05

  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 0.8
  • 0.8
  • 0.5
  • 0.5
  • 0.5
  • 0.05
  • 0.1
  • 0.03

0.05 0.1 0.3 0.3 0.5 1 1 1 1 1 1 1 1 1 1 Only part of distance data is needed, so we can truncate the distance. To get the surface behind the surface. The camera is moving!

TSDF

slide-25
SLIDE 25
  • 0.3

1 1

  • 0.2

0.5 0.05

  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 0.8
  • 0.8
  • 0.5
  • 0.5
  • 0.5
  • 0.05
  • 0.1
  • 0.03

0.05 0.1 0.3 0.3 0.5 1 1 1 1 1 1 1 1 1 1 To get the surface behind the surface. The camera is moving! 1 time update !

TSDF

Only part of distance data is needed to represent the

  • bject, so we can truncate

the distance.

slide-26
SLIDE 26

1 1

  • 0.2

0.5 0.05

  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 0.8
  • 0.8
  • 0.5
  • 0.5
  • 0.5
  • 0.05
  • 0.1
  • 0.03

0.05 0.1 0.3 0.3 0.5 1 1 1 1 1 1 1 1 1 1 To get the surface behind the surface. The camera is moving! 2 times update !

TSDF

Only part of distance data is needed to represent the

  • bject, so we can truncate

the distance.

slide-27
SLIDE 27

0.3 1 1

  • 0.2

0.5 0.05

  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 0.8
  • 0.8
  • 0.5
  • 0.5
  • 0.5
  • 0.05
  • 0.1
  • 0.03

0.05 0.1 0.3 0.3 0.5 1 1 1 1 1 1 1 1 1 1 Only part of distance data is needed to represent the

  • bject, so we can truncate

the distance. To get the surface behind the surface. The camera is moving! 3 times update !

TSDF

slide-28
SLIDE 28

Raw Depth Image Noise Reduction Bilateral Filtering Measurement Compute Surface Vertex and Normal Map Pose Estimation ICP Update Reconstruction TSDF Surface Prediction Ray-cast

rk Tgk Vk, Nk Rk Sk Vk, Nk

Pipeline

slide-29
SLIDE 29

Cast only, no chasing. Transfer the TSDF cube in to some thing the computer can understand, Vertex fusion. Take a photo using X-ray.

RAY CASTING

slide-30
SLIDE 30
  • 1

1 1

  • 0.2

0.5 0.05

  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 0.8
  • 0.8
  • 0.5
  • 0.5
  • 0.5
  • 0.05
  • 0.1
  • 0.03

0.05 0.1 0.3 0.3 0.5 1 1 1 1 1 1 1 1 1 1 Detect the sign change. Two scales search Linear regression

RAY CASTING

slide-31
SLIDE 31
  • 1

1 1

  • 0.2

0.5 0.05

  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 1
  • 0.8
  • 0.8
  • 0.5
  • 0.5
  • 0.5
  • 0.05
  • 0.1
  • 0.03

0.05 0.1 0.3 0.3 0.5 1 1 1 1 1 1 1 1 1 1 Detect the sign change. Two scales search Linear regression Normal Vectors

RAY CASTING

slide-32
SLIDE 32

Demo

Real-time Reconstruction

slide-33
SLIDE 33

Reference

[1] KinectFusion: Real-Time Dense Surface Mapping and Tracking. Microsoft Research [2] B. Curless and M. Levoy. A volumetric method for building complex models from range images. [3] M. Harris, S. Sengupta, and J. D. Owens. Parallel prefix sum (scan) with CUDA. In H. Nguyen, editor, GPU Gems 3, chapter 39, pages 851–876. Addison Wesley, August 2007. 3.5 [4] C. Tomasi and R. Manduchi. Bilateral filtering for gray and color images. In Proceedings of the ICCV, 1998. [5] C. Rasch and T. Satzger. Remarks on the O(N) implementation of the fast marching method. [6] Y. Chen and G. Medioni. Object modeling by registration of multiple range images. Image and Vision Computing (IVC), 10(3):145–155,1992 [7] Kok-Lim Low Linear Least-Squares Optimization for Point-to-Plane ICP Surface Registration

slide-34
SLIDE 34