What is the Range of Surface Reconstructions from a Gradient Field? - - PowerPoint PPT Presentation

What is the Range of Surface Reconstructions from a Gradient Field?

Writer: Amit Agrawal, Ramesh Raskar, and Rama Chellappa Presenter: Hosna Sattar Uni Saarland Milestones and Advances in Image Analysis, 2013

(ECCV 2006)

Milestones and Advances in Image Analysis, 2013

2

Motivation

motivation diffusion M-estimator Alpha-Surface Poisson solver General framework

Seamless Cloning Selection Editing Texture Flattening

• Compute gradient of image
• Manipulate the gradient field in order to

achieve the desired goal

Milestones and Advances in Image Analysis, 2013

3 3

Gradient field and its application

Second derivatives: and . They are identical! Right: Integration of gradient field ( , ) which is identical to original image.

Milestones and Advances in Image Analysis, 2013

4

Integrating the Modified Gradient Field

(x) f (x) f (x) (x) - f f (x)) (x), f curl (f f(x)) curl (

xy yx xy yx T y

x

     

In order to integrate the gradient field it should be curl-free:

xy

f

yx

f

xy

f

yx

f

• In fact, the modified gradient field might

even be non-integrable!

Milestones and Advances in Image Analysis, 2013

5

Integrating the Modified Gradient Field

Left: space of all solution right: add add correction gradient field to make it integrable.

 

y x   ,

x 

y 

• A common approach to achieve the surface

from the non-integrable gradient field is to minimized the last square error function:

• The goal is to obtain surface Z. p(x,y) and

q(x,y) are given non-integrable gradient field.

Milestones and Advances in Image Analysis, 2013

6

Problem statement

)dxdy

• q)

(Z

• p)

((Z J(Z)

y x



 

2 2

• Which can also write as:
• The Euler-Lagrange equation gives the

Poisson equation:

Milestones and Advances in Image Analysis, 2013

7

          q p div Z

2

                          y x q p Z Z

y x

 

Problem statement

Milestones and Advances in Image Analysis, 2013

8

content

motivation diffusion M-estimator Alpha-Surface Poisson solver General framework

Effect of outliers in 2D integration (a) True surface (b) Reconstruction using Poisson solver. (c) If the location of outliers were known, rest of the gradients can be integrated to obtain a much better estimate.

• Least square solution doesn't perform

well in presence of outliers:

Milestones and Advances in Image Analysis, 2013

9

Problem of Poisson equation

Milestones and Advances in Image Analysis, 2013

10

General framework



 dxdy , . . .) , q , p Z E(Z, p, q, J

d c d c b a

y x y x y x

A general solution can be obtained by minimizing the following n-th

• rder error functional:



                    ) dxdy Z Z E(Z, p, q, J if k n; k y x q q y x p p y x Z Z k, k - d k, c b a

y x d c k- y x d c k- y x b a k y x

d c d c b a

, 1 1 , , integer positive some for 1

1 1

Milestones and Advances in Image Analysis, 2013

11

General framework

(p,q) f ) y ,Z x (Z f y Z E (p,q), f ) y ,Z x (Z f x Z E : y Z E , x Z E ) y Z E , x Z E div( Z E 4 2 3 1 for form following consider we if : gives equation Lagrange

• Euler

the                   

(1) (2)

Milestones and Advances in Image Analysis, 2013

12

(p,q)) (p,q),f div(f Z E ) y ,Z x (Z f ) y ,Z x (Z f div 4 3 ) , (

2

1

   

: in to(1) (2) ing substituit by free curl is field gradient modified the while

In all solutions we assume Neumann boundary conditions given by:

.  



n Z

General framework

• To achieve Poisson equation from the

general solution its just need to assume:

Milestones and Advances in Image Analysis, 2013

13

Poisson solver

div(p, q) Z  2

(p,q)) (p,q),f div(f Z E ) y ,Z x (Z f ) y ,Z x (Z f div 4 3 ) , (

2

1

   

   Z E

) y ,Z x (Z f

1

) y ,Z x (Z f2

(p,q), f3 (p,q), f4

x Z

y Z

p

q

Milestones and Advances in Image Analysis, 2013

14

content

motivation diffusion M-estimator Alpha-Surface Poisson solver General framework

• Techniques for robust estimation:
• 1. α- Surface: Anisotropic scaling using

binary weights

• 2. Anisotropic scaling using continuous

weight

• 3. Affine transformation of gradient using

diffusion tensor

Milestones and Advances in Image Analysis, 2013

15

Continuum of solution

• Define initial spanning tree which is all

gradient correspond to edge and are inliers

Milestones and Advances in Image Analysis, 2013

16

α- Surface

          q Z y p Z x

y x

x 

y 

If α=0 we get our initial spanning tree and if α=1 we will get our poisson solver. By changing α one can trace a path in the solution space.

The α- Surface is a weighted approach where the weight are 1 for gradients in S and otherwise zero.

Milestones and Advances in Image Analysis, 2013

17

α- Surface formulation

dxdy

• q)

(Z b

• p)

(Z b J(Z) y x b

y y x x x



     

2 2 y y x

• .w.,

S, Z if 1 y) (x, b

• .w.,

S, Z if 1 ) , (

Corresponding Euler_ Lagrange is:

) , ( ) , ( q b p b div Z b Z b div

y x y y x x



• M- estimator: the effect of outliers is

reduced by replacing the squared error residual by another function of residual:

Milestones and Advances in Image Analysis, 2013

18

Anisotropic scaling using continuous weight

dxdy

• q)

y (Z k y w

• p)

x Z k x w J(Z)      2 ) 1 ( 2 )( 1 (  

• A method for image restoration from

noisy image.

• The Euler-Lagrange gives:

Milestones and Advances in Image Analysis, 2013

19

Affine transformation of gradient using diffusion tensor



                  dxdy q p Z Z D Z E

y x 2

) ( ) (

) ( ) . (           q p D div Z D div

Milestones and Advances in Image Analysis, 2013

20

                              q d p d q d p d div Z d Z d Z d Z d div y x d y x d y x d y x d D

y x y x 22 21 12 11 22 21 12 11 22 12 21 11

: below as combined lineary and scaled are gradients The ) , ( ) , ( ) , ( ) , (

Affine transformation of gradient using diffusion tensor

D is 2×2 symmetric , positive-definite matrix at each pixel.

Milestones and Advances in Image Analysis, 2013

21

results

Photometric Stereo on Vase: (Top row) Noisy input images and true surface (Next two rows) Reconstructed surfaces using various algorithms. (Right Column) One-D height plots for a can line across the middle of Vase. Better results are obtained using α-surface, Diffusion and M-estimator as compared to Poisson solver, FC and Regularization

Milestones and Advances in Image Analysis, 2013

22

results

Photometric Stereo on Mozart: Top row shows noisy input images and the true surface. Next two rows show the reconstructed surfaces using various algorithms. (Right Column) One-D height plots for a scan line across the Mozart face. Notice that all the features of the face are preserved in the solution given by α-surface, Diffusion and M-estimator as compared to other algorithms.

Milestones and Advances in Image Analysis, 2013

23

results

Photometric Stereo on Flowerpot: Left column shows 4 real images of a flower pot. Right columns show the reconstructed surfaces using various algorithms. The reconstructions using Poisson solver and Frankot-Chellappa algorithm are noisy and all features (such as top of flower pot) are not recovered. Diffusion, α-surface and M-estimator methods discount noise while recovering all the salient features.

Milestones and Advances in Image Analysis, 2013

24

results

Milestones and Advances in Image Analysis, 2013

25

results

• A. Agrawal, R. Raskar: Gradient Domain Manipulation Techniques in

Vision and Graphices. ICCV 2007 Course

• Advanced Image Analysis, Lecture 9 by Dr. Christian Schmaltz
• http://www.cfar.umd.edu/~aagrawal/eccv06/RangeofSurfaceReconstruct

ions.html

Milestones and Advances in Image Analysis, 2013

26

reference

Milestones and Advances in Image Analysis, 2013

27

content

motivation diffusion M-estimator Alpha-Surface Poisson solver General framework

Question?