Image Processing using Variational Data Assimilation Methods - - PowerPoint PPT Presentation

Image Processing using Variational Data Assimilation Methods

Dominique B´ er´ eziat, Isabelle Herlin May 6, 2008

Introduction

• Use a new methodology ...

◮ to solve ill-posed problems of Image Processing, ◮ to take into account temporal information, ◮ to efficiently manage missing data.

• Well-posed problem [Hadamard, 1923]:

1 uniqueness of solution, 2 the solution depends continuously on the input data.

• In the Image Processing field:

◮ low level: gradient estimation, contours detection. ◮ high level: segmentation, restoration, matching, motion estimation,

shape from shading, ...

Solution

• Constraining the solution’s space to obtain a

well-posed [Tikhonov, 1963] problem.

• Minimization of a cost function [Mumford et al, 1989]:

Argmin

f∈L2(Ω)

• Ω

   F(f, g)2

• link to observation

+ α∇f2

regularization

   dx Ω bounded domain in

• Euler-Lagrange equations.

Temporal sequence

• The previous method is easily extended to the temporal

dimension: Argmin

f∈L2(Ω)

• Ω

T   F(f, g)2

• same model

+ αΨ(∇3f)

• regularization in

   dxdt (1) with ∇3 =

• ∂

∂x , ∂ ∂y , ∂ ∂t

T

• Missing data are not taken into account:

◮ aberrant values are smoothed; ◮ aberrant values on large areas (common situation in remote

sensing): smoothing is not sufficient.

• No realistic model of image structures’ evolution. Only linear and

regular dynamics are correctly handled.

Data Assimilation

• Used in inverse modeling and simulation in the environmental

context.

• U(x, t)

V(x, t) x ∈ Ω t ∈ [0, T]

• System to be solved w.r.t. U:

     ∂U ∂t (x, t) +

U(x, 0) = U0(x) + εb(x)

• The evolution equation provides the description of the temporal

dynamics.

• εm, εb, εo quantify the model, initial condition and observation

errors.

Variational method

• Minimize the following cost function:

Argmin

U

• (Ut +

• x,t

Q−1 (Ut +

• x′,t′

dxdtdx′dt′+

• (U(., 0) − U0)T
• x

B−1 (U(., 0) − U0)

• x′

dxdx′+

• H(U, V)T
• x,t

R−1

• x′,t′

dxdtdx′dt′

• Q(x, t, x′, t′), B(x, x′), R(x, t, x′, t′) covariance matrices of εm, εb,

εo.

• Euler-Lagrange equations.

Assimilation of images

• How to apply the Data Assimilation framework to solve ill-posed

Image Processing problems?

link to observation ∇f bounded regularization

• Our strategy:

1 The operator

• bservation.

2 Regularization must be performed by the evolution equation.

• Important points:

1 To choose a suitable evolution equation: strongly dependent on the

problem.

2 To choose the matrices of covariance Q and R.

Covariance matrix Q

• Inverse of covariance matrix:
• C(x, x′′)C−1(x′′, x′)dx′′ = δ(x − x′)
• The Dirac covariance: C(x, x′) = δ(x − x′).

◮ Inverse: C−1(x, x′) = δ(x − x′) ◮

• UT(x)C−1U(x′)dx =
• U2dx

→ zero-order regularization.

• The exponential covariance: C(x, x′) = exp(|x−x′|

σ

).

◮ Inverse: C−1(x, x′) = 1

2σ δ(x − x′) − σ 2δ′′(x − x′)

◮

• UT(x)C−1U(x′)dx =

1 2σ U2 + σ 2∇U2

• dx

→ first-order regularization.

Covariance matrix R

• R is weighting the contribution of the observation.
• Missing data: the acquisition is partially missing or noisy.
• Defining R is the natural way to manage missing data.

◮ If data is missing at time t1 on a given region: the related

covariance R has to be given to 0.

◮ Consequently, the observation equation does not act. ◮ The computation of the state vector is then only obtain by the

evolution equation.

• R has to be invertible (in convolution sense).

Application to optical flow

• w = apparent motion (optical flow), ∂x

∂t = w.

• Observation equation: the transport of image brightness.

dI dt (x, t) = ∇ITw + ∂I ∂t =

• Standard method [Horn et al, 1981]:

Argmin

w

(ITw + ∂I ∂t )2 + α∇w2

• dx

Application to optical flow

• Evolution equation: transport of velocity.

dw dt (x, t) = ∇wTw + wt =

• Formulation in the Data Assimilation framework:

∂w ∂t + ∇wTw = εm ∂I ∂t = −∇ITw + εo w(x, 0) = w0(x) + εb(x) initial condition

• Gradient values are assimilated into a model of velocity transport.

Matrices of covariance

• Q: exponential covariance insuring a first order regularization of

the velocity.

• R: managing missing data using a function f characterizing the

quality of the observation:

◮ Observation equation is verified even if the image spatio-temporal

gradient are close to 0.

◮ In this case, equation observation is then degenerated and

• bservations should not be considered.

◮ Covariance R:

R(x, t, x′, t) = r0(1 − f(V(x, t))) + r1f(V(x, t))δ(x − x)δ(t − t′) and f(I(x, t)) = ∇3I(x, t)

Some results

Figure: Comparison Data Assimilation / Horn-Schunk – frame 3

Some results

Figure: Comparison Data Assimilation / Horn-Schunk – frame 6

Some results

Figure: Comparison Data Assimilation / Horn-Schunk – frame 9

... with missing data

(a) Image gradient not available (b) Image gradient available

Figure: Result without observation on frame 5 and with observation on adjacent frames

Some results

Figure: Cuve Coriolis (courtesy of LEGI): comparison Data Assimilation / Horn-Schunk – frame 3

Some results

Figure: Cuve Coriolis: comparison Data Assimilation / Horn-Schunk – frame 3

Perspectives

• Experiment diffusion as evolution equation.
• How to perform spatial regularization: covariance / evolution

equation.

• Using observation in the diffusion.