Motion Estimation by Affine Transforms Motion Estimation by Affine - - PowerPoint PPT Presentation

Motion Estimation by Affine Transforms Motion Estimation by Affine Transforms Motion Estimation by Affine Transforms Motion Estimation by Affine Transforms Based on Based on Codirectionality of Movements Codirectionality of Movements Codirectionality of Movements Codirectionality of Movements

Mohrekesh, S. Samavi, N. Karimi, S. Shirani, P. Behnamfar

Department of Electrical and Computer Engineering Department of Electrical and Computer Engineering Isfahan Isfahan University of Technology, Isfahan, Iran University of Technology, Isfahan, Iran McMaster McMaster University, Hamilton, University, Hamilton, Canada Canada McMaster McMaster University, Hamilton, University, Hamilton, Canada Canada University of British Colombia, Canada University of British Colombia, Canada

Outline Outline

• Introduction
• Motion vectors
• Block based
• Mesh based
• ABC

ABC

• Results
• Conclusion
• Conclusion

Importance of Video Compression

• Improvements in Video Processing
• Video Applications

pp

• Machine vision
• Medical imaging

Vid f i

• Video conferencing
• Remote learning
• Information storage/transmission in
• Information storage/transmission in

limited memory/bandwidth Impossible storage/transmission of Impossible storage/transmission of raw data Definite need for compression Definite need for compression

Introduction: Video Compression

• Video Compression Basis
• Motion estimation/compensation
• Time redundancy elimination
• Differences Between Methods

D fferences Between Methods

• Complexity
• Accuracy

Accuracy

• Time for calculation

Block Based Motion Estimation

• Best mach of current block in the

f f reference frame

• Current block displacement from

reference block Motion Vector

i

u x ) y , x ( f − =

v y ) y x ( g − =

⎥ ⎦ ⎤ ⎢ ⎣ ⎡

i i

v u

i

v y ) y , x ( g

⎦ ⎣

i

v

Reference Frame Current Frame

Error Criteria

( )

∑∑

− −

− =

1 N 2 1 N ij ij 2

R C N 1 MSE

( )

∑∑

= = i j j j 2

N

∑∑

− = − =

− =

1 N i 1 N j ij ij 2

R C N 1 MAE

= = i j

∑∑

− − 1 N 1 N

∑∑

= =

− =

i j ij ij

R C SAD

Block Based Shortcomings

• Just Translation
• Unable to identify motions such as:
• Rotation
• Shearing
• Zoom in/out
• Disability in Codirectionality
• Equal Motions for Pixels of a Block
• Equal Motions for Pixels of a Block
• Reconstructed Frame Discontinuities

Mesh Based Motion Estimation

• Various Motions Modeling
• Various Motions Modeling
• Using Transforms

F h

• Current Frame Mesh
• Mesh
• Regular: less accurate
• Irregular: more complex

g p

Transforms

• Affine

3 i 2 i 1 i

a y a x a ) y , x ( f + + =

• Affine

6 i 5 i 4 i

a y a x a ) y , x ( g + + =

• Ability to Model Different Motions

y

• Disability in Codirectionality
• More Complex
• More Complex

Proposed Method: ABC

• Affine transform Based on Codirectionality
• Current Frame Partitioning
• Assuming Block for Nodes

g

• Block Matching

Reference Frame Current Frame

ABC

• Finding Triangles Motion Vectors
• Triangle Partitioning (if needed)

Triangle Partitioning (if needed)

ABC

• Transforms
• Affine
• Affine
• Bilinear vector

interpolation interpolation

• Translation

ABC: Transform Selection Criteria

• Affine Domain
• Rotation
• Zoom in
• Zoom out

Almost equilateral triangles produced Vector difference lengths almost equal Selection based on closeness of vector differences

ABC: Finding Different Vector

a b c

One of differences smaller than half of average of the others smaller than half of average of the others

ABC ABC

{ }

A,B,C α,β ΔABC each for

1

∈ ∀

ABC ABC

( )

( ) ( )

( )

( ) ( )

( )

( )

( ) ( ) ( )

{ }

β α α,β γ d ,N N y y x x α,β d

β MV α MV β MV α MV 2 1 2 1 2 2

min arg ≠ ∈ ∀ = − + − =

γ

( ) ( ) ( )

{ }

( ) ( ) { } { }

γ d ,N N d if

γ 2 1

10 1 <

∑

γ

{ } { }

N F N N F N ,N N A,B,C F

2 2 1 1 2 1

2 , 2 + = ′ + = ′ − =

( ) ( ) ( )

N N ΔF N ΔFN e interpolat F ,MV N N ΔF translate

2 1 2 1 2 1

2 2 ′ ′ − ′ ′

( ) ( ) ( ) ( ) ( )

C ,MV B ,MV A ΔABC,MV affine else N N ΔF N ΔFN e interpolat

2 1 2 1

( ) ( ) ( ) ( )

for end if end ff

Results Results

Paris Paris

ABC

Mobile

Results Results

Mobile

ABC ABC

Foreman

Results Results

Foreman

ABC ABC

Hall Monitor

Results Results

Hall Monitor

ABC ABC

Results Results

Original frame frame MFMB result result ABC result

Vid C i I t Conclusion Conclusion

• Video Compression Importance
• Time redundancy
• Motion Estimation
• Block based method
• Mesh based method
• Proposed ABC Method

Proposed ABC Method

• Better performance in codirectionality
• Higher PSNR

Higher PSNR

• Results