ACCELERATED VISUAL TRACKING FRAMEWORK David Concha Gomez, Raul - - PowerPoint PPT Presentation

DEVELOPMENT OF A GPU ACCELERATED VISUAL TRACKING FRAMEWORK

David Concha Gomez, Raul Cabido Valladolid, Antonio Sanz Montemayor, Juan José Pantrigo (Universidad Rey Juan Carlos) {david.concha, raul.cabido, juanjose.pantrigo, antonio.sanz}@urjc.es

Agenda

 Introduction  2D Articulated Tracking  CUDA Adaptation  3D Tracking  3D Articulated Tracking

Agenda

 Introduction  2D Articulated Tracking  CUDA Adaptation  3D Tracking  3D Articulated Tracking

Introduction

 The goal is to build a framework that eases the

development of visual tracking systems

 Visual tracking estimates the evolution of a system

state through time

 Particle Filter (PF) uses a population of N estimations  It can be computationally expensive  But it is highly parallelizable

Introduction (particle filter)

 Initialization:  Create a new random population

Particle Population P1=(x1,y1,p1) P2=(x2,y2,p2) … Pk=(xk,yk,pk) … Pn=(xn,yn,pn) pi=??

P1 Pk P2 Pn

Introduction (particle filter)

 Evaluation:  Using the measurement model, each particle is weighted

Particle Population P1=(x1,y1,3) P2=(x2,y2,27) … Pk=(xk,yk,63) … Pn=(xn,yn,4)

P1 Pk P2 Pn

Introduction (particle filter)

 Estimation:  Particle with the highest weight is selected as the best

estimator at time step t

Introduction (particle filter)

 Selection and diffusion:  A new population is generated based on the system

estimate, and every particle is diffused to provide diversity

t+1 t t

Agenda

 Introduction  2D Articulated Tracking  CUDA Adaptation  3D Tracking  3D Articulated Tracking

2D Articulated Tracking

 Our 2D articulated tracking problem in

2009

 N segments  X=(x, y , q1, q2, q3,…, qn)

2D Articulated Tracking

 In each frame we know:  The previous state of the system  The blob information  The edge information

2D Articulated Tracking

 Each particle defines a image regions that are

measured and compacted in tiles (32x32)

X=(x, y , q1, q2, q3 , q4 , q5 , q6)

2D Articulated Tracking

 Because the evaluation of each particle is independent

• n each other, It is highly parallelizable

2D Articulated Tracking

 Reducing the compact texture results in the particles

weights

 Reducing all the weights give the best particle  A new population is created by a resampling technique

(or any other mechanism)

2D Articulated Tracking

629.1 585.1 584.3 284.1 260.5 258.0 94.4 79.1 79.1 0.0 100.0 200.0 300.0 400.0 500.0 600.0 700.0

Standard PF PF with Geometric Templates PF with Geom.Temp & Elitism

GTX260 and 640x480 Fps (higher is better) 256 1024 4096 # Particles

Agenda

 Introduction  2D Articulated Tracking  CUDA Adaptation  3D Tracking  3D Articulated Tracking

CUDA Adaptation

 OpenGL + Shaders:  Good performance  Reduction: requires power of 2 tiles (i.e. 32x32)  OpenGL for rasterizer, CUDA for Reduction:  Reduction: more efficient for arbitrary tile size  Interoperability impact performance  CUDA Rasterizer and Reduction:  For high number of tiles it is as efficient as OpenGL+Shader

without its drawbacks

Agenda

 Introduction  2D Articulated Tracking  CUDA Adaptation  3D Tracking  3D Articulated Tracking

3D Visual tracking

 Calibrated and

synchronized system

 In a multiview system,

many images simultaneously

 Increased

computational cost of evaluation

3D Visual tracking

 3 DOF (x,y,z) to a 3D Volume (8 vertices)

(xk,yk,zk) 8 vertices

3D Visual tracking

 3D Volume (8 vertices) to an Axis Aligned Bounding Box

8 vertices AAB B

For each camera







m m y

x y x l k t t i v

l k I

, , , ,

) , ( p

3D Visual tracking

 One thread block obtain

the weight of one particle

 The AABB approach

simplified computation but induce some error

 The remaining PF

stages are similar

3D Visual tracking

50 100 150 200 250 300 350 400 450 10 100 500 1000 FPS # Particles

Performance (higher is better)

CPU 1 thread CPU 2 thread CPU 4 thread CPU 8 thread GTX 780

20 40 60 80 100 120 CPU CPU+GPU(W) CPU+GPU(BS+W) GPU

Hybrid configuration 1000 particles

CPU 1 thread CPU 8 threads GPU

3D Visual tracking

FPS x1 x3.9 x21.8 x38.2 x46.4 x46.7 x49.7

3D Visual tracking

50 100 150 200 250 300 350 400 450 10 100 500 1000 Partículas

GPUs Comparison

GTX 680 GTX 780 GTX Titan Black FPS

Particles

3D Visual tracking

• 1500
• 1000
• 500

500 1000 1500 1 76 151 226 301 376 451 526 601 676 751 826 901

Distance (mm)

Ground Truth vs AABB

Ground Truth X Ground Truth Y

Agenda

 Introduction  2D Articulated Tracking  CUDA Adaptation  3D Object Tracking  3D Articulated Tracking (work in progress)

3D Articulated Tracking

 Articulated body model with more than 30

DOF

 larger particle population  very computationally expensive evaluation  Each particle is represented by multiple

regions (articulated) in many images (3D)

 Regions no longer axis aligned and

cylindrical

+

3D Articulated Tracking

 PF alone is not enough  Huge state space

Conclusions

 Particle filter is a scalable and parallel friendly method.  Its weight computation is the most demanding stage  2D articulated tracking  Up to 630 fps on GPU using shaders (256 particles)  3D object tracking  Improvement of almost x50 against CPU  Good accuracy with 100 particles running at 300 fps.  Now combining both ideas for the 3D articulated problem

Questions

Thank you for your attention

2004

SIGGRAPH poster First PF weighting on GPU

2006

SIGGRAPH poster Complete PF on GPU

2009

GPUTA 2009 2D articulated tracking

2012

GTC CUDA Rasterizer

2013

GTC 3D Object Tracking (Kinect+GPU)

2014

GTC High-Performance Multiview 3D Tracking Using Particle Filters

2014

JRTIP CPU vs GPU Performance study of PF

2015-16 ??

3D articulated