People-Tracking-by-Detection and People-Detection-by-Tracking - - PowerPoint PPT Presentation

▶
people tracking by detection and people detection by
SMART_READER_LITE
LIVE PREVIEW

People-Tracking-by-Detection and People-Detection-by-Tracking - - PowerPoint PPT Presentation

Oct 31, 2023 9 likes •902 views

People-Tracking-by-Detection and People-Detection-by-Tracking Mykhaylo Andriluka Stefan Roth Bernt Schiele Department of Computer Science TU Darmstadt People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008 Motivation

slide-1
SLIDE 1

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

People-Tracking-by-Detection and People-Detection-by-Tracking

Mykhaylo Andriluka Bernt Schiele Stefan Roth Department of Computer Science TU Darmstadt

slide-2
SLIDE 2

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Motivation

  • Challenges for detection:
  • Partial occlusions
  • Appearance variation
  • Data association difficult
  • Challenges for tracking:
  • Dynamic backgrounds
  • Multiple people
  • Frequent long term occlusions

2

  • Goal: Detection and tracking of people in complex scenes
slide-3
SLIDE 3

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Motivation

  • Challenges for detection:
  • Partial occlusions
  • Appearance variation
  • Data association difficult
  • Challenges for tracking:
  • Dynamic backgrounds
  • Multiple people
  • Frequent long term occlusions

3

  • Goal: Detection and tracking of people in complex scenes
slide-4
SLIDE 4

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Overview

4

Three stages of our multi-person detection and tracking system:

  • 1. Single-frame

detection

slide-5
SLIDE 5

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Overview

4

Three stages of our multi-person detection and tracking system:

  • 1. Single-frame

detection

  • 2. Tracklet detection
slide-6
SLIDE 6

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Overview

4

Three stages of our multi-person detection and tracking system:

  • 1. Single-frame

detection

  • 2. Tracklet detection
  • 3. Tracking through
  • cclusion
slide-7
SLIDE 7

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Previous Work

  • People Detection & Tracking:
  • [Fossati et al., CVPR 2007]: 3D articulated tracking aided by

detection, single person, ground plane needed.

  • [Leibe et al., ICCV 2007]: Detection of tracking of multiple people,

high viewpoint → no full-body occlusions.

  • [Ramanan et al., PAMI 2007]: Appearance model learned from

people detection, then used for tracking and data association.

  • [Wu & Nevatia, IJCV 2007]: Use detection for tracking, works for

multiple people → no articulations, detector not aided by tracking.

  • Here:
  • More people
  • Significant, long-term full-body occlusions
  • However: more restricted scenario (2-D, people in side views)

5

slide-8
SLIDE 8

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Overview

6

  • 1. Single-frame

detection

  • 2. Tracklet detection
  • 3. Tracking through
  • cclusion

Three stages of our multi-person detection and tracking system:

slide-9
SLIDE 9

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single-frame Detector: partISM

  • Appearance of parts:

Implicit Shape Model (ISM)

[Leibe, Seemann & Schiele, CVPR 2005]

7

slide-10
SLIDE 10

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single-frame Detector: partISM

  • Appearance of parts:

Implicit Shape Model (ISM)

[Leibe, Seemann & Schiele, CVPR 2005]

7

xo

slide-11
SLIDE 11

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single-frame Detector: partISM

  • Appearance of parts:

Implicit Shape Model (ISM)

[Leibe, Seemann & Schiele, CVPR 2005]

7

xo

slide-12
SLIDE 12

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single-frame Detector: partISM

  • Appearance of parts:

Implicit Shape Model (ISM)

[Leibe, Seemann & Schiele, CVPR 2005]

8

x1 x2 x3 x4 x5 x6 x8 x7 xo

slide-13
SLIDE 13

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single-frame Detector: partISM

  • Appearance of parts:

Implicit Shape Model (ISM)

[Leibe, Seemann & Schiele, CVPR 2005]

  • Part decomposition and inference:

Pictorial structures model

[Felzenszwalb & Huttenlocher, IJCV 2005]

8

x1 x2 x3 x4 x5 x6 x8 x7 xo

slide-14
SLIDE 14

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single-frame Detector: partISM

  • Appearance of parts:

Implicit Shape Model (ISM)

[Leibe, Seemann & Schiele, CVPR 2005]

  • Part decomposition and inference:

Pictorial structures model

[Felzenszwalb & Huttenlocher, IJCV 2005]

8

x1 x2 x3 x4 x5 x6 x8 x7 xo

slide-15
SLIDE 15

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single-frame Detector: partISM

  • Appearance of parts:

Implicit Shape Model (ISM)

[Leibe, Seemann & Schiele, CVPR 2005]

  • Part decomposition and inference:

Pictorial structures model

[Felzenszwalb & Huttenlocher, IJCV 2005]

8

p(L|E) ∝ p(E|L)p(L)

Body-part positions Image evidence

x1 x2 x3 x4 x5 x6 x8 x7 xo

slide-16
SLIDE 16

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

  • Structure of the prior distribution :
  • Articulation variable models correlations

between part positions.

  • Given articulation, prior on configuration

becomes a star model.

Part Decomposition

  • - configuration of

body parts

9

L = {xo, x1, . . . , x8} p(L) xi a xo a

articulation

  • bject center

part position

x1 x2 x3 x4 x5 x6 x7 x8 xo

slide-17
SLIDE 17

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

  • Structure of the prior distribution :
  • Articulation variable models correlations

between part positions.

  • Given articulation, prior on configuration

becomes a star model.

Part Decomposition

  • - configuration of

body parts

9

L = {xo, x1, . . . , x8} p(L) xi a xo a

articulation

  • bject center

part position

x1 x2 x3 x4 x5 x6 x7 x8 xo

slide-18
SLIDE 18

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

  • Structure of the prior distribution :
  • Articulation variable models correlations

between part positions.

  • Given articulation, prior on configuration

becomes a star model.

Part Decomposition

  • - configuration of

body parts

9

L = {xo, x1, . . . , x8} p(L) xi a xo a

articulation

  • bject center

part position

x1 x2 x3 x4 x5 x6 x7 x8 xo

slide-19
SLIDE 19

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

  • Structure of the prior distribution :
  • Articulation variable models correlations

between part positions.

  • Given articulation, prior on configuration

becomes a star model.

Part Decomposition

  • - configuration of

body parts

10

L = {xo, x1, . . . , x8} p(L) xi a xo

articulation

  • bject center

part position Covariance and mean part positions for .

p(xi|xo)

a

slide-20
SLIDE 20

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single Frame Detection

  • Detections at equal error rate:

11

HOG 4D-ISM partISM

slide-21
SLIDE 21

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single-frame Detection Results

12

TUD pedestrians data No occlusions

  • partISM clearly outperforms 4D-ISM [Seemann et al, DAGM’06].
  • Outperforms HOG [Dalal&Triggs, CVPR’05] with much less training

data (Note: we only use sideviews).

slide-22
SLIDE 22

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Overview

13

  • 1. Single-frame

detection

  • 2. Tracklet detection
  • 3. Tracking through
  • cclusion

Three stages of our multi-person detection and tracking system:

slide-23
SLIDE 23

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection in Short Subsequences

  • Given:
  • Want:
  • Posterior over positions and configurations:

14

E = [E1, . . . , Em]

frame m

...

frame 2 frame 1

x1 x2 x3 x4 x5 x6 x7 x8 xo

  • verlapping subsequences
slide-24
SLIDE 24

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection in Short Subsequences

  • Given:
  • Want:
  • Posterior over positions and configurations:

14

E = [E1, . . . , Em]

frame m

...

frame 2 frame 1

Xo∗ = [xo∗

1 , . . . , xo∗ m]

body positions

x1 x2 x3 x4 x5 x6 x7 x8 xo xo

  • verlapping subsequences
slide-25
SLIDE 25

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection in Short Subsequences

  • Given:
  • Want:
  • Posterior over positions and configurations:

14

E = [E1, . . . , Em]

frame m

...

frame 2 frame 1

Xo∗ = [xo∗

1 , . . . , xo∗ m]

body positions Y∗ = [y∗

1, . . . , y∗ m]

body configurations

x1 x2 x3 x4 x5 x6 x7 x8 xo

−200 −150 −100 −50 50 100 50 100 150 200 250

xo

  • verlapping subsequences
slide-26
SLIDE 26

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection in Short Subsequences

  • Given:
  • Want:
  • Posterior over positions and configurations:

14

E = [E1, . . . , Em]

p(Xo∗, Y∗|E) ∝ p(E|Xo∗, Y∗)p(Xo∗)p(Y∗).

frame m

...

frame 2 frame 1

Xo∗ = [xo∗

1 , . . . , xo∗ m]

body positions Y∗ = [y∗

1, . . . , y∗ m]

body configurations

x1 x2 x3 x4 x5 x6 x7 x8 xo

−200 −150 −100 −50 50 100 50 100 150 200 250

xo

  • verlapping subsequences
slide-27
SLIDE 27

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection in Short Subsequences

  • Given:
  • Want:
  • Posterior over positions and configurations:

14

Likelihood model (partISM)

E = [E1, . . . , Em]

p(Xo∗, Y∗|E) ∝ p(E|Xo∗, Y∗)p(Xo∗)p(Y∗).

frame m

...

frame 2 frame 1

Xo∗ = [xo∗

1 , . . . , xo∗ m]

body positions Y∗ = [y∗

1, . . . , y∗ m]

body configurations

x1 x2 x3 x4 x5 x6 x7 x8 xo

−200 −150 −100 −50 50 100 50 100 150 200 250

xo

  • verlapping subsequences
slide-28
SLIDE 28

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection in Short Subsequences

  • Given:
  • Want:
  • Posterior over positions and configurations:

14

speed prior (Gaussian) Likelihood model (partISM)

E = [E1, . . . , Em]

p(Xo∗, Y∗|E) ∝ p(E|Xo∗, Y∗)p(Xo∗)p(Y∗).

frame m

...

frame 2 frame 1

Xo∗ = [xo∗

1 , . . . , xo∗ m]

body positions Y∗ = [y∗

1, . . . , y∗ m]

body configurations

x1 x2 x3 x4 x5 x6 x7 x8 xo

−200 −150 −100 −50 50 100 50 100 150 200 250

xo

  • verlapping subsequences
slide-29
SLIDE 29

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection in Short Subsequences

  • Given:
  • Want:
  • Posterior over positions and configurations:

14

dynamical body model (hGPLVM) speed prior (Gaussian) Likelihood model (partISM)

E = [E1, . . . , Em]

p(Xo∗, Y∗|E) ∝ p(E|Xo∗, Y∗)p(Xo∗)p(Y∗).

frame m

...

frame 2 frame 1

Xo∗ = [xo∗

1 , . . . , xo∗ m]

body positions Y∗ = [y∗

1, . . . , y∗ m]

body configurations

x1 x2 x3 x4 x5 x6 x7 x8 xo

−200 −150 −100 −50 50 100 50 100 150 200 250

xo

  • verlapping subsequences
slide-30
SLIDE 30

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Modeling Body Dynamics

  • is very high-dimensional: Full body poses in frames.
  • Model the body dynamics using hierarchical Gaussian process

latent variable model (hGPLVM) [Lawrence&Moore, ICML 2007]

15

Y∗ m

slide-31
SLIDE 31

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Modeling Body Dynamics

  • is very high-dimensional: Full body poses in frames.
  • Model the body dynamics using hierarchical Gaussian process

latent variable model (hGPLVM) [Lawrence&Moore, ICML 2007]

15

Y Configuration

−200 −150 −100 −50 50 100 50 100 150 200 250

yi Y = [yi ∈ RD]

Y∗ m

slide-32
SLIDE 32

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Modeling Body Dynamics

  • is very high-dimensional: Full body poses in frames.
  • Model the body dynamics using hierarchical Gaussian process

latent variable model (hGPLVM) [Lawrence&Moore, ICML 2007]

15

Y Configuration

−200 −150 −100 −50 50 100 50 100 150 200 250

yi Y = [yi ∈ RD] Latent space Z Z = [zi ∈ Rq]

Y∗ m

slide-33
SLIDE 33

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Modeling Body Dynamics

  • is very high-dimensional: Full body poses in frames.
  • Model the body dynamics using hierarchical Gaussian process

latent variable model (hGPLVM) [Lawrence&Moore, ICML 2007]

15

Y Configuration

−200 −150 −100 −50 50 100 50 100 150 200 250

yi Y = [yi ∈ RD] Latent space Z Z = [zi ∈ Rq] Time (frame #) T T = [ti ∈ R]

Y∗ m

slide-34
SLIDE 34

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Modeling Body Dynamics

  • is very high-dimensional: Full body poses in frames.
  • Model the body dynamics using hierarchical Gaussian process

latent variable model (hGPLVM) [Lawrence&Moore, ICML 2007]

15

p(Y|Z, θ) =

D

  • i=1

N(Y:,i|0, Kz) Y Configuration

−200 −150 −100 −50 50 100 50 100 150 200 250

yi Latent space Z Time (frame #) T

Y∗ m

slide-35
SLIDE 35

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Modeling Body Dynamics

  • is very high-dimensional: Full body poses in frames.
  • Model the body dynamics using hierarchical Gaussian process

latent variable model (hGPLVM) [Lawrence&Moore, ICML 2007]

15

p(Y|Z, θ) =

D

  • i=1

N(Y:,i|0, Kz) p(Z|T, ˆ θ) =

q

  • i=1

N(Z:,i|0, KT) Y Configuration

−200 −150 −100 −50 50 100 50 100 150 200 250

yi Latent space Z Time (frame #) T

Y∗ m

slide-36
SLIDE 36

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Modeling Body Dynamics

  • is very high-dimensional: Full body poses in frames.
  • Model the body dynamics using hierarchical Gaussian process

latent variable model (hGPLVM) [Lawrence&Moore, ICML 2007]

15

p(Y|Z, θ) =

D

  • i=1

N(Y:,i|0, Kz) p(Z|T, ˆ θ) =

q

  • i=1

N(Z:,i|0, KT) training Y Configuration

−200 −150 −100 −50 50 100 50 100 150 200 250

yi Latent space Z Time (frame #) T

Y∗ m

slide-37
SLIDE 37

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection

  • Tracklets are local maxima of:
  • Local maxima can be found using standard non-linear
  • ptimization (e.g. conjugate gradients).
  • How can we provide good initial hypotheses for
  • ptimization?

16

p(Xo∗, Y∗|E) ∝ p(E|Xo∗, Y∗)p(Xo∗)p(Y∗).

slide-38
SLIDE 38

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection

17

slide-39
SLIDE 39

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection

17

slide-40
SLIDE 40

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection

17

slide-41
SLIDE 41

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection

17

propagate detection

slide-42
SLIDE 42

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection

17

propagate detection hGPLVM mean prediction

slide-43
SLIDE 43

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection

17

propagate detection hGPLVM mean prediction

slide-44
SLIDE 44

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection

17

propagate detection hGPLVM mean prediction

slide-45
SLIDE 45

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection

17

propagate detection hGPLVM mean prediction

slide-46
SLIDE 46

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracklet Detection

17

propagate detection hGPLVM mean prediction

pose

  • ptimization
slide-47
SLIDE 47

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single-Frame Detector vs. Tracklet Detector

  • At equal error rate:
  • Fewer false positives.
  • More robust detection of partially occluded people.

18

partISM Tracklet detector

slide-48
SLIDE 48

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single-Frame Detector vs. Tracklet Detector

  • At equal error rate:
  • Fewer false positives.
  • More robust detection of partially occluded people.

18

partISM Tracklet detector

slide-49
SLIDE 49

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single-Frame Detector vs. Tracklet Detector

  • At equal error rate:
  • Fewer false positives.
  • More robust detection of partially occluded people.

18

partISM Tracklet detector

slide-50
SLIDE 50

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single-Frame Detector vs. Tracklet Detector

  • At equal error rate:
  • Fewer false positives.
  • More robust detection of partially occluded people.

18

partISM Tracklet detector

slide-51
SLIDE 51

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Single-Frame Detector vs. Tracklet Detector

  • At equal error rate:
  • Fewer false positives.
  • More robust detection of partially occluded people.

18

partISM Tracklet detector

slide-52
SLIDE 52

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Detection Performance

  • Significant improvement over single-frame detector.
  • Also at high precision levels.

19

TUD campus data With occlusions (up to 50%)

slide-53
SLIDE 53

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Overview

20

  • 1. Single-frame

detection

  • 2. Tracklet detection
  • 3. Tracking through
  • cclusion

Three stages of our multi-person detection and tracking system:

slide-54
SLIDE 54

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

21

...

t

t + 1 t + 2 t + 3

slide-55
SLIDE 55

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

21

Candidate poses from all

  • verlapping tracklets

...

t

t + 1 t + 2 t + 3

slide-56
SLIDE 56

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

21

Candidate poses from all

  • verlapping tracklets

...

t

t + 1 t + 2 t + 3

slide-57
SLIDE 57

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

21

Candidate poses from all

  • verlapping tracklets

...

t

t + 1 t + 2 t + 3

slide-58
SLIDE 58

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

21

Candidate poses from all

  • verlapping tracklets

...

t

t + 1 t + 2 t + 3

slide-59
SLIDE 59

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

21

Candidate poses from all

  • verlapping tracklets

...

t

t + 1 t + 2 t + 3

slide-60
SLIDE 60

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

21

Candidate poses from all

  • verlapping tracklets

...

t

t + 1 t + 2 t + 3

slide-61
SLIDE 61

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

22

...

t

t + 1 t + 2 t + 3

slide-62
SLIDE 62

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

22

Viterbi Decoding

...

t

t + 1 t + 2 t + 3

slide-63
SLIDE 63

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

22

Viterbi Decoding

...

t

t + 1 t + 2 t + 3

slide-64
SLIDE 64

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

22

Viterbi Decoding

...

t

t + 1 t + 2 t + 3

slide-65
SLIDE 65

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

22

Viterbi Decoding

...

t

t + 1 t + 2 t + 3

slide-66
SLIDE 66

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

22

Viterbi Decoding

...

t

t + 1 t + 2 t + 3

slide-67
SLIDE 67

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Tracks from Overlapping Tracklets

22

Viterbi Decoding

...

t

t + 1 t + 2 t + 3

slide-68
SLIDE 68

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Finding Multiple Tracks

23

  • Find the best

track

  • Remove its

hypotheses

  • Repeat

...

t

t + 1 t + 2 t + 3

slide-69
SLIDE 69

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Finding Multiple Tracks

23

  • Find the best

track

  • Remove its

hypotheses

  • Repeat

...

t

t + 1 t + 2 t + 3

slide-70
SLIDE 70

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Finding Multiple Tracks

23

  • Find the best

track

  • Remove its

hypotheses

  • Repeat

...

t

t + 1 t + 2 t + 3

slide-71
SLIDE 71

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Occlusion Event

24

...

t

t + 1 t + 2 t + 3

slide-72
SLIDE 72

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Occlusion Event

24

...

“bad” detections

t

t + 1 t + 2 t + 3

slide-73
SLIDE 73

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Occlusion Event

24

...

“bad” detections

t

t + 1 t + 2 t + 3

slide-74
SLIDE 74

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Occlusion Event

24

...

“bad” detections

terminate if low-probability for any transition

t

t + 1 t + 2 t + 3

slide-75
SLIDE 75

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Appearance Model for Occlusion Recovery

  • Extract person-specific

appearance model for each limb:

  • Color histogram.
  • Require relatively accurate

pose estimate:

  • Pose from extracted tracks.
  • Appearance comparison

measure:

  • Bhattacharyya distance.

25

slide-76
SLIDE 76

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Appearance Model for Occlusion Recovery

  • Extract person-specific

appearance model for each limb:

  • Color histogram.
  • Require relatively accurate

pose estimate:

  • Pose from extracted tracks.
  • Appearance comparison

measure:

  • Bhattacharyya distance.

25

slide-77
SLIDE 77

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Appearance Model for Occlusion Recovery

  • Extract person-specific

appearance model for each limb:

  • Color histogram.
  • Require relatively accurate

pose estimate:

  • Pose from extracted tracks.

25

slide-78
SLIDE 78

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Appearance Model for Occlusion Recovery

  • Extract person-specific

appearance model for each limb:

  • Color histogram.
  • Require relatively accurate

pose estimate:

  • Pose from extracted tracks.
  • Appearance comparison

measure:

  • Bhattacharyya distance.

25

slide-79
SLIDE 79

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Occlusion Recovery

26

  • Greedily link partial tracks based on:
  • Motion & articulation compatibility.
  • Plus appearance compatibility.

time

slide-80
SLIDE 80

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Occlusion Recovery

26

  • Greedily link partial tracks based on:
  • Motion & articulation compatibility.
  • Plus appearance compatibility.

time

slide-81
SLIDE 81

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Occlusion Recovery

26

  • Greedily link partial tracks based on:
  • Motion & articulation compatibility.
  • Plus appearance compatibility.

time

slide-82
SLIDE 82

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Occlusion Recovery

26

  • Greedily link partial tracks based on:
  • Motion & articulation compatibility.
  • Plus appearance compatibility.

time

slide-83
SLIDE 83

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Occlusion Recovery

26

  • Greedily link partial tracks based on:
  • Motion & articulation compatibility.
  • Plus appearance compatibility.

time

slide-84
SLIDE 84

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Occlusion Recovery

26

  • Greedily link partial tracks based on:
  • Motion & articulation compatibility.
  • Plus appearance compatibility.

time

slide-85
SLIDE 85

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Occlusion Recovery

26

  • Greedily link partial tracks based on:
  • Motion & articulation compatibility.
  • Plus appearance compatibility.

time

slide-86
SLIDE 86

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008 27

slide-87
SLIDE 87

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008 28

slide-88
SLIDE 88

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

  • partISM: Extended the ISM detection framework to

part-based detection:

  • Improved detection
  • Basis for incorporating body dynamics.
  • Incorporated temporal continuity in a “tracklet” detection

framework:

  • hGPLVM dynamics model.
  • Improves occlusion robustness.
  • Reduces false positives.
  • Extracted and combined tracks across occlusion

events:

  • Person identification throughout entire sequences.

29

Summary

slide-89
SLIDE 89

People-Tracking-by-Detection and People-Detection-by-Tracking - CVPR 2008

Thanks!

  • Acknowledgements:
  • Neil Lawrence for his GPLVM code.
  • Mario Fritz for helpful discussions.
  • Partial funding through DFG GRK “Cooperative, Adaptive and

Responsive Monitoring in Mixed Mode Environments”

  • Travel funding from DFG.
  • Data available at:

30

http://www.mis.informatik.tu-darmstadt.de/