Introduction Tracking with static camera Tracking with moving camera
Tracking
H˚ akan Ard¨
- March 4, 2013
H˚ akan Ard¨
- Tracking
March 4, 2013 1 / 57
Tracking H akan Ard o March 4, 2013 H akan Ard o Tracking - - PowerPoint PPT Presentation
Introduction Tracking with static camera Tracking with moving camera Tracking H akan Ard o March 4, 2013 H akan Ard o Tracking March 4, 2013 1 / 57 Introduction Tracking with static camera Tracking with moving camera State
Introduction Tracking with static camera Tracking with moving camera
H˚ akan Ard¨
H˚ akan Ard¨
March 4, 2013 1 / 57
Introduction Tracking with static camera Tracking with moving camera State space Sliding Window Detection
1
Introduction State space Sliding Window Detection
2
Tracking with static camera Greedy Kalman filter Particle filter
3
Tracking with moving camera Self-motion
H˚ akan Ard¨
March 4, 2013 2 / 57
Introduction Tracking with static camera Tracking with moving camera State space Sliding Window Detection
State: position X = (X, Y , Z, 1), velocity, v = (vx, vy, vz, 0), ... Observation: detection in image x = (x, y, 1) Observation model: λx = PX Dynamic model: Xt+1 = Xt + vt and vt+1 = vt Sthocastic dynamic model: Introduce noise, random numbers q and w Xt+1 = Xt + vt + q vt+1 = vt + w Sthocastic observation model: Introduce noise, a random number r λx = PX + r
H˚ akan Ard¨
March 4, 2013 3 / 57
Introduction Tracking with static camera Tracking with moving camera State space Sliding Window Detection
State: position X = (X, Y , Z, 1), velocity, v = (vx, vy, vz, 0), ... Observation: detection in image x = (x, y, 1) Observation model: λx = PX Dynamic model: Xt+1 = Xt + vt and vt+1 = vt Sthocastic dynamic model: Introduce noise, random numbers q and w Xt+1 = Xt + vt + q vt+1 = vt + w Sthocastic observation model: Introduce noise, a random number r λx = PX + r
H˚ akan Ard¨
March 4, 2013 3 / 57
Introduction Tracking with static camera Tracking with moving camera State space Sliding Window Detection
State: position X = (X, Y , Z, 1), velocity, v = (vx, vy, vz, 0), ... Observation: detection in image x = (x, y, 1) Observation model: λx = PX Dynamic model: Xt+1 = Xt + vt and vt+1 = vt Sthocastic dynamic model: Introduce noise, random numbers q and w Xt+1 = Xt + vt + q vt+1 = vt + w Sthocastic observation model: Introduce noise, a random number r λx = PX + r
H˚ akan Ard¨
March 4, 2013 3 / 57
Introduction Tracking with static camera Tracking with moving camera State space Sliding Window Detection
State: position X = (X, Y , Z, 1), velocity, v = (vx, vy, vz, 0), ... Observation: detection in image x = (x, y, 1) Observation model: λx = PX Dynamic model: Xt+1 = Xt + vt and vt+1 = vt Sthocastic dynamic model: Introduce noise, random numbers q and w Xt+1 = Xt + vt + q vt+1 = vt + w Sthocastic observation model: Introduce noise, a random number r λx = PX + r
H˚ akan Ard¨
March 4, 2013 3 / 57
Introduction Tracking with static camera Tracking with moving camera State space Sliding Window Detection
State: position X = (X, Y , Z, 1), velocity, v = (vx, vy, vz, 0), ... Observation: detection in image x = (x, y, 1) Observation model: λx = PX Dynamic model: Xt+1 = Xt + vt and vt+1 = vt Sthocastic dynamic model: Introduce noise, random numbers q and w Xt+1 = Xt + vt + q vt+1 = vt + w Sthocastic observation model: Introduce noise, a random number r λx = PX + r
H˚ akan Ard¨
March 4, 2013 3 / 57
Introduction Tracking with static camera Tracking with moving camera State space Sliding Window Detection
State: position X = (X, Y , Z, 1), velocity, v = (vx, vy, vz, 0), ... Observation: detection in image x = (x, y, 1) Observation model: λx = PX Dynamic model: Xt+1 = Xt + vt and vt+1 = vt Sthocastic dynamic model: Introduce noise, random numbers q and w Xt+1 = Xt + vt + q vt+1 = vt + w Sthocastic observation model: Introduce noise, a random number r λx = PX + r
H˚ akan Ard¨
March 4, 2013 3 / 57
Introduction Tracking with static camera Tracking with moving camera State space Sliding Window Detection
H˚ akan Ard¨
March 4, 2013 4 / 57
Introduction Tracking with static camera Tracking with moving camera State space Sliding Window Detection
H˚ akan Ard¨
March 4, 2013 5 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
H˚ akan Ard¨
March 4, 2013 6 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 1
University of North Carolina at Chapel Hill Department of Computer Science
http://www.cs.unc.edu/~welch/kalmanLinks.html
H˚ akan Ard¨
March 4, 2013 7 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Sanjay Patil1 and Ryan Irwin2 Graduate research assistant1, REU undergrad2 Human and Systems Engineering
URL: www.isip.msstate.edu/publications/seminars/msstate/2005/particle/
Gentle Introduction to Particle Filtering
H˚ akan Ard¨
March 4, 2013 8 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 5
H˚ akan Ard¨
March 4, 2013 9 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 6
1 = z1
2 1 = σ 2 z1
Conditional Density Function
14 12 10 8 6 4 2
N(z1,σz1
2 )
z1
,
H˚ akan Ard¨
March 4, 2013 10 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 7
Conditional Density Function
z2
,
2 =...?
2 2 = ...?
14 12 10 8 6 4 2
N(z2,σz2
2 )
H˚ akan Ard¨
March 4, 2013 11 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 8
2
2 + σ z2 2
2
2 + σ z2 2
2
1 + K2 z2 − ˆ
1
2
2 + σ z2 2
H˚ akan Ard¨
March 4, 2013 12 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 9
2
2
2
H˚ akan Ard¨
March 4, 2013 13 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 10
2 = σ 2
2
2
14 12 10 8 6 4 2
Conditional Density Function N( σ 2)
x,ˆ
H˚ akan Ard¨
March 4, 2013 14 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 11
H˚ akan Ard¨
March 4, 2013 15 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Page 7 of 20 Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering algorithm step-by-step (1)
H˚ akan Ard¨
March 4, 2013 16 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Page 8 of 20 Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering step-by-step (2)
H˚ akan Ard¨
March 4, 2013 17 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Page 9 of 20 Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering step-by-step (3)
H˚ akan Ard¨
March 4, 2013 18 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Page 10 of 20 Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering step-by-step (4)
H˚ akan Ard¨
March 4, 2013 19 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Page 11 of 20 Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering step-by-step (5)
H˚ akan Ard¨
March 4, 2013 20 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Page 12 of 20 Particle Filtering – Gentle Introduction and Implementation Demo
Particle filtering step-by-step (6)
H˚ akan Ard¨
March 4, 2013 21 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 13
x x A A x x w w z z H H x x . . . . . . . . = + =
H˚ akan Ard¨
March 4, 2013 22 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 14
process state (mean) error covariance z y x
H˚ akan Ard¨
March 4, 2013 23 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 16
measurement model dynamic model
previous state next state state measurement
image plane
( u , v )
H˚ akan Ard¨
March 4, 2013 24 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 17
H˚ akan Ard¨
March 4, 2013 25 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 18
H˚ akan Ard¨
March 4, 2013 26 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 19
H˚ akan Ard¨
March 4, 2013 27 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 20
H˚ akan Ard¨
March 4, 2013 28 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 21
H˚ akan Ard¨
March 4, 2013 29 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 22
H˚ akan Ard¨
March 4, 2013 30 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 23
H˚ akan Ard¨
March 4, 2013 31 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 24
H˚ akan Ard¨
March 4, 2013 32 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 25
k
H˚ akan Ard¨
March 4, 2013 33 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 26
– = E[(xk- xk –)(xk - xk –)T]
H˚ akan Ard¨
March 4, 2013 34 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 27
H˚ akan Ard¨
March 4, 2013 35 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 28
state error covariance
H˚ akan Ard¨
March 4, 2013 36 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 29
H˚ akan Ard¨
March 4, 2013 37 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 30
–
–
H˚ akan Ard¨
March 4, 2013 38 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 31
– –
–
– –
H˚ akan Ard¨
March 4, 2013 39 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
UNC Chapel Hill Computer Science Slide 32
Time Update (Predict) Measurement (Correct) Update
H˚ akan Ard¨
March 4, 2013 40 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
H˚ akan Ard¨
March 4, 2013 41 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
H˚ akan Ard¨
March 4, 2013 42 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
H˚ akan Ard¨
March 4, 2013 43 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
H˚ akan Ard¨
March 4, 2013 44 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
H˚ akan Ard¨
March 4, 2013 45 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Current state
k , x(2) k , x(3) k , · · · , x(n) k
k+1 , x−(2) k+1 , x−(3) k+1 , · · · , x−(n) k+1
k+1, x(2) k+1, x(3) k+1, · · · , x(n) k+1
akan Ard¨
March 4, 2013 46 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Current state
k , x(2) k , x(3) k , · · · , x(n) k
k+1 , x−(2) k+1 , x−(3) k+1 , · · · , x−(n) k+1
k+1, x(2) k+1, x(3) k+1, · · · , x(n) k+1
akan Ard¨
March 4, 2013 46 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Current state
k , x(2) k , x(3) k , · · · , x(n) k
k+1 , x−(2) k+1 , x−(3) k+1 , · · · , x−(n) k+1
k+1, x(2) k+1, x(3) k+1, · · · , x(n) k+1
akan Ard¨
March 4, 2013 46 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Any sthocastic model from which we can sample f (xk+1 |xk ) Example: The dynamic model from the kalman filter xk+1 = Axk + wk f (xk+1 |xk ) = N(Axk, Q)
H˚ akan Ard¨
March 4, 2013 47 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Any sthocastic model from which we can sample f (xk+1 |xk ) Example: The dynamic model from the kalman filter xk+1 = Axk + wk f (xk+1 |xk ) = N(Axk, Q)
H˚ akan Ard¨
March 4, 2013 47 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Propagate each particle i, separately The prediction x−(i)
k+1 is choosen as a sample from
f
k
akan Ard¨
March 4, 2013 48 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Any sthocastic model from which we can calculate likelihoods f (zk |xk ) Example: The measurement model from the kalman filter zk = Hxk + vk f (zk |xk ) = N(Hxk, R)
H˚ akan Ard¨
March 4, 2013 49 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
Any sthocastic model from which we can calculate likelihoods f (zk |xk ) Example: The measurement model from the kalman filter zk = Hxk + vk f (zk |xk ) = N(Hxk, R)
H˚ akan Ard¨
March 4, 2013 49 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
The measurement model gives a weight for each particle, i w(i)
k
= f
k
Each corrected particle, x(i)
k , is choosen randomly
x(i)
k
= x−(j)
k
for some random j The probability of choosing sample j is w(j)
k
k
The same particle may be choosen several times
H˚ akan Ard¨
March 4, 2013 50 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
The measurement model gives a weight for each particle, i w(i)
k
= f
k
Each corrected particle, x(i)
k , is choosen randomly
x(i)
k
= x−(j)
k
for some random j The probability of choosing sample j is w(j)
k
k
The same particle may be choosen several times
H˚ akan Ard¨
March 4, 2013 50 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
The measurement model gives a weight for each particle, i w(i)
k
= f
k
Each corrected particle, x(i)
k , is choosen randomly
x(i)
k
= x−(j)
k
for some random j The probability of choosing sample j is w(j)
k
k
The same particle may be choosen several times
H˚ akan Ard¨
March 4, 2013 50 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
The measurement model gives a weight for each particle, i w(i)
k
= f
k
Each corrected particle, x(i)
k , is choosen randomly
x(i)
k
= x−(j)
k
for some random j The probability of choosing sample j is w(j)
k
k
The same particle may be choosen several times
H˚ akan Ard¨
March 4, 2013 50 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
The measurement model gives a weight for each particle, i w(i)
k
= f
k
Each corrected particle, x(i)
k , is choosen randomly
x(i)
k
= x−(j)
k
for some random j The probability of choosing sample j is w(j)
k
k
The same particle may be choosen several times
H˚ akan Ard¨
March 4, 2013 50 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
H˚ akan Ard¨
March 4, 2013 51 / 57
Introduction Tracking with static camera Tracking with moving camera Greedy Kalman filter Particle filter
H˚ akan Ard¨
March 4, 2013 52 / 57
Introduction Tracking with static camera Tracking with moving camera Self-motion
H˚ akan Ard¨
March 4, 2013 53 / 57
Introduction Tracking with static camera Tracking with moving camera Self-motion
Detect keypoints Track each keypoint separately Use RANSAC to find camera motion Compensate for camera motion (stiching)
H˚ akan Ard¨
March 4, 2013 54 / 57
Introduction Tracking with static camera Tracking with moving camera Self-motion
H˚ akan Ard¨
March 4, 2013 55 / 57
Introduction Tracking with static camera Tracking with moving camera Self-motion
H˚ akan Ard¨
March 4, 2013 56 / 57
Introduction Tracking with static camera Tracking with moving camera Self-motion
Detta projekt syftar till att utveckla mjukvara som automatiskt kan filma, f¨
ata tider f¨
avs dels kunskaper i matematisk bildanalys, men en del kunskaper i programmering kommer ¨ aven att beh¨
ar hj¨ alp av IFK Lund att spela in film och utv¨ ardera resultatet. Ett examensarbete syftar till att utveckla algoritmer f¨
f¨
an b¨
ar kameran stilla och efter starten skall den hela tiden f¨
m˚ aste kunna kompensera och den egna kamerar¨
Ytterligare ett examensarbete behandlar automatisk detektion av l¨
anda m˚ att kan l¨
aknas och diagram ¨
f¨
andrats genom loppet presenteras f¨
information efterfr˚ agas av aktiva l¨
m˚ allinje ¨ ar ¨ aven viktigt f¨
Contact: Petter Strandmark <petter@maths.lth.se>
H˚ akan Ard¨
March 4, 2013 57 / 57