SIFT Features and Hough Transform Various slides from previous - - PowerPoint PPT Presentation

▶
sift features and hough transform
SMART_READER_LITE
LIVE PREVIEW

SIFT Features and Hough Transform Various slides from previous - - PowerPoint PPT Presentation

Mar 20, 2023 172 likes •813 views

CS4501: Introduction to Computer Vision SIFT Features and Hough Transform Various slides from previous courses by: D.A. Forsyth (Berkeley / UIUC), I. Kokkinos (Ecole Centrale / UCL). S. Lazebnik (UNC / UIUC), S. Seitz (MSR / Facebook), J. Hays

slide-1
SLIDE 1

CS4501: Introduction to Computer Vision

SIFT Features and Hough Transform

Various slides from previous courses by: D.A. Forsyth (Berkeley / UIUC), I. Kokkinos (Ecole Centrale / UCL). S. Lazebnik (UNC / UIUC), S. Seitz (MSR / Facebook), J. Hays (Brown / Georgia Tech), A. Berg (Stony Brook / UNC), D. Samaras (Stony Brook) . J. M. Frahm (UNC), V. Ordonez (UVA).

slide-2
SLIDE 2
  • Corner Detection - Harris
  • Blob Detection – Laplacian of Gaussian / Difference of Gaussians (DoG)

Last Class – Interest Points

slide-3
SLIDE 3
  • Blog Detection – Difference of Gaussians
  • SIFT Feature descriptor – Feature Matching
  • Hough Transform -> For Line Detection

Today’s Class

slide-4
SLIDE 4

Basic idea

  • Convolve the image with a “blob filter” at multiple

scales and look for extrema of filter response in the resulting scale space

  • T. Lindeberg. Feature detection with automatic scale selection.

IJCV 30(2), pp 77-116, 1998.

Slide by Svetlana Lazebnik

slide-5
SLIDE 5

Blob filter

  • Laplacian of Gaussian: Circularly symmetric operator for blob

detection in 2D

2 2 2 2 2

y g x g g ¶ ¶ + ¶ ¶ = Ñ

slide-6
SLIDE 6

Blob detection

  • Find maxima and minima of blob filter response in

space and scale

*

=

maxima minima

Source: N. Snavely

slide-7
SLIDE 7

Blob at multiple scales – Option 1

*

=

*

=

*

=

slide-8
SLIDE 8

Apply Non-Max Suppression – Show blobs as circles

slide-9
SLIDE 9

Scale-space blob detector: Example

Slide by Svetlana Lazebnik

slide-10
SLIDE 10

Scale-space blob detector: Example

Slide by Svetlana Lazebnik

slide-11
SLIDE 11

Blog at Multiple Scales: Option 2

Slide by Svetlana Lazebnik

slide-12
SLIDE 12

Scale-space blob detector

  • 1. Convolve image with scale-normalized Laplacian at several scales
  • 2. Find maxima of squared Laplacian response in scale-space

Slide by Svetlana Lazebnik

slide-13
SLIDE 13

Scale-space blob detector: Example

Slide by Svetlana Lazebnik

slide-14
SLIDE 14
  • Approximating the Laplacian with a difference of Gaussians:

( )

2

( , , ) ( , , )

xx yy

L G x y G x y s s s = +

( , , ) ( , , ) DoG G x y k G x y s s =

  • (Laplacian)

(Difference of Gaussians)

Efficient implementation

Slide by Svetlana Lazebnik

slide-15
SLIDE 15

Efficient implementation

David G. Lowe. "Distinctive image features from scale-invariant keypoints.” IJCV 60 (2), pp. 91-110, 2004.

slide-16
SLIDE 16

Gaussian Pyramid – DoG pyramid

David G. Lowe. "Distinctive image features from scale-invariant keypoints.” IJCV 60 (2), pp. 91-110, 2004.

Figure from Workload analysis and efficient OpenCL-based implementation of SIFT algorithm on a smartphone

  • Guohui Wang, Blaine Rister, Joseph R. Cavallaro
slide-17
SLIDE 17

Gaussian Pyramid

David G. Lowe. "Distinctive image features from scale-invariant keypoints.” IJCV 60 (2), pp. 91-110, 2004.

Figure from Workload analysis and efficient OpenCL-based implementation of SIFT algorithm on a smartphone

  • Guohui Wang, Blaine Rister, Joseph R. Cavallaro
slide-18
SLIDE 18

Same results

Slide by Svetlana Lazebnik

slide-19
SLIDE 19

Locations + Scales + Orientations

  • D. Lowe, Distinctive image features from scale-invariant keypoints, IJCV 60 (2),
  • pp. 91-110, 2004.
slide-20
SLIDE 20

Eliminating rotation ambiguity

  • To assign a unique orientation to circular image

windows:

  • Create histogram of local gradient directions in the

patch

  • Assign canonical orientation at peak of smoothed

histogram

2 p Slide by Svetlana Lazebnik

slide-21
SLIDE 21

From keypoint detection to keypoint representation (feature descriptors)

Figure by Svetlana Lazebnik

slide-22
SLIDE 22

SIFT descriptors

  • Inspiration: complex neurons in the primary visual cortex
  • D. Lowe. Distinctive image features from scale-invariant keypoints. IJCV 60 (2),
  • pp. 91-110, 2004.
slide-23
SLIDE 23

From keypoint detection to keypoint representation (feature descriptors)

Figure by Svetlana Lazebnik

Compare SIFT feature vectors instead

slide-24
SLIDE 24

SIFT Feature Matching

Rice Hall at UVA

slide-25
SLIDE 25

JiaWang Bian, Wen-Yan Lin, Yasuyuki Matsushita, Sai-Kit Yeung, Tan Dat Nguyen, Ming-Ming Cheng GMS: Grid-based Motion Statistics for Fast, Ultra-robust Feature Correspondence IEEE CVPR, 2017 The method has been integrated into OpenCV library (see xfeatures2d in opencv_contrib).

slide-26
SLIDE 26

A hard keypoint matching problem

NASA Mars Rover images

slide-27
SLIDE 27

NASA Mars Rover images with SIFT feature matches Figure by Noah Snavely

Answer below (look for tiny colored squares…)

slide-28
SLIDE 28

Feature Descriptors Zoo

  • SIFT (under a patent) Proposed around 1999
  • SURF (under a patent too – I think)
  • BRIEF
  • ORB (seems free as it is OpenCV’s preferred)
  • BRISK
  • FREAK
  • FAST
  • KAZE
  • LIFT (Most recently proposed at ECCV 2016)
slide-29
SLIDE 29
slide-30
SLIDE 30

How to do Line Detection?

slide-31
SLIDE 31

How to do Line Detection?

x = 12px x = 38px x = 68px y = 16px y = 56px y = 20px

slide-32
SLIDE 32

Idea: Sobel First!

x = 12px x = 38px x = 68px y = 56px y = 20px y = 16px

slide-33
SLIDE 33

Idea: Sobel First!

x = 12px x = 38px x = 68px y = 56px y = 20px y = 16px

slide-34
SLIDE 34

Idea: Then Count Pixels that support each line hypothesis.

x = 12px x = 38px x = 68px y = 56px y = 20px y = 16px

slide-35
SLIDE 35

Idea: Then Count Pixels that support each line hypothesis.

x = 12px x = 38px x = 68px y = 56px

count (sum) count (sum)

y = 16px y = 20px

slide-36
SLIDE 36

Problem with this?

x = 12px x = 38px x = 68px y = 56px

count (sum) count (sum)

y = 16px y = 20px

slide-37
SLIDE 37

What if you’re given this instead?

slide-38
SLIDE 38

How do we get this?

slide-39
SLIDE 39
  • Furthermore. How do we get this!?

y= -2x + 30 y= -2x + 20 y= -2x + 10 y= x + 10 y= x - 5 y= x + 7

slide-40
SLIDE 40

Hough transform

  • An early type of voting scheme for Detecting Lines
  • General outline:
  • Discretize parameter space into bins
  • For each feature point in the image, put a vote in every bin in the parameter space that

could have generated this point

  • Find bins that have the most votes

P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int.

  • Conf. High Energy Accelerators and Instrumentation, 1959

Image space Hough parameter space

Slides by Svetlana Lazebnik

slide-41
SLIDE 41

Hough Transform: Let’s again apply Sobel first

slide-42
SLIDE 42

Hough Transform: Let’s again apply Sobel first

slide-43
SLIDE 43

Then let’s count but now in a 2D array

! = #$ + & # & Count all points intersecting with all lines with m = (0, inf), b = [-B-min, B-max]

slide-44
SLIDE 44
  • Problems with the (m,b) space:
  • Unbounded parameter domains
  • Vertical lines require infinite m

Parameter space representation

slide-45
SLIDE 45
  • Problems with the (m,b) space:
  • Unbounded parameter domains
  • Vertical lines require infinite m
  • Alternative: polar representation

Parameter space representation

r q q = + sin cos y x

slide-46
SLIDE 46

Then let’s count but now in a 2D array

! cos % + ' sin % = + + % Count all points intersecting with all lines with rho = (-diagonal, diagonal), theta = [0, 180]

slide-47
SLIDE 47

Hough Transform Algorithm outline

  • Initialize accumulator H to all zeros
  • For each feature point (x,y)

in the image For θ = 0 to 180 ρ = x cos θ + y sin θ H(θ, ρ) = H(θ, ρ) + 1 end end

  • Find the value(s) of (θ, ρ) where H(θ, ρ) is a local

maximum

  • The detected line in the image is given by

ρ = x cos θ + y sin θ

ρ θ

Slide by Svetlana Lazebnik

slide-48
SLIDE 48
  • Each point (x,y) in Image Space

will add a sinusoid in the Hough Transform (q,r) parameter space

slide-49
SLIDE 49

features votes

Basic illustration

slide-50
SLIDE 50

Hough Transform for an Actual Image

slide-51
SLIDE 51

Edges using threshold on Sobel’s magnitude

slide-52
SLIDE 52

Hough Transform (High Resolution)

! = −90& ! = 90& ' = − ℎ) + +) ' = ℎ) + +) ' = 0 ! = 0

slide-53
SLIDE 53

Hough Transform (After threshold)

! = −90& ! = 90& ' = − ℎ) + +) ' = ℎ) + +) ' = 0 ! = 0

slide-54
SLIDE 54

Hough Transform (After threshold)

! = −90& ! = 90& ' = − ℎ) + +) ' = ℎ) + +) ' = 0 ! = 0

Vertical lines

slide-55
SLIDE 55

Hough Transform (After threshold)

! = −90& ! = 90& ' = − ℎ) + +) ' = ℎ) + +) ' = 0 ! = 0

Vertical lines

slide-56
SLIDE 56

Hough Transform with Non-max Suppression

! = −90& ! = 90& ' = − ℎ) + +) ' = ℎ) + +) ' = 0 ! = 0

slide-57
SLIDE 57

Back to Image Space – with lines detected

! = − $%&' &()' * + , &()'

r q q = + sin cos y x

slide-58
SLIDE 58

Hough transform demo

slide-59
SLIDE 59

Incorporating image gradients

  • Recall: when we detect an

edge point, we also know its gradient direction

  • But this means that the line

is uniquely determined!

  • Modified Hough transform:
  • For each edge point (x,y)

θ = gradient orientation at (x,y) ρ = x cos θ + y sin θ H(θ, ρ) = H(θ, ρ) + 1 end

Slide by Svetlana Lazebnik

slide-60
SLIDE 60

Hough transform for circles

) , ( ) , ( y x I r y x Ñ +

x y

(x,y)

x y r

) , ( ) , ( y x I r y x Ñ

  • image space

Hough parameter space

Slide by Svetlana Lazebnik

slide-61
SLIDE 61

Hough transform for circles

  • Conceptually equivalent procedure: for each (x,y,r), draw the

corresponding circle in the image and compute its “support”

x y r Is this more or less efficient than voting with features?

Slide by Svetlana Lazebnik

slide-62
SLIDE 62

Questions?

62