Graph-based Segmentation Computer Vision CS 543 / ECE 549 - - PowerPoint PPT Presentation

Graph-based Segmentation

Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem

02/25/10

i

Images as graphs

• Fully-connected graph

– node for every pixel – link between every pair of pixels, p,q – similarity wij for each link

j wij

c Source: Seitz

Segmentation by Graph Cuts

• Break Graph into Segments

– Delete links that cross between segments – Easiest to break links that have low cost (low similarity)

• similar pixels should be in the same segments
• dissimilar pixels should be in different segments

w A B C Source: Seitz

Graph cuts segmentation

Markov Random Fields

å å

Î

=

edges j i j i i i

data y y data y data Energy

, 2 1

) , ; , ( ) , ; ( ) , ; ( q y q y q y

Node yi: pixel label Edge: constrained pairs Cost to assign a label to each pixel Cost to assign a pair of labels to connected pixels

Markov Random Fields

• Example: “label smoothing” grid

Unary potential 0 1 0 0 K 1 K 0 Pairwise Potential 0: -logP(yi = 0 ; data) 1: -logP(yi = 1 ; data)

å å

Î

=

edges j i j i i i

data y y data y data Energy

, 2 1

) , ; , ( ) , ; ( ) , ; ( q y q y q y

Solving MRFs with graph cuts

å å

Î

=

edges j i j i i i

data y y data y data Energy

, 2 1

) , ; , ( ) , ; ( ) , ; ( q y q y q y

Source (Label 0) Sink (Label 1) Cost to assign to 0 Cost to assign to 1 Cost to split nodes

Solving MRFs with graph cuts

å å

Î

=

edges j i j i i i

data y y data y data Energy

, 2 1

) , ; , ( ) , ; ( ) , ; ( q y q y q y

Source (Label 0) Sink (Label 1) Cost to assign to 0 Cost to assign to 1 Cost to split nodes

Graph cuts segmentation

• 1. Define graph

– usually 4-connected or 8-connected

• 2. Define unary potentials

– Color histogram or mixture of Gaussians for background and foreground

• 3. Define pairwise potentials
• 4. Apply graph cuts
• 5. Return to 2, using current labels to compute

foreground, background models

ï þ ï ý ü ï î ï í ì

• +

=

2 2 2 1

2 ) ( ) ( exp ) , ( _ s y c x c k k y x potential edge ÷ ÷ ø ö ç ç è æ

• =

) ); ( ( ) ); ( ( log ) ( _

background foreground

x c P x c P x potential unary q q

Moderately straightforward examples

… GrabCut completes automatically

GrabCut – Interactive Foreground Extraction 10

Difficult Examples

Camouflage & Low Contrast Harder Case Fine structure

Initial Rectangle Initial Result

GrabCut – Interactive Foreground Extraction 11