GNR607 Principles of Satellite Image Processing Instructor: Prof. - - PowerPoint PPT Presentation

GNR607 Principles of Satellite Image Processing

Instructor: Prof. B. Krishna Mohan CSRE, IIT Bombay bkmohan@csre.iitb.ac.in

Slot 2 Lecture 29-31 Introduction to Texture and Color October 7, 2014 10.35 AM – 11.30 AM

• Oct. 09, 2014 11.35 AM – 12.30 PM

October 13, 2014 9.30 AM – 10.25 AM

Gray Level Co-occurrence Matrix Approach

• GLCM is based on second order statistics (2-D

histogram)

• It is conjectured (B. Jules, a psychophysicist) that

textures differing in second order statistics are indeed

• different. (counter-examples provided later)
• Therefore numerical features were extracted from the

image in terms of the second-order statistics that were a measure of the underlying texture. IIT Bombay Slide 22 GNR607 Lecture 29-31 B. Krishna Mohan

Definition of GLCM

• The GLCM is defined by:

Pd(i,j) = ni,j = #{f(m,n) = i, f(m+dx, n+dy) = j; 1≤m≤M; 1≤n ≤N} – where nij is the number of occurrences of the pixel values (i,j) lying at distance d in the image. – The co-occurrence matrix Pd has dimension n× n, where n is the number of gray levels in the image. IIT Bombay Slide 23 GNR607 Lecture 29-31 B. Krishna Mohan

Construction of GLCM

• A co-occurrence matrix is a two-dimensional array, P, in

which both the rows and the columns represent a set of possible image values.

• A GLCM Pd[i,j] is defined by first specifying a

displacement vector d=(dx,dy) and counting all pairs of pixels separated by d having gray levels i and j.

• (dx,dy) define the directionality of texture; dx=1,dy=0

represents horizontal direction;dx=1,dy=1 represents diagonal direction IIT Bombay Slide 24 GNR607 Lecture 29-31 B. Krishna Mohan

Example

IIT Bombay Slide 25 GNR607 Lecture 29-31 B. Krishna Mohan

There are 16 pairs of pixels in the image which satisfy this spatial separation. Since there are only three gray levels – 0,1,2, P[i,j] is a 3×3 matrix.

Algorithm to construct GLCM

Count all pairs of pixels in which the first pixel has a value i, and its matching pair displaced from the first pixel by d has a value of j. This count is entered in the ith row and jth column of the matrix Pd[i,j] Note that Pd[i,j] is not symmetric in this form of counting, since the number of pairs of pixels having gray levels [i,j] does not necessarily equal the number of pixel pairs having gray levels [j,i]. IIT Bombay Slide 26 GNR607 Lecture 29-31 B. Krishna Mohan

Normalized GLCM

The elements of Pd[i,j] can be normalized by dividing each entry by the total number of pixel pairs.

Normalized GLCM N[i,j], defined by:

which normalizes the co-occurrence values to lie between 0 and 1, and allows them to be thought of as probabilities. IIT Bombay Slide 27 GNR607 Lecture 29-31 B. Krishna Mohan

∑∑

=

i j

j i P j i P j i N ] , [ ] , [ ] , [

Numeric Features from GLCM

Gray level co-occurrence matrices capture properties of a texture but they are not directly useful for further analysis, such as the comparison of two textures. Numeric features are computed from the co-

• ccurrence matrix that can be used to represent

the texture more compactly.

IIT Bombay Slide 28 GNR607 Lecture 29-31 B. Krishna Mohan

Haralick Texture Features

Haralick et al. suggested a set of 14 textural features which can be extracted from the co-

• ccurrence

matrix, and which contain information about image textural characteristics such as homogeneity, linearity, and contrast.

Haralick, R.M., K. Shanmugam, and I. Dinstein, "Textural features for image classification” IEEE Transactions on Systems, Man and Cybernetics: pp. 610-621. 1973. IIT Bombay Slide 29 GNR607 Lecture 29-31 B. Krishna Mohan

Features from GLCM: Angular Second Moment (ASM)

• Angular Second Moment ASM
• ASM =
• R is a normalizing factor
• ASM is large when only very few gray level pairs are

present in the textured image

• K is the number of gray levels

IIT Bombay Slide 30 GNR607 Lecture 29-31 B. Krishna Mohan

2 1 1

( , ) /

K K d i j

P i j R

= =

∑∑

Contrast (CON)

• Contrast CON
• CON =
• This feature highlights co-occurrence of

very different gray levels

IIT Bombay Slide 31 GNR607 Lecture 29-31 B. Krishna Mohan

2 1 1

( ) ( , ) /

K K d i j

i j P i j R

= =

−

∑∑

Entropy (ENT)

• ENT =
• ENT emphasises many different

co-occurrences

• P(i,j) is the normalized co-occurrence

matrix, each entry indicating probability of

• ccurrence of that gray level combination

IIT Bombay Slide 29-31 GNR607 Lecture 29-31 B. Krishna Mohan

1 1

1 [ , ]ln [ , ]

K K i j

P i j P i j

= =

   ÷  

∑∑

Inverse Difference Moment (IDM)

• Inverse Difference Moment IDM
• IDM =
• IDM emphasises co-occurrence of close

gray levels compared to highly different

• graylevels. m and r can user specified

IIT Bombay Slide 33 GNR607 Lecture 29-31 B. Krishna Mohan

1 1

[ , ] | ( ) | 1

r K K d m i j i j

P i j i j

= = ≠

− +

∑∑

Algorithm for image segmentation

• Specify a window of size wxw
• For the pixels in the window, compute the co-
• ccurrence matrix
• Derive the texture features from the co-
• ccurrence matrix
• Move the window by 1 pixel, and repeat the

procedure

• The procedure leads to texture images that may

be treated like additional bands, equal to the number of features computed.

IIT Bombay Slide 34 GNR607 Lecture 29-31 B. Krishna Mohan

IIT Bombay Slide 35 GNR607 Lecture 29-31 B. Krishna Mohan

Input Image

IDM

IIT Bombay Slide 36 GNR607 Lecture 29-31 B. Krishna Mohan

CLASSIFIED IMAGE (Mumbai)

GNR607 Lecture 29-31 B. Krishna Mohan IIT Bombay Slide 37

CLASSIFIED IMAGE (Mumbai)

LEGEND

WATER MARSHY LAND / SHALLOW WATER HIGHLY BUILT-UP AREA PARTIALLY BUILT-UP AREA OPEN AREAS/ GROUNDS

IIT Bombay Slide 38 GNR607 Lecture 29-31 B. Krishna Mohan

Other Features from GLCM

• More features are defined from GLCM by

Haralick et al. and by other researchers

• e.g., Sahasrabudhe and Nageswara Rao used

eigenvalues of GLCM as texture features

• 1st and 2nd eigenvalues of GLCM were shown to

be capable of good texture discrimination

• Limited utility due to computational intensive

nature of features

• Haralick et al. defined 28 features of which ASM,

ENT, CON, IDM were most effective

IIT Bombay Slide 39 GNR607 Lecture 29-31 B. Krishna Mohan

Fast Computation of GLCM

• Fast Computation of GLCM useful for efficient

application

• Basis for fast computation – number of pixel

pairs that are common to computation of GLCM / features at successive positions of the window

• Significant savings possible when window size is

large, and many features are computed

IIT Bombay Slide 40 GNR607 Lecture 29-31 B. Krishna Mohan

Redundant Computations

• When texture window moves by 1 pixel right,

– The first column moves out of the computation – The last column enters the computation – Many pixel pairs remain unchanged IIT Bombay Slide 41 GNR607 Lecture 29-31 B. Krishna Mohan

Efficiency Considerations

• Incremental Adjustments

– Deduct the pairs formed with elements of first column – Add pairs formed with elements of last column – New matrix is ready IIT Bombay Slide 42 GNR607 Lecture 29-31 B. Krishna Mohan

Efficiency Considerations

• Direct computation of features

– Examine each feature – Make modifications to the feature directly instead of to the GLCM

• ASM =
• Deduct from Pd (i,j) for column moving out
• Add to Pd (i,j) for column coming in

IIT Bombay Slide 43 GNR607 Lecture 29-31 B. Krishna Mohan

2 1 1

( , ) /

K K d i j

P i j R

= =

∑∑

Sum-Difference Histograms

Sum and Difference Histograms

• Michael Unser proposed a sum-and-difference

histogram approach as an approximation to the Co-occurrence matrix method to save computational time, without compromising much

• n the quality of results.
• Essentially, the texture information captured in

the co-occurrence matrices is approximated in two one-dimensional histograms

– Sum histogram – Difference histogram GNR607 Lecture 29-31 B. Krishna Mohan IIT Bombay Slide 44

Relationship with Co-occurrence Matrix

• The co-occurrence matrix represents the

joint probability of occurrence of gray levels in a given spatially adjacent position

• Pi,j = Ci,j / (R)
• R = number of possible gray level pairs in

the image

• Ci,j is the number of times gray levels i and

j co-occurred in a given spatial adjacency

GNR607 Lecture 29-31 B. Krishna Mohan IIT Bombay Slide 45

Sum and Difference Histograms

Unser’s approximation

• Consider a pair of random variables y1 and y2
• The covariance matrix associated with these random

variables is given by

• Cy = 1

ρ

1

ρ

2 y

σ

σy is the standard deviation and ρ is the correlation coefficient. GNR607 Lecture 29-31 B. Krishna Mohan IIT Bombay Slide 46

Sum and Difference Histograms

• The eigenvalues and eigenvectors of the

covariance matrix are given by

• λ1 =
• λ2 =
• The eigenvectors are given by
• u1 = (1/sqrt(2)).[1 1]t ;
• u2 = (1/sqrt(2)) [1 -1]t

2[1

]

y

σ ρ +

2[1

]

y

σ ρ −

GNR607 Lecture 29-31 B. Krishna Mohan IIT Bombay Slide 47

Sum and Difference Histograms

Therefore the new random variables

• z1 = (1/sqrt(2)).(y1 + y2)
• z2 = (1/sqrt(2)).(y1 – y2)

are decorrelated versions of the original input variables y1 and y2 This is the basis for the formulation of the sum and difference histogram approach proposed by Unser

GNR607 Lecture 29-31 B. Krishna Mohan IIT Bombay Slide 48

Sum and Difference Histograms

• For each pixel (i,j)
• define
• s(i,j) = f(i,j) + f(i+k,j+l)
• d(i,j) = f(i,j) – f(i+k,j+l)
• The histograms hs and hd together are

used to define the texture features

GNR607 Lecture 29-31 B. Krishna Mohan IIT Bombay Slide 49

Sum and Difference Histograms

• Interpretation of the histograms
• For a flat region (small tone variations),

the sum histogram will have a few entries

• f large magnitude, and difference

histogram will have large population close to 0

• For a highly textured region, sum and

difference histograms are more widely distributed

GNR607 Lecture 29-31 B. Krishna Mohan IIT Bombay Slide 50

Sum and Difference Histograms

• Unser defined features like: (9 suggested by him)
• mean: f1 =
• Contrast: f2 =

1 2

( )

s i

iP i µ =

∑

2 2 d j

j P

∑

The texture features can be computed for a moving window to generate texture images that can be classified or segmented; they are also computed for entire texture samples for discrimination purposes GNR607 Lecture 29-31 B. Krishna Mohan IIT Bombay Slide 51

Texture Features using GLCM and Sum- Diff Histograms

GNR607 Lecture 29-31 B. Krishna Mohan IIT Bombay Slide 52

GNR607 Lecture 29-31 B. Krishna Mohan IIT Bombay Slide 52

Sample Textures

Source: IEEE T-PAMI 8(1), Jan. 1986

GNR607 Lecture 29-31 B. Krishna Mohan IIT Bombay Slide 52 Source: IEEE T-PAMI 8(1), Jan. 1986

Selected slides used from

www.cs.washington.edu/education/courses/.. ./Texture_white.ppt