CFA Interpolation Detection Leszek Swirski October 15, 2009 - PowerPoint PPT Presentation

Outline CFA Introduction Interpolation Detection CFA Pattern Synthesis CFA Interpolation Detection Leszek ´ Swirski October 15, 2009 Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Interpolation Detection CFA Pattern Synthesis CFA Introduction Example Interpolation Interpolation Detection Methods Examples Identifying Forged Regions CFA Pattern Synthesis Reasoning Methods Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis CFA? ▶ “Colour Filter Array” ▶ Photosensors have no wavelength specificity ▶ So filter RGB onto array of photosensors ▶ e.g. Bayer filter Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Target Image Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Filters Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Sensor Data Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Coloured Sensor Data Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Coloured Sensor Data (Detail) Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Why interpolate? ▶ Each pixel is only R, G or B ▶ Want full colour, full size image ▶ So guess interpolate! Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Nearest Neighbour Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Nearest Neighbour Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Nearest Neighbour Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Bilinear (and other polynomials) and B R G ⎡ ⎤ ⎡ ⎤ 1 2 1 0 1 0 R ′ / B ′ = R / B ∗ 1 G ′ = G ∗ 1 2 4 2 1 4 1 ⎣ ⎦ ⎣ ⎦ 4 4 1 2 1 0 1 0 Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Bilinear (and other polynomials) Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Bilinear (and other polynomials) Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Smooth Hue Transition Seperate luminance ( G ) and chrominance ( R and B ). Interpolate G bilinearly ⎡ ⎤ 0 1 0 G ′ = G ∗ 1 1 4 1 ⎣ ⎦ 4 0 1 0 ij = R ij For R (and similarly for B ), interpolate the ratio R ′′ ij , and G ′ pointwise multiply by G ⎛ ⎡ ⎤ ⎞ 1 2 1 ⎝ R ′′ ∗ 1 R ′ ij = G ij × 2 4 2 ⎣ ⎦ ⎠ 4 1 2 1 ij Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Smooth Hue Transition Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Smooth Hue Transition Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Median filter ▶ Bilinearly filter R , G , B to get R ′′ , G ′′ , B ′′ . ▶ Calculate pairwise differences ( R ′′ − G ′′ , R ′′ − B ′′ , G ′′ − B ′′ ) ▶ Median filter these to get M rg , M rb , M gb ▶ Each resulting pixel is CFA pixel image plus/minus appropriate median. e.g. (1,0) is a green pixel in CFA, so R ′ 1 , 0 = G 1 , 0 + ( M rg ) 1 , 0 G ′ 1 , 0 = G 1 , 0 B ′ 1 , 0 = G 0 , 0 − ( M gb ) 1 , 0 Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Median filter Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Median filter Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Gradient-Based ▶ Want to preserve edges, so ‘adaptively’ interpolate G ▶ Approximate horizontal and vertical second derivatives of R and B and take absolutes, e.g. � � � � ∂ 2 R R i , j − 2 + R i , j +2 � � � � ≈ − R i , j � � � � ∂ x 2 2 � � � � i , j ▶ Compare these H and V . If H i , j < V i , j , (i,j) is a horizontal edge, so interpolate horizontally. ⎧ G i , j − 1 + G i , j +1 H i , j < V i , j  2 ⎨ G i − 1 , j + G i +1 , j G ′ i , j = H i , j > V i , j 2  G i , j − 1 + G i , j +1 + G i − 1 , j + G i +1 , j ⎩ H i , j = V i , j 4 Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Gradient-Based Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis Gradient-Based Leszek ´ Swirski CFA Interpolation Detection

Outline CFA Introduction Example Interpolation Detection Interpolation CFA Pattern Synthesis And more... ▶ Adaptive Colour Plane Has adaptive interpolation for G using first order derivative of luminance and second order derivative of chrominance, and uses adaptive interpolation again for R and B . ▶ Threshold-Based Variable Number of Gradients Eight gradient samples taken from a 5 × 5 neighbourhood of each pixel, averages are calculated for each gradient, gradients of values less than a (dynamic!) threshold are averaged, and averages are added to/subtracted from the CFA values. (Though both are similar in principle to the gradient-based) Leszek ´ Swirski CFA Interpolation Detection

Outline Methods CFA Introduction Examples Interpolation Detection Identifying Forged Regions CFA Pattern Synthesis Why detect? ▶ Image/Camera verification ▶ Identification of forged regions ▶ Recognition of PRCG (PhotoRealistic Computer Generated images) Leszek ´ Swirski CFA Interpolation Detection

Outline Methods CFA Introduction Examples Interpolation Detection Identifying Forged Regions CFA Pattern Synthesis General idea ▶ Interpolation creates correlation ▶ Most CFA interpolation is regular and approximately linear (especially for G ) ▶ If can determine some regular correlation, image is interpolated Leszek ´ Swirski CFA Interpolation Detection

Outline Methods CFA Introduction Examples Interpolation Detection Identifying Forged Regions CFA Pattern Synthesis EM Algorithm ▶ ‘Expectation-Maximisation’ ▶ Simulataneously estimate parameters of correlation (i.e. what interpolation is used) and which points are correlated to their neighbours ▶ Two-step iterative algorithm ▶ Creates a separable parameter space Leszek ´ Swirski CFA Interpolation Detection

Outline Methods CFA Introduction Examples Interpolation Detection Identifying Forged Regions CFA Pattern Synthesis A statistical Achilles’ Heel ▶ Interpolation creates correlation ▶ Correlation decreases variance ▶ Variance can be measured! ▶ Periodic low variance is indicative of interpolation Leszek ´ Swirski CFA Interpolation Detection

Outline Methods CFA Introduction Examples Interpolation Detection Identifying Forged Regions CFA Pattern Synthesis A statistical Achilles’ Heel 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 4 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 4 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 4 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 휎 2 4 4 4 Leszek ´ Swirski CFA Interpolation Detection

Outline Methods CFA Introduction Examples Interpolation Detection Identifying Forged Regions CFA Pattern Synthesis Gallagher and Chen ▶ First, high-pass filter: ⎡ ⎤ 0 1 0 h = 1 4 1 ⎣ ⎦ 0 1 0 ▶ Estimate variance using mean of absolutes along anti-diagonals ∑ ∣ ( h ∗ i ) x , y ∣ x + y = d m ( d ) = N d Leszek ´ Swirski CFA Interpolation Detection