Blind Detection of Photomontages Using Higher Order Statistics - - PowerPoint PPT Presentation
Blind Detection of Photomontages Using Higher Order Statistics Tian-Tsong Ng, Shih-Fu Chang Columbia University, New York, USA Qibin Sun Institute for Infocomm Research, Singapore Motivation: How much can we trust digital images? March
March 2003: A Iraq war news
Feb 2004: A photomontage
Adobe Photoshop: 5 million
the source side and verifying the mark integrity at the detection side.
authentication signature at the source side and verifying the image integrity by signature comparison at the receiver side.
Need a fully-secure trustworthy camera Need a common algorithm for the source and the detection side. Watermark degrades image quality
signature), verifying whether an image is authentic or fake.
at the source side
Photomontage: [Mitchell 94]
images
Spliced Image (see figure):
e.g. edge softening, etc.
Why interested in detecting image splicing?
photomontages and photomontaging is one of the main techniques for creating fake images with new semantics.
would include detection of post-processing operations and computer graphics techniques for detecting scene internal inconsistencies
spliced spliced
Natural-imaging Quality
Entailed by natural imaging process with real
imaging devices, e.g. camera
Effects from optical low-pass, sensor noise,
lens distortion, etc.
Natural-scene Quality
Entailed by physical light transport in real-world
scene with real-world objects
Results are real-looking texture, right shadow,
right perspective and shading, etc.
Examples:
Computer graphics and photomontages lack in
both qualities.
Computer Graphics photomontage
NIQ: Authentic images comes directly from camera
Image splicing introduces rough edges deviate
We characterize such NIQ using bicoherence Bicoherence (BIC):
A normalized bispectrum, a 3rd order moment spectra ) , ( ( 2 1 2 2 1 2 2 1 2 1 * 2 1 2 1
2 1
) , ( ] ) ( [ ] ) ( ) ( [ )] ( ) ( ) ( [ ) , (
ω ω
ω ω ω ω ω ω ω ω ω ω ω ω
b j
e b X E X X E X X X E b
Φ
= + + =
Magnitude Phase Numerator: Bispectrum Normalization according to
Cauchy-Schwartz Inequality
For signals of low-order moments like Gaussian, BIC
[Fackrell95b] Quadratic Phase Coupling (QPC) vs. BIC
A simultaneous occurrence of frequency harmonics at
(Quadratic Frequency Coupling -
QFC), with respective phase being
At with QPC,
BIC phase = 0 & BIC magnitude = ratio of QPC energy
A
1 2 1 2
, and ω ω ω ω +
1 2 1 2
, and φ φ φ φ +
1 1 2 2 1 2 1 2 1 2 3 3 1 2
( ) cos( ) cos( ) ( ) cos(( ) ( )) ( ) cos(( ) ) where is uncoupled with and
O C C UC UC
X t t t X t C t X t C t ω φ ω φ ω ω φ φ ω ω φ φ φ φ = + + + = + + + = + +
2 2 1 2 2 2
If ( ) ( ) ( ) ( ) ( , )
C O C UC X C UC
C X t X t X t X t BIC C C ω ω = + + ⇒ = +
1 2
If ( ) ( ) ( ) ( , )
O C X
X t X t X t BIC ω ω = + ⇒ ∠ =
1 2 1 2 1 2 1 1 1 1 2 2 2 2 1 2 1 2 1 2 2
If Y( ) ( ) ( ) Y( ) cos(2 2 ) co cos(( s(2 2 ) cos ) ( )) cos( (( ) ) co 1 s( ) ( ))
O O
t X t X t t t t t t t t ω ω φ ω φ φ ω φ ω φ ω ω φ φ ω φ + + + + = + ⇒ = + + + + + − + − + + + +
1 2
( , ) ω ω
Linear quadratic operation induces QPC
[Farid99]
Assuming that speech signal is originally
Nonlinearity associated with splicing
BIC features used for detecting the
average BIC magnitude Variance of the BIC phase histogram
BIC/BIS detects QPC/QFC as one form of non-linearity:
[Bullock97] Studying non-linearity in intracranial EEG signal [KimPowers79] Application in plasma physics [SatoSasaki77] Application in manufacturing [Hasselman63] Application in oceanography [Fackrell95a] Detecting fatigue crack in structure through vibration
BIC/BIS detect signal non-gaussianity
[Santos02] Detecting non-gaussianity in the cosmic microwave
background data
Image splicing introduces rough edges at splicing interface Image splicing can be considered as a bipolar perturbation
1 2 1 2
( ) ( ) with
k x x k x x k k δ δ = − + − − ∆ ⋅ < Difference between the jagged and the smooth signal
Theoretical analysis shows that
bipolar perturbation of a signal results in an increase in BIC magnitude and phase concentration at ±90o
An example of BIC phase histogram
128 128-points DFT (with zero padding and Hanning windowing)
⎟ ⎠ ⎞ ⎜ ⎝ ⎛ + ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ + =
k k k k k k k k k
X k X X k X X X k b
2 2 1 2 2 1 2 1 * 2 1 2 1
) ( 1 ) ( ) ( 1 ) ( ) ( ) ( 1 ) , ( ˆ ω ω ω ω ω ω ω ω ω ω
2 1 2 1
) ( ) (
+ =
i Vertical i N i Horizontal i N
M M fM
v h
2 1 2 1
) ( ) (
+ =
i Vertical i N i Horizontal i N
P P fP
v h
64 Overlapping segments
Negative Phase Entropy (P) ( )log ( )
n n n
P p p = Ψ Ψ
2 1 2
1 1 2 ( , )
Magnitude mean, ( , ) M b
ω ω
ω ω
∈Ω Ω
=
* To reduce noise effect, phase histogram is obtained from the BIC components with magnitude exceeding a threshold
exhibit concentration of energy in 2-D BIS at regions with frequencies corresponding to
energy implies QPC. Hence, Krieger97’s empirical observation predicts that image splicing detection using bicoherence magnitude and phase features would face a significant level of noise. A A
1 1 2 2
/ /
x y x y
f f f f =
1 y
f
2 y
f
1 x
f
2 x
f
natural image random noise
Source: [Krieger97]
distribution for our data set (described later)
greatly overlapped
Sample count Sample count BIC magnitude feature BIC phase feature
textured, textured-smooth and smooth-smooth
different interface types
determining interface types
Bicoherence Magnitude Features Bicoherence Magnitude Features Bicoherence Magnitude Features Textured-smooth Smooth-smooth Textured-textured Edge Percentage Edge Percentage Edge Percentage
The scatter plot for BIC phase feature is similar!
AC is similar to the spliced image except that it is
u = structure component (a edge-preserving function of bounded
variation)
v = fine-texture component (a oscillating function)
formulated as: a
2
2 2 2 2
inf ( ) ; , ( ), ( ), ( ),
BV G BV u
E u u f u f u v f L u BV v G u u λ ⎧ ⎫ ⎪ ⎪ = + − = + ∈ ∈ ∈ = ∇ ⎨ ⎬ ⎪ ⎪ ⎩ ⎭
R
R R R
2
( ) u BV ∈
( ) v G ∈
Structure Fine-texture
We approximate the authentic counterpart (AC) using
We assume that the structure component captures the
splicing invariant features, i.e., less contaminated by splicing
We assume that splicing artifacts (bipolar perturbation) are
captured by the fine-texture component
2 approaches for detecting image splicing
Detect the presence of splicing artifacts in the fine-texture
component (Does not work well because the value of BIC features of the fine-texture component vary in a very narrow range, hence not discriminative)
Detect the absence of splicing artifacts in the structure
Structure-Texture Decomposition
Extract Plain BI C features Extract Plain BI C features
I S
f c f − ⋅
S
I
Prediction Residue Features Plain BIC features computed are the magnitude and phase features. We learn the scaling factor, c, using linear Fisher discriminant analysis See slide with title “Extract Plain BIC features”
we assume to be authentic
and also vertical/horizontal strip.
Samples
Textured Smooth Textured Textured Smooth Smooth Textured Smooth
RBF kernel Support Vector Machine (SVM) on 933
3 evaluation metrics over 100 runs of classification:
Accuracy mean: Average precision: Average recall
i i A i S i A A i S S accuracy
| | | | 100 1
i i S i S S precision
| | 100 1
i i S i S S recall
| | 100 1
Features evaluated (all features below are 1-D)
BIC magnitude feature BIC phase feature BIC magnitude predication residue BIC phase prediction residue Edge pixel percentage
2 1 2
1 1 2 ( , )
( , ) M b
ω ω
ω ω
∈Ω Ω
=
( )log ( )
n n n
P p p = Ψ Ψ
Plain BIC features Prediction residue features Edge feature
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9
Accuracy Mean Average Percision Average Recall
Plain BIC Prediction Residue Plain BIC + Prediction Residue Plain BIC + Edge Prediction Residue + Edge Plain BIC + Prediction Residue + Edge
72% 62%
invariant component with respect to BIC
illumination consistency)
spliced spliced
intracranial EEG in sleep, wakefulness and seizures," EEG Clin Neurophysiol, vol. 103, pp. 661-678, 1997.
1995.
Bicoherence for the Detection of Quadratic Phase Coupling," IEEE-SP ATHOS Workshop on Higher-Order Statistics, Girona, Spain, Jnne 1995.
Interactions," IEEE Transactions on Plasma Science, vol. PS-7, pp. 120-131, June 1997.
Canada, July 21-23, 1997.
Processing, Singapore, Oct 24-27, 2004.
Review Letters, vol. 88, 2002.
to Contactless Diagnosis," Journal of the Acoustic Society America, vol. 62, pp. 382-387, 1977.
Patterns in Image Processing," UCLA C.A.M. Report 02-19, May 2002.
Transactions on Information Theory, vol. 42, pp. 943-958, May 1996.