Seeing the Invisibles: Recent Progress in Info. Forensics · Seeing the Invisibles: Recent Progress...

Seeing the Invisibles:Seeing the Invisibles:Recent Progress in Info. ForensicsRecent Progress in Info. Forensics

Min Wu

Media and Security Team (MAST)Media and Security Team (MAST) ECE Department / UMIACS

University of Maryland, College Park

Include joint work with Wei-Hong Chuang, Ravi Garg, Hongmei Gou,

http://www.ece.umd.edu/~minwu/research.html

Adi Hajj-Ahmad, K.J. Ray Liu, Hui Su, Ashwin Swaminathan, and Avinash Varna.

Questions about Digital Images Questions about Digital Images and Video …and Video …

Photographed? Scanned? Or computer generated?

Tampered? When and where captured? Who/where’s the leak? Post on YouTube or not?

Min Wu (UMD): Seeing Invisibles - Spring 2013 2Iranian Missile: Illustration by The New York Times; photo via Agence France-Presse

Many Forms of “Digital Fingerprints”Many Forms of “Digital Fingerprints”

Many types of fingerprints for multimedia protection & management

I C EI C EI. C. E.I. C. E.

Embedded FingerprintEmbed unique ID/signal as digital fingerprints to track individual copy and trace unauthorized use

C t t b d Fi i tContent-based FingerprintCompact content signature for content identification, and also useful for watermarking and content authentication

Intrinsic FingerprintExamine inherent traces left on multimedia by device or processing –Provide non intrusive forensics to determine origin integrity etc

Min Wu (UMD): Seeing Invisibles - Spring 2013 3

Provide non-intrusive forensics to determine origin, integrity, etc.






Whi HAlice

w1 LeakLeak


White House

Satellite Image

Bobw2

LeakLeak


Provide non-intrusive forensics to determine origin, integrity, etc.g

Carl

w3




Embedded FingerprintShazam app

Embed unique ID/signal as digital fingerprints to track individual copy and trace unauthorized use

C t t b d Fi i t

Shazam app for iPhone

Content-based FingerprintCompact content signature for content identification, and also useful for watermarking and content authentication




Intrinsic Traces in Images and VideoIntrinsic Traces in Images and Video

Represent group properties– “Digital / software” components of device or processing system

f– Ensemble properties of analog components: e.g. statistical noise profile of sensors

R ?R ?? ? ? ?? ?

R ?

? ?

R ?

? ?

? ?? ?

CandidateCFA pattern CFA InterpolationCFA pattern

Fitting errorFitting error

7Digital photograph Scanner model 1 Scanner model 2




Represent individuality of capturing device or environmentRepresent individuality of capturing device or environment– “Unreproducible / unclonable” individual properties

~ e.g. individual variation from “analog” part of sensors due to manufacturing variabilitymanufacturing variability

Min Wu (UMD): Seeing Invisibles - Spring 2013

8Samsung i760Apple iPhone 3G




Represent individuality of capturing device or environmentRepresent individuality of capturing device or environment– “Unreproducible / unclonable” individual properties

~ e.g. individual variation from “analog” part of sensors due to manufacturing variabilitymanufacturing variability

– Unique time-varying location-dependent conditions during capture

D t ti C t d t ti Detection vs. Counter-detection– Determine integrity, origin, time/location, processing history. etc.– Remove detectable/inferable traces for privacy sanitization


Forensic Forensic Questions Questions on “Time” and “Place”on “Time” and “Place”

500

600

-90

-80

300

400

500

me

(in s

econ

ds)

-120

-110

-100

90

9 6 10 10 4 10 8

100

200

Tim

-150

-140

-130

When was the video actually shot? And where? Was the sound track captured at the same time as the

9.6 10 10.4 10.8Frequency (in Hz)

Was the sound track captured at the same time as the picture? Or super-imposed afterward?

Explore fingerprint influenced by power grid onto sensorExplore fingerprint influenced by power grid onto sensor recordings

10Min Wu (UMD): Seeing Invisibles - Spring 2013

Ubiquitous Forensic Fingerprints from Power GridUbiquitous Forensic Fingerprints from Power Grid

400

500

onds

) -40

-20

0.7

0.8

0.9

effic

ient

400

500

600

nds) -100

-90

-80

100

200

300

Tim

e (in

sec

o

-100

-80

-60

30 20 10 0 10 20 30

0.3

0.4

0.5

0.6

Cor

rela

tion

co

100

200

300

Tim

e (in

sec

o

-150

-140

-130

-120

-110

49.5 50 50.5 51 51.5Frequency (in Hz)

-30 -20 -10 0 10 20 30Time frame lag

ENF matching result demonstrating similar variations in the ENF

Video ENF signal Power ENF signal Normalized correlation9.6 10 10.4 10.8

Frequency (in Hz)

Electric Network Frequency (ENF): 50/60 Hz nominal

g gsignal extracted from video and from power signal recorded in India

Varies slightly over time; main trends consistent in same grid Can be “seen” or “heard” in sensor recordingsHelp determine recording time/location detect tampering etc

Min Wu (UMD): Seeing Invisibles - Spring 2013

Help determine recording time/location, detect tampering, etc.

Ref: Ravi-Varna-Wu paper in ACM Multimedia 2011 11

Tampering DetectionTampering Detection

ENF signal from Video

ENF matching result demonstrating the detection of video tampering based on the ENF traces

10

10.1

10.2

10.3

eque

ncy

(in H

z)

Insertedli

160 320 480 640 800 96010

Time (in seconds)

Fre

50.2n H

z)

Ground truth ENF signal

clip

160 320 480 640 80049.9

5050.1

Ti (i d )

Freq

uenc

y (in

Adding a clip between the original video leads to discontinuity in the ENF signal extracted from videoCli i ti l b d t t d b i th id ENF

Time (in seconds)

Clip insertion can also be detected by comparing the video ENF signal with the power ENF signal at corresponding time


Forensic Binding of Audio and Visual TracksForensic Binding of Audio and Visual Tracks ENFs in audio and video tracks captured at the same time have

high correlation

Research questions ahead: (1) How to accurately estimate and match weak and noisy ENF? (2) Can ENF be removed? Tampered? (3) How to prevent anti forenics on ENF?be removed? Tampered? (3) How to prevent anti‐forenics on ENF? (4) New applications: smart grid monitoring; ……

13Min Wu (UMD): Seeing Invisibles - Spring 2013Ref: Chuang-Ravi-Wu paper in ACM CCS 2012

Explore Machine Learning to Infer LocationExplore Machine Learning to Infer Location Inter-Grid location-of-recording estimation from sensing

signals containing ENF traces– Preliminarily identified useful features for average 85% accuracy

mat

es (H

z)

mat

es (H

z)

mat

es (H

z)

requ

ency

Est

im

requ

ency

Est

im

eque

ncy

Estim

FrFr

Time (secs) Time (secs) Time (secs)

Fre

LEBANON INDIA Eastern US


Can ENF Pinpoint to Locations Within a Grid?Can ENF Pinpoint to Locations Within a Grid? Main trend of ENF is known to be same in a grid “Microscopic” traces due to localized effect

– Small variations aren’t felt between places far apart– Dynamic distributed control to stabilize power grid has a response

propagation speed of about 500 miles per secondp p g p p

Our multi-location studies in U.S. east and west grids

Min Wu (UMD): Seeing Invisibles - Spring 2013 15(a) ENF signals from different locations

of US Eastern grid(b) Correlation between ENF signals

after high-pass filtering

Include joint work with Wei-Hong Chuang, Ravi Garg, Hongmei Gou, Adi Hajj-Ahmad, K.J. Ray Liu, Hui Su, Ashwin Swaminathan, and Avinash Varna.


Seeing the Invisibles: Recent Progress in Info. Forensics · Seeing the Invisibles: Recent Progress...

Documents

Transcript of Seeing the Invisibles: Recent Progress in Info. Forensics · Seeing the Invisibles: Recent Progress...