Multimedia (Social Forensics)
-
Upload
sebastiano-battiato -
Category
Presentations & Public Speaking
-
view
184 -
download
0
Transcript of Multimedia (Social Forensics)
![Page 1: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/1.jpg)
Multimedia Forensics:
discovering the history of
multimedia contents.
Prof. Sebastiano BattiatoDipartimento di Matematica e Informatica,
Università di Catania
Image Processing LAB – http://iplab.dmi.unict.it
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 2: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/2.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
Multimedia
Forensics
- Source identification
- Integrity verification/tampering detection
Techniques from multimedia forensics merely provide a way to
test for the authenticity and source of digital sensor data. In this
sense is not about analyzing the semantics of digital or
digitized media objects.
![Page 3: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/3.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 4: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/4.jpg)
Original File: Special Cases
• Recapture: create a fake and then take a
picture with the camera we want to
pretend the picture was taken with
• Staging: the image file is authentic, but
the content has been staged
In these cases an authentic file does not
imply an authentic content.
![Page 5: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/5.jpg)
Multimedia Forensics
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 6: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/6.jpg)
Multimedia Forensics (in practice)
• Source Identification
• Integrity/Authenticity
• Enhancement/Restoration
• Interpretation and Content Analysis– Plate Recognition
– Dynamic Reconstruction (car crashes, etc.)
– Antropomethric issues
– …
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 7: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/7.jpg)
“Forensics Image (Video) analysis is
the application of IMAGE SCIENCE
and DOMAIN EXPERTISE to interpret
the content of an image or the image
itself in legal matters” (SWGIT –
www.fbi.gov)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 8: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/8.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
Recent documents:
• 2016-02-08 SWGDE Best Practices for Photographic Comparison for AllDisciplines
• 2016-02-08 SWGDE Image Processing Guidelines Version1.0
• 2016-02-08 SWGDE Proposed Techniques for Advanced Data Recoveryfrom Security Digital Video Recorders v1-1
• 2016-02-08 SWGDE Training Guidelines for Video Analysis, ImageAnalysis and Photography V1-1
https://www.swgde.org/
![Page 9: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/9.jpg)
ISO Guidelines
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 10: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/10.jpg)
Fantasy/Fiction
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 11: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/11.jpg)
CSI Effect
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 12: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/12.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 13: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/13.jpg)
Esper Blade Runner
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 14: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/14.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 15: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/15.jpg)
Reality
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 16: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/16.jpg)
I Need That Plate! No Way...
![Page 17: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/17.jpg)
I Need That Plate! No Way...
![Page 18: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/18.jpg)
Boston Marathon
“The FBI, reportedly has more than 2,000 agents looking at the publicly
available evidence,”
![Page 19: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/19.jpg)
Challenging Problems
Prof. Sebastiano Battiato – CF 2015-2016
![Page 20: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/20.jpg)
![Page 21: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/21.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
(source Interpol)
![Page 22: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/22.jpg)
Multimedia Forensics is based on the idea
that inherent traces (like digital fingerprints)
are left behind in a digital media during both
the creation phase and any other
successively process.
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 23: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/23.jpg)
• Example:• Forensic analysis of a smartphone: which pictures have been generated
on the device and which ones have been generated by other devices
and sent by messaging application or saved from the internet
• We can identify:• Type of device
• Maker and model
• Specific exemplar
Camera BallisticsWhich Device Has Created This Picture?
![Page 24: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/24.jpg)
Device Identification
![Page 25: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/25.jpg)
Model Identification
http://snapsnapsnap.photos/how-does-the-iphone-6-camera-compare-to-previous-iphone-cameras/
![Page 26: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/26.jpg)
Camera Identification
![Page 27: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/27.jpg)
Source Identification Noise Based
Sensor output carries not only pure signal
but also various noise components. Sensor
noise model could be used as a
representative feature for cameras.
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 28: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/28.jpg)
PRNU as a camera fingerprint
![Page 29: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/29.jpg)
PRNU Estimation
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 30: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/30.jpg)
Sensor Identification Using
Pattern Noise
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
[Lukas2006] J. Lukas, J. Fridrich, and M. Goljan, “Digital Camera identification from sensor pattern noise” IEEE
Transaction Inf. Foren.Sec. Vol. 1, 205–214 (2006).
![Page 31: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/31.jpg)
Sensor Identification Using
Pattern NoiseThis method provide good results, and is
quite reliable also using:
–images with different level of JPEG
compression (low, medium)
–images processed using point-wise operator
such as brightness/contrast adjustment or
gamma correction.
–images acquired by two cameras of the same
brand and model.
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 32: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/32.jpg)
Integrity: What is a Forgery?• “Forgery” is a
subjective word.
• An image canbecome a forgery
based upon the
context in which
it is used.
• An image altered for fun or someone who has taken an badphoto, but has been altered to improve its appearancecannot be considered a forgery even though it has beenaltered from its original capture.
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 33: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/33.jpg)
Altering Images
The concepts have movedinto the digital world byvirtue of digital camerasand the availability ofdigital image editingsoftware
The ease of use of digital image editing software, which doesnot require any special skills, makes image manipulation easyto achieve.
circa 1860: This nearly iconic portrait of U.S. President Abraham Lincoln is a composite of Lincoln's head and the Southern politician John Calhoun's body.
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 34: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/34.jpg)
Who Cares?
media
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 35: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/35.jpg)
Who Cares?
geopolitics…
…and political propaganda2nd Meeting EU IAI – Interpol Headquarter
(Lyon) – October 2016
![Page 36: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/36.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 37: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/37.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
Advertisement
![Page 38: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/38.jpg)
More (and more) examples
Photo Tampering through History
http://www.fourandsix.com/photo-tampering-history/
Photoshopdisaster
http://www.photoshopdisasters.com/
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 39: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/39.jpg)
Image Editing
Malicious image editing alters the image semantic
content, mainly:
Adding information
Removing information
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
Piva 2013
![Page 40: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/40.jpg)
Image Editing• Splicing (two images)
– Also called cut and paste, compositing
– Used to add information
• Cloning (single image)
– Also called copy and paste, copy move, region duplication
– Used to add or remove information
– Can be exact, or the clone can be resized, rotated…)
• Inpainting (kind of intelligent clone)
– Seam carving, content aware resize, content aware fill, content dependent crop
– Used to remove information
• Retouch (local editing)
– Dodge and burn, healing tool…
• Image enhancement/filtering
– Histogram equalization, contrast enhancement, median filtering, denoise, smooth…
• Image editing (geometric transformation)
– Resize, crop, zoom, shear
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 41: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/41.jpg)
How To Authenticate An Image?
• Visual Inspection
• File AnalysisFile Format and Structures
Metadata (EXIF)
Compression Parameters (Quantization
Tables)
• Global AnalysisPixel and compressed data statistics
• Local AnalysisFinding inconsistencies of pixel statistics
across the image
![Page 42: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/42.jpg)
Image Forensics Methods
Passive Methods: Using the alterations ofthe underlying statistics produced by digitalforgeries on an image:
PHYSICS BASEDCAMERA BASED
PIXEL BASEDGEOMETRIC BASED
FORMAT BASED2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 43: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/43.jpg)
Physics-BasedLighting inconsistencies can used for revealing traces of
digital tampering.
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 44: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/44.jpg)
Camera-Based
INTERPOLATION
LENS CFA SENSOR
POST PROCESSINGDIGITAL IMAGESTORAGE
Processing and Storage
ORIGINALIMAGE
Acquisition
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 45: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/45.jpg)
Types Of Analysis: Signal Level
Based on statistical features of pixel values; need good quality image
• Clone detection
– Cloned image blocks
– Similar couples of key points
• Resampling detection
– For resize, rotate, but also when splicing or cloning
• Enhancement Detection
– Specific for algorithms (median, histogram equalization, color adjustment)
• Seam carving detection
• General intrinsic footprints
• Inconsistencies from acquisition and coding fingerprints
– CFA, PRNU, DCT, ELA…
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 46: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/46.jpg)
Format-BasedJPEG compression engine
(for both luminance and chrominance channels):
the input image ispartitioned into 8x8non-overlapping blocks
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 47: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/47.jpg)
Format-BasedJPEG forgery engine
2nd Meeting EU IAI – Interpol Headquarter
(Lyon) – October 2016
![Page 48: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/48.jpg)
Periodic artifact introduced by Double JPEG quantizations (2)
A. C. Popescu and H. Farid, Statistical tools for digital forensics, in Proc. 6th Int. Workshop Information Hiding, Berlin,
Germany, 2004, pp. 128–147, Springer-Verlag.
Z. Lin, J. He, X. Tang, and C.-K. Tang, Fast, automatic and fine-grained tampered JPEG image detection via DCT
coefficient analysis, Pattern Recognition, vol. 42, no. 11, pp. 2492–2501, Nov. 2009.
If q2<q1, then n(u2) =0 for some u2, hence the histogram related to the double
quantization can show periodically missing values. On the contrary, if q2>q1 the
histogram can have some periodicity in terms of peaks and valleys pattern.
2nd Meeting EU IAI – Interpol Headquarter
(Lyon) – October 2016
![Page 49: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/49.jpg)
THE TYPICAL PIPELINEFOR A COPY-PASTE
OPERATION
+
=
original image
QF(1) = q1
resulting image
QF(3) = q3
2nd image
QF(2) = q2
duplicating
resizing
2nd Meeting EU IAI – Interpol Headquarter
(Lyon) – October 2016
![Page 50: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/50.jpg)
F. Galvan, G. Puglisi, A. R. Bruna, S. Battiato, First Quantization Matrix Estimation from Double Compressed JPEG Images, IEEE Transactions on Information Forensics and Security, 2014.
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 51: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/51.jpg)
Image Alignment for Tampering Detection
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
S. Battiato, G. M. Farinella, E. Messina, G. Puglisi - Robust Image Alignment for Image
Authentication and Tampering Detection – IEEE Transactions on Information Forensics & Security,
Vol. 7 – Issue 4, pp. 1105-1117, 2012.
![Page 52: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/52.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
http://revealproject.eu/
http://www.rewindproject.eu/
http://maven-project.eu/#_=_
https://s-five.eu/
The final draft of the FIVE Best Practice
Manual is publically available from
December 8, 2015 ("October/DIWG2015
version"): DRAFT_BPM_FIVE_20151009
![Page 53: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/53.jpg)
Use Cases
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 54: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/54.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 55: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/55.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 56: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/56.jpg)
Image Manipulation: Case “Mozzarella Blu”
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 57: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/57.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 58: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/58.jpg)
Evidence on the web
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 59: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/59.jpg)
Forgery on Biomedical Images
Corriere della Sera – Ottobre 2013
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 60: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/60.jpg)
Forgery on Science
“What’s in a picture? The temptation of image manipulation.,” J. Cell Biol., vol. 166, no. 1, pp.
11–5, Jul. 2004.2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 61: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/61.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
Current Trends And
Challenges
![Page 62: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/62.jpg)
Current Trends: Point&Shoot
and Share…
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 63: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/63.jpg)
Future of Imaging
Nikon
![Page 64: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/64.jpg)
Sharing
![Page 65: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/65.jpg)
The Social Picture
S. Battiato, G. M. Farinella, F. L. M. Milotta, A. Ortis, L. Addesso, A. Casella, V. D'amico, G. Torrisi, The Social Picture, ACM International Conference on Multimedia Retrieval 2016
![Page 66: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/66.jpg)
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
Social (Multimedia) Forensics
• Image and Video Phylogeny
ReVeal project
![Page 67: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/67.jpg)
Social (Multimedia) Forensics
• Uploading an image on a Social Network
- The process alters images
- Resize
- Rename
- Meta-Data deletion/editing
- Re-Compression
- NEW JPEG file Structure
M. Moltisanti, A. Paratore, S. Battiato, L. Saravo - Image Manipulation on Facebook for Forensics
Evidence – ICIAP 2015, LNCS 2015;
O. Giudice, A. Paratore, M. Moltisanti, S. Battiato - A Classification Engine for Image Ballistics of
Social Data – (Arxiv 2016 No. 1699257) http://arxiv.org/abs/1610.06347
![Page 68: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/68.jpg)
Social (Multimedia) Forensics (2)
• Uploading an image on a Social Network- The process alters images
- Each Social Network Service do different alterations
Resized
Proportionally
Squared
Image
O. Giudice, A. Paratore, M. Moltisanti, S. Battiato - A Classification Engine for Image Ballistics of
Social Data – (Arxiv 2016 No. 1699257) http://arxiv.org/abs/1610.06347
![Page 69: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/69.jpg)
Social (Multimedia) Forensics (2)
• Uploading an image on a Social Network- The process alters images
- Each Social Network Service makes differentalterations
O. Giudice, A. Paratore, M. Moltisanti, S. Battiato - A Classification Engine for Image Ballistics of
Social Data – (Arxiv 2016 No. 1699257) http://arxiv.org/abs/1610.06347
![Page 70: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/70.jpg)
Social (Multimedia) Forensics (2)
• Uploading an image on a Social Network- The process alters images
- Each Social Network Service makes different alterations
O. Giudice, A. Paratore, M. Moltisanti, S. Battiato - A Classification Engine for Image Ballistics of
Social Data – (Arxiv 2016 No. 1699257) http://arxiv.org/abs/1610.06347
![Page 71: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/71.jpg)
Social (Multimedia) Forensics (2)
• Uploading an image on a Social Network- The process alters images
- Each Social Network Service makes differentalterations
![Page 72: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/72.jpg)
Social (Multimedia) Forensics (2)
• Uploading an image on a Social Network- The process alters images
- Each Social Network Service makes differentalterations
Social Network
Fingerprint
on Uploaded
Images
![Page 73: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/73.jpg)
Social (Multimedia) Forensics (2)
• Uploading an image on a Social Network- The process alters images
- Each Social Network Service makes differentalterations
- Alterations are dependent to uploading client
![Page 74: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/74.jpg)
Social (Multimedia) Forensics (2)
• Social Image Ballistics (recover image history)
Uploaded images
dataset
![Page 75: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/75.jpg)
Social (Multimedia) Forensics (2)
• Social Image Ballistics (recover image history)
Social Altered image dataset- 10 Social Platforms
- Facebook, Google+, Instagram, Flickr, Tumblr,
Twitter, Imgur, Tinypic, Telegram, Whatsapp
- 2720 JPEG Images representing different
subjects (natural, indoor, outdoor)
- Dataset available at: http://iplab.dmi.unict.it/DigitalForensics/social_image_forensics/
![Page 76: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/76.jpg)
Social (Multimedia) Forensics (2)
• Social Image Ballistics (recover image history)
- On which Social Network was uploaded image I?
IGiven a JPEG image I, the Social Image Ballistics task has the objective of
defining:
1) if there is a compatibility between the non-related JPEG elements of I
(i.e. filename, EXIF data) and the processing pipeline of SNSs;
2) if there is a compatibility between the JPEG elements of I and the
processing pipeline of SNSs;
3) which SNS is compatible with the JPEG elements of the image, with a
certain degree of confidence, and what is the uploading source in
terms of operating system (OS) and application. Input Image
O. Giudice, A. Paratore, M. Moltisanti, S. Battiato - A Classification Engine for Image Ballistics of
Social Data – (Arxiv 2016 No. 1699257) http://arxiv.org/abs/1610.06347
![Page 77: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/77.jpg)
Social (Multimedia) Forensics (2)
• Social Image Ballistics (recover image history)
- On which Social Network was uploaded image I?
I
Input Image
Feature
Extraction
O. Giudice, A. Paratore, M. Moltisanti, S. Battiato - A Classification Engine for Image Ballistics of
Social Data – (Arxiv 2016 No. 1699257) http://arxiv.org/abs/1610.06347
![Page 78: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/78.jpg)
Social (Multimedia) Forensics (2)
• Social Image Ballistics (recover image history)
- On which Social Network was uploaded image I?
I
Input Image
Feature
Extraction
O. Giudice, A. Paratore, M. Moltisanti, S. Battiato - A Classification Engine for Image Ballistics of
Social Data – (Arxiv 2016 No. 1699257) http://arxiv.org/abs/1610.06347
![Page 79: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/79.jpg)
Social (Multimedia) Forensics (2)
• Social Image Ballistics (recover image history)- On which Social Network was uploaded image I?
Representation of whole Dataset
O. Giudice, A. Paratore, M. Moltisanti, S. Battiato - A Classification Engine for Image Ballistics of
Social Data – (Arxiv 2016 No. 1699257) http://arxiv.org/abs/1610.06347
![Page 80: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/80.jpg)
Social (Multimedia) Forensics (2)
• Social Image Ballistics (recover image history)
- On which Social Network was uploaded image I?
Input Image
Feature
Extraction
• DQTs coeffs
• Image Size
• # EXIF
• # JPEG Markers
Anomaly
Detection
The Anomaly Detector excludes images not processed
by Social Network Platforms
Given a Similarity measure between features extracted
from images:
It is possible to build a distance matrix D of size N×N
where the element dij is equal to the distance
between the images Ii and Ij.
The Anomaly Detector is then defined as:
![Page 81: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/81.jpg)
Social (Multimedia) Forensics (2)
• Social Image Ballistics (recover image history)
- On which Social Network was uploaded image I?
Input Image
Feature
Extraction
• DQTs coeffs
• Image Size
• # EXIF
• # JPEG Markers
Anomaly
Detection
SNS
Classification
Upload Client
Classification
Output: Not in our dataset
The image probably is not altered by a SNS
Image does not come from considered platforms
O. Giudice, A. Paratore, M. Moltisanti, S. Battiato - A Classification Engine for Image Ballistics of
Social Data – (Arxiv 2016 No. 1699257) http://arxiv.org/abs/1610.06347
![Page 82: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/82.jpg)
Social (Multimedia) Forensics (2)
• Social Image Ballistics (recover image history)
- On which Social Network was uploaded image I?
O. Giudice, A. Paratore, M. Moltisanti, S. Battiato - A Classification Engine for Image Ballistics of
Social Data – (Arxiv 2016 No. 1699257) http://arxiv.org/abs/1610.06347
![Page 83: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/83.jpg)
O. Giudice, A. Paratore, M. Moltisanti, S. Battiato - A Classification Engine for Image Ballistics of
Social Data – (Arxiv 2016 No. 1699257) http://arxiv.org/abs/1610.06347
![Page 84: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/84.jpg)
Conclusions
• Multimedia Forensics is now a
consolidated field but new intriguing
challenges emerge every day.
• Among other current trends include:
– Big Data analysis (e.g. Social Network) by
«deep» paradigm?
– Advanced Video Synopsis (First-person-
Vision)
– Semantic Exploitation of user-generated
content
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 85: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/85.jpg)
Surveys
• Matthew C. Stamm, Min Wu and K. J. Ray Liu, Information
Forensics: An Overview of the First Decade (2013), in: IEEE
Access, 1(167-200)
• Alessandro Piva, An Overview on Image Forensics (2013), in:
ISRN Signal Processing, 2013 (Article ID 496701, 22 pages)
• C. Baron - Adobe Photoshop Forensics – Sleuths, Thruts, and
Fauxtography – Thomson Course Tehcnology - 2009
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 86: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/86.jpg)
On line Resources• Tutorial by Prof. Hany Farid - Digital Image Forensics:
lecture notes, exercises, and matlab code for a survey
course in digital image and video
forensics. http://www.cs.dartmouth.edu/farid/downloads/tut
orials/digitalimageforensics.pdf
• SOFTWARE: Amped5, Authenticate, Adroit, Four&Six,
Izitru, Ghiro, …
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 87: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/87.jpg)
Other related works
• Wang W. and Dong J. and Tan T.: Exploring DCT CoefficientQuantization Effects for Local Tampering Detection, IEEETransactions on Information Forensics and Security, 9, 10,1653–1666, (2014)
• Liu Q. and Sung A.H. and Chen Z. and Chen L.: ExposingImage Tampering with the Same Quantization Matrix,Multimedia Data Mining and Analytics, 327–343, (2015)
• C. Pasquini, F. Perez-Gonzlez, Giulia Boato: A Benford-Fourier JPEG compression detector. ICIP 2014:
• C. Pasquini, G. Boato, F. Perez-Gonzlez Multiple JPEGcompression detection by means of Benford-Fouriercoefficients. WIFS 2014
2nd Meeting EU IAI – Interpol
Headquarter (Lyon) – October 2016
![Page 88: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/88.jpg)
Main Scientific PublicationsM.Moltisanti, A.Paratore, S. Battiato, L. Saravo - Image Manipulation on Facebook for
Forensics Evidence – ICIAP 2015, LNCS 2015;
F. Galvan, G. Puglisi, A. R. Bruna, S. Battiato, First Quantization Matrix Estimation from
Double Compressed JPEG Images, IEEE Transactions on Information Forensics and
Security, 2014
S. Battiato, G. M. Farinella, E. Messina, G. Puglisi - Robust Image Alignment for Image
Authentication and Tampering Detection – IEEE Transactions on Information Forensics
& Security, Vol. 7 – Issue 4, pp. 1105-1117, 2012.
S. Battiato, G. M. Farinella, G. Puglisi, D. Ravì – Aligning Codeboooks for Near
Duplicate Image Detection – Multimedia Tools and Applications - Springer 2013.
S. Battiato, G. Messina - Digital Forgery Estimation into DCT Domain - A Critical Analysis
- In Proceedings of ACM Multimedia 2009 - Workshop Multimedia in Forensics - Bejing
(China), October 2009.
S. Battiato, G.M. Farinella, G.C. Guarnera, T. Meccio, G. Puglisi, D. Ravì, R. Rizzo - Bags
of Phrases with Codebooks Alignment for Near Duplicate Image Detection – In
Proceedings of ACM Multimedia – Workshop Multimedia in Forensics, Security and
Intelligence (MiFor 2010) – Florence (Italy), October 2010;
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 89: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/89.jpg)
IISFA memberbook• S. Battiato, G. Messina, R. Rizzo - Image Forensics - Contraffazione Digitale e
Identificazione della Camera di Acquisizione: Status e Prospettive - Chapter in IISFA
Memberbook 2009 DIGITAL FORENSICS - Eds. G. Costabile, A. Attanasio - Experta, Italy
2009;
• S. Battiato, G.M. Farinella, G. Messina, G. Puglisi - Digital Video Forensics: Status e
Prospettive - Chapter in IISFA Memberbook 2010 DIGITAL FORENSICS - Eds. G. Costabile,
A. Attanasio - Experta, Italy 2010
• S. Battiato, G.M. Farinella, G. Puglisi - Image/Video Forensics: Casi di Studio - Chapter in
IISFA Memberbook 2011 DIGITAL FORENSICS - Eds. G. Costabile, A. Attanasio - Experta,
Italy 2012.
• S. Battiato, M. Moltisanti – Tecniche di Steganografia su Immagini Digitali – Chapter in
IISFA Memberbook 2012 DIGITAL FORENSICS - Eds. G. Costabile, A. Attanasio - Experta,
Italy (2013)
• S.Battiato, F. Galvan, M. Jerian, M. Salcuni - Linee Guida per l'autenticazione Forense di
Immagini – Chapter in IISFA Memberbook 2013 DIGITAL FORENSICS - Eds. G. Costabile,
A. Attanasio - Experta, Italy (2013)
• S. Battiato, A. Catania, F. Galvan, M. Jerian, L.P. Fontana – Acquisizione ed Analisi
Forense di Sistemi di Videosorveglianza - Chapter in IISFA Memberbook 2014 DIGITAL
FORENSICS - Eds. G. Costabile, A. Attanasio - Experta, Italy 2015
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 90: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/90.jpg)
Sicurezza e Giustizia• S.Battiato, F. Galvan - Introduzione alla Image/Video Forensics - Sicurezza e
Giustizia - Numero I/MMXIII - pp. 42-43 – 2013.
• S.Battiato, F. Galvan - La Validità Probatoria Delle Immagini e dei Video-
Sicurezza e Giustizia - Numero II/MMXIII - pp. 30-31 – 2013
• S.Battiato, F. Galvan - Ricostruzione Di Informazioni 3d A Partire Da Immagini
Bidimensionali - Sicurezza e Giustizia ( n.IV_MMXIII ) – 2014
• S.Battiato, F. Galvan - Verifica dell'Attendibilità di un Alibi Costituito da
Immagini o Video - Sicurezza e Giustizia - Numero II/MMXIV - pp. 47-50 – 2014.
• Rundo, E. Tusa, S. Battiato - Medical Image Enhancement nei Procedimenti
Giudiziari Medico-Legali in ambito Oncologico - Sicurezza e Giustizia - Numero
I/MMXVI - pp. 53-56 - 2016
• - See more at: http://www.sicurezzaegiustizia.com/
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 91: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/91.jpg)
Prof. Sebastiano Battiato
Dipartimento di Matematica e Informatica
University of Catania, Italy
Image Processing LAB – http://iplab.dmi.unict.it
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016
![Page 92: Multimedia (Social Forensics)](https://reader036.fdocuments.us/reader036/viewer/2022062412/58ef5de01a28abfd4c8b4611/html5/thumbnails/92.jpg)
Main Contacts
Further Info
Image Processing Lab
Università di Catania
www.dmi.unict.it/~iplab
2nd Meeting EU IAI – Interpol Headquarter (Lyon) – October 2016