BANK TREASURY COURSE .pptx Developed & Presented by Mss. TCHATA CATHERINE
Course Outline - Columbia Universitycylin/course/mss/e6886-mss-L1.pdf · Course Outline Multimedia...
Transcript of Course Outline - Columbia Universitycylin/course/mss/e6886-mss-L1.pdf · Course Outline Multimedia...
1
1/18/2006 | Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. © 2006 Columbia University
EE 6886: Topics in Signal Processing
-- Multimedia Security System
Lecture I: Introduction
Ching-Yung Lin
Columbia University
New York, NY 10027, USA
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 2
E 6886 Topics in Signal Processing: Multimedia Security Systems
Course Outline
�Multimedia Security :
� Multimedia Standards – Ubiquitous MM
� Encryption and Key Management – Confidential MM
� Watermarking – Uninfringible MM
� Authentication – Trustworthy MM
�Security Applications of Multimedia:
� Audio-Visual Person Identification – Access Control, Identifying Suspects
� Surveillance Applications – Abnormality Detection
� Media Sensor Networks – Event Understanding, Information Aggregation
2
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 3
E 6886 Topics in Signal Processing: Multimedia Security Systems
About this course
� Instructor: Ching-Yung Lin
� Email: [email protected]
�Office Hour: Wednesday 6:40 – 7:10 pm, Mudd 1312, or by appointment
� Course Webpage: http://www.ee.columbia.edu/~cylin/course/mss/
� Course Time: Wednesday 4:10 – 6:40 pm
� Course Format: Lecture 100 – 120 mins + Presentation/Discussion 20 – 40 mins
� Location: Mudd 535
� TA: TBD
� TA Office Hour: TBD
�Grading: 3 homeworks: 50%, final project: 50%
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 4
E 6886 Topics in Signal Processing: Multimedia Security Systems
Homeworks
�HW #1: Image compression and encryption experiments.
�HW #2: Video watermarking experiments
�HW #3: Audio speaker authentication experiments
�Software Requirements: C, C++, Java, Matlab, or others
�Hardware Requirements: Windows, Unix/Linux or Mac
3
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 5
E 6886 Topics in Signal Processing: Multimedia Security Systems
Final Project
�Team work is encouraged (1 – 3 students)
� Implement components of multimedia security systems or surveys of
emerging technologies
�Oral presentations at the mid-term project proposal and the final
presentation.
�Final project report due at the end of semester.
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 6
E 6886 Topics in Signal Processing: Multimedia Security Systems
Examples of Final Project Topic
� Digital Rights Management in Mobile Environment
� Steganography and steganoanalysis
�Multimedia Forensics
� Human Vision Systems – implementations and experiments
� Art authentication � Types of paintings: modern, abstract, impression, etc.
� Tampering detection, Natural / CG detection
� Face recognition in videos
� Fingerprint recognition
� Human behavior authentication:� Keyboard
� Email records
� Event detection from camera(s)
� Audio/Visual Sensor Network
4
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 7
E 6886 Topics in Signal Processing: Multimedia Security Systems
Outline -- Introduction
�Multimedia Security :
� Multimedia Standards – Ubiquitous MM
� Encryption and Key Management – Confidential MM
� Watermarking – Uninfringible MM
� Authentication – Trustworthy MM
�Security Applications of Multimedia:
� Audio-Visual Person Identification – Access Control, Identifying Suspects
� Surveillance Applications – Abnormality Detection
� Media Sensor Networks – Event Understanding, Information Aggregation
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 8
E 6886 Topics in Signal Processing: Multimedia Security Systems
Multimedia Standards: Towards Knowledge Management and Transaction Enrichment for Digital Media
Media mining
Decision support
IPMP (rights)
Modeling and classification
Personalization and summarization
Similarity searching
Object- and feature-based coding
Compression
Coding
Communications
MPEG-21
Electronic content
Digital items
MPEG-7
Semantic-based retrieval and filtering
Enterprise content mgmt.
MPEG-4,-7
Content-based retrieval
Multimedia filtering
Content adaptation
MPEG-1,-2,-4
Video storage
Broadband
Streaming video delivery
Problems and Innovations
Applications
Media mining
Decision support
Modeling and classification
Personalization and
Similarity searching
Object- and feature-
Compression
Coding
Communications
MPEG-21
E-commerce of
MPEG-7
Semantic-based retrieval
MPEG-4,-7
Content-based retrieval
Multimedia filtering
Content adaptation
MPEG-1,-2,-4
Video storage
Broadband
Streaming video
Problems and Innovations
Applications
Signals Features Semantics KnowledgeSignals Features Semantics Knowledge
Nearfuture
RecentpastRecentpast
MPEG - 1, - 2MPEG - 4
MPEG - 7MPEG - 21
MPEG - 1, - 2MPEG - 4
MPEG - 7MPEG - 21
5
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 9
E 6886 Topics in Signal Processing: Multimedia Security Systems
MPEG-1,2 Overview
� Intraframe: I frames� Interframe: P and B frames
�MPEG-1: 352x240 or 352x264 – for VCD�MPEG-2: (1) multiple resolutions, e.g., 1024x768 – for compatibility with TV. (2) field-based compression
�MPEG-1 Audio Layer 3 – MP3
I B B P B B P B B P B B I
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 10
E 6886 Topics in Signal Processing: Multimedia Security Systems
MPEG-4 Overview
�Object-based compression --- x
�Low-bit rate coding for mobile applications
�Natural-Synthetic hybrid compression --- x
�The latest MPEG-4 standard: H.264/AVC
6
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 11
E 6886 Topics in Signal Processing: Multimedia Security Systems
MPEG-7 OverviewXML Metadata for Multimedia Content Description
� Rich, highly granular descriptions:� Video segments, moving regions, shots, frames, …
� Audio-visual features: color, texture, shape, …
� Semantics: people, events, objects, scenes, …
Acquisition
Authoring
Editing
Browsing
Navigation
Filtering
Management
Transmission
Retrieval
Streaming
Coding
Compression
Searching
Indexing
MPEG-1,-2,-4
MPEG-7
Reference Region Reference Region
Motion Motion
Reference Region
Motion
Segment Tree
Shot1 Shot2 Shot3
Segment 1
Sub-segment 1
Sub-segment 2
Sub-segment 3
Sub-segment 4
segment 2
Segment 3
Segment 4
Segment 5
Segment 6
Segment 7
Event Tree
• Introduction
• Summary
• Program logo
• Studio
• Overview
• News Presenter
• News Items
• International
• Clinton Case
• Pope in Cuba
• National
• Twins
• Sports
• Closing
Time
Axis
� MPEG-7 Normative elements:� Descriptors and Description Schemes
� DDL for defining Description Schemes
� Extensible for application domains
DDL
DS DS
D
D D D
D
DS
Applicationdomain
MPEG-7Standard
i.e., Medical imagingRemote-sensing imagesSurveillance videoComputer animations and graphics
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 12
E 6886 Topics in Signal Processing: Multimedia Security Systems
MPEG-21 Multimedia Framework“Transactions of Digital Items”
� Users and participants in the content value chain
seamlessly exchange content in form of “digital
items” across networks and devices
� Framework supporting all forms of electronic
content/intellectual property (video, music, learning
objects, on-line reports, etc.)
� Digital Item = bundling of:
� Essence (i.e., media resources)
� Metadata
� Rights expressions
� Identifiers
Descriptor
Item
Item
Item
Music package
(one song plus
two photos)
music.mp3
photo1.jpg
photo2.jpg
text/plain
DescriptorMPEG-7
Creation
Informationtext/xml
Item
Example: Digital music package
7
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 13
E 6886 Topics in Signal Processing: Multimedia Security Systems
COMPUTER SECURITY
NETWORK SECURITY
CONTENT SECURITY++
�� Computer SecurityComputer Security� Protect data on a computer
�� Network SecurityNetwork Security� Protect data during transmission
�� Content SecurityContent Security� Protect intellectual property
� Provide Trustworthiness
Types of Security
Multimedia Security
Courtesy of Prof. Kundur, Texas A&M
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 14
E 6886 Topics in Signal Processing: Multimedia Security Systems
Multimedia Security
Content
Provider
Content
Receiver???
� Data Authentication:
-- assure the credibility of multimedia content.
� Confidentiality:
-- secure content transmission privacy.
� Copy Control:
-- protect multimedia data from illegal distribution and theft.
8
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 15
E 6886 Topics in Signal Processing: Multimedia Security Systems
Digital Rights Management (DRM) System
�Definition (from Iannella, 2001)
� Digital Rights Management (DRM) involves the description,
identification, trading, protection, monitoring, and tracking of all forms
of rights usages over both tangible and intangible assets – both in
physical and digital form – including management of Rights Holders
relationships.
�Digital management of use rights to content
� Links specific user rights to media to control access, viewing,
duplication, and sharing.
� Ideally, balances information protection, usability, and cost to provide a
beneficial environment for all parties involved.
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 16
E 6886 Topics in Signal Processing: Multimedia Security Systems
An Example of Digital Rights Management System
Creator
Log
Creation
Description
Unique
Number
IPR dataIPR data
Creation
Provider
RightsRights
HolderHolder
CreationCreation
Value
ValueValue
Value
Assignment of Rights
Assignment of Rights
IPR
Database
Unique
Number
Issuer
Purchaser
Certifi-
cation
Authority
CreationCreation
CreationCreation
Value
LogLog
IPR Info
IPR Info
Check MD
Check MD
authorisation
authorisation
Check ID of MD
Check ID of MD
Check PCheck P
identityidentity
Value
Value
Appl. for licence
Appl. for licence
Conditional Licence
Conditional Licence
T & Cs of use
Media
Distributor
Monitoring
Service
Provider
IMPRIMATUR DRM Model
Unique?
Number
Issuer
IPR
Database
Media
Distributor
9
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 17
E 6886 Topics in Signal Processing: Multimedia Security Systems
Multiple Aspects of DRM
�Technical
� Enforcement by engineering mechanisms/systems
�Business
� Commercially viable products/services
�Social
� User privacy, limits on user behavior, etc.
�Legal
� Enforcement by legislation
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 18
E 6886 Topics in Signal Processing: Multimedia Security Systems
Outline -- Introduction
�Multimedia Security :
� Multimedia Standards – Ubiquitous MM
� Encryption and Key Management – Confidential MM
� Watermarking – Uninfringible MM
� Authentication – Trustworthy MM
�Security Applications of Multimedia:
� Audio-Visual Person Identification – Access Control, Identifying Suspects
� Surveillance Applications – Abnormality Detection
� Media Sensor Networks – Event Understanding, Information Aggregation
10
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 19
E 6886 Topics in Signal Processing: Multimedia Security Systems
Security Services (X.800)
�Person Authentication
� Assurance that communicating entity is the one claimed
�Access Control
� Prevention of unauthorized use of a resource
�Data Confidentiality
� Protection of data from unauthorized disclosure
�Data Integrity
� Assurance that data received is as sent
�Non-Repudiation
� Protection against denial by the parties in a communication
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 20
E 6886 Topics in Signal Processing: Multimedia Security Systems
� Encryption
� Real-time requirements
� Integration with compression
EAVESDROPPING
ConfidentialityConfidentiality
Courtesy of Prof. Kundur, Texas A&M
11
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 21
E 6886 Topics in Signal Processing: Multimedia Security Systems
Access ControlAccess Control
� Encryption
� Authentication
Query???Courtesy of Prof. Kundur, Texas A&M
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 22
E 6886 Topics in Signal Processing: Multimedia Security Systems
RevocationRevocation
� Early detection and reaction
Courtesy of Prof. Kundur, Texas A&M
12
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 23
E 6886 Topics in Signal Processing: Multimedia Security Systems
Conventional Encryption Algorithms
�Data Encryption Standard (DES)
� The most widely used encryption scheme
� DES is a block cipher – the plaintext is processed in 64-bit blocks
� The key is 56-bits in length
� Based on Feistel Cipher Structure
�Triple DES
� Effective key length of 112/168 bits
�Advanced Encryption Standard (AES)
� 128-bit data, 128/192/256-bit keys
� Stronger & faster than Triple-DES
�Public key encryption: asymmetric key
plaintext plaintext
ciphertext
key key
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 24
E 6886 Topics in Signal Processing: Multimedia Security Systems
Outline -- Introduction
�Multimedia Security :
� Multimedia Standards – Ubiquitous MM
� Encryption and Key Management – Confidential MM
� Watermarking – Uninfringible MM
� Authentication – Trustworthy MM
�Security Applications of Multimedia:
� Audio-Visual Person Identification – Access Control, Identifying Suspects
� Surveillance Applications – Abnormality Detection
� Media Sensor Networks – Event Understanding, Information Aggregation
13
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 25
E 6886 Topics in Signal Processing: Multimedia Security Systems
Piracy ProtectionPiracy Protection
� Compliant Players
Courtesy of Prof. Kundur, Texas A&M
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 26
E 6886 Topics in Signal Processing: Multimedia Security Systems
Copyright Protection and Copy Control
ScrambleTx Rx
Descramble
VCR
� Ownership Identification, Copy Control have to survive multi-stage
transcoding
� Use robust watermarking
Because of content-preserving transcoding:
14
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 27
E 6886 Topics in Signal Processing: Multimedia Security Systems
Watermarking
Tx Rx
WatermarkVerify the
watermark
PIL or content- based
feature codes
• Embedding
Visible/Invisible Codes
in Multimedia Data for
(or not for) Security
Purpose
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 28
E 6886 Topics in Signal Processing: Multimedia Security Systems
Visible Watermark
� Purpose:
� Claim the ownership and prevent content piracy.
� Properties:
� Robust: Watermarks must be very difficult, if not impossible, to be
removed.
� Non-obtrusive: Watermarks must not affect the audiovisual contents
too much.
� Visible: It must be visible, but it had better to be insensible.
15
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 29
E 6886 Topics in Signal Processing: Multimedia Security Systems
Visible Watermark Example
� Description:� Pixel brightness were altered pixel by pixel.
Depending on the brightness of each mask pixel, if it is larger than a threshold, then we add a value to the corresponding pixel of the original image to make it brighter. Otherwise, if it is smaller than a threshold, then we subtract a value to make the corresponding pixel darker.
� Alternation Function:
Watermark mask
♦ Robustness: – Randomly set the adjustment factor.
– Randomly locate the mask.
– The mask pixels may depend on the image contents.
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 30
E 6886 Topics in Signal Processing: Multimedia Security Systems
Invisible Watermark
� Purpose:
� Protect ownership and trace illegal use.
� Properties -- Transmit a bitstream through a very noisy channel, i.e. the
original picture.
� Robust: The watermark must be very difficult, if not impossible, to
remove. It must be able to survive manipulations to the images, such
as: lossy compression, format transformation, shifting, scaling,
cropping, quantization, filtering, xeroxing, printing, and scanning.
� Invisible: The watermark should not visually affect the image/video
content.
16
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 31
E 6886 Topics in Signal Processing: Multimedia Security Systems
What is Watermarking ? –Multimedia as a Communication Channel
Information
W Encoder Decoder W
� Analog Communication --
� Encoder/ Decoder:
� Amplitude Modulation (AM),
� Frequency Modulation (FM).
� Multiplexing: use different carrier frequencies.
� Channel: air, wire, water, space, …
Channel
Information
W Encoder Decoder W
Image/Video/Audio
� Watermarking:
� Basic communication system:
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 32
E 6886 Topics in Signal Processing: Multimedia Security Systems
Watermarking -- Multimedia as Communication Channel
S: Source Image (Side Information)W: Embedded InformationX: Watermark (Power Constraint: P)Z: Noise (Power Constraint: N )
W Encoder
S
Decoder W
Z ≤N
Source Image
Information
SwX ≤P Sw
Perceptual
Model
Private/ Public
Distortion
Model
�Encoder may include two stages: Coding and Modulation.
�Coding: Error Correction Codes, Scrambling (use cryptographic keys).
�Modulation:
�Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Code Division Multiple Access (CDMA).
�Spread Spectrum is a CDMA technique, which needs modulation keys for Frequency Hopping or other specific codes.
17
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 33
E 6886 Topics in Signal Processing: Multimedia Security Systems
Example: Watermark surviving Print-and-Scan
After PS, Crop to 360x240 &
JPEG CR: 95:1 => ρ=0.64, Z=4.30After Print & Scan, Crop to
402x266 => ρ=0.80, Z=6.46
Original Image [384x256] Watermarked Image, PSNR 43.8dB, ρ=0.84, Z=7.02
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 34
E 6886 Topics in Signal Processing: Multimedia Security Systems
Example: IBM Digital Music Content Platform Project
ADADADADConversion
DADADADAConversion
Real-time
Watermark
Embedder
Content ID
Audio-in Audio-out
ThinkPad
Host
Audio
Stream
Watermarked
Audio
Stream
Monitoring Center
Content Authoring
(Watermark Embedding)
(Watermark Detection)
Broadcast Monitoring
FM Radio Broadcast
•Automatic generation of cue sheets using audio watermarking
•Secure and easy distribution of music content
•Japanese government funding the project
•Cooperation by popular FM radio stations and major Japanese labels
18
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 35
E 6886 Topics in Signal Processing: Multimedia Security Systems
DMCP Audio Data Hiding Performance
� Data Payload� Standard Version : 72bit/30second (for STEP and DMCP)
� Short Window Detection : 27bit/2-5second
� Benchmarking: STEP – JASRAC (Japan), BIEM, CISAC (France)
� Robustness:
� Acoustic Quality� DVD-Audio quality Psycho-acoustic model
� Joint evaluation test at a Audio device maker’s DVD-Audio studio in 2000
� Reliability� Controllable false alarm rate (e.g. 10-5-10-12)
� Low bit error rate achieved through built-in Error Correcting Code
Noise addition (-30dB)
PCM broadcastAM broadcastFM broadcast
Dynamic range compressionWindows Media Audio compression (64kbps)
RealAudio compression (64kbps)
ATRAC compressionAAC compression 128kbpsMP3 compression (96kbps)
ATRAC3 compressionPitch shifting (10%)Time scaling (10%)
Down-sampling (16kHz)Down-mixing (2ch->1ch)Analogue Conversion
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 36
E 6886 Topics in Signal Processing: Multimedia Security Systems
Outline -- Introduction
�Multimedia Security :
� Multimedia Standards – Ubiquitous MM
� Encryption and Key Management – Confidential MM
� Watermarking – Uninfringible MM
� Authentication – Trustworthy MM
�Security Applications of Multimedia:
� Audio-Visual Person Identification – Access Control, Identifying Suspects
� Surveillance Applications – Abnormality Detection
� Media Sensor Networks – Event Understanding, Information Aggregation
19
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 37
E 6886 Topics in Signal Processing: Multimedia Security Systems
A Photographer’s Shot
President Clinton and First Lady strolled in the White House
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 38
E 6886 Topics in Signal Processing: Multimedia Security Systems
Somebody Manipulates It….
Another proof of their relationship ???
20
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 39
E 6886 Topics in Signal Processing: Multimedia Security Systems
Hillary’s Revenge???
Hillary and Ching-Yung walked together???
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 40
E 6886 Topics in Signal Processing: Multimedia Security Systems
IntegrityIntegrity
� Hash Functions
� Traditional approaches sensitive to format conversion and minor bit
changes
� Existing software tools enable seamless tampering
TAMPERINGCourtesy of Prof. Kundur, Texas A&M
21
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 41
E 6886 Topics in Signal Processing: Multimedia Security Systems
Person AuthenticationPerson Authentication
� Digital signatures
� Biometrics
IMPERSONATIONCourtesy of Prof. Kundur, Texas A&M
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 42
E 6886 Topics in Signal Processing: Multimedia Security Systems
NonrepudiationNonrepudiation
� Digital certificates
� Biometrics
Courtesy of Prof. Kundur, Texas A&M
22
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 43
E 6886 Topics in Signal Processing: Multimedia Security Systems
(Robust) Digital Signature
Digital Camera
ImageHash
Private Key EncryptedDigital
Signature
Digital Image
Image Authenticator
Digital
Signature
Digital Image
Public Key Decrypted
Hash
Comparator
Result
Feature
Codes
Extraction
Feature
Codes
Extraction
� Digital Signature, Diffie and Hellman (1976).
� Verify the data integrity which is endorsed by the signer.
� Trustworthy Digital Camera, Friedman (1993).
� Non-Repudiation Signature to prove reality
� Content-Based Digital Signature, Schneider, Lin, Chang (1996, 1997)� Using content-related feature codes instead of the hash values.
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 44
E 6886 Topics in Signal Processing: Multimedia Security Systems
original image
add
watermark
manipulation
image after crop-and-replacement
and JPEG lossy compression
watermarked SARI
image
authentication
authentication
& recovery
Self Authentication-and-Recovery Images (SARI)
23
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 45
E 6886 Topics in Signal Processing: Multimedia Security Systems
Other Applications
�Semantic Authentication
� Image/Document Forensics
� Who is the actual author of a picture or a novel?
� Is the media data manipulated?
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 46
E 6886 Topics in Signal Processing: Multimedia Security Systems
Outline -- Introduction
�Multimedia Security :
� Multimedia Standards – Ubiquitous MM
� Encryption and Key Management – Confidential MM
� Watermarking – Uninfringible MM
� Authentication – Trustworthy MM
�Security Applications of Multimedia:
� Audio-Visual Person Identification – Access Control, Identifying Suspects
� Surveillance Applications – Abnormality Detection
� Media Sensor Networks – Event Understanding, Information Aggregation
24
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 47
E 6886 Topics in Signal Processing: Multimedia Security Systems
Biometric Features for Person Authentication
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 48
E 6886 Topics in Signal Processing: Multimedia Security Systems
Example: Fingerprint-based Authentication
Fingerprint minutiae
Fingerprint Match
25
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 49
E 6886 Topics in Signal Processing: Multimedia Security Systems
Speaker Recognition
�Speaker Verification and Speaker Identification:
� IBM system
Feature
ExtractionClustering GMM
Basic Model Unit
GMM + T
Gradient
Descent TPopulation
Counts
Transformation
Enhanced
Model Unit
� Features: (TREC2002) 38 dimensional MFCC, (Journal 1/03) 19
� GMM: (TREC 2002) 1536 or 2048 mixtures, (Journal 1/03) 64 mixtures
� Universal Background model (UBM): [TREC2002] trained by 60
speakers (two minutes for training, one minute for testing),
Speech
Signal
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 50
E 6886 Topics in Signal Processing: Multimedia Security Systems
Outline -- Introduction
�Multimedia Security :
� Multimedia Standards – Ubiquitous MM
� Encryption and Key Management – Confidential MM
� Watermarking – Uninfringible MM
� Authentication – Trustworthy MM
�Security Applications of Multimedia:
� Audio-Visual Person Identification – Access Control, Identifying Suspects
� Surveillance Applications – Abnormality Detection
� Media Sensor Networks – Event Understanding, Information Aggregation
26
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 51
E 6886 Topics in Signal Processing: Multimedia Security Systems
Event Understanding
Another View
� Event Understanding:
� Objects:
� Visual Objects:, Tree, Person, Hands, …
� Audio Objects: Music, Speech, Sound, …
� Scenes:
� Background: Building, Outdoors, Sky
� Relationships:
� The (time, spatial) relationships between objects & scenes
� Activities:
� Holding Hand in Hand, Looking for Stars
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 52
E 6886 Topics in Signal Processing: Multimedia Security Systems
Example: Low-cost Multimodality Sensors for Sleep Situation Monitoring
� Understand human night-time activity –Sleep
� What we have done:� Using visual, audio, heartbeat, infrared
sensors to monitor a person’s sleep patterns
� Measurement of sleep quality
� Logging and retrieval of sleep situation
� What we are going to do:� Early detection and long term monitoring of
sleep related diseases
27
Business Unit or Product Name
© 2003 IBM Corporation
IBM Research
Learning and Mining from Multimodal Sensing Signals | Ching-Yung Lin | IBM © 2005 IBM Corporation
Distributed Video Signal Understanding
Face
Outdoors
Indoors
PE1: 9.2.63.66: 1220
PE2: 9.2.63.67
PE3: 9.2.63.68
Female
Male
Airplane
Chair
Clock
PE4: 9.2.63.66:1235
PE5: 9.2.63.66: 1240
PE6: 9.2.63.66
PE7: 9.2.63.66
PE100: 9.2.63.66
(Server) Concept Detection
Processing Elements
CDS
Features
(Distributed Smart Sensors) Block diagram of the
smart sensors
Meta-
data
600
bps
Control
Modules
Resource
Constraints
User
Interests
Display and
Information
Aggregation
Modules
1.5
Mbps
MPEG-
1/2 GOP
Extraction
Event
Extraction 2.8
Kbps
Feature
Extraction320
Kbps
22.4
Kbps
Encod-
ing
Sensor 1Sensor 2
Sensor NSensor 3
TV broadcast,
VCR,
DVD discs,
Video
File Database,
Webcam
Smart Cam
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 54
E 6886 Topics in Signal Processing: Multimedia Security Systems
References
• K. Hill, “Imprimatur Business Model,”MCPS 2nd Consensus Forum, Stockholm, May 1997.
• ISO/IEC JTC 1/SC 29/WG 11 N4269, “Information Technology – Multimedia Framework
(MPEG-21) – Part 4: Intellectual Property Management and Protection,” July 2001.
� G. Kesden, “Introduction on Content Scrambling System,” Lecture on Operating Systems:
Design and Implementation, CMU, Dec. 2000.
�R. Tachibana, “Audio Watermarking for Live Performance,” SPIE Security and
Watermarking of Multimedia Contents V, San Jose, January 2003.
�J. A. Bloom, I. J. Cox, T. Kalker, J.-P. Linnartz, M. L. Miller and B. Traw, “Copy
Protection for DVD Video,” Proceedings of the IEEE, July 1999.
�J. A. Bloom, M. L. Miller and I. J. Cox, “Digital Watermarking”, Morgan Kaufmann
Publishers, 2001.
�C.-Y. Lin and S.-F. Chang, “Semi-Fragile Watermarking for Authenticating JPEG Visual
Content,” SPIE Security and Watermarking in Multimedia Contents II, Vol. 3971, Jan. 2000.
�U. Uludag, S. Pankanti, S. Prabhakar and A. K. Jain, “Biometric Cryptosystems: Issues and
Challenges,” Prof. of the IEEE, June 2004.
28
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 55
E 6886 Topics in Signal Processing: Multimedia Security Systems
M.S. Research Projects Available (Spring 2006, 3Pts)
�Developing Distributed Smart Video Cameras (Phase II):
� Large-Scale Data Mining, Community Modeling, Human Behavior
Modeling
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 56
E 6886 Topics in Signal Processing: Multimedia Security Systems
Other Related Research Issues� Issues in multimedia security
� copyright protection, authentication, fingerprinting: system, theory and
techniques
� public watermarking techniques, watermarking attacks, quality
evaluations and benchmarks
� perceptual models, noise models, information theoretical models
� conditional access
� Traitor tracing
� legal aspects
� watermarking protocols
� security in JPEG2000, MPEG-4, MPEG-7 or MPEG21
� biometrics and multimedia security
� watermarking/ information hiding applications
http://www.research.ibm.com/people/c/cylin
Acknowledgment: Thanks for Prof. Deepa Kundur’s (Texas A&M U.) assistance on the
graphical examples of MM security objectives.
29
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 57
E 6886 Topics in Signal Processing: Multimedia Security Systems
Columbia and IBM
1945-1953(612 W116th St.)
1953-1970 (612 W115th St.)
1970 –(Yorktown Heights)
� The first computer science course
(1947)
� The founding of ACM (1947)
� The first “personal” computer (1948)
� The first
supercomputer
(1954)
� T. J. Watson Research Center
1/18/06: Lecture 1 -- Introduction © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ. 58
E 6886 Topics in Signal Processing: Multimedia Security Systems
Open Discussion