A Practical Real-Time Video WatermarkingScheme Robust Against Downscaling Attack

Kyung-Su Kim1, Dong-Hyuck Im1, Young-Ho Suh2, and Heung-Kyu Lee1

1 Department of EECS, Korea Advanced Institute of Science and Technology,Guseong-dong, Yuseong-gu, Daejeon, Republic of Korea

{kskim, iammoni, hklee}@mmc.kaist.ac.kr2 Digital Content Research Division, Electronics and Telecommunications Research

Institute, Gajeong-dong, Yuseong-gu, Daejeon, Republic of Koreasyh@etri.re.kr

Abstract. In this paper, we propose a practical real-time video water-marking scheme for downscaling attack. We embed watermark into anarbitrary size of a host video and extract it from the watermarked videounder a preset false error rate. Both watermark embedding and extrac-tion are done in the uncompressed domain. Also, blind detector is used(i.e., no original video needs during the detection). As our watermarkingsystem uses a video coder and decoder according to a format of an inputvideo, it can be applicable to DCT-based coding videos (e.g., MPEG-2)as well as MPEG-1 and MPEG-4. Moreover, for a practical use both em-bedder and detector are developed as directshow filters that make it easyto connect after another within directshow applications. The payload ofthis system is totally 192 bits. PSNR after embedding is around 45dBfor the host video that ranges from QVGA to High Definition (HD) size.Finally, several experimental results prove that the proposed scheme hasgood robustness.

1 Introduction

The development of digital multimedia technique and commercial market placeswhere digital media are broadcast through the high-speed Internet service havealready made any one has access to digital media such as audio, images andvideo. Since digitalized media are perfectly copied and easily modified, contentsproviders (CP) and sellers have difficulty protecting their copyrights. Accord-ingly, it is very important to detect copyright violations and control access tothe digital media. Digital watermarking is used as one of popular techniques forauthentication and copyright protection of the digital media [1].

Figure 1 shows a general digital right management (DRM) system in whichdigital watermarking technique is adopted. At first, User A chooses a favor-able content to buy and requests it to DRM center. After being confirmed thepayment from DRM center through payment gateway, CP verify the requestedcontent and then packages it into the protected content by watermarking. Af-ter that, the user can access the protected content through the Internet service.

2

DRM Center User AUser Authentication

Content Request

Contents

Provider

Billing

List

Payment

Gateway

Paymen

t

Approval

Paymen

t

Request

Authorize

d Co

ntent

with so

me information

User B

Redis

tribu

tion

Distorted

Contents

Processing

Red

istribution

Fig. 1. A general DRM system: a solid line stands for legal flow and a dotted linestands for illegal or unauthorized flow.

There are two ways to communicate between CP and the user: transmittingstreaming format and using downloading service with a secure web browser. Inhere, watermark embedding process should be done in real-time, so watermark-ing algorithm should have low cost and low computational complexity. From theaspect of visual quality, the authenticated user has to want to watch it withoutany degradation. Moreover, the user can record it into own storages and mod-ify for playing portable devices. In order to meet real-time, visual quality, androbustness requirements, watermark is embedded in real-time and human visualsystem (HVS) should be considered during watermark embedding process.

One of possible attacks in the above system is geometric transformations suchas rotation, scale and translation (RST) attack. There are some watermarkingmethods resistant to these attacks such as embedding watermark into invariantdomain [2, 3], using template (template-based synchronization) [4, 5], and usingself-synchronizing watermark (autocorrelation) [6, 7]. First of all, embedding wa-termark into a RST invariant domain spends considerable computing time con-verting into the invariant domain and extracting feature points. Next, templateinsertion is additional distortion to digital data because the template is intendedfor only synchronization. Moreover, even if the synchronization is obtained, therecan be failure to correctly extract watermark. Finally, self-synchronizing water-mark is specifically designed to have both synchronization and watermark in-formation. It has periodic peaks by using autocorrelation function, so detectormeasures attack parameters and recovers as close as possible to the original data.However, since the detector correlates the received watermarked data with itself,computational cost is too high when a high resolution video (e.g., 1920 × 1080or 1280 × 720) is received as an input. Thus, it is difficult to satisfy real-timeembedding or extraction of watermark and simultaneously make watermark ro-

3

bust against the attacks.Some distortions affecting viewing angles such as translation and rotation pro-

cessing are beyond of our scope because it does not commonly happen, so wefocused on downscaling attack. As it is easy for people to purchase portableelectronic devices such as a personal digital assistant (PDA) and a portablemultimedia player (PMP), people want to watch the downloaded video clip intheir own devices by resizing the video clip and converting into other file formatsusing a specific tool. The downloaded video clip is originally adjust to playing ona personal computer with sufficient performance to show high quality and highresolution. However, a PDA is typically about 5 inches in height and 3 inchesin width. Due to this limited size most PDAs have a small amount of memory,include slow processors, and feature small display screens. In order to play it onthe PDAs it is not only required downscaling transformation but also encodedat low-bit rate. Despite these transformations embedded watermark must stillhave robustness. This is why downscaling attack is preferred to other geometricattacks.

In [8], they have suggested a robust video watermarking technique for down-scaling attack in the compressed domain. According to them, spatial downsizingis related to the truncation of full DCT with the same geometrical size. Basedon this criterion their watermarking technique is robust against downscaling at-tack about all DCT-based coding videos. However, if matched video encoderand decoder are available, a robust video watermarking can be accomplished indecoded video frames.

For the convenience we use an image watermarking technique and then extendit to a single decoded video frame. This paper is organized as follows. Section2 begins by explaining a real-time video watermarking algorithm in the uncom-pressed domain. Experimental results demonstrate the performance in section3. Section 4 concludes.

2 The Proposed Watermarking Scheme

Now, we introduce our watermarking scheme. For embedding watermark themain idea is to fix a size of basic pattern and scale the basic pattern withrespect to a size of a host video. Then, the scaled pattern is embedded into thehost video in additive way. If the watermarked video is received as an input atdetector, we recover a basic pattern which has the same size as the original atthe embedder by using the size of the received video. Thus, the recovered patternis correlated with the original basic pattern to extract watermark. We explainwatermark insertion and extraction in detail as follows.

2.1 Watermark Insertion

Let Xi be a set of host video frames, Wi be a watermark sequence, and Yi bea set of watermarked video frames (i = 1, . . . , n). The first step in the inser-tion process is basic pattern generation. This step consists of two parts: (1) a

4

codebook generation and (2) deciding the size of basic pattern. The codebookcomprises M floating random patterns of length N , where the random patternsare identically independent distribution (i.i.d) Gaussian pseudorandom patternswith zero mean and unit variance. The patterns are generated by private key K.So messages to hide are encoded each random pattern Wi. The size of the basicpattern is decided by how many messages are embedded into a single decodedframe and how many the encoded pattern is repeated within it. We define thelength of messages as m bytes and the number of repetition as R times. Thus,we obtain the fixed size of the basic pattern m × N × R and make it of a 2-Ddimension pattern x × y. To resist resizing attack, the size of the 2-D patternshould be smaller than the size of attacked videos. All variables are also knownto detector.

The next step begins with acquiring the size of the host video. Then, Wi is en-larged according to width and height of the host video. That is, Wi becomes thelow-frequency spatial watermark SWi with scaled width Sx and scaled height Sy.If both Sx and Sy are bigger than 3.0, as we know, the low-frequency watermarkcauses blocking artifacts, so an imperceptible embedding technique should beneeded. After SWi has been generated, perceptual model is applied to both thescaled watermark pattern for imperceptible embedding and the original framefor calculating a local strength. The perceptual model that we used consists oftwo components: (1) variable dithering using pixel-by-pixel method and (2) theoptimized noise visibility function (NVF) based on the HVS. To achieve former,SWi is modified to decrease the blocking effect as stated [9, 10]. Dithering matrixis originally used for printing on a 1-bit printer, but we utilize it to reduce thevisibility (see Fig. 2(c)). Pre-defined matrices have different sizes that a value ofa smooth region of 2-D scaled watermark is taken from the dithered value usinglarge size, whereas a value of a detail region is taken using small size. For latterthe spatially perceptual masking is adopted to the original frame and then wecalculate local scaling factor α by pixel-by-pixel estimation. Finally, the water-marked frames Yi are obtained by adding the dithered version of the watermarksto host video frames Xi.

For boosting detector performance the same watermark is inserted into a fixednumber of t consecutive frames. Since it should need high computational coststo estimate α and add the watermark to the host frames, we utilize MMX tech-nology to accomplish real-time insertion process [11]. Figure 2 illustrates ourproposed watermark embedding scheme as mentioned above.

2.2 Watermark Extraction

In the extraction process, we use blind watermark detector performed by normal-ized cross correlation. The first step of the watermark extraction is base patterngeneration with same secret key K at the embedder. It only includes codebookgeneration because the same variables as the embedder are already known to thedetector. Next, the watermarked video frame Yi is sent to a denoising filter sothat we obtain the estimated watermark Wi by subtracting the estimated origi-nal video frame from Yi. Adaptive wiener filter [12] is used as the denosing filter

5

Xi

Yi

Video

Stream

Decoder

Extractluminancecomponent

Encoder

YUV toRGB

Converter

Host Video

Frames

Watermarked

Video Frames

Basic Pattern

Generation

Size

Watermark Embedder

Perceptual

Model

Masking

DitheringSW

K

m

Wi

W

(a)

Wi

SWi

Dithering

Xi

Optimized

HVS

Yi

Modified SWi

(i,j)α

Sx > 3.0

Sy > 3.0

x

y

Sx

Sy

(b)

3 0

21

0 8 2 10

12 4 14 6

3 11 1 9

15 7 13 5

3

15

X

3

8

X

6 8 4

1 0 3

5 2 7

(c)

Fig. 2. Block diagram of proposed watermark embedding scheme: (a) overview of wa-termark embedding into video streams, (b) the details of the watermark embedder,and (c) used 2×2, 3×3, and 4×4 matrices as dithering.

6

and a 3× 3 window is applied to computing mean and variance of an individualpixel.

W (i, j) =VW (i, j)

VW (i, j) + VY (i, j)[Y (i, j)−MY (i, j)] (1)

Equation (1) represents how to compute the estimated watermark. V (i, j)and M(i, j) are the local variance and the local mean for the (i, j) locationrespectively. Since detector has no knowledge of probability distribution andproperties of watermark pattern, we can replace VW (i, j) with the mean valueof VY (i, j). Although the performance of the adaptive wiener filter is lower thana few denosing filters such as phase only filter (POF) and binary phase onlyfilter (BPOF)[13], it is sufficient to our system. To be robust against variousattacks and enhance the watermark energy, we accumulate each extracted noise-like signal from the correspondence frame during t consecutive frames and theestimated watermark is constructed simultaneously by folding and summing ev-ery t frames. Finally, watermark can be extracted by cross correlating it withthe codebook. If estimated correlation value C exceeds threshold T , there is awatermark and then decode the hidden message. Otherwise the detector is failedto extract. This means that the detector has no guarantee its reliability underfalse positive error rate fp. T is slightly different every t frames because ourdetection approach is based on fp. For example, T under the fixed fp = 10−6 isgiven by

fp =12· erfc(

T −mean√2 · var

) = 10−6 (2)

T ≥ mean + 3.36√

2 · var

where T can be computed by means of the mean and the variance of theestimated watermark. The watermark extraction process is depicted in Fig. 3.

Yi

Video

Stream

Decoder

ExtractLuminanceComponent

Video

Renderer

YUV toRGB

Converter

Watermarked

Video Frames

Codebook

Size

K

Watermark Extractor

Denoising

FilterAccumulator

Fold&

Sum

.

.

.W

i

C > T

Decode message

Yi

C

1

M

Fig. 3. Block diagram of proposed watermark extraction scheme.

7

2.3 Implementation of Directshow Filter for Practical Use

Directshow is composed of two objects, filters and filter graphs. The filter graphis a collection of filters and consists of three kinds of filters such as source fil-ters, transform filters, and renderer filters. Among them the transform filter iscommonly used to handle tasks such as data compression and decompression,video decoding, stream splitting, and so on. So, it is possible for desirable re-sults or effects to add specific operations to the filter graph. Also, each filter ofthe filter graph is derived from a modular design; it hides its internal operationand we don’t need to understand complicated interfaces between filters. Usingthese features both embedder and detector are developed as transform filters.This means that some applications built with directshow can make use of itto achieve watermark insertion and detection process. In our case, watermarkembedder and detection filters are connected after a video decoder within ourMPEG video player.

3 Experimental Results

Our experimental system is composed of an Intel Pentium IV CPU with a 3.6Ghzcore, 2 GB DDR2, and ATI X1600 with 256 MB graphics memory. The hostvideos we have tested are 3 categories of video clips with length 17 seconds at30 frame per second (fps): (1) TV drama, (2) documentary, and (3) music showas depicted in Fig. 4. All measured correlation values in this section average

Fig. 4. Snapshot examples of test videos.

three of them. The payload of watermark is totally 192 bits (24 bytes) and samewatermark is repeated 2 times in whole frames. Prior to the watermark insertionand extraction we have set each variable, N = 1024, m = 2 bytes, R = 12 times,t = 30 frames, and fp = 10−6. Thus, the size of 2-D basic pattern becomesdimensions 256×96.

Table 1 reveals that we accomplish real-time processing with our MPEG video

8

Table 1. Execution time and PSNR value comparison between the proposed water-marking scheme and Kang et al. [11]. This table shows a performance of our MPEGvideo player with insertion and extraction when an input video is 1920 × 1080 HDMPEG-2.

[sec.]

Operations Kang et al.[11] Proposed

Frame decoding 0.01317 0.01317

Watermark insertion / extraction 0.011 / 0.004 0.011 / 0.004

Displaying video 0.00481 0.00481

Total insertion time / extraction time 0.029 / 0.022 0.029 / 0.022

Average PSNR 43dB 45dB

player for decoding a video frame, watermark insertion or extraction, and dis-playing video compared to the previous scheme. Note that real-time processingmeans that for a 30fps video a single frame is decoded, processed, and displayedon a screen in around 0.03 sec. Also, an average PSNR value after embeddingis around 45dB for all kinds of test videos. Results show that our watermarkingscheme can produce high-fidelity watermarked videos. Besides the analysis of ex-ecution time and visual quality, correlation values after various attacks are shownin Table 2 when the 1920 × 1080 HD MPEG-2 watermarked video is receivedas an input. The correlation differences between two schemes are approximately0.04 due to adding dithered values instead of watermark values itself for visualquality. However, our watermark is also survived with the acceptable correlationvalues as maintaining visual quality.

Table 2. Normalized correlation value comparisons between the proposed watermark-ing scheme and Kang et al. [11]. After various attacks, watermark is survived with thefollowing values.

Kang et al. [11] Proposed

Original(1920×1080) 0.71 0.67

Resize to VGA 0.46 0.41

Cropping 0.60 0.56

Frame rate change (30fps → 24fps) 0.60 0.56

Color conversion (RGB to Gray) 0.58 0.55

MPEG-1 conversion 0.53 0.49

MPEG-4 conversion 0.47 0.44

In order to prove our robustness to downscaling attack we have constrained6 host video sizes for simplicity, e.g., 1920 × 1080, 1280 × 720, 720 × 480, 640 ×480, 352 × 288, and 320 × 240. We remind that the detector has no informa-tion about the original size of an input video. From Fig. 5 to Fig. 10 shows the

9

downscaling performance of the system; x-axis stands for the resized video fromthe watermarked video and y-axis stands for normalized cross correlation. Thedetection threshold is normally estimated 0.13 subject to fp = 10−6. Althoughsome values are below the threshold on the spot of a second, the whole messagesare correctly extracted since the repetition time of same watermark is two. Ifthe length of video is more longer than the tested video, it can carry out betterresults.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 170

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Seconds

Nor

mal

ized

Cor

rela

tion

Val

ue

No Resizing to 1280x720 to 720x480 to 640x480 to 352x288

Threshold

Fig. 5. Performance against downscaling attack (Input Video: 1920 × 1080).

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 170

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Seconds

Nor

mal

ized

Cor

rela

tion

Val

ue

No Resizing to 720x480 to 640x480 to 352x288

Threshold

Fig. 6. Performance against downscaling attack (Input Video: 1280× 720).

10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 170

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Seconds

Nor

mal

ized

Cor

rela

tion

Val

ue

No Resizing to 640x480 to 352x288 to 320x240

Threshold

Fig. 7. Performance against downscaling attack (Input Video: 720 × 480).

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 170

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Seconds

Nor

mal

ized

Cor

rela

tion

Val

ue

No Resizing to 352x288 to 320x240

Threshold

Fig. 8. Performance against downscaling attack (Input Video: 640 × 480).

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 170

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Seconds

Nor

mal

ized

Cor

rela

tion

Val

ue

No Resizing to 320x240

Threshold

Fig. 9. Performance against downscaling attack (Input Video: 352 × 288).

11

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 170

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Seconds

Nor

mal

ized

Cor

rela

tion

Val

ue

No Resizing

Threshold

Fig. 10. Performance against downscaling attack (Input Video: 320 × 240).

Besides downscaling attack, for each video size the scheme is also robustagainst various attacks such as cropping, format conversion, frame rate change,color-to-gray conversion attack as shown in Table 3. All results indicate that notonly the proposed scheme has good robustness against downscaling attack andother malicious attacks but also it allows us to be confident that the specifiederror rate will not be exceeded.

Table 3. Summarization of robustness to other attacks (threshold = 0.13).

Input video MPEG-1 MPEG-4 Frame rate change Color conversion

1280× 720 0.42 0.35 0.51 0.51

720× 480 0.51 0.42 0.61 0.62

640× 480 0.44 0.41 0.59 0.59

352× 288 0.39 0.38 0.55 0.54

320× 240 0.35 0.35 0.51 0.49

4 Conclusions

In this paper, we present a practical real-time video watermarking scheme fordownscaling attack. We mainly focus on the robustness to downscaling attackbecause more portable devices spread out, more downscaled and distorted videodata are in public. In addition to this attack, the proposed scheme is also robustagainst not only spatial attacks such as cropping, format conversion, and color-to-gray conversion but also temporal attacks such as frame rate change. Bothwatermark insertion and the extraction are done in uncompressed domain, so the

12

proposed scheme can be straightforwardly applied to all video coding if there arecorresponding video coder and decoder. Moreover, using the beneficial featuresof directshow our scheme is suitable for directshow built-in applications.

Acknowledment

This work was in part supported by the KOSEF grant NRL program funded bythe Korea government(MOST) (No. R0A-2007-000-20023-0), and the IT R&Dprogram of MIC/IITA(2007-S017-01, Development of user-centric contents pro-tection and distribution technology)

References

1. T.Sencar, M.Ramkumar, N.Akansu: Data Hiding Fundamentals and Application.Elsevier, Oxford, UK (2004)

2. H.Y.Kim, Y.J.Baek, H.K.Lee: Rotation, scale, and translation invariant watermarkusing higher order spectra. The Journal of SPIE 42 (2003) 340–349

3. H.Y.Lee, H.S.Kim, H.K.Lee: Robust image watermarking using invariant features.The Journal of SPIE 45 (2006) 1–11

4. D.Delannay, B.Macq: A method for hiding synchronization marks in scale androtation resilient watermarking schemes. In: Security and Watermarking of Mul-timedia Contents IV. Volume 4675 of Proceedings of SPIE. (2002) 548–554

5. S.Pereira, J.Ruanaidh, F.Deguillaume, G.Csurka, T.Pun: Template-based recoveryof fourier-based watermarks using log-polar and log-log maps. In: InternationalConference Multimedia Computing and Systems, IEEE CS Press (1999) 870–874

6. C.H.Lee, H.K.Lee: Improved autocorrelation function based watermarking withside information. The Journal of SPIE. 14 (2005) 1–13

7. M.Kutter: Watermarking resisting to translation, rotation, and scaling. In: Securityand Watermarking of Multimedia Contents. Volume 3657 of Proceedings of SPIE.(1999) 423–431

8. Y.Wang, A.Pearmain: Blind mpeg-2 video watermarking robust against geometricattacks: a set of approaches in dct domain. IEEE Transaction on Image Processing15 (2006) 1536–1543

9. J.W.Park, C.H.Lee, H.K.Lee: A visual quality enhancing method for low frequency.In: Proceeding of International Workshop on Image Analysis for Multimedia In-teractive Services, IEE&EURASIP (2006) 49–52

10. Z.Hel-Or, M.Zhang, A.Wandell: Adaptive clust dot dithering. The Journal of SPIE8 (1999) 133–144

11. I.K.Kang, D.H.Im, H.K.Lee: Implementation of real-time watermarking scheme forhigh quality video. In: Proceeding of the ACM Multimedia and Security Workshop.(2006) 124–129

12. I.Karybali, K.Berberidis: Efficient spatial image watermarking via new perceptualmasking and blind detection scheme. IEEE Transaction on Information Forensicsand Security 1 (2006) 256–274

13. Y.Liu, J.Zhao: A new filtering method for rst invariant image watermarking. In:The 2nd IEEE Internatioal Workshop. (2003) 101–106