A Well-Behaved TV Logo RecognitionMethod Using Heuristic Optimality Checked

Basis Pursuit Denoising

Jie Xu(B), Min He, and Xiuguo Bao

National Computer Network and Information SecurityAdministrative Center, Beijing, China

xujie@cert.org.cn

Abstract. Since video logos are embedded into complex backgroundand normally hollow or translucent, traditional video logo recognitiontechniques are facing issues such as difficulties in extracting image fea-tures, high complexity of matching algorithm and feature dimensiondisaster. A well-behaved TV logo recognition method using heuristicoptimality checked basis pursuit denoising is firstly proposed in thepaper. Original pixels are directly used for recognition in this method,avoiding the difficulties in extracting image features as well as being ableto recognize hollow and translucent logos with relatively better robust-ness. Experiments have shown that by applying this method, averagelogo recognition time is within 30 ms for a single-frame video with recog-nition accuracy more than 98 %.

Keywords: Image recognition · Pursuit algorithm · Video signalprocessing

1 Introduction

With the rapid development of Internet technology and widespread use of largescale memory, multimedia information in primary form of video is growing explo-sively. How to classify and administrate the huge number of videos is a challeng-ing question. Identification marks can appear anywhere in a video and containimportant semantic information, such as the TV station logos, vehicle licenseplates and traffic signs. Recognizing identification marks in a video image rapidlyand accurately is very important for content based video retrieval and harmfulinformation filtering. For example, information such as TV station name, pro-gram orientation can be retrieved by video logo recognition; compared withother general identification marks, position and size of the video logo are rela-tively fixed and easy to recognize. However, currently in the academic circles,video logo recognition remains a challenging research topic. The performance ofvideo logo recognition is affected by many factors, such as complex background,translucent and hollow logo, etc.

c© Springer International Publishing Switzerland 2015Y.-J. Zhang (Ed.): ICIG 2015, Part I, LNCS 9217, pp. 529–540, 2015.DOI: 10.1007/978-3-319-21978-3 46

530 J. Xu et al.

Traditional approaches of video logo recognition are categorized as thefollowing: (1) Color histogram-based method [1]. The method recognizes logo byits tone, extracts color histogram from the logo, and matches with histograms oflogos in the library. Color information does not change with size or viewing angleand is easy to separate. However, affected by the background, colors of hollowor translucent logos are not stable and may change quite often. (2) Shape-basedmethod [2]. The shape-based method segments the logo from a video completely,and then recognizes the logo by shape description methods such as shape invari-ant and Fourier descriptor. Unfortunately, existing image segmentation tech-niques cannot accurately segment hollow or translucent logo from the video.(3) Machine learning-based method [3]. The machine learning-based method suchas support vector machines and artificial neural networks is restricted to the per-formance of the feature extraction algorithm as well as size of training samples.(4) Local invariant feature based method [4]. Local invariant features mean thatthe recognition and description of local features remain unchanged in response toimage changes such as geometric transformation, illumination changes and view-ing angle changes. It is widely used in object recognition as well as image andvideo retrieval. Existing research methods based on local invariant features arelack of interpretation on the semantic level; moreover, extracted local invariantfeatures can be located on the logo, on the complex background structure or onthe boundary between the logo and background. Therefore, TV logo recognitionwith invariant features is still facing tremendous challenges.

A basis pursuit denoising (BPDN)-based TV logo recognition algorithm isproposed in this paper. Basis-pursuit-method is a signal reconstruction algorithmbelonging to mathematical optimization category [5]. The method acquires thesparsest representation of the signal from an over-complete function set, repre-sents the original signal with minimal basis as accurately as possible, so as toobtain the intrinsic characteristics of the signal. Exact solutions to basis pursuitdenoising are often the best computationally approximation of an undeterminedsystem of equations. Basis pursuit denoising has potential applications in chan-nel coding [6], compressive sensing [7,8], computer vision [9], signal recovery[10], sparse coding [11], image denoising [12], etc. In this paper, BPDN is firstlyused to achieve the purpose of TV logo recognition. A low-dimensional linearsubspace is spanned by the training samples from different subjects. Trainingsamples which belong to the same subject are adjacent in the subspace andhighly correlated. Given a new query logo, BPDN is used to seek the sparsestrepresentation of it with respect all the training samples. The dominant nonzerocoefficients in the sparse representation reveal the true subject class the querylogo belongs to. We also firstly put forward a heuristic optimality checked in-crowd algorithm to accelerate the convergence process of basis pursuit denoising,because speed is more important for practical application.

The rest of the paper is organized as follows. In Sect. 2, the heuristic optimal-ity checked in-crowd algorithm is described. Section 3 details the proposed TVlogo recognition method. Section 4 presents experimental results of the proposedmethod. Finally, Sect. 5 provides conclusions and references.

TV Logo Recognition 531

2 Heuristic Optimality Checked In-Crowd Algorithm

2.1 Basis Pursuit Denoising

Basis pursuit (BP) is an optimization problem of the form [5]:

(BP )minx

‖ x ‖1 s.t.Ax = y, (1)

where A is a m × n measurement matrix (m � n), x is a n × 1 solution vectorof the original signal and y is a m × 1 vector of measurements.

BP is relaxed to obtain the constrained basis pursuit denoising (BPDN)problem:

(BPDN)minx

‖ x ‖1 s.t. ‖ Ax − y ‖22≤ σ (2)

and its variant form of unconstrained basis pursuit denoising (QPλ) problem:

(QPλ)minx

12

‖ Ax − y ‖22 +λ ‖ x ‖1 . (3)

The parameter σ is used to estimate the noise level of the original signal andλ controls the trade-off between sparsity and reconstruction fidelity. It becomesa standard BP problem when σ equals to zero or λ → 0. BPDN solves a regu-larization problem with a trade-off between having a small residual and makingx sparse in the L1 sense. It can be thought of finding the sparsest explanationcapable of accounting for the observation.

Basis pursuit denoising is a convex quadratic problem, so it can be solvedby many general solvers, such as interior point methods [13], homotopy [14], thespectral projected gradient for L1 minimization (SPGL1) [15], fixed-point con-tinuation (FPC) [16], and in-crowd algorithm [17], etc. The in-crowd algorithmis a numerical method for solving basis pursuit denoising problem. Although it isfaster than any other algorithm for large-scale sparse problem [17], the need forfurther accelerating the speed of convergence process is still of greatest impor-tance. Therefore, we propose an improved in-crowd algorithm using heuristicoptimality check to solve BPDN quickly and apply this method to TV logorecognition problem in this paper.

2.2 Solving BPDN by Heuristic Optimality CheckedIn-Crowd Algorithm

In-crowd algorithm is introduced in [17] to find the exact solution to BPDNby discovering a sequence of subspaces guaranteed to arrive the correct globalsolution of �1-regularized least squares problems. Since every time the in-crowdalgorithm performs a global search and adds up L components to the activeset, it can be a factor of L faster than the best alternative algorithms whenthis search is computationally expensive. A theorem [17] guarantees that theglobal optimum is reached in spite of the many-at-a-time nature of the in-crowd

532 J. Xu et al.

Algorithm 1. Heuristic optimality checked in-crowd algorithm

Input: A full rank matrix A = [ν1, · · · , νn] ∈ Rm×n, m � n, a vector y ∈ R

n.

1. Declare x to be 0, so the unexplained residual r = y.2. Declare the active set I to be the empty set.3. Calculate the usefulness uj = | < r,Aj > | for each component in Ic.4. S = supp(x), compute approximate solutions ω to AT

Sω = sign(xS).5. If (no |uj > λ| on Ic) or

(‖ AT ω ‖∞≈ 1 and AS xS ≈ b and (‖ x ‖1 −bT w)/ ‖ x ‖1≈ 0),then terminate, x is an optimal solution.

6. Otherwise, add L ≈ 25 components to I based on their usefullness.7. Solve basis pursuit denoising exactly on I, and throw out any component of I whose

value attains exactly 0.8. Update r = y − Ax .9. Go to step 3.

Output: x.

algorithm. When it comes to solving BPDN, an important practical concernstill is the rate of convergence. Especially when it is used to solve the TV logorecognition problem, speed is crucial.

The distribution of sparse coefficients are completely enough for TV logorecognition problem, hence no exact solutions are required. We use the heuristicoptimality check (HOC) proposed in [18] which allows for “jumping” to an approx-imate optimal point of BPDN, saving time. The Algorithm 1 is the procedure ofthe heuristic optimality checked in-crowd algorithm. In Step 5, if the condition ismet, the proposed algorithm will early terminate the iterative process.

When HOC is included into the in-crowd algorithm, the outer loop executionfrequency is probably reduced by 30 %. Considering the load of HOC, the averageperformance of the in-crowd algorithm could be improved about 25 % empiricallyby our method.

3 TV Logo Recognition Using the Accelerated BPDN

The basic task of logo recognition is to determine which class a new query logobelongs to. If we use BPDN to solve this problem, the task can be expressedas finding the sparse representation of the query logo on the known-type logocollections. We cut out logo areas from lots of videos and put together the onesfrom the same TV program. These logo areas are used to make the training set.Assume the total quantity of training samples is n, the quantity of training sam-ples in the ith class is ni(i = 1, 2, ..., k), the area of each logo is w×h, where w isthe width and h is the height. vi,j(j = 1, 2, ..., ni) is the column vector represen-tation of the jth training sample belonging to the ith class. The training samplesof the ith class are highly correlated with each other and can be representedby a matrix Ai

.= [vi,1,vi,2, ...,vi,ni] ∈ R

m×ni , where m = w × h. All trainingsamples can be represented as A .= [A1,A2, ...,Ak] = [v1,1,v1,2, ...,vk,nk

]. If the

TV Logo Recognition 533

Step. 1 Preparation Process

Query Logos

Training Samples

Linear Subs ace

Test Logos

Coefficients Distribution

Step. 2 Recognition Process Step. 3 Recognition Results

Fig. 1. The framework of TV logo recognition.

test logo samples y belongs to the ith class, then it can be considered as thelinear combination of samples from the ith class:

y = αi,1vi,1 + αi,2vi,2 + ... + αi,nvi,ni. (4)

Equation (4) can be represented in a matrix form:

y = Ax0 ∈ Rm, (5)

where x = [0, ...0, αi,1, αi,2, ..., αi,ni, 0, ..., 0]T ∈ R

n is the sparse representationof the test logo sample y on training samples A. Except for elements associatedwith the ith class, all other elements are approximately 0 (Fig. 2). Above processshows that basis pursuit-based logo recognition can be considered as solving thesolution vector x from (5), and then determining the type of the query logobased on the distribution of non-zero coefficients or dominant coefficients in x.

534 J. Xu et al.

The logo has a relatively small area (m � n), so (5) is an underdeterminedequation has no unique solution. According to the theory of compressive sensing[8], if A satisfies the restricted isometry property (RIP), (5) can be solved bybasis pursuit. In this paper, A is constructed with random collected logo samples,so it satisfies RIP with high probability. Therefore, we can convert (5) into an�1 norm minimization problem and solve it with basis pursuit.

Equation (5) is the recognition method under the ideal case. In practicalapplication, the noise caused by pixels surrounding the logo and behind thetranslucent logo area is the main reason of recognition inaccuracy. In this paper,σ is used to denote the noise level caused by the background image and thesparse coefficients of the test logo on the training set is solved by the proposedheuristic optimality checked basis pursuit denoising in Sec. 2.

The TV logo recognition proposed in this paper is executed in the followingsteps (see Fig. 1):

Step 1. Capture TV programs, cut out logo areas, normalize the logo areas toa fixed size of w × h, put together logo areas from the same TV program, andcreate a logo training set A with k classes.Step 2. Normalize a new query logo area y to the same size, then acquirethe sparse representation x of it on the training set by the proposed heuristicoptimality checked basis pursuit denoising algorithm.Step 3. Analyze the distribution of x on the space spanned by the trainingsamples. In this paper, each class has the same number of training samples.ni denotes the quantity of dominant coefficients in the ith class. If nk is themaximum, then the query logo belongs to the kth class.

The type of the query logo can be recognized using the original pixels directly,which need not pre-processing video images, extracting the region features of thelogo or differentiating between solid, hollow and translucent logos. Therefore, ourapproach is superior to traditional methods.

In the TV logo recognition tasks, there is a tradeoff between how fast a taskcan be performed and how many mistakes are made in performing the task.That is, a recognition method can either perform the task very slow with veryfew errors or very fast with a large number of errors. In the following section, wewill first optimize accuracy, optimize speed and combine the two by adjustingthe noise level δ, and then compare the performance of the proposed methodwith other typical logo recognition methods in both speed and accuracy.

4 Experimental Results

In this section, we describe the experimental results of basis pursuit denoisingfor TV logo recognition. The experimental environment is Intel(R) Core(TM)i7-3520M CPU with 4G RAM, Gehua HD set-top box, Tianmin video capturecard UT840, running windows 7 with Matlab R2011b.

To validate the method proposed in this paper, videos each with 1000 frameswere acquired in real time from the Dragon TV, Heilongjiang TV, Hunan TV

TV Logo Recognition 535

0.02

0.04

0.06

0.08

0.1

0.12

(a) Dragon

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

(b) Heilongjiang

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

(c) Hunan

0.02

0.04

0.06

0.08

0.1

(d) CCTV Sports

Fig. 2. Sparse coefficient.

(a) Dragon (b) Heilongjiang (c) Hunan (d) CCTV Sports

Fig. 3. TV logos.

and CCTV Sports Channel to serve as a training data set and a test data set.Among the TV stations, Dragon TV has a typical solid logo; Heilongjiang TVand Hunan TV have hollow logos; the logo of CCTV Sports is both hollow andtranslucent (Fig. 3). Therefore, the experimental data set was representative,covering the vast majority of TV logo characteristics.

Two hundred and fifty frames of the images were randomly selected to set upthe logo library while the remaining 750 frames were reserved for testing. Size ofeach TV logo region was normalized to 20 × 20 using cubic spline interpolation.Therefore, the size of the training set was 400 × 1000, where column 1–250 was

536 J. Xu et al.

0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

50

100

150

200

250

300

time(ms)

σ0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

0.5

0.625

0.75

0.875

1

Precision

(a) Dragon

0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

100

200

300

400

500

600

time(ms)

σ0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

0.8

0.825

0.85

0.875

0.9

0.925

0.95

0.975

1

Precision

(b) Heilongjiang

0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

20

40

60

80

100

120

140

time(ms)

σ0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

0.95

0.955

0.96

0.965

0.97

0.975

0.98

0.985

0.99

0.995

1Precision

(c) Hunan

0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

100

200

300

400

500

600

700

800

900

time(ms)

σ0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

0.5

0.625

0.75

0.875

1

Precision

(d) CCTV Sports

Fig. 4. Average recognition accuracy (in percentage) and average run time (in milli-second).

TV Logo Recognition 537

from Dragon TV, column 251–500 from Heilongjiang TV, column 501–750 fromthe Hunan TV and column 751–1000 from CCTV Sports.

4.1 Speed-Accuracy Tradeoff Under Different Noise Level

The recognition speed and accuracy are affected by the background image whichis around and behind the query logo. Figure 4 shows the performance of theproposed method under different noise level σ. We plotted both recognitionspeed and accuracy for the test TV logos under different noise level σ in Fig. 4.As can be seen, when σ increases, the speed also increases exponentially, butthe accuracy firstly increases and then decreases. Judging from the reams ofevidence presented, we can safely conclude that the speed-accuracy tradeoff isaffected by the noise level. The question here is under which circumstances wouldthe proposed method be both fast and accurate? By observing the experimentaldata, we found that when σ is approximately equal to 1.5 the proposed methodhas the best performance on Dragon, Heilongjiang and Hunan TV logo; whenσ is approximately equal to 2, it is optimal to recognize CCTV sports logo interms of both speed and accuracy. So if there are more background pixels inthe TV logo area, especially for the hollow and translucent ones such as CCTVsports, we need to set a higher noise level to improve the speed and accuracy atthe same time. By choosing appropriate noise level, the average logo recognitiontime is within 30 ms for a single-frame video with recognition accuracy morethan 98 %.

4.2 Performance Comparison

We compared our method to the histogram based method (HIST), the Cannyedge-based method [19], the spatial histogram of gradient (HOG) based method[20] and the scale invariant feature transform (SIFT) based method [21].

RGB histogram was extracted from the logo area as recognition features forHIST-based method. Canny edge features were extracted directly for the Canny-based method. Spatial histogram of gradient was extracted from TV logos whichwere normalized to 64 × 64 for the HOG-based method. In order to effectivelyextract the feature points, TV logos were normalized to 180× 180 for the SIFT-based method. For the above four methods, we used support vector machine(SVM) as classifier. For our method, we chose σ = 1.5 for Dragon, Heilongjiangand Hunan logos, and σ = 2 for CCTV Sports logos.

The average recognition accuracy and run time of the five method are shownin Fig. 5. Clearly, the proposed method achieves the best overall performancein recognition accuracy. Its recognition rate is almost 100 %. Its run time iswithin 30 ms, although it carries a penalty that the speed is slower than theHIST-based method, the Canny-based method and the HOG-based method. TheHIST-based method is the fastest, but its performance in recognition accuracy isthe worst, especially for the Heilongjiang TV logo which is hollow and the CCTVsports logo which is translucent. The Canny-based method and the HOG-basedmethod have similar run time, but the latter has higher recognition accuracy.

538 J. Xu et al.

Dragon Heilongjiang Hunan CCTV Sports0

10

20

30

40

50

60

70

80

90

100

Ave

rage

Rec

ogni

tion

Acc

urac

y(%

)

Our Method HIST Canny HOG SIFT

(a) Average Recognition Accuracy (In Percentage)

Dragon Heilongjiang Hunan CCTV Sports0

9

18

27

Ave

rage

Run

ning

tim

e(m

s)

Our Method HIST Canny HOG SIFT

(b) Average Run Time (In Millisecond)

Fig. 5. Performance comparison.

The speed of SIFT-based method is the slowest and its recognition accuracy isunsatisfactory. In the experiment, we found that it is difficult to find a sufficientnumber of sift feature points on the small logo region.

TV Logo Recognition 539

5 Conclusions

A heuristic optimality checked in-crowd algorithm is proposed and used for TVlogo recognition in this paper. Original pixels are directly used for recognitionof TV station logos by the method first set out in the paper, avoiding such dif-ficulties as image pre-processing, logo features extraction and feature dimensiondisaster, which demonstrated a relatively better robustness in identification ofhollow and translucent logos. Massive experiments have shown that the proposedlogo recognition method has high recognition accuracy with low average recogni-tion time, and can be applied in real-time video analyzing and detecting system.Our further research work will be focusing on optimizing the performance oflogo recognition by setting the background noise of the logo σ in a self-adaptivemanner.

References

1. Sizintsev, M., Derpanis, K.G., Hogue, A.: Histogram-based search: a comparativestudy. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8.IEEE Press, Anchorage (2008)

2. Wang, J., Duan, L., Li, Z., Liu, J., Lu, H., Jin, J.S.: A robust method for TV logotracking in video streams. In: IEEE International Conference on Multimedia andExpo, PP. 1041–1044. IEEE Press, Toronto (2006)

3. Maldonado-Bascon, S., Lafuente-Arroyo, S., Gil-Jimenez, P., Gomez-Moreno, H.,Lpez-Ferreras, F.: Road-sign detection and recognition based on support vectormachines. IEEE Trans. Intell. Transp. Syst. 8(2), 264–278 (2007)

4. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEETrans. Pattern Anal. Mach. Intell. 27(20), 1615–1630 (2005)

5. Chen, S., Donoho, D.D., Saunders, M.: Atomic decomposition by basis pursuit.SIAM J. Sci. Comput. 20, 33–61 (1999)

6. Cands, E., Tao, T.: Decoding by linear programming. IEEE Trans. Info. Th.51(12), 4203–4215 (2005)

7. Cands, E., Romberg, J., Tao, T.: Robust uncertainty principles: Exact signal recon-struction from highly incomplete frequency information. IEEE Trans. Info. Th.52(2), 489–509 (2006)

8. Donoho, D.: Compressed sensing. IEEE Trans. Info. Th. 52(4), 1289–1306 (2006)9. Zhang, D., Yang, M., Feng, X.: Sparse representation or collaborative representa-

tion: which helps face recognition?. In: IEEE International Conference on Com-puter Vision, pp. 471–478. IEEE Press (2011)

10. Bruckstein, A., Donoho, D., Elad, M.: From sparse solutions of systems of equationsto sparse modeling of signals and images. SIAM Rev. 51(1), 34–81 (2009)

11. Zheng, M., Bu, J., Chen, C.: Graph regularized sparse coding for image represen-tation. IEEE Trans. Image Process. 20(5), 1327–1336 (2011)

12. Barthel, K.U., Cycon, H.L., Marpe, D.: Image denoising using fractal-and wavelet-based methods. In: Photonics Technologies for Robotics, Automation, and Manu-facturing, pp. 39–47. International Society for Optics and Photonics (2004)

13. Kim, S.J., Koh, K., Lustig, M., Boyd, S., Gorinevsky, D.: An interior-point methodfor large-scale l1-regularized least squares. IEEE J. Sel. Topics Signal Process. 1(4),606–617 (2007)

540 J. Xu et al.

14. Malioutov, D.M., Cetin, M., Willsky, A.S.: Homotopy continuation for sparse signalrepresentation. In: IEEE International Conference on Acoustics, Speech, and SignalProcessing, pp. 733–736. IEEE Press (2005)

15. SPGL1: A solver for large-scale sparse reconstruction. http://www.cs.ubc.ca/labs/scl/spgl1

16. Hale, E.T., Yin, W., Zhang, Y.: Fixed-point continuation for l1-minimization:methodology and convergence. SIAM J. Optim. 19(3), 1107–1130 (2008)

17. Gill, P., Wang, A., Molnar, A.: The in-crowd algorithm for fast basis pursuit denois-ing. IEEE Trans. Signal Process. 59, 4595–4605 (2011)

18. Lorenz, D.A., Pfetsch, M.E., Tillmann, A.M.: Solving Basis Pursuit: Subgradientalgorithm, heuristic optimality check, and solver comparison. Optimization OnlineE-Print ID 2011–07-3100 (2011)

19. Beeran Kutty, S., Saaidin, S., Yunus, M., Ashikin, P.N., Abu Hassan, S.: Evaluationof canny and sobel operator for logo edge detection. In: International Symposiumon Technology Management and Emerging Technologies, pp. 153–156 (2014)

20. Zhang, X., Zhang, D., Liu, F., Zhang, Y., Liu, Y., Li, J.: Spatial HOG basedTV logo detection. In: the Fifth International Conference on Internet MultimediaComputing and Service, pp. 76–81. ACM (2013)

21. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Com-put. Vis. 60(2), 91–110 (2004)