VIDEO SURVEILLANCE USING RASPBERRY PI GPUeprints.utm.my/id/eprint/79581/1/WongYanYinPFKE2018.pdf ·...
-
Upload
duonghuong -
Category
Documents
-
view
219 -
download
0
Transcript of VIDEO SURVEILLANCE USING RASPBERRY PI GPUeprints.utm.my/id/eprint/79581/1/WongYanYinPFKE2018.pdf ·...
Replace this page with the Cooperation Declaration form, which can beobtained from SPS or your faculty. This page is OPTIONAL when yourresearch is done in collaboration with other institutions that requirestheir consent to publish the finding in this document.]
VIDEO SURVEILLANCE USING RASPBERRY PI GPU
WONG YAN YIN
A project report submitted in partial fulfilment of the
requirements for the award of the degree of
Master of Engineering (Computer and Microelectronic System)
Faculty of Electrical Engineering
Universiti Teknologi Malaysia
JUNE 2018
iv
ACKNOWLEDGEMENT
At the first place I would like to express my gratitude to my project supervisor,
Prof. Madya Muhammad Mun’im bin Ahmad Zabidi. Thank you on his patience,
motivation, guidance and the continuous support given throughout the completion of
this project. He had provide countless helpful advice and gave handy guidance which
resulting to a better project.
Besides my project supervisor, I would like to say thank you to my family.
They have been always there to support me and make me feel better when I was under
stress. I could not have done this without their support.
v
ABSTRACT
Today, surveillance system is being utilized and deployed in many places to
provide supervision and bring security to people. The most commonly used technology
currently is Closed-Circuit Television (CCTV). However, there are several defects with
the technology such as anomalies cannot be identified automatically and expensive.
This project proposed to use the GPU in Raspberry Pi for video surveillance task.
Raspberry Pi is a powerful single-board computer which features an ARM processor
and a VideoCore IV graphics processing unit (GPU). It is sufficiently powerful to work
as a video surveillance system and relatively cheap compared to CCTV. Furthermore,
GPU is optimized for parallel computing of video data. It can theoretically provides
better performance and have higher efficiency in video processing compared to CPU.
Hence, the GPU in Raspberry Pi should provides large performance gain by porting the
algorithm from CPU-only reference to works on GPU. The objective of this project is
to explore on how the GPU can be programmed for the purpose of video surveillance.
vi
ABSTRAK
Dalam zaman ini, sistem pengawasan telah dipasang dan digunakan di
banyak tempat untuk mengawas dan menjamin keselamatan orang ramai. Teknologi
yang paling biasa digunakan ialah Televisyen Circuit Tertutup (CCTV). Walau
bagaimanapun, terdapat beberapa kelemahan dengan teknologi tersebut seperti
anomali tidak dapat dikenal pasti secara automatik dan mahal. Projek ini
mencadangkan penggunaan GPU dalam Raspberry Pi untuk tugas pengawasan video.
Raspberry Pi sebenarnya merupakan sebuah komputer yang kuat, ia mempunyai
pemproses ARM dan unit pemprosesan grafik VideoCore IV. Raspberry Pi mempunyai
performasi yang cukup untuk berfungsi sebagai sistem pengawasan video dan agak
murah berbanding dengan CCTV. Selain itu, GPU adalah sistem yang dioptimumkan
untuk pengkomputeran data video selari. Ia secara teorinya boleh memberikan prestasi
yang lebih baik dan mempunyai kecekapan yang tinggi dalam pemprosesan video
berbanding dengan penggunaan CPU. Oleh itu, GPU dalam Raspberry Pi sepatutnya
memberikan keuntungan prestasi yang besar dengan mengalihkan algoritma dari
rujukan CPU sahaja kepada kerja-kerja GPU. Objektif projek ini adalah untuk
meneroka bagaimana GPU boleh diprogramkan untuk tujuan pengawasan video.
vii
TABLE OF CONTENTS
CHAPTER TITLE PAGE
DECLARATION ii
DEDICATION iii
ACKNOWLEDGEMENT iv
ABSTRACT v
ABSTRAK vi
TABLE OF CONTENTS vii
LIST OF TABLES x
LIST OF FIGURES xi
LIST OF ABBREVIATIONS xiii
1 INTRODUCTION 1
1.1 Problem Background 1
1.2 Problem Statement 2
1.3 Objective 2
1.4 Scope of Work 3
1.5 Thesis Outline 3
2 LITERATURE REVIEW 4
2.1 Embedded Surveillance System Using Background
Subtraction and Raspberry Pi 4
2.2 Real-time Face Detection and Tracking Using Haar
Classifier on SoC 6
2.3 Low Cost Real-Time System Monitoring Using
Raspberry Pi 7
2.4 Embedded Video Processing on Raspberry Pi 8
viii
2.5 Accelerated FFT Computation for GNU Radio
Using GPU of Raspberry Pi 9
2.6 Accelerating Real-time Face Detection on a
Raspberry Pi Telepresence Robot 11
2.7 A GPU- Accelerated Framework for Image Pro-
cessing and Computer Vision 13
2.8 Fast Image Processing with i.MX 6 Series 14
2.9 Chapter Summary 14
3 RESEARCH METHODOLOGY 15
3.1 Overall Project Flow 15
3.2 Tools and Platforms 16
3.3 Program Flow 16
3.4 The chosen algorithms 20
3.4.1 Frame Difference 20
3.4.2 Image Convolution 22
3.4.3 Local Binary Pattern 24
3.5 Implementation of chosen algorithms to run on RP
CPU 24
3.5.1 CPU Single Threaded Design 26
3.5.2 CPU Four Threaded Design 31
3.5.3 OpenCV Design 31
3.6 Implementation of chosen algorithms to run on RP
GPU 33
3.6.1 QPULib Design 35
3.7 Benchmark result between RP CPU and GPU 39
3.8 Chapter Summary 39
4 RESULTS AND DISCUSSION 40
5 CONCLUSION 46
5.1 Future Works 46
x
LIST OF TABLES
TABLE NO. TITLE PAGE
2.1 Achieved accuracies of [1] and comparison with [2] 5
2.2 Performance comparison between Haar and LBP 9
2.3 Execution time of FFT using CPU in Raspberry Pi in seconds 10
2.4 Execution time of FFT using Intel Core i5 in seconds 10
2.5 Execution time of FFT using GPU in Raspberry Pi in seconds 11
4.1 Frame Difference table result 42
4.2 Image Convolution (Sharpen) table result 43
4.3 Image Convolution (Edge Detection) table result 44
xi
LIST OF FIGURES
FIGURE NO. TITLE PAGE
2.1 Overall Program Flow for Background Subtraction 5
2.2 Flowchart of face detection system 6
2.3 Algorithm of Frame Differencing 7
2.4 Flowchart of motion detection system 8
2.5 Full Program Flow 12
3.1 Overview of Project Flow 17
3.2 Overall project timeline 18
3.3 Development system setup 18
3.4 Specifications of Raspberry Pi 19
3.5 Program Flow 21
3.6 Image Convolution formula 22
3.7 Illustration of Image Convolution 23
3.8 Edge Detection and Sharpen kernel 23
3.9 Illustration of LBP codes extraction 25
3.10 Examples of images after LBP extraction 25
3.11 Single Threaded Frame Difference Coding 27
3.12 Single Threaded Edge Detection Coding 27
3.13 Single Threaded Sharpen Coding 28
3.14 Single Threaded LBP Coding 28
3.15 Four Threaded Frame Difference Coding 29
3.16 Four Threaded Edge Detection Coding 29
3.17 Four Threaded Sharpen Coding 30
3.18 Four Threaded LBP Coding 30
3.19 OpenCV Frame Difference Coding 31
3.20 OpenCV Edge Detection Coding 31
xii
3.21 OpenCV Sharpen Coding 31
3.22 OpenCV LBP Coding 32
3.23 QPU Core Pipeline 34
3.24 QPULib Frame Difference Coding 36
3.25 QPULib Edge Detection Coding 37
3.26 QPULib Sharpen Coding 37
3.27 QPULib LBP Coding 38
4.1 Frame Difference bar chart result 42
4.2 Image Convolution (Sharpen) bar chart result 43
4.3 Image Convolution (Edge Detection) bar chart result 44
A.1 Frame difference algorithm in action 50
A.2 Edge Detection algorithm in action 51
A.3 Sharpen algorithm in action 51
A.4 LBP algorithm in action 52
xiii
LIST OF ABBREVIATIONS
CCTV - Closed Circuit Television
CMOS - Complementary Metal Oxide Semiconductor
CPU - Central Processing Unit
DSP - Digital Signal Processing
FFT - Fast Fourier Transform
FPS - Frame Per Second
GFLOPS - Giga Floating Point Operations Per Second
GPU - Graphics Processing Unit
LBP - Local Binary Pattern
PC - Personal Computer
PGC - Programmable Graphic Core
PTU - Pan and Tilt Unit
QPU - Quad Processing Unit
ROI - Region Of Interest
SDRAM - Synchronous Dynamic Random Access Memory
SIMD - Single Instruction Multiple Data
SOC - System On Chip
XML - Extensible Markup Language
CHAPTER 1
INTRODUCTION
1.1 Problem Background
Surveillance system is a system which monitor and observe. The main purpose
of surveillance system is to provide supervision and security to people. In this modern
era, surveillance system is very common and it can be found easily in many places
such as shopping mall, bank and even in certain housing areas [3].
The most commonly used technology for surveillance system is CCTV (Closed
Circuit Television) and the most common type of CCTV is analogue CCTV. Analogue
CCTV is a well-stable, established technology which is supported by all major CCTV
manufacturers. Despite the popularity, analogue CCTV has a few downsides. First
of all, it is dumb. It does not mean literally dumb, in this context it is saying the
analogue CCTV is lacking some intelligence [4]. Analogue CCTV can only perform
surveillance task solely. It cannot do anything other than passive monitoring and
recording the video. Secondly, analogue CCTV requires installation and maintenance.
Analogue CCTV has to be installed and it involves a lot of wiring work, not to mention
it still has to be maintained from time to time.
Because of all the mentioned disadvantages of CCTV, a new trend of low
cost embedded video surveillance system using Raspberry Pi emerged [3, 5, 6, 7].
Raspberry Pi is being used for several reasons. First, it has a small form factor. The
size of Raspberry Pi is only pocket size large. This makes it suitable for surveillance
purpose because it can be hide effortlessly. Second, it has a relatively powerful
hardware. As matter of fact, Raspberry Pi is essentially a mini computer and it can
2
turned into a smart video surveillance system through programming to perform motion
or face detection. Third, Raspberry Pi has a cheap pricing. It is affordable for most
people.
1.2 Problem Statement
There are a few problem statement in this project. First problem statement is
there is very limited exploration on GPU based video surveillance system [8] especially
on Raspberry Pi. As of today, not much research has been done on the Raspberry Pi
GPU needless to mention those research specialized in video surveillance system using
Raspberry Pi GPU.
The second problem statement is high user demand for high performance and
intelligent video surveillance system. In computing world, the expectation level of
performance for devices is ever increasing, user is constantly expecting devices to run
faster and perform better. This applies to Raspberry Pi as well, even though Raspberry
Pi already has a powerful enough hardware but as time passed by the computation
power of Raspberry Pi may not meet user expectation anymore.
The third problem statement is no previous work has been done to compare
the video surveillance system performance on Raspberry Pi GPU and CPU. Currently,
there is no research to study the performance difference in Raspberry Pi GPU and
CPU based video surveillance system. This is saddening because it should be very
interesting to study on the performance gain of switching from CPU implementation
to GPU implementation. From the learning, it can determine whether the switch is
worthwhile.
1.3 Objective
The objectives to be achieved in this project are:
3
1. To implement video surveillance algorithms on Raspberry Pi.
2. To improve the performance of Raspberry Pi video surveillance system in terms
of FPS or latency.
3. To benchmark the results between Raspberry Pi CPU and GPU platform.
1.4 Scope of Work
There are many different types of Raspberry Pi in the market. In this project,
a non-overclock Raspberry Pi model 2B is used. Raspberry Pi 2B has a ARMv7 four
cores processors which is running at 900Mhz and a Broadcom VideoCore IV GPU
which consists of QPU, VPU and ISP.
In order for Raspberry Pi to have computer vision capability, it needs to be
programmed with video surveillance algorithms. The video surveillance algorithms
are limited to Local Binary Patterns (LBP) [9], Frame Differencing and Image
Convolution.
A normal USB Logitech webcam instead of the official Pi camera module is
used for the video capturing purpose.
1.5 Thesis Outline
This thesis consists of five chapters, they are introduction, literature review,
research methodology, results and discussion, and conclusion. Chapter 1 gives the
motivation and background of the project, raises the problem statement, finding the
objective and set the scope for the project. Chapter 2 contains the study from literature
reviews from all related field. Chapter 3 describes the design methodology of the
project in details. The results and discussion can be found in chapter 4. Lastly, chapter
5 summarize the overall findings and the achievement of this project.
REFERENCES
1. Cocorullo, G., Corsonello, P., Frustaci, F., Guachi, L. and Perri, S. Embedded
surveillance system using background subtraction and Raspberry Pi. AEIT
International Annual Conference (AEIT). IEEE, 2015: 1–5. doi:10.1109/
AEIT.2015.7415219.
2. Guachi, L., Cocorullo, G., Corsonello, P., Frustaci, F. and Perri, S. A novel
background subtraction method based on color invariants and grayscale levels.
2014.
3. Jyothi, S. N. and Vardhan, K. V. Design and implementation of real time
security surveillance system using IoT. Proceedings of the International
Conference on Communication and Electronics Systems, ICCES 2016, 2016.
doi:10.1109/CESYS.2016.7890003.
4. S, K. H., R, G. S., B, K. S. and Sangmesh, P. Smart Video Surveillance. 2015.
3(1): 109–112.
5. Menezes, V. Surveillance and Monitoring System Using Raspberry Pi
and SimpleCV. 2015 International Conference on Green Computing and
Internet of Things (ICGCIoT), 2015: 1276–1278. doi:10.1109/ICGCIoT.2015.
7380661.
6. Shifani, S. A. Security System Using Raspberry Pi. 2017.
7. Jacob, T. P. Motion Activated Security Camera using Raspberry Pi. 2017:
1598–1601.
8. Mijat, R., Computing, V. and Manager, M. Understanding the Role of
Integrated GPUs in Vision Applications. 2015. (May).
9. PietikAinen, M. Local Binary Patterns. Scholarpedia, 2010. 5(3): 9775.
ISSN 1941-6016. doi:10.4249/scholarpedia.9775. URL http://www.
scholarpedia.org/article/Local_Binary_Patterns.
48
10. Tripathy, R. and Daschoudhury, R. N. Real-time face detection and tracking
using Haar classifier on SoC. International Journal of Electronics and
Computer Science Engineering, 2014. 3(2): 177.
11. Viola, P. and Jones, M. Robust Real-time Face Detection. Eighth IEEE
International Conference on Computer Vision, 2001. 20: 7695. ISSN 0920-
5691. doi:10.1109/ICCV.2001.937709.
12. Huu-Quoc Nguyen, Ton Thi Kim Loan, Bui Dinh Mao and Eui-Nam Huh.
Low cost real-time system monitoring using Raspberry Pi. 2015 Seventh
International Conference on Ubiquitous and Future Networks, 2015: 857–
859. doi:10.1109/ICUFN.2015.7182665. URL http://ieeexplore.
ieee.org/document/7182665/.
13. Sabarinath, S., Shyam, R., Aneesh, C., Gandhiraj, R. and Soman, K. P.
Accelerated FFT Computation for GNU Radio Using GPU of Raspberry
Pi. 2015. 32: 657–664. doi:10.1007/978-81-322-2208-8. URL http:
//link.springer.com/10.1007/978-81-322-2208-8.
14. Janard, K. and Marurngsith, W. Accelerating real-time face detection on a
raspberry pi telepresence robot. 5th International Conference on Innovative
Computing Technology, INTECH 2015, 2015. (INTECH): 136–141. doi:10.
1109/INTECH.2015.7173482.
15. Allusse, Y., Horain, P., Agarwal, A. and Saipriyadarshan, C. GpuCV:
A GPU-Accelerated Framework for Image Processing and Computer
Vision. Advances in Visual Computing SE - Lecture Notes in Computer
Science, 2008. 5359: 430–439. doi:doi:10.1007/978-3-540-89646-3 42.
URL citeulike-article-id:7187795%5Cnhttp://dx.doi.
org/10.1007/978-3-540-89646-3_42.
16. Semiconductor, F. Fast Image Processing with i . MX 6 Series. 2013: 1–41.
17. Culjak, I., Abram, D., Pribanic, T., Dzapo, H. and Cifrek, M. A
brief introduction to OpenCV. MIPRO, 2012 Proceedings of the 35th
International Convention, 2012: 1725–1730. doi:978-1-4673-2577-6.
URL http://ieeexplore.ieee.org/xpls/abs_all.jsp?
arnumber=6240859.
49
18. BroadCom. VideoCore ® IV 3D Architecture Reference Guide. Architecture
Guide, 2013. URL https://www.broadcom.com/docs/support/
videocore/VideoCoreIV-AG100-R.pdf.
19. Mn416. QPULib. URL https://github.com/mn416/QPULib.