AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical...
Transcript of AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical...
![Page 1: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/1.jpg)
AN OPTIMISED FRAME WORK FOR MOVING TARGET DETECTION FOR
UAV APPLICATION
Md. Shahid, Pooja HR#
Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO),
Bangalore - 560075
# Siddaganga Institute of Technology(SIT), Tumkur
![Page 2: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/2.jpg)
Agenda
ADE: UAV Scenario
Introduction
Problem Statement
Acquisition and Tracking
Proposed Frame work
Evaluation Criteria
Results Analysis
Conclusion
Scope of future work
Discussion
![Page 3: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/3.jpg)
ADE: UAV Scenario
LAKSHYA – Aerial target NISHANT – TUAV RUSTOM-1 UAV
![Page 4: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/4.jpg)
Introduction
Electro Optics (EO) Payload
DTV Camera
FLIR Camera
Electronics package
Unmanned Aerial Vehicles (UAVs) are increasingly being used for reconnaissance and surveillance.
GROUND CONTROL STATION (GCS)
![Page 5: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/5.jpg)
Problem Statement
Acquisition: – Moving target detection under platform disturbances and delays.
Tracking: – Tracking target/vehicle independent of speed/maneuvering.
– Not limited to number of targets.
Target Acquisition
Target Tracking
Manual Automatic
![Page 6: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/6.jpg)
Why moving target detection?
Acquiring moving targets from airborne platform is difficult task due to associated delays.
– Video Downlink: 200 mSec
– Commands uplink: 400 mSec
– Object displacement: ~ 100 pixels (in acquisition)
Ground Control Station(GCS) Antenna Vehicle
Video downlink (200 mSec)
Command uplink
(400 mSec)
![Page 7: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/7.jpg)
Acquisition Problem
![Page 8: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/8.jpg)
Tracking
![Page 9: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/9.jpg)
Proposed Frame work for Acquisition
Input
Video
Moving targets
detected video
* MATLAB 2013a and its tool boxes
Interest point Detection
(Eligibility Criteria)
Registration (Scene lock)
Background Subtraction
Clutter reduction
![Page 10: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/10.jpg)
Interest Point Detection
Definition: – Local image structure around the interest point is rich in terms of
local information contents.
Examples : Corner, blob, ridge, edge etc.
Corner Detection Techniques:
– Harris Detector.
– Moravec Operator.
– Features From Accelerated Segment Test (FAST).
– Median Method.
Interest point Detection
(Eligibility Criteria)
Registration (Scene lock)
Background Subtraction
Clutter reduction
![Page 11: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/11.jpg)
Corner detection results
(a) (b) (a) (b)
(c) (d)
Figure : Corners detected by (a) Harris Detector (b) Moravec Operator (c) FAST algorithm (d) Median algorithm
![Page 12: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/12.jpg)
Complexity Analysis
Complexity in terms of computations and memory (per pixel), is as follows
ALGORITHM
Multiplication operation
Arithmetic operation
Division/ Comparision
operation
Memory requirement
Robustness
Harris Detector
49 43 - 18
Excellent
Moravec Operator
72 136 - 1
Moderate
Fast Algorithm
1 18 - 2
Good
Median Algorithm
- 6 1/26 1 Moderate
![Page 13: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/13.jpg)
Eligibility Criteria
A3(2x2)
A2(2x2)
A1(2x2)
A0(2x2)
Restricts the eligible candidates(pixels) to be under process for further corner detection.
Considering the pixel as a centre for its 5x5 size block A and its four sub-blocks.
Sub-blocks are of 2x2 size each
Threshold : D2 > Mean(A)
Reduces computational burden significantly.
3210 2 AAAAH
0213 2 AAAAV
222 VHD
![Page 14: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/14.jpg)
Robustness of eligibility criteria
Eligibility criteria is robust under
– Translation
– Rotation
– Scaling
– Noisy environment
Computational saving
Algorithm/ Technique
Processing time (all pixels)
Processing time (eligible pixels)
Computational Saving
Harris Detector
~1.5458 sec ~0. 5825 sec 62%
Moravec Operator ~0.5449 sec
~0.2075 sec 61%
FAST Algorithm
~0.1059 sec
~0.0399 sec
62%
![Page 15: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/15.jpg)
Results of eligibility criteria
(a) (b) (c) (d)
(e) (f) (g) (h)
Figure : (a) Original image of size 256x256. (b) Corners detected by Harris Detector. (c) Rotated image (-100).
(d) Corners detected for rotated image. (e) Image affected by Gaussian noise. (f) Corners detected for Image affected by Gaussian noise. (g) Original image resized to 128x128. (h) Corners detected for resized image.
![Page 16: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/16.jpg)
Results of eligibility criteria(UAV image)
(a) (b) (c) (d)
(e) (f) (g) (h) Figure : (a) Original image of size 256x256. (b) Corners detected by Harris Detector. (c) Rotated image (-100). (d)
Corners detected for rotated image. (e) Image affected by Gaussian noise. (f) Corners detected for Image affected by
Gaussian noise. (g) Original image resized to 130x130. (h) Corners detected for resized image.
![Page 17: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/17.jpg)
Registration
Why Registration?
– Arresting the background against platform movement
– Required for moving or dynamic platform
Assumptions
– Background forms most part of the scene
– Background interest points moves slower
than foreground
Multiple target tracking
– Circularization and correlation matching
– Restricting to least movement, r2 = x2 + y2
– Proportional weightage X = (4*x1 + 3*x2)/7,
Y = (4*y1 + 3*y2)/7
– Discarding poor target & Best target weigh more
Unregistered
video clip
Registered
video clip
Interest point Detection
(Eligibility Criteria)
Registration (Scene lock)
Background Subtraction
Clutter reduction
![Page 18: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/18.jpg)
Background Subtraction
Background subtraction
– Key aspect of the frame work
Type of backgrounds
– Dynamic backgrounds
– Gradual illumination changes
– Sudden illumination changes
– Moved object
– Shadows
Various methods
– Pixel or region based methods
– Parametric or nonparametric methods
– Recursive or non-recursive methods
Interest point Detection
(Eligibility Criteria)
Registration (Scene lock)
Background Subtraction
Clutter reduction
![Page 19: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/19.jpg)
Background Subtraction Methods
Bayesian histogram
Morphological filtering
Sigma-delta(∑-∆) motion detection
Visual Background Extractor (ViBe)
![Page 20: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/20.jpg)
Static platform
(a) (b) (c)
(d) (e) (f)
Figure : a) Input video frame of static camera. b) Ground truth. c) Bayesian histogram. d) Morphological Filtering. e) ∑-∆ motion detection. f) ViBe method. Courtesy: video sequence Highway II (available at http://cvrr.ucsd.edu/aton/shadow/data/highwayII-raw.avi)
![Page 21: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/21.jpg)
Moving platform
(a) (b) (c)
(d) (e) (f)
Figure : (a) Input video frame of UAV. (b) Ground truth. (c) Bayesian histogram. (d) Morphological Filtering. (e)∑-∆ motion detection. f) ViBe method. (Note: Clutter has not been removed)
Clutter
![Page 22: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/22.jpg)
Clutter reduction algorithm
Background subtracted image
Remove the area containing less than 200 pixels
Find boundaries
Find, A o = Area occupied by object
& A B = Filled area of bounding box
Fill Ratio, A = A o/ A B
Aspect Ratio
A r =Width/Height
( A > 0.5) ?
Clutter
No
Moving Objects
(3.5 > A r > 1 )? No
Yes
Yes
Background Subtraction 1
2
Interest point Detection
(Eligibility Criteria)
Registration (Scene lock)
Background Subtraction
Clutter reduction
![Page 23: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/23.jpg)
Evaluation Criteria
Subjective: Visual inspection
Objective :
– Percentage of Correct Classification(PCC)
– False Positive Rate (FPRate)
Where,
True positives(TP): Number of correctly detected foreground pixels
False positives(FP): Number of background pixels incorrectly classified as foreground
True negatives(TN): Number of correctly classified background pixels
False negatives(FN): Number of foreground pixels incorrectly classified as background
FNFPTNTP
TNTPPCC
TNFP
FPFPRate
![Page 24: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/24.jpg)
Results Analysis
![Page 25: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/25.jpg)
Conclusion
MATLAB helped us all the way to develop this frame work for real time UAV application
Extensively utilized following MATLAB Tool boxes
– Computer vision system tool box
– Image Processing tool box
– DSP System Processing tool box
– Statistics tool box
Quick study of various methodologies
Not limited to number of moving targets
Complexity is independent of target speed
Reduced time to develop this framework
![Page 26: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/26.jpg)
Scope of future work
Immediate:
– Target merged to clutter
– Fill ratio criteria
(a) (b)
Figure : (a) Targets merged with clutter. (b) Target failing to fill ratio criteria.
Next:
– Clutter is more due to rolling
– Replacement of Morphological operation
![Page 27: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/27.jpg)
References
• O. Barnichand M. Van Droogen broeck, "ViBe: A Universal Background Subtraction Algorithm for Video Sequences", IEEE Transactions on image processing, Vol 20, no.6, June 2011.
• A Robust and Computationally Efficient Motion Detection Algorithm Based on ∑-∆Background Estimation. A. Manzanera J. C. Richefeu. ENSTA/LEI, 32 Bd VictorF-75739 PARIS CEDEX 15, july 6, 2011.
• “FASTER and better: A machine learning approach to corner detection” in IEEE Trans. Pattern Analysis and Machine Intelligence, Edward Rosten, Reid Porter and Tom Drummond, vol 32, pp. 105-119, 2010.
• Saad Ali and Mubarak Shah, COCOA - TRACKING IN AERIAL IMAGERY. Computer Vision Lab,School of Computer Science,University of Central Florida, 2006.
• A Background Subtraction Model adapted to Illumination changes. Julio Cezar Silveira Jacques Jr., Claudio Rosito Jung and Soraia Raupp Musse, IEEE Transactions, 1-4244-0481-9/06/$20.00 C 2006.
• C. Harris and M. Stephens. A Combined Corner and Edge Detector. Proc. Alvey Vision Conf., Univ. Manchester, pp. 147-151, 1988.
• H. P. Moravec. Towards Automatic Visual Obstacle Avoidance. Proc. 5th International Joint Conference on Artificial Intelligence, pp. 584, 1977.
![Page 28: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/28.jpg)
Acknowledgement
Aeronautical Development Establishment (ADE),
Defence Research and Development Organization (DRDO),
Bangalore, India.
Department of Electronics & Communication Engg.,
Siddaganga Institute of Technology (SIT), Tumkur, India.
CoreEL Technologies, Bangalore, India.
![Page 29: AN OPTIMISED FRAME WORK FOR MAXIMUM ACTIVITY …€¦ · Md. Shahid, Pooja HR# Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO), ...](https://reader034.fdocuments.us/reader034/viewer/2022043002/5f7faff7a1868f5b4f22547a/html5/thumbnails/29.jpg)
Discussion