S9545 - Using the Deepstream SDK for AI-Based Video Analytics · )5$0(:25. )25 $1$
AN INTRODUCTION TO DEEPSTREAM SDK - NVIDIA · 2018. 4. 23. · NVIDIA CONFIDENTIAL. DO NOT...
Transcript of AN INTRODUCTION TO DEEPSTREAM SDK - NVIDIA · 2018. 4. 23. · NVIDIA CONFIDENTIAL. DO NOT...
AN INTRODUCTION TO DEEPSTREAM SDKKaustubh Purandare
March 2018
2NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
AGENDA
• Introduction to DeepStream SDK
• DeepStream SDK Basic Building Blocks
• Setup & Installation
• Application Examples
• Performance Analysis
• 3rd Party Integration
• Q&A
3NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
WHAT IS DEEPSTREAM SDK
1) NVIDIA DeepStream simplifies the development of scalable intelligent video analytics (IVA) applications
2) Developers can now use this to quickly build new applications to transform video into valuable insight.
3) Applications for the DeepStream SDK include image classification, scene understanding, video categorization, content filtering etc..
4NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
DEEPSTREAM SDK FOR INTELLIGENT VIDEO ANALYTICS
Public Transit
Retail Analytics
Traffic Engineering
Law Enforcement
Access control
Securing Critical Infrastructure
Parking Management
In-Vehicle Analytics
5NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
DEEPSTREAM SDK
NVIDIA Platforms
Linux, CUDA
DeepStream SDK
Sample
DNNs &
plug-ins
3rd party
application
plugins
Modular,
Scalable,
Architecture
Development
Tools
Retail analytics & checkout
Intelligent traffic systems
Law
enforcemen
t
Smart parking
Access control
User Applications
Solution
adaptation
guide
* Early release and limited availability.
TensorRT
cuDNN
Multimedia
Multi-stream Encode &
Decode
Imaging
Image capture &
processing
Deep Learning
6NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
DEEPSTREAM SDK BUILDING BLOCKS
7NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
SETUP & INSTALLATION
A] Jetson
1) Latest JetPack https://developer.nvidia.com/embedded/jetpack
2) DeepStream on Tegra SDK https://developer.nvidia.com/deepstream-jetson
3) Ubuntu 16.04 64-bit operating system (host)
4) Jetson TX1 / TX2 Development Platform
B] Running the Sample Application
nvgstiva-app -c <HOME_dir>/configs/<Config.txt> \
-i /home/nvidia/<path_to_input_stream>
8NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
Tracking plugin
Image capture and
process plugin
Primary detection
plugin
(TensorRT)
Car color(TensorRT)
Car type(TensorRT)
Car make(TensorRT)
License plate recognition
(3rd party plugin)
SAMPLE APPLICATION
Conso
lidati
ng u
niq
ue ID
Meta
data
export
Display/ OSD APIs
Situational awareness -• Identify cars, pedestrians, and two-wheelers• Classify Make (Mercedes, BMW, Audi, …)• Classify Type (SUV, Sedan, Truck, …)• Identify Color (Black, Blue, …)• Read license plate
Smart storage
9NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
10NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
APPLICATION FLOW
Parse application configuration file
Create graphs
(add/don’t add elements, set properties
based on configuration)
Set application graphs to playing
Metadata generated Callback
(Called per frame with metadata
generated by the graphs – object co-
ordinates, secondary labels, unique
tracking id, etc)
Overlay Graphics Callback
(Overlay custom text / graphics per
frame)
Destroy graphs
(Free up resource)
Loop
11NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
• Organized as groups and key-
value pairs
• Groups for graphs stages
• Key-value pair to configure each
stage
• Very high granularity for
configuring each stage
CONFIGURATION FILE
12NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
APPLICATION FLOW
Parse application configuration file
Create graphs
(add/don’t add elements, set properties
based on configuration)
Set application graphs to playing
Metadata generated Callback
(Called per frame with metadata
generated by the graphs – object co-
ordinates, secondary labels, unique
tracking id, etc)
Overlay Graphics Callback
(Overlay custom text / graphics per
frame)
Destroy graphs
(Free up resource)
Loop
13NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
GRAPHS CREATED
14NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
APPLICATION FLOW
Parse application configuration file
Create graphs
(add/don’t add elements, set properties
based on configuration)
Set application graphs to playing
Metadata generated Callback
(Called per frame with metadata
generated by the graphs – object co-
ordinates, secondary labels, unique
tracking id, etc)
Graphics Overlay
(Overlay custom text / graphics per
frame)
Destroy graphs
(Free up resource)
Loop
15NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
CREATING SIMPLE GRAPHSS
1) Gstreamer provides powerful tool like gst-launch to create trial / experimental graphs as per use cases.
2) File stream with Primary object detection and OnScreen Display
• gst-launch-1.0 uridecodebin uri=file:///home/nvidia/video.mp4 !
nvinfer <primary-infer-properties> ! queue ! nvosd <osd-
properties> ! nveglglessink
3) RTSP stream with primary object detection + tracking + secondary classification labels + OnScreen Display
• gst-launch-1.0 uridecodebin uri=rtsp://10.24.1.1/video0 ! nvinfer
<primary-infer-properties> ! queue ! nvtracker ! queue ! nvinfer
<secondary-infer-properties> ! queue ! nvosd <osd-properties> !
nveglglessink
16NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
PERFORMANCE ANALYSIS
17NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
APPLICATION SOFTWARE STACK
Linux, CUDA
Sample DL Networks ISV/ 3rd party solutions
NVIDIA Platforms
Applications
Deep Learning
TensorRT
cuDNN
Multimedia
Multistream
Encode & Decode
Imaging
DeepStream
Image capture &
processing
Plugin based compute Graph ● Gstreamer Framework
Custom DL Networks
18NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
START DEVELOPING WITH DEEPSTREAM
DeepStream Early Access program . Explore Metropolis . Intelligent Video Analytics Forums
19
THANK YOU