"What's New in Tools for Vision Application Design and Development?," a Presentation from BDTI

Copyright © 2014 BDTI 1

Jeff Bier, President, BDTI

May 29, 2014

What’s New in Tools for Vision

Application Design and Development?


About BDTI

• BDTI provides:

• Product development engineering services

• Emphasis on optimization for performance, cost and power

• Benchmarking and evaluation

• For technology selection, feasibility studies, competitive

analysis and proof points

• Focused on:

• Algorithm-intensive applications: vision, video, audio, wireless

• Embedded processors, tools and techniques

“BDTI’s deliverables were orders of magnitude better than other

vendors’. BDTI has set a new standard for quality.”

– Group Program Manager, Fortune 500 Systems & Software Co.

See our fun vision demos in the Technology Showcase!


The Right Tool for the Job

Concept Proof of

Concept Prototype Product

Priority on rapid experimentation Priority on efficient implementation

Which tools you use, and how you use them, depends on where you

are in the product development process


Fundamental Trade-offs

Breadth: Applications,

Algorithms, Data Types

Maturity,

Quality

Portability Across

Processor Types,

Brands, Models

Developer

Productivity


• OpenCV is a popular open source computer vision software component

library

• Supports Windows, Linux, Android, iOS

• Advantages:

• Large library; free

• Easy to use; good abstraction from details

• Open source with liberal (BSD) license

• Large, active community

• Disadvantages:

• Not all algorithms are best-in-class

• Not a standard

• OpenCV 3.0 release due in June

• Most significant enhancement: automatic acceleration via OpenCL

• Complements existing optimizations for CUDA, ARM NEON, x86

OpenCV 3.0


• A Khronos standard API and language for implementing data-parallel

kernels

• Provides portability across different types of platforms (GPU, FPGA,

other)

• Most efficient when code is customized for a specific target

platform

• OpenCL 2.0 (spec released in November) moves towards Heterogeneous

System Architecture (HSA)

• Expands the utility of OpenCL for applications requiring CPU-GPU

interaction and task-based parallelism

• Key features:

• Shared Virtual Memory (SVM)

• Dynamic Parallelism

• Generic Address Space

OpenCL 2.0


• Strengths:

• Portability

• Beginning to see strong traction in industry and academia

• Limitations/Weaknesses:

• Single Program, Multiple Data paradigm can be difficult for

developers

• And not all algorithms naturally exhibit data parallelism

• Relative immaturity compared to other parallel programming

frameworks or APIs (e.g. CUDA)

• Vendor-specific extensions can provide valuable functionality but break

portability

• E.g. FPGA extensions allow threads to communicate across

workgroups

OpenCL Strengths and Weaknesses


• The HSA Foundation is developing a standards-based unifying framework

for heterogeneous computing

• Goal: “Write once, run anywhere for heterogeneous systems”

• Encompasses system architecture spec and software stack

• A unified memory space and programming methodology for

programming CPU and associated compute engines (GPU, DSP, FPGA, …)

• Initiated by AMD, which has been most active in implementing it

• Overlaps with OpenCL; OpenCL can be implemented on top of HSA

• Developers will likely use HSA indirectly via

optimized OpenCL drivers

• Breadth and speed of adoption unclear

Heterogeneous System Architecture (HSA)

Foundation


• An emerging Khronos standard API providing a

vision hardware acceleration (abstraction) layer

• Focus on enabling real-time vision on mobile and

embedded systems

• For use by high-level libraries or apps

• Promises to relax the trade-off between

portability and efficiency

• Targets support for heterogeneous systems,

including CPU, GPU, DSP, FPGA, etc.

• Developer specifies data flow graph of vision

algorithm kernels

• OpenVX framework partitions the graph

among processing engines, handles

memory transfers, etc.

• Provisional specification released November 2013

• Breadth and speed of adoption TBD

OpenVX

Open source sample implementation?

Hardware vendor implementations

OpenCV open source library

Other higher-level CV libraries

Application


• Not a new concept (see, e.g., TI’s VLIB, released in 2008), but the

number and scope of offerings is increasing rapidly

• Multiple levels of abstraction:

• Building-block function libraries (e.g., HOG, optical flow)

• NVIDIA VisionWorks Toolkit for Tegra K1

• CEVA-CV library for CEVA-MM3101 IP processor core

• Qualcomm FastCV: Supports all ARM-based application

processors; accelerated using co-processors on Qualcomm chips

• Various accelerated OpenCV subsets (ARM, AMD, …)

• Function-specific

• SoftKinetic iisu for gesture recognition

• Application-domain-specific

• Qualcomm Vuforia for mobile augmented reality

• LabView Vision for machine vision

Optimized Vision Software Libraries for

Specific Processors


• Vision-oriented development boards and kits are increasingly

available for embedded processors; many at very low prices

• Avnet FinBoard (BF609)

• NVIDIA Jetson (TK1)

• Xilinx Zynq-7000 SoC Video and Imaging Kit

• Qualcomm DragonBoard (SnapDragon)

• Enable getting a sense for the capabilities of the

processor and its tools

• Facilitate early processor-aware algorithm and software design

• Enable testing algorithms in real-world conditions

• Often used for prototyping, and sometimes initial production

Development Kits


• Many tools of many kinds; they’re all moving (some quickly)

• Choosing tools has big implications for programming style, choice of

processor, efficiency of implementation, and other important factors.

• Ideally, think about processor choice early, so initial development

on a PC can use tools and techniques that make it easier to

implement on the target processor

• Choice of tools has big implications for portability and can lock you into

a hardware platform

• Source code is key for libraries: enables learning, debugging,

customization

• With many developers tackling vision for the first time, easy on-ramps

are very valuable:

• Development kits, application examples, tool flow tutorials, etc.

Lessons Learned


• Tools and libraries that relax the trade-offs between productivity,

portability, and flexibility

• Tools for exploring, evaluating, and

selecting algorithm options

• More complete, higher-quality libraries

• More complete reference designs

• Better guidance for developers on tool selection (and co-selection of

tools, algorithms and processors)

Wish List


• "Tools for 'Democratizing' Computer Vision,” http://bit.ly/1tpD9ty

• “Start Developing OpenCV Applications Immediately Using the BDTI

Quick-Start OpenCV Kit,” http://bit.ly/1pmn8a4

• “Designing Visionary Mobile Apps Using the Tegra Android Development

Pack,” http://bit.ly/1jvwbgV

• “Getting Started With GPU-Accelerated Computer Vision Using OpenCV

and CUDA,” http://bit.ly/1oMwJEG

• “The open standard for parallel programming of heterogeneous

systems,” https://www.khronos.org/opencl/

• “Hardware acceleration API for Computer Vision applications and

libraries,” https://www.khronos.org/openvx

Resources

http://bit.ly/1tpD9ty

http://bit.ly/1tpD9ty

http://bit.ly/1pmn8a4

http://bit.ly/1pmn8a4

http://bit.ly/1jvwbgV

http://bit.ly/1jvwbgV

http://bit.ly/1oMwJEG

http://bit.ly/1oMwJEG

https://www.khronos.org/opencl/

https://www.khronos.org/opencl/

https://www.khronos.org/openvx

"What's New in Tools for Vision Application Design and Development?," a Presentation from BDTI

Technology

Transcript of "What's New in Tools for Vision Application Design and Development?," a Presentation from BDTI