Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

26
Xilinx Confidential Developing Video Applications on Xilinx FPGAs

TAGS:

Transcript of Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Page 1: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx Confidential

Developing Video Applications on Xilinx FPGAs

Page 2: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar2

Xilinx Design FlowVideo Hardware Development

Develop Executable Spec in Simulink

Create Hardware Model using

SysGen

Verify Hardware using HW Cosim

Integrate Hardware into Embedded

System

Page 3: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar3

Simulink Executable Spec

• Purely algorithmic• Use abstract blocks • Define desired system response

Page 4: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar4

Executable Spec Demo

Page 5: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar5

Xilinx Design FlowVideo Hardware Development

Develop Executable Spec in Simulink

Create Hardware Model using

SysGen

Verify Hardware using HW Cosim

Integrate Hardware into Embedded

System

Page 6: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar6

Design a Hardware Architecture

• Floating-point numbers• Defining basic elements of Hardware Architecture• Compare to Executable Spec

Page 7: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar7

Define Fixed-Point Quantization

• Compare fixed-point error to golden source

Page 8: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar8

Redefine Dataflow for Hardware

• Convert from frames to serial streaming• Consistent with CMOS video camera outputs

Page 9: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar9

Redefine design using the Xilinx DSP Blockset

• Xilinx DSP Blockset includes over 100 DSP building blocks that have been optimized for efficient results on Xilinx devices

• Define partition using GatewayIn / GatewayOut blocks

Gateway blocks define the FPGA boundary

SysGen token block enables use of netlist generation scripts

Page 10: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar10

Model Based Design Demo

Page 11: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar11

Xilinx Design FlowVideo Hardware Development

Develop Executable Spec in Simulink

Create Hardware Model using

SysGen

Verify Hardware using HW Cosim

Integrate Hardware into Embedded

System

Page 12: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar12

Video Hardware Verification Flow

Reference Model

Golden Input Sequences & Test Cases

Design Capture

Verification

Validation

50+ Test Sequences

Golden Test Vector Suite

Test Cases-Pedestrian Crossing-Abandon Object-Privacy Regions-Object Removal

Test Cases-Pedestrian Crossing-Abandon Object-Privacy Regions-Object Removal

Golden Test Vectors = ?

The Key is Fast Algorithm Confirmation• Simulink and the Video and Imaging blockset• Early System Architecture• Bit True, Not cycle accurate

Micro-Architecture• IO Definitions• Automated TB Generation• API Definition• Early FPGA Characterization

• HDL Simulation• Back Annotated HDL Sim• SysGen CoSim

• SysGen-VSK HwCoSim• SysGen-VSK-VFBC HwCoSim• Live Validation, Camera>VSK>Display

Page 13: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar13

Accelerating Verification through Hardware

• Hardware co-verification removes simulation bottleneck– Up to 1000x simulation performance

improvement– Automates FPGA and board setup

process

DesignSimulation Time (Seconds)

Software HW Co-Sim Increase

Beamformer 113 2.5 45X

OFDM BER Test 742 .75 989X

DUC CFR 731 23 32X

Color Space Converter 277 4 69x

Video Scalar 10422 92 113X

Page 14: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar14

Eliminating IO Bottlenecks using Hardware Frame Passing

System Generator includes specials “Shared Memory Read / Write” blocks to allow large amounts of data to be efficiently passed to hardware from Simulink

Boost HW co-sim performance by bundling input data samples together thus reducing simulation to hardware transactions.

- Frame size =2880

Boost HW co-sim performance by bundling input data samples together thus reducing simulation to hardware transactions.

- Frame size =2880

Page 15: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar15

Simulation Runtime Improvements using Hardware Co-simulation

Abandon Object DesignSim Time(seconds)

Time / frame(seconds)

Performance Improvement

Original Simulink Abstract Model .5 .1 N/A

With SysGen block* 50 10 baseline

SysGen with HW co-sim no frames* 165 33 3X slower

SysGen with HW co-sim with frames* 15 3 3X

With input from .mat file* 10 2 5X

With in/output to .mat file* 4.2 .8 12X

* For bit-true hardware accurate simulation models

Page 16: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar16

Additional Data from Xilinx Video Development Team

Frames800x600 1280x720 1920x1080

Non-Accelerated HW Accelerated Non Accelerated HW Accelerated Non-accelerated HW Accelerated

1 1243 sec 18 sec 2238 sec 25 sec 5310 sec 37 sec

2 2579 sec 25 sec 4728 sec 35 sec NA 62 sec

5 NA 43 sec NA 65 sec NA 128 sec

• Notes:– 100 Mbps Ethernet link, Effective rate ~5.1 Mbps

• ML506 Virtex-5 SXT development platform– Payload = 2 * N_Frames * Frame_Size * 32-bits– Load N frames of data, Process N Frames, Store N Frames

Page 17: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar17

Hardware Co-Simulation Demo

Page 18: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar18

Xilinx Design FlowVideo Hardware Development

Develop Executable Spec in Simulink

Create Hardware Model using

SysGen

Verify Hardware using HW Cosim

Integrate Hardware into Embedded

System

Page 19: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar19

Why Video Systems?• Video designs generally include embedded processing

for:– Video system control and dataflow control– Table and memory updates– Low performance video processing

• Xilinx embedded processors allow high-performance video systems on a single chip– Lower cost– Higher performance– Obsolescence proof

Page 20: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar20

The VSK Base System

Processor System Reset

Clock Generator

MicroBlaze Processor

MDM UARTGIP

DIP SwitchesGPIO

Push ButtonsGPIOLEDS

MPMC System ACE XPS IIC XPS IIC

DDR2

PLBARB

BlockRAM

ILMB DLMB

IXCL

DXCL

PLB

XCL

• Embedded base system provided with the VSK• Forms the framework from which video designs are created• Includes one MicroBlaze embedded processor• Customized using Platform Studio

Page 21: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar21

System Generator to Embedded

• System Generator automatically generates DSP accelerators for use with the Xilinx embedded development environment (XPS)– placed into embedded IP Catalog– Supports PLB or FSL bus– Supports async clocking – Includes driver files and

documentation

• XPS project can be imported into SysGen for system debug

System Generator

Embedded Hardware

Platform Studio

Embedded Developers Kit

Embedded IP Catalog

pcores XPS Project

Page 22: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar22

Video Starter Kit Reference IPCore Name

Use in Reference Designs

DVI Pass-through Camera Frame Buffer Frame Buffer

DVI_IN and DVI_OUT Yes Yes Yes

Camera processing Yes Yes

DE_GEN Yes Yes

VIDEO_TO_VFBC Yes Yes

Video Frame Buffer Controller (VFBC)

Yes Yes

• Provided as a library of “drag and drop” IP for use with the video base system

• Provides abstraction to the video interface details– Includes SW driver files

Platform Studio IP Catalog

Generated by SysGen

Reference IP included with the VSK

Page 23: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar23

Abstracting the Processor Interface

• “Shared” registers, RAMs and FIFOs are used to create HW / SW abstraction – DSP design connects to a “to”

or “from” memory– Memory maps and interface

logic is added during RTL generation

– Software drivers and documentation are created for easy programming

Page 24: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar24

System Design Integration Demo

Page 25: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar25

Video Example #1 - VFBC

Processor System Reset

Clock Generator

MicroBlaze PRocessor

MDM UARTGIP

DIP SwitchesGPIO

Push ButtonsGPIOLEDS

MPMC

System ACE XPS IIC XPS IIC

PLBARB

BlockRAM

ILMB DLMB

PLB

Camera Input

Camera Processing

GammaVideo to

VFBCDisplay

ControllerDVI_OUTCamera Video Out

System Generator Design

VFBC

VFBC

Embedded System

User Created Video Accelerator

Video Interface Reference IP

Page 26: Xilinx Confidential Developing Video Applications on Xilinx FPGAs.

Xilinx / Avnet / Mathworks Video Seminar26

Getting Started with VSK Reference Designs

Simplest Frame Buffer Data TransferDVI Input

Frame Buffer

DVI Output

Basic “real-time” video processingImage

ProcessingDVI Input

DVI Output

“Real-time” Frame Buffer Based Video Processing

Image ProcessingCamera

InputFrame Buffer

DVI Output


Xilinx XAPP852 RLDRAM II Memory Interface for Virtex-5 FPGAs

Xilinx XAPP852 RLDRAM II Memory Interface for Virtex-5 FPGAs

Analog for Xilinx (R) FPGAs Selection Guide - 2015 (Rev. B)

Analog for Xilinx (R) FPGAs Selection Guide - 2015 (Rev. B)

Xilinx CPLDs and FPGAs Lecture L1.1. CPLDs and FPGAs XC9500 CPLD Spartan II FPGA Virtex FPGA.

Xilinx CPLDs and FPGAs Lecture L1.1. CPLDs and FPGAs XC9500 CPLD Spartan II FPGA Virtex FPGA.

XILINX CONFIDENTIAL. Design AXI Master Page 1. XILINX CONFIDENTIAL. Understanding Zynq AXI Master IP axi_user_npi Page 2 Agenda © Copyright 2012 Xilinx.

XILINX CONFIDENTIAL. Design AXI Master Page 1. XILINX CONFIDENTIAL. Understanding Zynq AXI Master IP axi_user_npi Page 2 Agenda © Copyright 2012 Xilinx.

Xilinx XAPP951 Configuring Xilinx FPGAs with SPI … Xilinx FPGAs with SPI Serial Flash Author: Stephanie Tapp SPI Basics XAPP951 (v1.3) September 23, 2010 2 The master FPGA device

Xilinx XAPP951 Configuring Xilinx FPGAs with SPI … Xilinx FPGAs with SPI Serial Flash Author: Stephanie Tapp SPI Basics XAPP951 (v1.3) September 23, 2010 2 The master FPGA device

FLIPPER SEU Fault Injection in Xilinx FPGAs

FLIPPER SEU Fault Injection in Xilinx FPGAs

Xilinx XAPP753 Interfacing Xilinx FPGAs to TI DSP Platforms … · 2019-10-10 · Interfacing Xilinx FPGAs to TI DSP Platforms XAPP753 (v2.0.1) January 29, 2007 Xilinx is disclosing

Xilinx XAPP753 Interfacing Xilinx FPGAs to TI DSP Platforms … · 2019-10-10 · Interfacing Xilinx FPGAs to TI DSP Platforms XAPP753 (v2.0.1) January 29, 2007 Xilinx is disclosing

7 Series FPGAs Configuration - Xilinx - All Programmable Series FPGAs Configuration User Guide

7 Series FPGAs Configuration - Xilinx - All Programmable Series FPGAs Configuration User Guide

Programming Module for Xilinx FPGAs Overvie · Programming Module for Xilinx ® FPGAs Revised July 22, 2014 Overview The Joint Test Action Group (JTAG)-SMT2 is a compact, complete

Programming Module for Xilinx FPGAs Overvie · Programming Module for Xilinx ® FPGAs Revised July 22, 2014 Overview The Joint Test Action Group (JTAG)-SMT2 is a compact, complete

Xilinx XAPP753 Interfacing Xilinx FPGAs to TI DSP · PDF fileR Interfacing Xilinx FPGAs to TI DSP Platforms Using the EMIF Application Note XAPP753 (v2.0.1) January 29, 2007

Xilinx XAPP753 Interfacing Xilinx FPGAs to TI DSP · PDF fileR Interfacing Xilinx FPGAs to TI DSP Platforms Using the EMIF Application Note XAPP753 (v2.0.1) January 29, 2007

Xilinx UG429 7 Series FPGAs Migration Methodology Guide

Xilinx UG429 7 Series FPGAs Migration Methodology Guide

Developing Video Applications on Xilinx FPGAs

Developing Video Applications on Xilinx FPGAs

Xilinx Memory Interfaces Made Easy with Xilinx FPGAs and ... · FPGAs provide I/O blocks and logic resources that make the interface design easier and more reliable. Nonetheless,

Xilinx Memory Interfaces Made Easy with Xilinx FPGAs and ... · FPGAs provide I/O blocks and logic resources that make the interface design easier and more reliable. Nonetheless,

ANALOG SOLUTIONS FOR XILINX FPGAs - Maxim IntegratedAnalog Solutions for Xilinx FPGAs A message from the Vice President, Portfolio and Solutions Marketing, Xilinx, Inc. Dear Customers,

ANALOG SOLUTIONS FOR XILINX FPGAs - Maxim IntegratedAnalog Solutions for Xilinx FPGAs A message from the Vice President, Portfolio and Solutions Marketing, Xilinx, Inc. Dear Customers,

Xilinx UG480 7 Series FPGAs and Zynq-7000 All Programmable ...

Xilinx UG480 7 Series FPGAs and Zynq-7000 All Programmable ...

High Performance Computing Using FPGAs - Xilinx

High Performance Computing Using FPGAs - Xilinx

Proven Power Reduction with Xilinx UltraScale FPGAs (WP466) · White Paper: UltraScale FPGAs WP466 (v1.1) October 15, 2015 Proven Power Reduction with Xilinx UltraScale FPGAs By:

Proven Power Reduction with Xilinx UltraScale FPGAs (WP466) · White Paper: UltraScale FPGAs WP466 (v1.1) October 15, 2015 Proven Power Reduction with Xilinx UltraScale FPGAs By:

Xilinx CPLDs and FPGAs

Xilinx CPLDs and FPGAs

Programming and roadmap for Xilinx FPGAs · Programming and roadmap for Xilinx FPGAs Kees Vissers Xilinx. Contents Processors and FPGAs BDTI The Video Starter Kit ... Xilinx FPGA

Programming and roadmap for Xilinx FPGAs · Programming and roadmap for Xilinx FPGAs Kees Vissers Xilinx. Contents Processors and FPGAs BDTI The Video Starter Kit ... Xilinx FPGA

Semicon2012 Keynote - FPGAs Xilinx

Semicon2012 Keynote - FPGAs Xilinx