Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This...

18
Combination of Altera OpenCL kernel with video IP cores Kostyantyn Pelekh GlobalLogic Inc. 1

Transcript of Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This...

Page 1: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

Combination of Altera OpenCL kernel with video IP cores Kostyantyn Pelekh GlobalLogic Inc.

1

Page 2: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

•  Solution Overview •  Design Details •  Summary •  Additional information

2

Page 3: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

Design Challenge

3

Even though it is low cost it has to do a real calculation. Like medical device scanning part of your body or network device doing packet filtering, etc. OpenCL helps here.

Many companies consider using Intel® Cyclone® V SoC chip and Boards based on it because it is very affordable.

On the other hand in majority of cases it is mandatory for device to provide UI and allow user to operate it over touch screen for example.

There is a dilemma what to program the FPGA for: video or calculations. We will investigate a solution which optimally combines both.

Page 4: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

Hardware Description

4

Cyclone® V SoC Development Kit offers a quick and simple approach to develop custom ARM® processor-based SoC designs accompanied by Altera’s low-power, low-cost Cyclone V FPGA fabric.

The DE0-Nano-SoC-HDMI Development Kit presents a robust hardware design platform built around the Altera System-on-Chip (SoC) FPGA, which combines the latest dual-core Cortex-A9 embedded cores with industry-leading programmable logic for ultimate design flexibility.

The Terasic HDMI_TX_HSMC is a HDMI transmitter daughter board with HSMC (High Speed Mezzanine Connector) interface. Host boards, supporting HSMC-compliant connectors, can control the HDMI_TX_HSMC daughter board through the HSMC interface.

Page 5: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

Altera Video Design Framework

5

This is a combination of IP cores, interface standards and system level design tools that are developed to enable a plug-and-play video system design flow. Altera provides a comprehensive suite of video function IP blocks that can be connected together to design and build video systems, allowing replacement of Altera IP with own custom function block

Memory Controller

VIP Framereader

VIP Clocked Video Output

VIP Suite Function 1..n

HPS

HDMI/SDI Controller

Avalon-MM

Avalon-MM

Avalon-MM

Avalon-ST

Clocked Video Standard

User Interaction/Status

Video Subsystem

Quartus (HDL)

Page 6: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

Altera OpenCL SDK

6

•  Allows to use the software development flow instead of the traditional hardware FPGA development flow

•  Provides tools to easy develop apps onto FPGAs by abstracting away the complexities of FPGA design

•  Allows software programmers to write hardware-accelerated kernel functions in OpenCL C

Programming model: •  Device programming language based on C99 with extensions to

support parallelism •  Application Programming Interface (API) for device management Performance: •  CPU offload •  Software to leverage silicon acceleration

Page 7: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

Resulting Solution

7

Is a design where OpenCL hardware is combined with customized Video subsystem based on Altera VIP framework, which allows to show the calculation results on a display. The design demonstrates a framework for rapid development of video and image processing systems using Altera VIP Suite.

DE0-Nano-SoC-HDMI

Terasic HDMI_TX_HSMC

OpenCL Kernel with Mandelbrot algorithm running

Cyclone® V SoC

Page 8: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

Features

8

•  Accelerate calculation with any algorithm that fits the CycloneV SX

•  Calculation results output to HDMI / DVI display

•  Support of eight common resolutions from VGA to HD1080

•  Resolution change on the fly without calculation interruption

•  Turn on/off the hardware acceleration. Display frame rate in real-time

Page 9: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

Performance Metrics of Mandelbrot

9

0

2

4

6

8

10

12

SW Render (FPS) HW Render (FPS)

640x480 720x480 800x600 1024x768 1280x720 1280x1024 1600x1200 1920x1080

Page 10: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

•  Solution Overview •  Design Details •  Summary •  Additional information

10

Page 11: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

Top-Level Design

11

OpenCL Kernel Qsys

HPS

VIP Qsys

OS

DDR

HDMI

§  Common open Avalon data and control interfaces are used to facilitate connection of video functions chain and video system modeling

§  Video submodule is connected to the HPS and DDR memory through the H2F Lightweight and F2H ports, respectively

§  Operating System configures all VIP and PLL cores through the HPS ports to work with appropriate screen resolution

§  Operating System prepares frame window in the specific DDR memory region for video output stream

Page 12: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

Use HPS For Configuration

12

640x480 720x480 800x600 1024x768 1280x720 1280x1024 1600x1200 1920x1080

§  HPS initializes all possible resolution modes in the Clocked Video Output (CVO) core

§  HPS changes the frame window size in the Framereader VIP core that will cause automatic mode switch in the CVO

§  HPS sets appropriate video sync frequency by configuring the PLL

H2F LW

I2C

HPS

OS

VIP Frame Reader

VIP Clocked

Video Out

Reconfig PLL

HDMI Controller

Page 13: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

FPGA Design Partitions

13

§  HPS

§  Mandelbrot Kernel

§  OpenCL Kernel

§  Video Subsystem

18390

732

4829

Logic (ALM) 20,833 / 41,910 (50 %)

Total block memory bits 405,656 / 5,662,720 (7 %)

Total RAM Blocks 99 / 553 (18 %)

Total DSP Blocks 28 / 112 (25 %)

Total PLLs 4 / 12 (33 %)

Total DLLs 1 / 4 (25 %)

Page 14: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

Lessons Learned

14

Clock setup issues

•  Framereader VIP core and Clocked Video Output core should be initialized in Qsys editor with maximum ratings. If you have several resolution modes, then the cores should be initialized with the highest resolution settings.

•  All VIP cores should be connected to one clock domain. The main cores’ clock should be equal or bigger than video sync clock.

•  To work with high resolutions the PLL should be initialized in Qsys editor to generate integer frequency value without decimal part.

FPGA space optimizations

•  Removed the additional Nios-based submodule that was configuring PLL, HDMI and VIP cores during the HPS launch. Switch all configuration procedures to HPS.

Page 15: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

•  Solution Overview •  Design Details •  Summary •  Additional information

15

Page 16: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

Summary and Next Steps •  Motivated the need of combined OpenCL & Video Framebuffer

•  Demonstrated the combined design architecture

•  Described the benefits and performance metrics

•  Provided development constraints and limitations

•  The next steps -  Search for applied areas and define use cases

-  Create OpenCL kernels for other purposes

-  Work on solution optimization for lower space and better performance

-  Add video layering functionality

16

Page 17: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

•  Solution Overview •  Design Details •  Summary •  Additional information

17

Page 18: Combination of Altera OpenCL kernel with video IP cores ......Altera Video Design Framework 5 This is a combination of IP cores, interface standards and system level design tools that

Additional Sources of Information

•  Altera OpenCL SDK: https://www.altera.com/products/design-software/embedded-software-developers/opencl/overview.html

•  Altera Video Design Framework: https://www.altera.com/solutions/technology/dsp/dsp-video-solutions/dsp-1080p-hd-video.html

•  Mandelbrot algorithm: https://en.wikipedia.org/wiki/Mandelbrot_set

18


Altera University Program Video IP Cores - UMass … Altera University Program (UP) ... 5 megapixel CCD camera and LCD screen with touch panel ... The VGA controller core displays

Altera University Program Video IP Cores - UMass … Altera University Program (UP) ... 5 megapixel CCD camera and LCD screen with touch panel ... The VGA controller core displays

Integer Arithmetic IP Cores User Guide · 2020. 12. 20. · 2016.06.10 Design Example Files 1-3 Integer Arithmetic IP Cores Altera Corporation Send Feedback. LPM_COUNTER (Counter)

Integer Arithmetic IP Cores User Guide · 2020. 12. 20. · 2016.06.10 Design Example Files 1-3 Integer Arithmetic IP Cores Altera Corporation Send Feedback. LPM_COUNTER (Counter)

MAX 10 FPGA Configuration User Guide - U of T Physicsastummer/Archives... · Related IP Cores • Altera Dual Configuration IP Core—used in the remote system upgrade feature. •

MAX 10 FPGA Configuration User Guide - U of T Physicsastummer/Archives... · Related IP Cores • Altera Dual Configuration IP Core—used in the remote system upgrade feature. •

Introduction to Altera IP Cores

Introduction to Altera IP Cores

Fixed-Point IP Cores (ALTERA FIXED-POINT …...1 About Fixed-Point IP Cores The fixed-point IP cores (ALTERA_FIXED-POINT_FUNCTIONS) allow you to implement simple fixed-point functions

Fixed-Point IP Cores (ALTERA FIXED-POINT …...1 About Fixed-Point IP Cores The fixed-point IP cores (ALTERA_FIXED-POINT_FUNCTIONS) allow you to implement simple fixed-point functions

Altera University Program Video IP Cores · PDF fileThe Altera University Program (UP) ... that can be connected to any memory device in the Qsys system. In this example, the SRAM/SSRAM

Altera University Program Video IP Cores · PDF fileThe Altera University Program (UP) ... that can be connected to any memory device in the Qsys system. In this example, the SRAM/SSRAM

MagAmp cores MagAmp Cores

MagAmp cores MagAmp Cores

Altera FPGA FIFO master Programming Guide - FTDI Altera FPGA … · GuideAN_377 Altera FPGA FIFO master Programming ... Document Title: AN_377 Altera FPGA FIFO master Programming

Altera FPGA FIFO master Programming Guide - FTDI Altera FPGA … · GuideAN_377 Altera FPGA FIFO master Programming ... Document Title: AN_377 Altera FPGA FIFO master Programming

EPICS on Altera NIOS2 FPGA Softcore€¦ · Traditional VME/cPCI backplanes ›Now they are becoming a bottleneck There are Altera IP cores for ›System processor ›Ethernet interface

EPICS on Altera NIOS2 FPGA Softcore€¦ · Traditional VME/cPCI backplanes ›Now they are becoming a bottleneck There are Altera IP cores for ›System processor ›Ethernet interface

Cores, cores, everywhere

Cores, cores, everywhere

Altera University Program Video IP Cores

Altera University Program Video IP Cores

Altera University Program Video IP Cores...The Altera University Program (UP) Video IP cores facilitate decoding, processing and display of video data. They They are designed for use

Altera University Program Video IP Cores...The Altera University Program (UP) Video IP cores facilitate decoding, processing and display of video data. They They are designed for use

SDI Audio IP Cores User Guide - Altera · PDF fileSDI Audio IP Cores User Guide Last updated for Quartus Prime Design Suite: 16.0 Subscribe Send Feedback UG-SDI-AUD 2016.05.30 101

SDI Audio IP Cores User Guide - Altera · PDF fileSDI Audio IP Cores User Guide Last updated for Quartus Prime Design Suite: 16.0 Subscribe Send Feedback UG-SDI-AUD 2016.05.30 101

Internal Memory (RAM and ROM) User Guide - Intel … 2014 Altera Corporation Internal Memory (RAM and ROM) User Guide 2. Customizing Internal Memory IP Cores Installing and Licensing

Internal Memory (RAM and ROM) User Guide - Intel … 2014 Altera Corporation Internal Memory (RAM and ROM) User Guide 2. Customizing Internal Memory IP Cores Installing and Licensing

Arria 10 Triple Speed Ethernet and Native PHY with … › content › dam › altera-www › global › en...Speed Ethernet (TSE) and Intel® Arria 10 Transceiver Native PHY IP cores

Arria 10 Triple Speed Ethernet and Native PHY with … › content › dam › altera-www › global › en...Speed Ethernet (TSE) and Intel® Arria 10 Transceiver Native PHY IP cores

Intel FPGA Integer Arithmetic IP Cores User Guide - Altera · Intel FPGA Integer Arithmetic IP Cores User ... These functions offer more efficient logic synthesis and ... The LPM_COUNTER

Intel FPGA Integer Arithmetic IP Cores User Guide - Altera · Intel FPGA Integer Arithmetic IP Cores User ... These functions offer more efficient logic synthesis and ... The LPM_COUNTER

Altera Video and Image Processing Suite...The Video and Image Processing Suite features cores that range from simple building block functions, such as color-space conversion to sophisticated

Altera Video and Image Processing Suite...The Video and Image Processing Suite features cores that range from simple building block functions, such as color-space conversion to sophisticated

EMBEDDED MACHINE VISION - STEMMER IMAGING · Texas Instruments Keystone •ARM + DSP Altera (Intel) Cyclone V •ARM + FPGA Start with evaluation board Make your own board ARM cores

EMBEDDED MACHINE VISION - STEMMER IMAGING · Texas Instruments Keystone •ARM + DSP Altera (Intel) Cyclone V •ARM + FPGA Start with evaluation board Make your own board ARM cores

Fixed-Point IP Cores (ALTERA FIXED-POINT FUNCTIONS) … · .spd - Simulation startup scripts * .cmp - VHDL component declaration .v or

Fixed-Point IP Cores (ALTERA FIXED-POINT FUNCTIONS) … · .spd - Simulation startup scripts * .cmp - VHDL component declaration .v or

Altera Ships 100,000-Gate PLDebook.pldworld.com/_Semiconductors/Altera/literature/_nv/... · 2003-01-09 · Altera Ships 100,000-Gate PLD Altera is now shipping the EPF10K100 device,

Altera Ships 100,000-Gate PLDebook.pldworld.com/_Semiconductors/Altera/literature/_nv/... · 2003-01-09 · Altera Ships 100,000-Gate PLD Altera is now shipping the EPF10K100 device,