Xilinx XAPP058 Xilinx In-System Programming Using an Embedded
ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx...
Transcript of ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx...
![Page 1: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/1.jpg)
ARM64 + FPGA and more:Linux on the Xilinx ZynqMPOpportunities and challenges from a powerful and complex chip
Luca Ceresoli, AIM [email protected]://lucaceresoli.netFOSDEM 2018
![Page 2: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/2.jpg)
About me
• Embedded Linux engineerat AIM Sportlinehttp://www.aim-sportline.com/
• Develop real products on customhardware
• Kernel, bootloader, drivers• Integration, build system
• Open source enthusiast• Contributor to Buildroot and a
few other projects
1
![Page 3: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/3.jpg)
Agenda
• Introduction• Development tools• Linux on your FPGA design• Booting• GPU• Video Codec Unit• Conclusion
2
![Page 4: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/4.jpg)
Introduction
![Page 5: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/5.jpg)
SoC+FPGA: what is it?
• A SoC and an FPGA on a single chip• Connected on-chip• Other technologies:
• 2 chips: a SoC and an FPGA, connected via pins• An FPGA with a synthesized “soft core” CPU
• A good introduction to SoC+FPGA:Introduction to SoC+FPGA, Marek Vašut, ELC-E 2017https://elinux.org/images/e/ed/Elce-2017-socfpga.pdf
https://youtu.be/R3gJhnGjjWY
3
![Page 6: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/6.jpg)
Current Linux-capable SoC+FPGAs
• Xilinx• 1st generation: Zynq 7000• 2nd generation: Zynq UltraScale+ MPSoC (aka ZynqMP)
• Intel (Altera)• 1st generation: Cyclone V and Arria V• 2nd generation: Stratix 10
4
![Page 7: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/7.jpg)
ZynqMP block diagram (simplified)
CortexA53 (x4)
FullPower
Interconnects
Fast
Peri
phera
ls
GPU
CortexR5 (x2)
LowPower
Interconnect
DDR Controller
FPGA
VCUVery Fast
Peripherals
Peri
phera
ls
CSU
PMU
Slave
Master
5
![Page 8: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/8.jpg)
Development tools
![Page 9: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/9.jpg)
Documentation
• A lot of documentation by Xilinx• Start from the Xilinx ZynqMP Documentation hub
https://www.xilinx.com/support/documentation-navigation/design-hubs/
dh0070-zynq-mpsoc-design-overview-hub.html
6
![Page 10: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/10.jpg)
FPGA development
6
![Page 11: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/11.jpg)
FPGA Development Tools — Xilinx
Xilinx Vivado Design Suite
• Vivado: design FPGA to bitstream• XSDK: eclipse IDE for firmware development• Runs on Linux• Has a zero-cost version (has most features, not the advanced
ones)• Closed source, huge, has some bugs and issues
7
![Page 12: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/12.jpg)
FPGA Development Tools — Open source
• No open source FPGA toolchain available• Bitstream reverse engineering in progress
(https://symbiflow.github.io/)• On why FPGA open source toolchains matter:
https://blog.elphel.com/2013/10/fpga-is-for-freedom/
8
![Page 13: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/13.jpg)
BSP components
8
![Page 14: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/14.jpg)
U-Boot
• Xilinx is active in mainlining• Discussion: [email protected]• Mainline U-Boot enough to boot• Newer boards are available at
https://github.com/xilinx/u-boot-xlnx
• Note: ZynqMP cannot boot U-Boot the “normal” way, seeBooting section later
9
![Page 15: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/15.jpg)
Linux kernel
• Xilinx is active in mainlining• Discussion: [email protected]• Mainline Linux: partially implemented• Development in progress at
https://github.com/xilinx/linux-xlnx(especially the master branch)
• “Hard” silicon features• Recent UltraScale+ IPs from Xilinx, mostly video-related
10
![Page 16: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/16.jpg)
Build system
10
![Page 17: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/17.jpg)
Available workflows
Typical ways to build the software stack for Xilinx products:
• The “Xilinx workflow”• Officially supported by Xilinx
• The “Community workflow”• Similar to other open source projects• Supported by the community (with Xilinx contributions)
• Other/mixed workflows
11
![Page 18: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/18.jpg)
The Xilinx workflow
• FPGA: Vivado• Baremetal and bootloaders: XSDK• Petalinux
• A Xilinx-specific embedded build system• Nowadays internally uses Yocto
• Yocto layers on https://github.com/xilinx• A meta-xilinx-bsp fork• meta-xilinx-tools to use Xilinx tools during the build• meta-petalinux: a distro layer• Note: these layers will soon be moved to subdirs of
meta-xilinx
12
![Page 19: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/19.jpg)
The Community workflow
• FPGA: Vivado• A little bit of XSDK• Yocto meta-xilinx-bsp layer
https://git.yoctoproject.org/cgit/cgit.cgi/meta-xilinx/subdir meta-xilinx-bsp
• Until a few weeks ago: in the top dir, and called meta-xilinx
• Goal: follow the common practices in FOSS/Yocto• Not all features supported
13
![Page 20: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/20.jpg)
Other resources
• meta-xilinx mailing list• https://lists.yoctoproject.org/listinfo/meta-xilinx• Discussion on the Yocto layers• Also for general discussion about Linux on Xilinx hardware
• https://github.com/topic-embedded-products/meta-topic• Support for boards by Topic Embedded• But has very useful code (see Booting section later)
14
![Page 21: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/21.jpg)
Buildroot
• Work in progress• Patches to add basic support under discussion on the
Buildroot mailing-list
15
![Page 22: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/22.jpg)
Linux on your FPGA design
![Page 23: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/23.jpg)
Build your own SoC!
• FPGA ↔ CPU interface is AXI4• AXI = Advanced eXtensible Interface (part of AMBA)• Peripherals in FPGA are accessible on the physical address
space• Like hard peripherals and traditional SoCs
16
![Page 24: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/24.jpg)
Typical workflow
• Workflow with Vivado1. IP integration2. Address editor3. Constraints
17
![Page 25: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/25.jpg)
Step 1: IP integration
17
![Page 26: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/26.jpg)
Vivado: block design
18
![Page 27: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/27.jpg)
Vivado: customize the PS block
19
![Page 28: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/28.jpg)
Vivado: customize an IP block
20
![Page 29: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/29.jpg)
Linux: instantiate a device
Linux drivers for Xilinx IPs:wiki.xilinx.com/Linux+Drivers
gpio: gpio@a0000000 {#gpio-cells = <2>;compatible = "xlnx,xps-gpio-1.00.a";gpio-controller;xlnx,gpio-width = <3>;xlnx,all-inputs = <1>;/*...*/
};
21
![Page 30: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/30.jpg)
Vivado: interrupts
22
![Page 31: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/31.jpg)
Linux: interrupts
#include <dt-bindings/interrupt-controller/arm-gic.h>#include "zynqmp-irqs.dtsh" // (*)
gpio: gpio@a0000000 {#gpio-cells = <2>;compatible = "xlnx,xps-gpio-1.00.a";gpio-controller;interrupt-parent = <&gic>;interrupts = <GIC_SPI PL_PS_GRP0_IRQ_0
IRQ_TYPE_LEVEL_HIGH>;/*...*/
};
(*) From github.com/xilinx/qemu-devicetrees/blob/master/
23
![Page 32: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/32.jpg)
Step 2: Address editor
23
![Page 33: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/33.jpg)
Vivado: Address editor
24
![Page 34: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/34.jpg)
Linux: address map
gpio: gpio@a0000000 {#gpio-cells = <2>;compatible = "xlnx,xps-gpio-1.00.a";gpio-controller;reg = <0x0 0xa0000000 0x0 0x10000>;/*...*/
};
25
![Page 35: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/35.jpg)
Step 3: Constraints
25
![Page 36: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/36.jpg)
Vivado: constraints
• Connect nets to pins: placement, IO standard, pull-up/down…• No visible effect in Linux
26
![Page 37: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/37.jpg)
Booting
![Page 38: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/38.jpg)
Simple ARM32 booting
• Good old simple booting on ARM32:1. Boot ROM loads SPL into internal RAM2. SPL initializes SDRAM, loads U-Boot3. U-Boot loads kernel
• But waking up in an ARM64 world is much more complex…
27
![Page 39: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/39.jpg)
The Platform Management Unit (PMU)
• Duties:• Power gates peripherals,
power islands, power domains• Clock gates peripherals
• Can’t boot without it• Similar to other ARM64 boards
• E.g. the ARM Juno board
CortexA53 (x4)
FullPower
Interconnects
Fast
Peri
phera
ls
GPU
CortexR5 (x2)
LowPower
Interconnect
DDR Controller
FPGA
VCUVery Fast
Peripherals
Peri
phera
ls
CSU
PMU
Slave
Master
28
![Page 40: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/40.jpg)
PMU firmware
• PMU is a Microblaze core with 128 kB RAM• Executes a firmware that can be reprogrammed
• But the one in ROM is not enough, so it must bereprogrammed
• Source at https://github.com/xilinx/embeddedsw• meta-xilinx-bsp master can build it (not the Xilinx
branches)
• The PMUFW needs a configuration object• Tells which master (CPU) owns which slave (peripheral)• Depends on how the hardware is configured in Vivado
29
![Page 41: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/41.jpg)
ARM Trusted Firmware
A secure monitor reference implementation
Cortex A53Cortex R5
PMU
EL3
EL0
/1
ATF
Linux /U-Boot
xilpm
Bare-metalapplication
SM
CPM
API
PM
API
30
![Page 42: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/42.jpg)
Booting — The Xilinx workflow
30
![Page 43: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/43.jpg)
Boot sequence
CSU
A53
PMURelease
CSU
Load FSBL
LoadPMUFW
Tampermonitoring
FSBL: loadcfg_obj, ATF, U-Boot ATF U-Boot
Powermanagement
CFG OBJ
31
![Page 44: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/44.jpg)
Building the pieces
VivadoH
DF
psu_init*
XSDK
PMUFW BSP
PMUFW
FSBLBSP
FSBL
pm_cfg_obj
pmufw.elf
fsbl.elf
Petalinux
ATF
U-Boot
bootgen
BOOT.BIN
fpga.bit
SD card32
![Page 45: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/45.jpg)
Pros and cons
Pros
• The Xilinx tools make it easy• FSBL is easy to understand and debug
Cons
• FSBL is slow (~3 seconds to load a 4 MB FPGA bitstream)• The Xilinx tools: big and heavy, hard to automate• Proprietary bootgen tools needed to generate BOOT.BIN• Non-standard (w.r.t. other SoCs)
33
![Page 46: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/46.jpg)
Booting — The Community workflow
33
![Page 47: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/47.jpg)
FSBL → U-Boot SPL
Can U-Boot SPL replace Xilinx FSBL?
• SPL cannot load FPGA• but U-Boot can (~10x faster)
• SPL loads U-Boot proper, not ATF!• but there’s a trick for that
• SPL cannot load PMUFW configuration object• but there’s a workaround for that
34
![Page 48: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/48.jpg)
FSBL → U-Boot SPL
Can U-Boot SPL replace Xilinx FSBL?
• SPL cannot load FPGA• but U-Boot can (~10x faster)
• SPL loads U-Boot proper, not ATF!• but there’s a trick for that
• SPL cannot load PMUFW configuration object• but there’s a workaround for that
34
![Page 49: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/49.jpg)
FSBL → U-Boot SPL
Can U-Boot SPL replace Xilinx FSBL?
• SPL cannot load FPGA• but U-Boot can (~10x faster)
• SPL loads U-Boot proper, not ATF!• but there’s a trick for that
• SPL cannot load PMUFW configuration object• but there’s a workaround for that
34
![Page 50: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/50.jpg)
FSBL → U-Boot SPL
Can U-Boot SPL replace Xilinx FSBL?
• SPL cannot load FPGA• but U-Boot can (~10x faster)
• SPL loads U-Boot proper, not ATF!• but there’s a trick for that
• SPL cannot load PMUFW configuration object• but there’s a workaround for that
34
![Page 51: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/51.jpg)
SPL: loading ATF
• U-Boot SPL must load U-Boot proper• But ATF must be there before U-Boot proper• The trick for SPL to load both is:
configs/xilinx_zynqmp_*_defconfig
CONFIG_SPL_OS_BOOT=y // aka Falcon Mode
include/configs/xilinx_zynqmp.h
/* u-boot is like dtb */#define CONFIG_SPL_FS_LOAD_ARGS_NAME "u-boot.bin"#define CONFIG_SYS_SPL_ARGS_ADDR 0x8000000
/* ATF is my kernel image */#define CONFIG_SPL_FS_LOAD_KERNEL_NAME "atf-uboot.ub"
35
![Page 52: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/52.jpg)
SPL: loading ATF
• U-Boot SPL must load U-Boot proper• But ATF must be there before U-Boot proper• The trick for SPL to load both is:
configs/xilinx_zynqmp_*_defconfig
CONFIG_SPL_OS_BOOT=y // aka Falcon Mode
include/configs/xilinx_zynqmp.h
/* u-boot is like dtb */#define CONFIG_SPL_FS_LOAD_ARGS_NAME "u-boot.bin"#define CONFIG_SYS_SPL_ARGS_ADDR 0x8000000
/* ATF is my kernel image */#define CONFIG_SPL_FS_LOAD_KERNEL_NAME "atf-uboot.ub"
35
![Page 53: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/53.jpg)
SPL: loading ATF
• U-Boot SPL must load U-Boot proper• But ATF must be there before U-Boot proper• The trick for SPL to load both is:
configs/xilinx_zynqmp_*_defconfig
CONFIG_SPL_OS_BOOT=y // aka Falcon Mode
include/configs/xilinx_zynqmp.h
/* u-boot is like dtb */#define CONFIG_SPL_FS_LOAD_ARGS_NAME "u-boot.bin"#define CONFIG_SYS_SPL_ARGS_ADDR 0x8000000
/* ATF is my kernel image */#define CONFIG_SPL_FS_LOAD_KERNEL_NAME "atf-uboot.ub"
35
![Page 54: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/54.jpg)
Loading PMUFW configuration object
• SPL cannot load PMUFW configuration object• The current best workaround is:
• Link pm_cfg_obj.c in the PMUFW• Load it during PMUFW startup• Original patches on the meta-topic layer• A trivial script to apply the patches and build PMUFW:
https://github.com/lucaceresoli/zynqmp-pmufw-builder
• Pros: it works!• Cons: must rebuild PMUFW when configuration changes
• But in the Xilinx workflow you’d have to rebuild FSBL anyway
36
![Page 55: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/55.jpg)
Boot sequence
CSU
A53
PMURelease
CSU
Load SPL
LoadPMUFW
Tampermonitoring
SPL: loadATF, U-Boot ATF U-Boot
Powermanagement
CFG OBJ
37
![Page 56: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/56.jpg)
Peripheral initialization
• Pinctrl and most peripherals must be configured by thebootloader
• To do this in U-Boot:1. In Vivado: File → Export → Export hardware...
• Generates a .HDF file, actually a ZIP file2. Extract the HDF file, get
psu_init_gpl.c and psu_init_gpl.h3. Put them in the U-Boot sources
• in board/xilinx/zynqmp/• and make sure it’s not using the bundled ones (see the source
code)
38
![Page 57: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/57.jpg)
Building the pieces
VivadoH
DF
XSDK
FSBLBSP
pm_cfg_obj
Yocto
ATF
PMU FWU
-Bootpsu_init*
fpga.bit
u-boot.bin
SPL boot.bin
SD card
fpga.bin
39
![Page 58: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/58.jpg)
GPU
![Page 59: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/59.jpg)
The GPU
• ARM Mali-400 MP2• Official software support from ARM/Xilinx:
• Requires a binary blob• Lacks some features
• Open source alternatives:• limadriver.org: reverse engineering, abandoned• github.com/yuq/mesa-lima: new Mesa driver in progress
40
![Page 60: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/60.jpg)
Official software support
• A kernel module• GPLv2• Not in mainline (perhaps never)
• A userspace library• Where the interesting stuff is done• Implements OpenGL ES APIs• Proprietary, binary only• SoC-specific
41
![Page 61: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/61.jpg)
MALI kernel module
• Yocto recipe on meta-xilinx-bsp
• Sources from ARM• + 10 patches
• ZynqMP customizations• Updates for recent kernels
• Works fine
42
![Page 62: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/62.jpg)
libmali-xlnx: find it
• Yocto recipe in the Xilinx fork (GitHub)• Not in the “official” meta-xilinx-bsp
(master on yoctoproject.org)• To stay on the master path:
• Copy the whole recipe from Xilinx’s rel-v2017.4 in your layer
43
![Page 63: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/63.jpg)
libmali-xlnx: fetch the sources
• The recipe fetches from gitenterprise.xilinx.com→ No address associated with hostname• An internal Xilinx repo?
• docs/MALI-binaries has the procedure:1. Register on xilinx.com2. Download a tarball, extract it3. Read the docs there, extract a 2nd tarball4. Extract a 3rd tarball (oh, looks like a bare git repo)5. Set SOURCE_MIRROR_URL to that path
• Small improvement: extract the 3rd tarball in DL_DIR, skipstep 5
44
![Page 64: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/64.jpg)
libmali-xlnx: use it
• Two versions to choose from:• fbdev fullscreen
• Enough for many embedded products• No multi-screen support
• X11• Multi-screen does not seem to work
45
![Page 65: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/65.jpg)
Video Codec Unit
![Page 66: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/66.jpg)
The VCU
• VCU = Video Codec Unit• H.264 and H.265 encoder and decoder in
hardware• Floating in FPGA, not connected to the
hard interonnects
CortexA53 (x4)
FullPower
Interconnects
FPGA
VCU
46
![Page 67: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/67.jpg)
Hardware/Software stack
OpenMAX
gstreamer
Driver
libvcu
VCU FWVCU
libomxil-xlnx
HW
Xilinx, binary
Xilinx, source
3rd party, open
47
![Page 68: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/68.jpg)
Instantiate the VCU
1. Instantiate in Vivado (pretend it’s an IP block)2. Devicetree in Linux (in the linux-xlnx kernel)
48
![Page 69: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/69.jpg)
Software
• All recipes on meta-petalinux• Not usable in the Community workflow or with recent Yocto:
• forces gstreamer 1.8 (Yocto 2.4 has 1.12)• points to patches it does not ship
• some not in poky anymore• some don’t apply on gstreamer 1.8
• But easy to use manually, see next slide
49
![Page 70: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/70.jpg)
Shopping list
1. Take Xilinx specific recipes as-is• kernel-module-vcu_git.bb• vcu-firmware_git.bb• libvcu-xlnx_git.bb• libomxil-xlnx_git.bb
2. Take gstreamer-omx tweaks from Xilinx as-is• gstreamer1.0-omx_%.bbappend• Switches to Xilinx fork• Adds build flags
3. install gstreamer1.0-omx
50
![Page 71: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/71.jpg)
Use it
gst-launch-1.0 -ve v4l2src device=/dev/video0 \! 'video/x-raw,format=NV12,
width=1920,height=1080,framerate=(fraction)60/1' \! omxh265enc \! filesink location=video.h265
51
![Page 72: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/72.jpg)
Conclusion
![Page 73: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/73.jpg)
Conclusion
• Very powerful, flexible hardware• But very complex• Software support is complex (partly avoidable)• Xilinx works to support users
• Slowly moving towards standard tools
• Mainlining effort
52
![Page 74: ARM64 + FPGA and more: Linux on the Xilinx ZynqMP ... · ARM64 + FPGA and more: Linux on the Xilinx ZynqMP Opportunities and challenges from a powerful and complex chip Luca Ceresoli,](https://reader030.fdocuments.us/reader030/viewer/2022040204/5ecd8847d525a4298018ed5b/html5/thumbnails/74.jpg)
Thank you for your attention
Questions?
Luca [email protected]
http://lucaceresoli.net
© Copyright 2018, Luca Ceresoli. Slides released underCreative Commons Attribution - Share Alike 3.0 Licensehttps://creativecommons.org/licenses/by-sa/3.0/
53