P4080DS U-boot & Linux - NXP Semiconductors - … · P4080DS U-boot & Linux ... Pre-positioning of...
Transcript of P4080DS U-boot & Linux - NXP Semiconductors - … · P4080DS U-boot & Linux ... Pre-positioning of...
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
FTF-NET-F0683
P4080DS U-boot & Linux®
June 22, 2010
Srikanth Srinivasan & Ted PetersNMG Systems and Applications Engineering
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 2
Board-bring up is a key activity for new silicon. Firmware and OS are the first software components run on the reference platforms. Initial bring-up involves debug of silicon, board and the software.
This presentation details the important parts of firmware (u-boot) and OS (Linux) porting/bring-up for P4080DS reference platform. Freescale’s Board Support Package (BSP) provides software that can be modified to run on the customer development systems. The talk will drive home the strategies, steps, concepts, and highlight relevant parts of the code needed most commonly during this effort.
Session Introduction
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 3
►Learn more about the Freescale BSP for P4080/P4080DS
►Details of u-boot and Linux porting, and hypervisor concepts.
• Obtain and build u-boot and Linux from source• Peruse source trees for making changes specific to your
platforms• Identify separation of tasks between SW components• Identify key components of the reference platform as well as
the software• Learn multi-core boot sequence• Hypervisor capabilities and use
Session Objectives
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 4
► Introduction – 5 min
►U-boot – 30 min
►Hypervisor – 25 min
►Linux – 40 min
►LWE – 5 min
Agenda
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 5
SRIOMessage
UnitDMA
PCIe
18-Lane 5GHz SERDES
PCIe SRIOPCIe
CoreNet™
1024-KbyteFrontsideL3 Cache
64-bitDDR-2 / 3
Memory ControllerP4080 Power Architecture™e500-mc Core
D-Cache I-Cache
128-KbyteBacksideL2 Cache
SRIO
WatchpointCrossTrigger
PerfMonitor
CoreNetTrace
Aurora
Real Time DebugSecurity
4.0
PatternMatchEngine
2.0
Queue Mgr.
Buffer Mgr.
eLBIU
M2SB
TestPort/SAP
Frame Manager
1GE 1GE
1GE 1GE10GE
Parse, Classify,Distribute
Buffer
32-Kbyte 32-Kbyte 1024-KbyteFrontsideL3 Cache
P4080 Block Diagram
64-bitDDR-2 / 3
Memory Controller
PAMU
Coherency FabricPAMUPAMUPAMU PAMU
1GE 1GE
1GE 1GE10GE
Parse, Classify,Distribute
Buffer
Frame Manager
Peripheral Access Mgmt Unit
eOpenPIC
Power Mgmt
2x USB 2.0/ULPI
SD/MMC
Clocks/Reset
DUART
2x I 2C
SPI
GPIO
PreBoot Loader
Security MonitorInternal BootROM
CCSR
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 6
EC1 – USB1EC2 – FM1[dTSEC2] 10/100/1G RGMII SerDes Lanes [0-17]:• Slot 1: Bank1 A-B (PEX x2)• Slot 2: Bank1 C-D (PEX x2)• Slot 3: Bank1 E-H (4x SGMII x1, PEX x4, SRIO
x4)• --------: Bank1 I-J (Aurora x2 only)• Slot 4: Bank 2 A-D (4x SGMII x1 or XAUI x4)• Slot 5: Bank 3 A-D (4x SGMII x1 or XAUI x4)
u-boot RCW[srds_prtcl] = 0x10 (RSX configuration)
PCIe1 (5/2.5Gb) PCIe3 (5/2.5Gb) 4x SGMII (1)
FM2 dTSEC[1:4]Debug (5/2.5Gb) XAUI
FM2 10GEC4x SGMII FM1 dTSEC[1:4]
Bank 1 Bank 2 Bank 3
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 7
Getting Started
► Master u-boot repository• http://git.denx.de
► Master linux repository• http://git.kernel.org
► BSP release• http://www.freescale.com/powerbsp• Follow instructions in the BSP documentation to extract software
packages
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 8
P4080 SDK Beta 2.1.0 Components
► Board Support Package (BSP) • GNU Toolchain
gcc-4.3.2, eglibc2.8, binutils 2.18, CodeSourcery release 4.3.74-6
• U-Boot Boot Loader
• Linux 2.6 kernel
Linux device drivers with P4080 Datapath Acceleration Architecture (DPAA) support
• Embedded Hypervisor Software
• Light Weight Executive (LWE)
LWE Libraries (LWE Libs)
LWE Example Applications (LWE Apps)
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 9
Extracting Sources And Building Images
► System requirements• x86 linux with cross-tools
► LTIB $ mount –o loop <BSP.iso> /mnt/p4080ds-ltib$ cd my-ltib-dir$ /mnt/p4080ds-ltib/install
From the installed directory :$ ./ltib –m prep –p u-boot (Sources available in rpm/BUILD/u-boot)$ ./ltib –m scbuild –p u-boot
To configure packages :$./ltib --configure
Local directory
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 10
git
► Version control done through git• SCM of choice for many open-source projects including Linux
► Clone the tree locally• git clone git://git.denx.de/u-boot.git• Replicates the remote tree with change logs and tags
► When ready to contribute1. Commit changes locally2. Create a patch and mail it to the u-boot list3. Peer review4. Patch applied to master tree for everyone to use5. Clean up patch for external world and repeat steps 2-4
► A few keepers for the master tree• Ensures safety of code base while still allowing group contribution and higher
software quality• Very convenient for large distributed projects
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 11
Summary Of Boot Flow From System-Wide Reset
Load hypervisor image and at least one guest OS image. Load device trees for guests.
Hypervisor initializes and then starts one or more guests. It passes them a device tree from which they get information on resources allocated to them (and more).
One guest is designated to do global initialization of BMan, QMan, and FMan since they are used by many guests. Other guests await this initialization.
Guests can bring up their network interfaces.
►uboot
►Guest OS
►Hypervisor
A guest can be authorized to act as a partition manager. It can load and run more guests or reboot them.
Example: Linux
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 12
P4080DS u-boot
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 13
Introduction
► What is u-boot?• Open source firmware for hardware platforms• A boot loader widely used in embedded space
► What does it do?• Brings the board to a sane state out of reset• Load OS image onto board and start OS
► Origins• DENX Software Engineering• More at “http://www.denx.de/wiki/U-Boot/WebHome”
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 14
U-boot Directories
Relevant directories for modifications
u-boot/. u-boot/includeu-boot/board u-boot/cpuu-boot/lib_ppc u-boot/common
fs
./u-boot
board common cpu drivers include lib_* netdisk
rtc toolsdoc examples postdtt
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 15
Boot Sequence
► Reset vector points to FLASH (0xfffffffc)
► Begins execution in FLASH
► Initializes DUARTs
► Checks for information on version numbers, clocks and displays on screen
► Initializes DDR
► Relocates itself to RAM
► Continues execution in RAM
► Releases other cores
► Initializes higher level devices – ethernet, PCIe, SATA, USB, etc.
► Begins command interpreter
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 16
U-boot Memory Map for P4080DS
/** Memory map** 0x0000_0000 0x7fff_ffff DDR 2G Cacheable** 0x8000_0000 0x9fff_ffff PEX 1 Mem 512M non-cacheable* 0xa000_0000 0xbfff_ffff PEX 2/SRIO Mem 512M non-cacheable* 0xc000_0000 0xdfff_ffff PEX 3 Mem 512M non-cacheable** 0xf800_0000 0xf802_ffff PCI IO range 256K non-cacheable* 0xf400_0000 0xf41f_ffff Bman 2M* 0xf420_0000 0xf43f_ffff Qman 2M** Localbus non-cacheable** 0xe000_0000 0xe80f_ffff Promjet/free 128M non-cacheable* 0xe800_0000 0xefff_ffff FLASH 128M non-cacheable* 0xffdf_0000 0xffdf_7fff PIXIS 32K non-cacheable TLB0** 0xffd0_0000 0xffd0_3fff L1 for stack 16K Cacheable TLB0* 0xfe00_0000 0xfeff_ffff CCSR 16M non-cacheable*/
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 17
Initialization
► MMU, LAW setup ► Clocks► Serial► DDR► Misc CPU init► Relocate to DDR space► Flash► PCIe► Misc init/MP init► FM init
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 18
Stack In Cache
► Drivers written in C codeNeed stack space to runLock the cacheUnlock before linux boot
* 0xe800_0000 0xefff_ffff FLASH 128M non-cacheable* 0xffdf_0000 0xffdf_7fff PIXIS 32K non-cacheable TLB0** 0xffd0_0000 0xffd0_3fff L1 for stack 16K Cacheable TLB0* 0xffe0_0000 0xffef_ffff CCSR 1M non-cacheable
Files:cpu/mpc85xx/start.S
Code Snippets:
lis r6,FSL_BOOKE_MAS0(1, 14, 0)@hori r6,r6,FSL_BOOKE_MAS0(1, 14, 0)@l
lis r7,FSL_BOOKE_MAS1(1, 1, 0, 1, BOOKE_PAGESZ_16K)@hori r7,r7,FSL_BOOKE_MAS1(1, 1, 0, 1, BOOKE_PAGESZ_16K)@l
lis r8,FSL_BOOKE_MAS2(CONFIG_SYS_INIT_RAM_ADDR, 0)@hori r8,r8,FSL_BOOKE_MAS2(CONFIG_SYS_INIT_RAM_ADDR, 0)@l
lis r9,FSL_BOOKE_MAS3(CONFIG_SYS_INIT_RAM_ADDR, 0, (MAS3_SX|MAS3_SW|MAS3_SR))@hori r9,r9,FSL_BOOKE_MAS3(CONFIG_SYS_INIT_RAM_ADDR, 0, (MAS3_SX|MAS3_SW|MAS3_SR))@l
::tlbwe
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 19
MMU, LAW Setup
► Configuring memory map
► Files:
include/configs/P4080DS.h, corenet_ds.hboard/freescale/corenet_ds/tlb.c, law.ccpu/mpc85xx/tlb.cdrivers/misc/fsl_law.cinclude/asm-ppc/immap_85xx.h
► Code Snippets:/* *I*G* - Flash, localbus *//* This will be changed to *I*G* after relocation to RAM. */SET_TLB_ENTRY(1, CONFIG_SYS_FLASH_BASE, CONFIG_SYS_FLASH_BASE_PHYS,
MAS3_SX|MAS3_SR, MAS2_W|MAS2_G, 0, 2, BOOKE_PAGESZ_256M, 1),
SET_LAW(CONFIG_SYS_FLASH_BASE_PHYS, LAW_SIZE_256M, LAW_TRGT_IF_LBC),
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 20
SYSCLK► SYSCLK value needed to get started
• Platform frequency
► FPGA registers SCLK0, 1, 2• Input to ICS307 clock generator code
► Obtain the board and ddr clocks
► Files:
include/configs/corenet_ds.hboard/freescale/corenet_ds/corenet_ds.c
► Code Snippets:
/* Calculate frequency being generated by ICS307-02 clock chip based upon* the control bytes being programmed into it. */static unsigned long ics307_clk_freq(unsigned char cw0, unsigned char cw1,
unsigned char cw2){
const unsigned long InputFrequency = CONFIG_ICS307_REFCLK_HZ;unsigned long VDW = ((cw1 << 1) & 0x1FE) + ((cw2 >> 7) & 1);
:::}
unsigned long calculate_board_sys_clk(ulong dummy){
ulong val;u8 *pixis_base = (u8 *)PIXIS_BASE;
val = ics307_clk_freq(in_8(pixis_base + PIXIS_VSYSCLK0),
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 21
Serial► 2 DUARTs ( 4 UARTs) on P4080
• 2 serial outputs on DS
► u-boot provides driver for NS16550 compatible DUARTs• Needs platform frequency
► Serial port very useful for early debug
► Files:
include/configs/corenet_ds.hdrivers/serial/ns16550.c, serial.c
► Code Snippets:
#define CONFIG_CONS_INDEX 2#undef CONFIG_SERIAL_SOFTWARE_FIFO#define CONFIG_SYS_NS16550#define CONFIG_SYS_NS16550_SERIAL#define CONFIG_SYS_NS16550_REG_SIZE 1#define CONFIG_SYS_NS16550_CLK (get_bus_freq(0)/2)
#define CONFIG_SYS_NS16550_COM1 (CONFIG_SYS_CCSRBAR+0x11C500)#define CONFIG_SYS_NS16550_COM2 (CONFIG_SYS_CCSRBAR+0x11C600)#define CONFIG_SYS_NS16550_COM3 (CONFIG_SYS_CCSRBAR+0x11D500)#define CONFIG_SYS_NS16550_COM4 (CONFIG_SYS_CCSRBAR+0x11D600)
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 22
DDR Setup
► Relocating code
► Supports multiple controllers and SPD-based (I2C eeprom) initialization
► Supports Interleaving
► Files:cpu/mpc8xxx/ddrinclude/configs/corenet_ds.hboard/freescale/corenet_ds/ddr.c board specific parameters
► Code Snippets:
#define CONFIG_NUM_DDR_CONTROLLERS 2#define CONFIG_DIMM_SLOTS_PER_CTLR 1#define CONFIG_CHIP_SELECTS_PER_CTRL (2 * CONFIG_DIMM_SLOTS_PER_CTLR)
#define CONFIG_DDR_SPD#define CONFIG_FSL_DDR3
#define CONFIG_SYS_SPD_BUS_NUM 1#define SPD_EEPROM_ADDRESS1 0x51#define SPD_EEPROM_ADDRESS2 0x52
Memory Controller
1
Memory Controller
2
CS0
CS0
CS1
CS1
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 23
Misc CPU Init► Set up local bus memory map
• BRn, ORn
► Enable timebase register
► Files:cpu/mpc85xx/cpu_init.c
► Code Snippets:
void cpu_init_f (void){
volatile ccsr_lbc_t *memctl = (void *)(CONFIG_SYS_MPC85xx_LBC_ADDR);
::#ifdef CONFIG_FSL_CORENET
corenet_tb_init();#endif
init_used_tlb_cams();
/* Invalidate the CPC before DDR gets enabled */invalidate_cpc();
}
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 24
Initialization
► MMU, LAW setup► Clocks ► Serial► DDR► Misc CPU init► Relocate to DDR space► Flash► PCIe► Misc init, MP init► FM init
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 25
Relocate To DDR Space► Relocates u-boot code to top of memory
• For 2GB on P4080DS, in the last 1MB• Set up of stack space, etc.
► Continues to run from DDR after relocation
► Files:cpu/mpc85xx/start.Slib_ppc/board.c
► Code Snippets:
/** void relocate_code (addr_sp, gd, addr_moni)** This "function" does not return, instead it continues in RAM* after relocating the monitor code.** r3 = dest* r4 = src* r5 = length in bytes* r6 = cachelinesize*/
.globl relocate_coderelocate_code:
mr r1,r3 /* Set new stack pointer */mr r9,r4 /* Save copy of Init Data pointer */mr r10,r5 /* Save copy of Destination Address */
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 26
Initialization
► MMU, LAW setup► Clocks ► Serial► DDR► Misc CPU init► Relocate to DDR space► Flash► PCIe► Misc init, MP init►FM init
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 27
NOR Flash► CFI flash support► Reads on POR
• Sets up structures much later in the driver
* Localbus non-cacheable** 0xe000_0000 0xe80f_ffff Promjet/free 128M non-cacheable* 0xe800_0000 0xefff_ffff FLASH 128M non-cacheable* 0xffdf_0000 0xffdf_7fff PIXIS 32K non-cacheable TLB0*
► Files:include/configs/corenet_ds.hdrivers/mtd/cfi_flash.c
► Code Snippets:
#define CONFIG_FLASH_CFI_DRIVER#define CONFIG_SYS_FLASH_CFI
#define CONFIG_SYS_FLASH_AMD_CHECK_DQ7#define CONFIG_SYS_FLASH_BANKS_LIST {CONFIG_SYS_FLASH_BASE_PHYS + 0x8000000,
CONFIG_SYS_FLASH_BASE_PHYS}/* Protect RCW and FMAN microcode sectors for the NOR flash boot bank */#define CONFIG_SYS_FLASH_AUTOPROTECT_LIST { {CONFIG_SYS_FLASH_BASE + 0x8000000, 1}, \
{CONFIG_SYS_FMAN_FW_ADDR, 1} }
This information is used by the driver to detect the flash using CFI query at both possible addresses and populate its info table
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 28
PCIe
► Check for PCIe interfaces on the SOC► Basic PCIe enumeration
• Useful when pcie devices are needed at boot time. e.g. RTC/NVRAM on nVIDIA 1575
/** 0x8000_0000 0x9fff_ffff PEX 1 Mem 512M non-cacheable* 0xa000_0000 0xbfff_ffff PEX 2 Mem 512M non-cacheable* 0xc000_0000 0xdfff_ffff PEX 3 Mem 512M non-cacheable
► Files:include/configs/corenet_ds.hdrivers/pci/fsl_pci_init.cboard/freescale/corenet_ds/corenet_ds.c
► Code Snippets:
void fsl_pci_init(struct pci_controller *hose){
::
}
#define CONFIG_SYS_PCIE1_ADDR (CONFIG_SYS_CCSRBAR+0x200000)#define CONFIG_SYS_PCIE2_ADDR (CONFIG_SYS_CCSRBAR+0x201000)#define CONFIG_SYS_PCIE3_ADDR (CONFIG_SYS_CCSRBAR+0x202000)
::
#define CONFIG_SYS_PCIE3_MEM_PHYS 0xc000000#define CONFIG_SYS_PCIE3_IO_PHYS 0xf8020000
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 29
Initialization
► MMU, LAW setup► Clocks ► Serial► DDR► Misc CPU init► Relocate to DDR space► Flash► PCIe► Misc init, MP init► FM init
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 30
Misc Init/MP Init►Power-on Reset
• Core0 comes out of reset at the reset vector 0xFFFF_FFFC and all other cores are in boot hold off mode
Core0 runs u-boot– After setting BSTRL/BSTAR to secondary core’s start page, core0 kicks off each
additional secondary core– Each secondary core comes out of reset at __secondary_start_page(boot page),
initialize resources specific that that core (caches, MMU, etc.) and enters a spin loop
Core0 loads Linux® image and device tree and boots Linux
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 31
Multicore Boot Architecture
► ePAPR describes specifics on how secondary CPUs are booted for a system with multiple CPUs
► Default boot architecture• The boot program releases all CPUs from hardware reset• One CPU is designated to be the client program’s boot CPU• All other CPUs are secondary and are placed into loop where the CPUs spin,
waiting for a spin table field to change that directs them where to go• Control is transferred to the client program on the boot CPU• When the client program is ready for secondary cores to start, it releases them
by writing the spin table field with the desired address
► The architecture allows for other custom-defined secondary CPU release mechanisms as well
► Reference: ePAPR (Power.org Standard for Embedded Power Architecture™ Platform Requirements (ePAPR). power.org, 2008.)
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 32
SMP Boot Process: U-boot Boot Process
► In the u-boot source, the entry point is the reset vector 0x0_FFFF_FFFC containing a simple branch to _start_e500, which is at the base of default boot page at 0x0_FFFF_F000; both are found in cpu/mpc85xx/start.S
_start_e500() will:
Reset vector call _start_e500()
_start_e500()
1.Enable L1 caches
2.Config interrupt vectors
3.Config MMU and LAWs
4.Config L1D RAM
5.Config local bus
6.Config DDR
7.Relocate to DDR
Prepare device trees
1.Copy HW device tree to DDR
2.Copy HV config tree to DDR
3.Copy hv.uImage to DDR
4.Launch hypervisor
Boot other coresStart command interpreter
1.Setup BSTRL and BSTAR
2.Release all other cores from boot holdoff using BRR and wait for them to boot.
cpu/mpc85xx/mp.c
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 33
Initialization
► MMU, LAW setup► Clocks ► Serial► DDR► Misc CPU init► Relocate to DDR space► Flash► PCIe► Misc init, MP init ► FM init
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 34
FM Init
► FMan Independent mode to get basic networking for tftp downloads
► Needs configuration data► Configuration data resides at 0xEF000000
MAC(RX data)
FM BMI(put MAC data in internal buffers)
FM BMI(allocate internal context buffers)
Enable
Rx Data
RISC(generate DMA to read
data from BDs)RISC(generate DMA to write
data to BDs)
DMA(read data)
DMA(write data)
BMI(read data)
MAC(transmit TX data)
FM BMI(free internal buffers)
END
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 35
FM Init – Independent mode
► Files:include/configs/corenet_ds.hdrivers/net/fm/*board/freescale/corenet_ds/corenet_ds.c
► Code Snippets:
int board_eth_init(bd_t *bis){
::
p4080_setup_serdes();
/* FM 1 DTSEC */if (is_pcie_configured(prtcl, SGMII_FM1)) {
printf("FM1 SGMII\n");#ifdef CONFIG_FM1_ETH0
SET_STD_FM_ETH_INFO(fm1_info[fm1_idx], 1, 0)fm1_idx++;
#endif#ifdef CONFIG_FM1_ETH1
SET_STD_FM_ETH_INFO(fm1_info[fm1_idx], 1, 1)fm1_idx++;
#endif#ifdef CONFIG_FM1_ETH2
SET_STD_FM_ETH_INFO(fm1_info[fm1_idx], 1, 2)fm1_idx++;
#endif#ifdef CONFIG_FM1_ETH3
SET_STD_FM_ETH_INFO(fm1_info[fm1_idx], 1, 3)fm1_idx++;
#endif::
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 36
Initialization
► MMU, LAW setup► Clocks ► Serial► DDR► Misc CPU init► Relocate to DDR space► Flash► PCIe► Misc init, MP init ► FM init
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 37
Other New Pieces
► BMAN, QMAN, FMAN• LIODNs• Device tree fixups for HV/linux
cpu/mpc85xx/portals.c, liodn.c
► CoreNet Platform Cache• Enabled by a flag in
include/configs/corenet_ds.h (CONFIG_SYS_FSL_CPC)
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 38
Building u-boot
►Using LTIB
$ ./ltib –m scbuild –p u-boot
► From command line• Set the compiler toolchain to compile u-boot
Set PATH variable on the build machineUse ARCH=ppc and CROSS_COMPILE flags
make CROSS_COMPILE=powerpc-linux-gnu- ARCH=ppc distcleanmake CROSS_COMPILE=powerpc-linux-gnu- ARCH=ppc P4080DS_configmake CROSS_COMPILE=powerpc-linux-gnu- ARCH=ppc
► u-boot.bin
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 3939
P4080DS NOR Flash Map Cheat Sheet(from Bank 0 perspective after U-Boot DDR relocation)
SW7[1:4] = 0b0111U-Boot PBL @ 0xE0000000
SW7[1:4] =0b0110
SW7[1:4] = 0b0101
SW7[1:4] = 0b0100
SW7[1:4] = 0b0011
SW7[1:4] = 0b0010
SW7[1:4] = 0b0001
SW7[1:4] = -0b0000 U-boot @ 0xEFF80000
0xE8000000 – 0xE8FFFFFF
0xE9000000 – 0xE9FFFFFF
0xEA000000 – 0xEAFFFFFF
0xEB000000 – 0xEBFFFFFF
0xEC000000 – 0xECFFFFFF
0xED000000 – 0xEDFFFFFF
0xEE000000 – 0xEEFFFFFF
0xEF000000 – 0xEFFFFFFF
Bank 2Bank 5 PBL Image
Bank 0
Bank 1
Bank 3
Bank 4
Bank 5
Bank 6
Bank 7Bank 0 PBL Image
Bank 1 PBL Image
BANK 2 PBL Image
Bank 3 PBL Image
Bank 4 PBL Image
Bank 6 PBL Image
Bank 7 PBL Image
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 40
Common (Helpful) Commands
⇒ help
⇒ Environment variables⇒ printenv⇒ setenv⇒ saveenv
⇒ NOR Flash⇒ flinfo⇒ erase <start addr> <end addr>⇒ cp.b <from addr > <to addr in flash> <#bytes>
⇒ Network⇒ setenv [ipaddr][gatewayip][netmask][serverip] <ip address>⇒ tftp <dest address in RAM> <file on server>
⇒ Other⇒ imls - Lists all u-boot recognizable images in flash⇒ i2c - i2c commands to display, read, write i2c devices ⇒ pci - display, read, write PCI/PCIe space
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 41
U-boot Hooks To Launch Hypervisor
► 36-bit addressing• U-boot uses 36-bit addressing but is limited to 4GB
HV uses 36-bit addressing to support multiple partitions and resources• Physical addresses of the IO are moved to the end of the 64GB address
spaceDDR address space starts at 0; now has room to expand
► New commands• Device trees for HW and configuration
Pass HW device tree address to “bootm” commandLocation of configuration device tree is passed “config-addr” env variable
=> setenv bootcmd bootm e8700000 - e8800000=> setenv bootargs config-addr=0xfe8900000
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 42
P4080DS Hypervisor
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 43
Terminology
►SMP – Symmetric Multiprocessing►ASMP – Asymmetric Multiprocessing (aka. AMP)►CAMP – Cooperative AMP►Guest OS – OS running in guest supervisor state under a hypervisor►Hypervisor - Manages globally shared resources, more privileged
than operating systems, enforces system security, virtualizes some resources
►Light Weight Executive (LWE) - Provides a communications mechanism between an application running in a partition and the hypervisor software
►Device Tree - A data structure used for representing a partition’s physical and virtual devices
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 44
Partitioning With A Hypervisor
CPUCPU CPU CPU
partition
AppApp
Memory
partition
AppApp
partition
AppApp
Memory
Memory Memory
I/O I/O I/O I/O
Hypervisor
Shared Cache
I/O
Interrupt Controller
Linux® RTOS Legacy OS
Multicore System
Hardware
►Hypervisor Software• Analogous to role of an operating
system kernel in managing user processes
• More privileged than operating systems• Enforces system security• Manages globally shared resources• Virtualizes some resources– e.g.
interrupt controller, UART
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 45
e500mc Privilege Levels – Guest State (GS) MSR bit
OS
AppApp
partition
User MSR[PR=1][GS=1]
Kernel/Supervisor MSR[PR=0][GS=1]
Hypervisor MSR[PR=0][GS=0]
Under Hypervisor
OS
AppApp
partition
User MSR[PR=1][GS=0]
Kernel/Supervisor MSR[PR=0][GS=0]
Bare Metal
CPU
Memory
I/O
CPU
Memory
I/O
Hypervisor
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 46
Hypervisor Features
virtual CPU (e500vcpu)
services boot services(ePAPR)
Emulation(privileged instructions)
guest operating system
debug stub
Hypervisor
Debug console
hypercalls device tree
system hardware
UART
device tree
directI/O
mux
UART
Doorbells
GPIOPIC
IOMMUByte
ChannelsPartition
Mgmt
►Operating System sees a virtual core plus hypervisor services
• Virtual CPU (like e500mc minus hypervisor features)
• Services via hypercall
• Debug stub interface for debugging guest operating systems
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 47
Virtual CPU
►Like e500mc minus the hypervisor extensions►Full emulation is used►MMU
• tlb access instructions are emulated• Guest receive and manage TLB miss interrupts directly
►Timer• DEC, FIT, watchdog, timebase synchronization
►Cache control• L1,L2
►Intra-partition signaling (msgsnd/msgclr)►Debug services (DBCRx, IACx, DACx, etc)►Core performance monitor
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 48
Virtual CPU – Emulation Example
►Operating system manages it’s own page tables. Hypervisor has no knowledge of OS page table format.
►Hypervisor traps and authenticates all TLB updates
► lwz r3,(r4)
► Application
► OS► (tlb miss handler)tlbwe
► tlb miss ► privilege trap
► Hypervisor
► Hypervisor► validate► tlb writes
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 49
Hypervisor Services – hcalls
►Interrupt controller (MPIC)►Byte-channels – character I/O stream►Inter-partition signaling – doorbell►Partition management
• Start/stop/image-loadingIn future: Create/destroy
• Partition management interrupts
►Power management– change clock frequency, power states►GPIO– supports partitioning of GPIO pins►IOMMU– supports create/destroy mappings
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 50
Device Trees
u-boot
hypervisor
HWDev tree
Guest #2
dynamicallycreated andloaded intoguest memory
Loaded into hypervisor memory by u-boot. Hardware dev tree /chosennode points to HV config tree
Guest #1
Guest Dev tree Guest Dev tree
HV config tree
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 51
Boot Process
►Hypervisor initialization• PAMU initialization• Coherence domain setup– set up LAWS, CSDIDs• Error Configuration– e.g. single bit ECC error thresholds• Driver initialization
DDRCPCCCMUART
• Release secondary CPUs• Partition instantiation/creation
The boot CPU for each partition takes care of partition creation• Control transfer to guest OS
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 52
HV Commands
► Commands list and uses
HV> ?
Commands:
help (?) - Print command usage information
version - Print the hypervisor version
list-partitions (lp) - List partitions
guest-device-tree (gdt) - Guest device tree operation
master-device-tree (mdt) - Display master device tree
paact - Dump PAMU's PAACT table entries
start - Start a partition
restart - Start a partition
stop - Stop a partition
pause - Stop a partition
resume - Stop a partition
HV>
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 53
P4080DS Linux
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 54
Overview
► P4080 Linux Overview• P4080 BSP Linux – Bare Metal and Guest OS• Linux Kernel Configuration• Device Tree• SMP Boot Process
► DPAA and the Linux Ethernet Driver• DPAA Components• Life of a Packet
IngressEgress
• Linux Driver Structure
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 55
P4080 Linux
►Bare metal Linux• Linux runs directly on HW• User state MSR[PR]=1 MSR[GS]=0• Supervisor state MSR[PR]=0 MSR[GS]=0
►Guest OS Linux• Hypervisor SW layer between Linux and HW• Guest User state MSR[PR]=1 MSR[GS]=1• Guest Supervisor state MSR[PR]=0 MSR[GS]=1• Hypervisor state MSR[PR]=0 MSR[GS]=0• Guest OS requires porting to run on Hypervisor
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 56
Bare Metal SMP Linux
Multicore System
Hardware
SMP Linux®
Shared Cache
I/O
Interrupt Controller
I/OI/O I/O I/O
CPU CPU CPUCPU
AppApp
Memory
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 57
Guest OS Linux
Multicore System
Hardware
SMP Linux® RTOS Legacy OS
AppApp
Memory
partition partition partition
Hypervisor
Shared Cache
I/O
Interrupt Controller
I/O I/O I/O I/O
Memory Memory Memory
CPU CPU CPUCPU
AppApp
AppApp
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 58
Porting Guest OS (Linux) to Hypervisor
► Resources controlled by Hypervisor:• Physical memory map• Core: caches, MMU, SPRs (Guest OS sees a virtual CPU via emulation)• Platform: CPC, MPIC, Partion management, Doorbell, Byte-channels,
IOMMU, GUTs, DDR (Guest OS access via HCALLs)
► Direct resources (Direct I/O access by Guest OS)► Device Tree aware (per ePAPR, libfdt library available)► State of HW when Guest OS Runs (per ePAPR)► Hypercall API (HCALLs for shared services, ie VMPIC)► Traps (emulation - vCPU)
► Reference: ePAPR (Power.org Standard for Embedded Power Architecture™ Platform Requirements (ePAPR). power.org, 2008.)
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 59
Building Linux
► LTIB BSP• Configure default LTIB settings
$ ./ltib
• Configure menu for P4080DS, then automatically build $ ./ltib --configure
• In configure menu, select:Select “Configure kernel” to alter the default kernel build configurationSelect “Leave the sources after building” to leave the linux source in rpm/BUILD/linux
► From GIT tree• make CROSS_COMPILE=powerpc-linux-gnu- ARCH=powerpc menuconfig• make CROSS_COMPILE=powerpc-linux-gnu- ARCH=powerpc uImage
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 60
P4080 Linux Kernel Configuration
►The easiest way to create a .config file is to simply copy the appropriate config file in ./arch/powerpc/configs to .config.
• cp arch/powerpc/configs/p4080_ds_defconfig .config
►Typing “make menuconfig” results in a console-based, menu-driven utility that can be used to specify the desired configuration.
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 61
Linux Directory Structure
net
./linux-2.6.xx
Doc. arch block drivers fs include initcrypto
scripts securitykernel mm soundlib
ipc usr
►arch/powerpc/boot/dts/p4080ds.dts►arch/powerpc/configs/p4080_ds_defconfig►arch/powerpc/boot/►arch/powerpc/platforms/85xx/
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 62
P4080 Linux Device Tree
►Baremetal Linux► Use the device tree compiler to build the appropriate dts file in the
./arch/powerpc/boot/dts directory.dtc -I dts -O dtb -R 8 -b 0 -S 0x3000 p4080ds.dts > p4080ds.dtb ORmake p4080ds.dtb (from linux-2.6/)
►Guest OS Linux• The device tree for each partition is generated by Hypervisor from a system
configuration file that maps the system partitioning and HW.
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 63
Kernel Configuration Options - Hypervisor
► CONFIG_P4080_HV - Freescale P4080 Hypervisor platform
► CONFIG_FSL_HYPERVISOR - Freescale P4080 Hypervisor
► CONFIG_VMPIC - Freescale P4080 Hypervisor - Virtualized OpenMPIC driver
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 64
SMP Boot Process: Linux® SMP Boot Process
smp_prepare_cpu()
smp_init()
smp_ops->probe()
cpu_up() smp_ops->kick_cpu()
smp_ops->setup_cpu()
...
in main.c
►In the Linux® kernel, the entry point is _start found inarch/powerpc/kernel/head_fsl_booke.S
start _kernel() is in init/main.c, it will:
_start call _start_kernel()
_start_kernel()
Kernel_init()
setup_arch()
smp_cpus_done()
… mpc85xx_smp_init() {
smp_ops = &smp_85xx_ops;
} ……
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 65
SMP Boot Process: Core1 Start Up (continued)
►Let’s look at smp_mpc85xx_kick_cpu() found in: arch/powerpc/platforms/85xx/mpc85xx_smp.c
smp_mpc85xx_kick_cpu() {…….
/* Get the BPTR */bptr_vaddr = ioremap(get_immrbase() + MPC85xx_BPTR_OFFSET, 4);
/* Set the BPTR to the secondary boot page */oldbptr = in_be32(bptr_vaddr);bptr = (BPTR_EN | (__pa((unsigned)__secondary_start_page) >> 12));out_be32(bptr_vaddr, bptr);
/* Kick that CPU */smp_85xx_release_core(nr);
/* Wait a bit for the CPU to take the exception. */while ((__secondary_hold_acknowledge != nr) && (++n < 1000))
mdelay(1);
/* Restore the BPTR */out_be32(bptr_vaddr, oldbptr);
……}
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 66
SMP Boot Process: Core1 “Kick-Off” Process
►Core 0 kicks off core 1 by setting EEBPCR[CPU1_EN]►Let’s look at smp_85xx_release_core() in
arch/powerpc/mpc85xx/mpc85xx_smp.c
smp_85xx_release_core(int nr){
……/** Startup Core #nr.*/
ecm_vaddr = ioremap(get_immrbase() + MPC85xx_ECM_OFFSET,MPC85xx_ECM_SIZE);
pcr = in_be32(ecm_vaddr + (ECM_PORT_CONFIG_OFFSET >> 2));pcr |= EEBPCR_CPU1_EN;out_be32(ecm_vaddr + (ECM_PORT_CONFIG_OFFSET >> 2), pcr);
} EEBPCR
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 67
SMP Boot Process: Core1 Kernel Code Flow
►Core1 starts up from __secondary_start_page defined in arch/powerpc/kernel/head_fsl_booke.S
• __secondary_start_page:4KThe code in this page must not exceed 1023 instruction. The1024th is a branch instruction.It will initialize I-cache and D-cacheUse TLB1[1] to map 16M memoryJump to __early_start
• __early_startIf it is core1, it will not call start_kernel(), but jump to __secondary_start
• __secondary_startjumps to start_secondary()
►start_secondary() defined in arch/powerpc/kernel/smp.c• Call smp_ops->setup_cpu()• ……• Call cpu_idle()
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 68
Linux DPAA (Ethernet) Driver
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 69
Multicore Datapath Issues and Requirements
►Multicore SoCs, like the P4080, have a number of new requirements related to packet processing when compared to single core SoCs:
• Load spreading of arriving packets across pools of cores for parallel processing
• Packet ordering issues after processing• Pipelined processing of packets using
cores• Share network I/O between cores• “Virtualizes” hardware accelerators• Inter-core communication
CoreD$ I$
Network I/O
HardwareAccelerator
Network
CoreD$ I$CoreD$ I$CoreD$ I$
CoreD$ I$
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 70
More Multicore Datapath Requirements
►Addressing these requirements can lead to new requirements:• Hardware managed queues lead to the need for
hardware-supported active queue management• Network interfaces must able to parse, classify, and distribute (PCD)
frames.
►High-bandwidth network I/O as found on P4080 also drive datapath requirements:
• Queue congestion driven flow control • Resource depletion driven flow control• Hardware buffer management
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 71
Frame Manager1GE 1GE
1GE 1GE10GE
QorIQ™ DPAA Components
► “Infrastructure” components• Queue Manager (QMan)• Buffer Manager (BMan)
►Network I/O• Frame Manager (FMan)
►Hardware accelerators• SEC – cryptographic accelerator• PME – Pattern matching engine
►Cores►CoreNet is not part of the DPAA
but it provides the interconnect between the cores and the DPAA infrastructure as well as access to memory (DRAM)
D$ I$
D$ I$L2$ e500mc
CoreD$ I$
Multi-Lanes SERDES
CoreNet™Coherency Fabric
Sec 4.0 PME 2
BufferMgr
Frame Manager1GE 1GE
1GE 1GE10GE
D$ I$
D$ I$L2$ e500mc
CoreD$ I$
QueueManager
…
…
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 72
FMan/QMan/BMan Ingress Packet Processing
10G 1G 1G 1G 1G
Packets Arriving
Buffer Acquisition Request
Buffer ReferencePackets
in process
FMan
Buffer
QManPacket Data written to main memory subsystem
BMan
Frontside Cache DDR SDRAM
…
References to Packet(Frame Descriptors)
Packet Data Stored in H/W managed buffers
1
2
4
3References to Buffers
Parse, classify/hash, select queue 5
Think ARP request
6 Software dequeues from QMan Software Portal and releases buffer to BMan
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 73
Priority based packet scheduling
FMan Egress Packet Processing
10G1G 1G 1G 1G
Packets Transmitted
FMan Releases Buffer to BManPackets
in process
FMan Buffer
QMan
Bman
Frontside Cache DDR SDRAM
…
4
5
Class scheduler
8 Priority Work Queues
Packet Data read from main memory subsystem
36
Optional packet response
2
Think ARP response
1 Software Acquires buffer from BMan
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 74
Aggregate Driver Structure
DPA net_device Linux Driver(fsl-dpa)
drivers/net/dpa
BMan Linux Driver QMan Linux Driver FMan/Port Linux Driver
FMan/MAC Linux Driver
drivers/hwalloc drivers/hwqueuedrivers/net/dpa/NetComSw drivers/net/dpa
Low Level Driver Low Level Driver
FMan/PCD/Port(FMD) (FMD)
MAC
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 75
Kernel Configuration Options – Bare Metal
► CONFIG_FSL_BMAN - Freescale BMan support
► CONFIG_FSL_BMAN_PORTAL - BMan portal support - compiles support to detect and support BMan software CoreNet portals (as provided by the DT)
► CONFIG_FSL_BMAN_CONFIG - BMan device management - if this Linux image is running natively, you need this option. If this Linux image is running as a guest OS under the hypervisor, only one guest OS ("the control plane") needs this option and the DT node
► CONFIG_FSL_QMAN - Freescale QMan support
► CONFIG_FSL_QMAN_PORTAL - QMan portal support - compiles support to detect and support QMan software CoreNet portals (as provided by the DT)
► CONFIG_FSL_QMAN_CONFIG - QMan device management - if this Linux image is running natively, you need this option. If this Linux image is running as a guest OS under the hypervisor, only one guest OS ("the control plane") needs this option and the DT node
► CONFIG_FSL_QMAN_FQALLOCATOR - QMan FQ allocator - implemented using BMan BP 0 as a system wide simple FQ allocator/manager
► CONFIG_DPA - Freescale data-path (and FMan) Ethernet driver
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 76
Light Weight Executive
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 77
Light Weight Executive (LWE)
►LWE is a set of lower level platform libraries used for development of data plane networking applications
►LWE libraries enable application software to run in OS-less environment.
►Applications based on LWE run in a partitioned environment under the control of Freescale Embedded Hypervisor.
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 78
LWE Application Programming Model
►Single application per partition
►Run to completion
►Thread model
►Global variables
►Per CPU variables
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 79
LWE Applications
►LWE Apps• Provide an example of how to use LWE Libraries• Out-of-box benchmark code• Starting point for users to develop their own high performance data path
code
►LWE Apps Examples• Ethernet Bridging• Hello World• Simple Queueing (Pktwire)• Queueing Torture Test• Simple Crypto • IPSecFwd • FMan Test• Simple PME
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 80
Partition Scenarios► hv-4p
• 2-core Linux partition• Single core Linux partition• 4-core LWE partition• Single core LWE partition• Requires use of Hypervisor
CPUCPUCPU CPU CPU CPUCPUCPU
White Board Shared Memory
Linux LWE
Data Path Shared Memory
Private Memory
Private Memory
Linux LWE
Private Memory
Private Memory
Ethernet Shared Memory
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 81
Session Summary
►U-boot, Hypervisor, Linux for P4080DS BSP• Concepts• Porting• Building and deploying
TM
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 84
What is DPAA and Why do I Need It?
► A multicore SoC must be able to:
• Load spread RX’ed packets across multiple cores to facilitate “parallel processing”• Ensure packets are TX’ed in the same order that they were RX’ed especially after
parallel processing• Support “pipelining” of cores• Support inter-core communication• Share network interfaces between cores
On ingress when the different cores providing different servicesOn egress
• Share hardware offload accelerators (e.g. SEC, PME) between multiple cores• Address performance issues that arise from high bandwidth network interfaces (e.g.
10GE) prevalent on such an SoC• Continue to support key functionality that exists in network interfaces today e.g.
IEEE1588QoSLoss-less flow control
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 85
Addressing Multi-Core Datapath Requirements
► Many of these requirements (“share …”, “pipeline …”, “… inter-core communication …”) result in the need for the ability to define communications paths between all the many blocks in the “datapath”
• Thus … QMan► Load spreading packets and sharing network interfaces require hardware to “parse,
classify, and distribute”• FMan’s PCD functionality
► Effective load spreading requires that multiple “consumers” can share queued data• Thus QMan’s shared queues
► Support for high bandwidth network interfaces (10GE) • FM policing• QMan’s WRED support • BMan – Helps offload overhead associated with managing buffers
► The intent of the DPAA is to address multicore requirements and provide optional software offload
TM