LLRF 2019 Roger Kalt Paul Scherrer Institut...CompactPCI Serial is a standard for modular industrial...
1
Next Generation Digital LLRF Platform at PSI LLRF 2019 Abstract The High-Intensity Proton Accelerator (50, 150 MHz CW) and the Swiss Light Source (500 MHz, CW and long-pulsed) analog LLRF system require an upgrade to a digital system. As a key requirement, the selected digital processing platform for LLRF systems must fit into the lab’s control system environment. Three different generic digital processing platforms were evaluated with respect to the LLRF requirements. They include CPCI-S.0, MTCA.4 and a full-custom in-house crate design. The pro’s and con’s of the different approach are compared. Roger Kalt – Paul Scherrer Institut Next Platform Evaluation Candidates for LLRF Conclusion / Outlook Out of the two remaining candidates for the new LLRF systems a selection is planned for Q4-2019. Criteria for this selection are the risk (not meeting functionality or time plan requirements), the priority of other PSI-internal non-LLRF projects and the overall cost. Then the goal for 2020 is to implement a prototype LLRF system for lab characterization and installation at the SLS 500 MHz cavity test stand in 2020/21. Revision 2, Roger Kalt [email protected] , 24.09.2019, presented at LLRF Workshop 2019 (LLRF2019, arXiv:1909.06754) Figure 1: MTCA.4 Crate, Schoff/nvent Figure 2: CPCI-S Crate, ADLinkTech Figure 3: DBPM3 Crate, PSI MicroTCA.4 CompactPCI Serial PSI DBPM3 Short Description MicroTCA is a standard for modular embedded computer systems. The standard is governed by the PICMG group. The extension MicroTCA for Physics (MTCA.4) defines a direct rear I/O without passing the backplane for the purpose of integration of the analog frontend into the same crate on rear boards. The use of an additional specific LLRF / RF backplane e.g. for ADC or LO clocks distribution is an optional feature. CompactPCI Serial is a standard for modular industrial computer systems. The standard is governed by the PICMG group. The standard backplane specifies only a part of the connectors and leaves freedom to define custom rear I/O. For more than 200 new BPM in the SLS2.0 and SwissFEL, PSI designed a custom crate that is more cost-effective and performant than standard crate solutions. Up to six RF front-end modules with integrated ADC/DACs can be hot- plugged from the rear side, with coplanar 25Gbps connectors to a single digital-backend board with a Xilinx Zynq UltraSCale+ MPSoC/ FPGA and flexible clocking. Up to 6 RFFEs with 2 JESD204B ADC (Rx) and 2 DAC (Tx) lanes each can be directly interfaced to the back-end. Availability Status LLRF specific COTS hardware is available from some few manufacturers. Small sized MTCA.4 crates are rather new (2019). Standard computer- or I/O type COTS hardware for 3U systems is available from many manu- facturers. Prototype of daughter board with ADC and additional BPM RF frontend available now. Prototype of crate and mainboard available in Q4-2019. Pros • Most hardware required for a 500 MHz LLRF system is available as COTS hardware from manufacturers: Proven working solution. • Allows integrated analog RF frontend – less cabling. • For all the standard computer or I/O type hardware, COTS is available. • Many manufacturers provide COTS for the same functionality. • PSI NUM department has built about 8 years ago a FPGA processing card for the purpose of detector readout and has already experience with the bus system. • Cost-effective low-noise / low-drift crate design (for BPMs and similar systems). • Share development effort for generic hard- ware, BSP, control- and event-system int- egration. • Allows integrated analog RF frontend – less cabling. Cons • It seems that the BSP for the FPGA digitizer boards is one of the weak points: There are alternative commercial BSP providers or labs implemented their own BSP for the same digitizer board. • Only a few providers of COTS hardware with the same functionality. • Own hardware development would need to build up internal knowledge to avoid pitfalls. • LO / CLK generation own development • Currently there is no suitable digitizer board available as COTS hardware on the market: Risk of high cost, late delivery. • We should not try to imitate MTCA.4 with an integrated RF backplane or an integrated analog RF frontend: The effort too high • This then also results in keeping the scheme of separated pizza-boxes for the RF front- ends and cabling in between. • Up to 12 Rx and 12 Tx lanes for JESD204B ADCs and DACs per one crate. Two crates required for a 24-channel LLRF. • The use of ADCs/DACs with parallel IOs would require an additional FPGA on the RF frontend acting as serialize/deserializer. • PSI-specific crate standard, not COTS boards available (yet). Architecture of 500 MHz LLRF Implementation Proposed Selection After the pre-selection for CPCI-S, it is already known that MTCA.4 is not considered for LLRF . The strategy is, to minimize the different type of systems that require long-term support for BSP, OS, drivers, control system integration. Remaining Candidate. Remaining Candidate. Pre-Selection Up to now the PSI bus standard used in the control systems in the machines HIPA, SLS, Proscan and SwissFEL is VME. Analysis of the current installations showed, that the majority of the VME 21- and 7-slot crates are used less than 50%. The majority of the usage is still slave I/O cards for slow and standard type I/O like 0-10V, 4-20mA or 0V/5V/24V. Nowadays such type of standard I/O with a slow sampling rate up to several hundred Hz is possible to implement with industrial type network attached I/O devices like e.g. from Wago or Beckhoff. On the other hand the VME slave boards hardware for such application type are discontinued and no longer available on the market. Bus Platform Evaluation for Control System For the remaining applications and systems which need to be covered by the control system toolbox, a bus platform study group was created. Based on definition of criteria and weights agreed on by the PSI management, CompactPCI Serial is presently being evaluated as possible future bus- and crate-standard for PSI. For this comparable new standard, different manufacturers for COTS hardware like CPU system controllers, XMC carriers, AIO, and DIO cards are already available, and significant future growth is expected. The partly PSI specific selection criteria for the crate standard evaluation include future usage & perspectives, complexity, features and technology. Proposal for detailed evaluation new platform: 1. CompactPCI Serial 2. VPX 3. VME64x 4. MTCA.4 5. MTCA.0/AMC Data Processing Platform for all Applications The use of a popular CPU architecture is an essential point. Compared to the currently used PowerPC in the PSI VME bus system controllers, other architectures like ARM or Intel receive better support from the Open Source community. This is important for the generic part like OS (e.g. Linux kernel) or hardware device drivers (e.g. for network PHY chips). Since the control system already nowadays runs with many EPICS softIOC’s on virtual servers, the Intel X86_64 CPUs allows to minimize additional internal effort for creation and maintenance of dedicated board-support- packages. Selection : Architecture CPU-only systems: Intel x86_64 FPGA-only systems: Xilinx (all families) FPGA+CPU systems Xilinx Zynq US+ For deeply embedded applications which are not located inside the CPCI-S crate but need CPU based data processing, the selection was made to Xilinx Zynq UltraScale+ FPGAs with embedded hardcore ARM cores. Dependent on the required number and type of I/Os, COTS SoM (System-on-Module) can be used. 1x vector modulator Power Supply Front Rear AMC RTM CPU MCH Digitizer Digitizer DWC8VM1 DWC8VM1 CLK Generation RTM MTCA Backplane 8 8 16 ADC / DAQ channel MTCA.4 Crate CLKs: RF reference, LO, sampling LO / CLK generation unit RF reference Digitizer DWC8VM1 1x vector modulator RF reference input Power Supply Front Rear CPU FPGA FPGA FPGA I/O I/O I/O Backplane 16 ADC / DAQ channels CPCI-S Crate 8ch ADC 8ch ADC 8ch ADC Analog Rx Frontend 8 8 16 Analog Tx Frontend +DAC JESD204B 2 DBPM3 crate Bay 1 IQM Bay 2 DWC+ADC Bay 3 DWC+ADC Bay 4 DWC+ADC Bay 5 DWC+ADC Bay 6 DWC+ADC 10x ADC / DAQ channel 1x vector modulator RF reference input Sampling + RF ref clk CLK gen
Transcript of LLRF 2019 Roger Kalt Paul Scherrer Institut...CompactPCI Serial is a standard for modular industrial...
Next Generation Digital LLRF Platform at PSI
LLRF 2019AbstractThe High-Intensity Proton Accelerator (50, 150MHz CW) and the Swiss Light Source (500 MHz,CW and long-pulsed) analog LLRF systemrequire an upgrade to a digital system. As a keyrequirement, the selected digital processingplatform for LLRF systems must fit into the lab’scontrol system environment.Three different generic digital processingplatforms were evaluated with respect to theLLRF requirements. They include CPCI-S.0,MTCA.4 and a full-custom in-house cratedesign. The pro’s and con’s of the differentapproach are compared.
Roger Kalt – Paul Scherrer Institut
Next Platform Evaluation Candidates for LLRF
Conclusion / OutlookOut of the two remaining candidates for the new LLRF systems a selection is planned for Q4-2019. Criteria for this selection are the risk (notmeeting functionality or time plan requirements), the priority of other PSI-internal non-LLRF projects and the overall cost. Then the goal for 2020is to implement a prototype LLRF system for lab characterization and installation at the SLS 500 MHz cavity test stand in 2020/21.
Revision 2, Roger Kalt [email protected] , 24.09.2019, presented at LLRF Workshop 2019 (LLRF2019, arXiv:1909.06754)
Figure 1: MTCA.4 Crate, Schoff/nvent Figure 2: CPCI-S Crate, ADLinkTech Figure 3: DBPM3 Crate, PSI
MicroTCA.4 CompactPCI Serial PSI DBPM3Short Description
MicroTCA is a standard for modular embeddedcomputer systems. The standard is governed bythe PICMG group.The extension MicroTCA for Physics (MTCA.4)defines a direct rear I/O without passing thebackplane for the purpose of integration of theanalog frontend into the same crate on rearboards.The use of an additional specific LLRF / RFbackplane e.g. for ADC or LO clocks distributionis an optional feature.
CompactPCI Serial is a standard for modularindustrial computer systems. The standard isgoverned by the PICMG group.The standard backplane specifies only a part ofthe connectors and leaves freedom to definecustom rear I/O.
For more than 200 new BPM in the SLS2.0 andSwissFEL, PSI designed a custom crate that ismore cost-effective and performant thanstandard crate solutions. Up to six RF front-endmodules with integrated ADC/DACs can be hot-plugged from the rear side, with coplanar25Gbps connectors to a single digital-backendboard with a Xilinx Zynq UltraSCale+ MPSoC/FPGA and flexible clocking. Up to 6 RFFEs with2 JESD204B ADC (Rx) and 2 DAC (Tx) lanes eachcan be directly interfaced to the back-end.
Availability Status
LLRF specific COTS hardware is available fromsome few manufacturers. Small sized MTCA.4crates are rather new (2019).
Standard computer- or I/O type COTS hardwarefor 3U systems is available from many manu-facturers.
Prototype of daughter board with ADC andadditional BPM RF frontend available now.Prototype of crate and mainboard available inQ4-2019.
Pros
• Most hardware required for a 500 MHz LLRFsystem is available as COTS hardware frommanufacturers: Proven working solution.
• Allows integrated analog RF frontend – lesscabling.
• For all the standard computer or I/O typehardware, COTS is available.
• Many manufacturers provide COTS for thesame functionality.
• PSI NUM department has built about 8years ago a FPGA processing card for thepurpose of detector readout and hasalready experience with the bus system.
• Cost-effective low-noise / low-drift cratedesign (for BPMs and similar systems).
• Share development effort for generic hard-ware, BSP, control- and event-system int-egration.
• Allows integrated analog RF frontend – lesscabling.
Cons
• It seems that the BSP for the FPGA digitizerboards is one of the weak points: There arealternative commercial BSP providers or labsimplemented their own BSP for the samedigitizer board.
• Only a few providers of COTS hardware withthe same functionality.
• Own hardware development would need tobuild up internal knowledge to avoid pitfalls.
• LO / CLK generation own development
• Currently there is no suitable digitizer boardavailable as COTS hardware on the market:Risk of high cost, late delivery.
• We should not try to imitate MTCA.4 withan integrated RF backplane or an integratedanalog RF frontend: The effort too high
• This then also results in keeping the schemeof separated pizza-boxes for the RF front-ends and cabling in between.
• Up to 12 Rx and 12 Tx lanes for JESD204BADCs and DACs per one crate. Two cratesrequired for a 24-channel LLRF.
• The use of ADCs/DACs with parallel IOswould require an additional FPGA on the RFfrontend acting as serialize/deserializer.
• PSI-specific crate standard, not COTS boardsavailable (yet).
Architecture of 500 MHz LLRF Implementation
Proposed Selection
After the pre-selection for CPCI-S, it is alreadyknown that MTCA.4 is not considered for LLRF.The strategy is, to minimize the different typeof systems that require long-term support forBSP, OS, drivers, control system integration.
Remaining Candidate. Remaining Candidate.
Pre-SelectionUp to now the PSI bus standard used in thecontrol systems in the machines HIPA, SLS,Proscan and SwissFEL is VME. Analysis of thecurrent installations showed, that the majorityof the VME 21- and 7-slot crates are used lessthan 50%. The majority of the usage is stillslave I/O cards for slow and standard type I/Olike 0-10V, 4-20mA or 0V/5V/24V.Nowadays such type of standard I/O with aslow sampling rate up to several hundred Hz ispossible to implement with industrial typenetwork attached I/O devices like e.g. fromWago or Beckhoff. On the other hand the VMEslave boards hardware for such applicationtype are discontinued and no longer availableon the market.
Bus Platform Evaluation for Control SystemFor the remaining applications and systemswhich need to be covered by the controlsystem toolbox, a bus platform study groupwas created. Based on definition of criteria andweights agreed on by the PSI management,CompactPCI Serial is presently being evaluatedas possible future bus- and crate-standard forPSI. For this comparable new standard,different manufacturers for COTS hardware likeCPU system controllers, XMC carriers, AIO, andDIO cards are already available, and significantfuture growth is expected. The partly PSIspecific selection criteria for the crate standardevaluation include future usage & perspectives,complexity, features and technology.
Proposal for detailed evaluation new platform:1. CompactPCI Serial2. VPX3. VME64x4. MTCA.45. MTCA.0/AMC
Data Processing Platform for all ApplicationsThe use of a popular CPU architecture is anessential point. Compared to the currentlyused PowerPC in the PSI VME bus systemcontrollers, other architectures like ARM orIntel receive better support from the OpenSource community. This is important for thegeneric part like OS (e.g. Linux kernel) orhardware device drivers (e.g. for network PHYchips).Since the control system already nowadaysruns with many EPICS softIOC’s on virtualservers, the Intel X86_64 CPUs allows tominimize additional internal effort for creationand maintenance of dedicated board-support-packages.
Selection: ArchitectureCPU-only systems: Intel x86_64FPGA-only systems: Xilinx (all families)FPGA+CPU systems Xilinx Zynq US+
For deeply embedded applications which arenot located inside the CPCI-S crate but needCPU based data processing, the selection wasmade to Xilinx Zynq UltraScale+ FPGAs withembedded hardcore ARM cores. Dependent onthe required number and type of I/Os, COTSSoM (System-on-Module) can be used.
1x vector modulator
Po
wer
Su
pp
ly
Fro
nt
Rea
rA
MC
R
TM
CP
U
MC
H
Dig
itiz
er
Dig
itiz
er
DW
C8
VM
1
DW
C8
VM
1
CLK
G
ener
atio
n
RTM
MTCA Backplane
8 8
16 ADC / DAQ channel
MTCA.4 Crate
CLK s: RF reference, LO, sampling
LO / CLK generation unit
RF reference
Dig
itiz
erD
WC
8V
M1
1x vector modulator
RF reference input
Po
wer
Su
pp
ly
Fro
nt
Rea
r
CPU
FPG
A
FPG
A
FPG
A
I/O
I/O
I/O
Backplane
16 ADC / DAQ channels
CPCI-S Crate 8ch
A
DC
8ch
A
DC
8ch
A
DC
Analog Rx Frontend
8 8
16
Analog Tx Frontend+DAC JESD204B
2
DBPM3 crate
Bay
1IQ
M
Bay
2D
WC
+A
DC
Bay
3D
WC
+A
DC
Bay
4D
WC
+A
DC
Bay
5D
WC
+A
DC
Bay
6D
WC
+A
DC
10x ADC / DAQ channel1x vector modulator
RF reference input
Sampling + RF ref clk
CLK gen
![PICMG 1.3 Backplane · PICMG 1.3 Backplane 26 PICMG 1.3 BACKPLANE PBPE-06P2 - Fit for Node chassis - Four USB ports 6-slot [PCIe x8 (1, x4 signal), PCIe x16 (2, x8 signal), PCI (2)]](https://static.fdocuments.us/doc/165x107/611a455fbe4d45595d007bf8/picmg-13-backplane-picmg-13-backplane-26-picmg-13-backplane-pbpe-06p2-fit-for.jpg)
