KEKB Accelerator Control System KEKB Controls Group 2001-08-29 by T. Katoh.
-
Upload
grant-chambers -
Category
Documents
-
view
222 -
download
0
Transcript of KEKB Accelerator Control System KEKB Controls Group 2001-08-29 by T. Katoh.
KEKB Accelerator Control System
KEKB Controls Group
2001-08-29
by T. Katoh
Control System Control Computer System Timing Control System Beam Gate Control System Personnel Safety Control System Communication System Control Room and Operators Consoles
Contents Construction History of KEKB
Accelerator Control System KEKB Control System Design System Hardware Configuration System Software Configuration Central Control Room
Construction History1994.4 Started System Design for KEKB1995.2 Controls Systems Evaluation1995.5 Committee Recommendation to adopt EPICS1995.8 First Version of Computer System Specificati
ons1996.1 Started to Dismantle TRISTAN Main-Ring1996.3 Final Version of Specifications1996.6 Opened the Bids1997.3 abco1 and 15 IOCs were Installed1997.4 Hardware Specifications were Fixed1997.10 BT Lines were Commissioned1998.3 The Rest of IOCs were Installed
Construction History(contd.)1998.4 Started Software Installation with Link-Pers
ons and Engineers from the Company
1998.12 Commissioning of KEKB Accelerators
1999.3 Commissioning with the BELLE Detector
1999.4 Started Designing PF-AR Control System
2000.10 Installation of abco2 and 10 IOCs for PF-AR
2001.2 Started to Dismantle PF-AR Devices
2001.7 Started to Install or Re-install Equipment
2002.1 Commissioning of PF-AR
KEKB Control System Design
System Requirements
Constraints
Basic Concepts
System Architecture
System Requirements Sources
– Equipment Groups• Magnet and Power Supply• RF• Beam Monitor• Vacuum• Beam Transport• Feedback, Facilities, Physics, etc.
– Operations Group– Accelerator Physicists or Commissioning Group– Controls Group
System Requirements(contd.)
Requirements– All the data that are possible to take should be
taken.
– All the data that are taken should be saved for
later analyses.
– All the operation should be recorded for later
inspection.
– All the machine parameters and information about
the machine components should be stored in the
database.
System Requirements(contd.)
Requirements(contd.)
– The Man-Machine Interface should be Operator-
Friendly.
– The Programming Environment should be
Programmer-Friendly.
– The Overall Response Time to an Operator’s
Request should be less than a Second unless
the Progress of the Process is Indicated.
Constraints Use CAMAC as an Equipment Interface
Schedule– Control Systems Final Design January, 1995– Removal of MR equipment January,
1996– Control Systems Installation September, 1996– Completed Hardware Installation March,
1997– KEKB Commissioning December, 1998
Constraints(contd.) Limited Man-Power
– 9 KEKB Controls Group Members• 3 Physicists and 6 Engineers
– 12 Link-Persons from Other Groups• 2 from Magnet and Power Supplies• 2 from RF• 2 from Vacuum• 2 from Beam Transport• 2 from Beam Monitor and Feedback• 1 from Linac, 1 from Physics
Basic Concepts
Standard Model Architecture
International Standards
– CAMAC, VME, VXI, GPIB, FDDI, etc.
Existing Software Environment : EPICS
Separate Computer-bus and Field-buses
“Link-person” System : Applications
“Out-Sourcing” : 5 People from Companies
Standard Model Architecture Presentation Layer (Server)
– Man-Machine Interface(OPI)– Logging, Analyses, Alarm Displays, Datab
ase, Global Feedbacks Equipment Control Layer (IOCs)
– Data Acquisition, Sequence Control, Local Feedbacks
Device Interface Layer (Field-buses)– Hardware Interfaces
The most Important Points Presentation Layer
– Software Development Tools• User/Programmer Friendly Tools
– Ergonomic Consoles• LCDs, Macintosh’s, PCs, PDPs, Flat Desks
Equipment Control Layer– VMEbus : High Reliability, Flexibility
Device Interface Layer– CAMAC : High Reliability, Well-known
Presentation Layer Operator’s Consoles : X-Terminals Database Management : ORACLE 7 Alarm Generation/Recording Data Logging Data Display : CATV Network Simulation : SAD Program High-Speed Network
– FDDI, Distributed Shared-Memory Network, – 100 Base/TX and 10 Base/T
Gateway to KEK Laboratory Network : acsad
Equipment Control Layer Provides Standard Interfaces
– CAMAC Serial Highway Drivers(Hytec)– MXI-bus Drivers for VXI Main Frames(HP)– ARCNET Drivers, GPIB Drivers(NI), etc.
Computers : VMEbus based IOCs– FORCE CPU-40(MC68040)– FORCE CPU-60(MC68060)– FORCE PowerCore6603(PPC 603e)– FORCE PowerCore6750(PPC 750)
Operating System : VxWorks
Device Interface Layer Standard Interfaces
– CAMAC : RF and Vacuum– VXI : Beam Position Monitors– GPIB : RF, Vacuum, Magnet Readouts, et
c.– ARCNET : Magnet Power Supplies
TV Signal Switches– MODBUS+ : PLCs– RS232: Vacuum Measuring Instruments
System Architecture(1) Functional Configuration
Database
Console Console Console
Simulation Logging Alarm Gateway Display
Laboratory Network
Console
CAMAC
VME
CAMAC
VME
CAMAC
CAMAC
Presentatikon Layer
Equipment Control Layer
Device Interface Layer
High-Speed Network
CATV
CAMAC
System Architecture(2) Actual Configuration
Gateway Display
Laboratory Network
CAMAC
VME
CAMAC
VME
CAMAC
CAMAC
Presentation Layer
Equipment Control Layer
Device Interface Layer
CATV
CAMAC
FDDI Network Switch
X-Term X-Term X-Term X-Term
Alarm, Logging, Database, Console, Simulation
Consoles
F/E F/E
System Configuration
Power Supply UnitARCNET Driver Module
GPIB Controller Module
CAMAC Serial Highway Driver Module
CPU Module
Various Equipment
Magnet Power Supplies
FDDI SwitchServer
Workstation
Consoles & Peripherals
Measuring Instruments
VXI Main Frame
RS232 Module
Central Control Building
Local Control Building
VME-MXIbus Driver Module
CAMAC Crates
Central Control Room
Cisco Catalyst 1200 Bridge Cisco Catalyst 1200 Bridge
Phaser 550JX2 QMS2425
Server abco1 ME/RK 460
Cisco C1400 Concentrator
FDDI GIGA Switch
26 Local Control Rooms
Lattis System
810M
X Terminals for Consoles
VME IOC Timing, etc.
abco1 Server Workstation PA-RISC 7200 Architecture
– 120MHz CPU Clock– 4 CPUs– 2GB of Main Memory– 4GB Hard Disk Drives– FDDI Interface– 20GB RAID Disk
acsad Server Workstations Compaq Alpha Server
– 4 Alpha CPUs of 440MHz– 6 Alpha CPUs of 330MHz– 1 GB Memory– 50 GB RAID– FDDI Network Interface– True 64 Unix Operating System
Former KEKB Control System
abco1
acsad
Laboratory Network
KEKB Control Network94 IOCs
PCs
KEKB Accelerator OperationEPICS Software DevelopmentRelational Database
KEKB Accelerator OperationBeam Optics Simulation
Operators’ Consoles
Real-time Control
KEKB/PF-AR Control System
abco2
acsad
Laboratory Network
KEKB Control Network104 IOCs
PCs
KEKB & PF-AR Accelerator OperationEPICS Software Development
Relational Database
KEKB Accelerator OperationBeam Optics Simulation
Operators’ Consoles
Real-time Controlabco1
abco2 Server Workstation PA-RISC 8500 Architecture
– 440MHz CPU Clock– 2 CPUs– 1GB of Main Memory– 36GB Hard Disk Drives– FDDI Interface– 140GB RAID Disk
IOC Configuration
Power Supply Module
ARCNET Driver
GPIB Controller( NI GPIB 1014)
CAMAC SD( HYTEC VSD 2992)
CPU Module ( PowerCore 6750 )
CAMAC Crates
Measuring Instruments
Magnet Power Supplies
FDDI ー Ethernet Bridge
Cisco Catalyst 1200
Ethernet HUB
Lattis 810M
Measuring Instruments
VXI Main Frame
RS-232C Module
MXIbus Interface Module
Terminal Server
Cisco 2509/2511
System Monitor Module
X TerminalsRS-232CEthernet Ethernet
From CCR
VME IOC System Subrack
– Schroff 16 Slots Subrack Power Supply Module
– 259 Watts, 5V 35A, 12V 5A, -12V 2A– 14 HP Wide, Schroff MPS8-7746
System Monitor Module– Mitsubishi DRSJ-01
• Remote System Reset• DC Power Lines Monitor
VME Modules Used CPUs
– FORCE CPU-40 MC68040 33MHz– FORCE CPU-60 MC68060 66MHz– FORCE PowerCore 6603e PPC 603e– FORCE PowerCore 6750 PPC-750 266MHz– FORCE PowerCore 6750 PPC-750 400MHz
CAMAC Serial Highway Driver– HYTEC VSD 2992
VME Modules Used (contd.) GPIB Controller
– National Instruments GPIB 1014 ARCNET Driver
– Advanet ARCNET-4 VME-MXI Driver
– Hewlett Packard VME-MXI– National Instruments VME-MXI II
MODBUS+ Interface Controller
Number of IOCs and VME Modules
MG RF MO VA BT MV EN TI PH LI Total
IOC 8 8 20 12 4 20 20 2 1 1 96
CAMAC - 8 - 12 2 - - 2 - 1 25
GPIB 8 - - 12 6 20 20 - - - 66
RS-232C 1 - - 12 - - - - - - 13
ARCNET 96 - - - 4 - - - - - 100
MXI - - 20 - 1 - - - - - 21
Total 105 8 20 36 13 20 20 2 0 1 225
Equipment Connected 2,517 Magnet Power Supplies
– 176 ARCNET Segments 800 Beam Position Monitors
EPICS Records on IOCs 242,597 EPICS Records on 94 IOCs 25,147 EPICS Records on IOCMGD06 2,788.5 EPICS Records in Average
48,149 kB Max. Allocated Memory 1,324 kB Min. Allocated Memory 12,142 kB Ave. Allocated Memory
Hardware Summary Use Standard Buses
– Reliability : VME, Compact PCI, CAMAC• Not ISA, EISA, PCI, Proprietary Buses
– Separate CPU Bus from Field Buses• Isolation, Analog Signal Handling
Choose Components Carefully– Power Supplies– Capacitors, Fans, Connectors
Use Ergonomic Equipment
Control Consoles
TRISTAN Consoles
KEKB and PF-AR Consoles
KEKB Consoles Voice Generator using Power Macintosh Multiple Screen Display for Consoles
– DELL Optiplex NX1– Intel Pentium II 400MHz, 1GHz– Colorgraphic Evolution 4– Macintosh with Multiple Video Cards
X-terminal using IBM Network Station– Power PC 406(133MHz)– 48MB Memory– Disk-less / Fan-less System
Control Consoles 18” TFT LCDs(1280x1024 pixels) 16” TFT LCDs(1280x1024 pixels) 14” TFT LCDs(1024x800 pixels) Book-type Personal Computers
– DELL Optiplex NX1 and IBM Network Station– Multi-Screen Display Controllers– Wireless Keyboard/Mouse
Power Macintosh with 2/4 Video Controllers Six 40” Plasma Displays as TV Monitors Easy to change Configuration
Software
Software Two Language Architecture
– TRISTAN Experiences : NODAL and PCL– Interpretive Language for Applications
• SAD and python• Short Turn-around Time• Safe Debuging Environment• Not Dedicated Application Software Programmer
– Compiler Language for Low-Level Software• C or C++ for Low-Level Software• Quick Response Time• Runs Fast• Realizes Everything You Want
Software (contd.) Relational Database Software
– From the Designing Stage– Put All the Accelerator Information
• Cable Connection Lists• Equipment Parameters
– Physical Parameters : Sizes, Weight, …– Calibration Constants, Fitting Curve Parameters, …
• Histories– Installation Dates, Repair Histories, Costs, ….
• Property Numbers and more …
– Generates EPICS Database Automatically
Software (contd.) EPICS Software Toolkit
– medm• Generated from Relational Database
– Ah : Alarm– Ar : Archiving– SNL : State Notation Language
EPICS Software Used medm SAD python
medm Applications
python Applications
SAD Applications
EPICS Databse Files
Linac Portable CA Server
Linac Console
Linac Main Controls
Linac Sub-Controls
Linac Equipment KEKB Equipment
KEKB IOCs
KEKB CA Clients
Linac CA Server
Linac Portable Channel Access
Application Software in Use SAD 141 medm 74 python 42 Miscellaneous 6 Total 263
EPICS Records on IOCs 242,597 EPICS Records on 94 IOCs 25,147 EPICS Records on IOCMGD06 2,788.5 EPICS Records in Average
48,149 kB Max. Allocated Memory 1,324 kB Min. Allocated Memory 12,142 kB Ave. Allocated Memory
Software Summary EPICS R3.13 Toolkit on HP-UX 10.2 medm for Man-Machine Interface X-Window Server Software on the Terminals python for Application Programming SAD for Accelerator Simulation / Operation VxWorks on IOCs Tornado Cross Software Development Sys. CAPFAST for EPICS Runtime Database Dev. ORACLE 7 for Relational Database Portable Channel Access Server for Linac Controls
VME CPU Board Benchmarks
CPU Board Benchmark Test
FORCE Pcore 6604 FORCE CPU-40
CPU Power PC 604e MC 68040
Clock 200 MHz 25 MHz
Memory 16 MB 16 MB
L2 Cache 512 kB --
CPU Load Arising from Scanning Database
PCore 6604 CPU-40
1.0 sec 0.60 % 5.8 %
0.5 sec 1.2 % 12 %
0.2 sec 3.0 % 27 %
0.1 sec 6.1 % 56 %
Transactions Time Required for Channel Access
PCore 6604 CPU-40
ca_search 953 us 1,189 us
ca_put 23 us 109 us
ca_get 58 us 118 us
ca_put&ca_get 76 us 246 us
VME-MXI Problems
VME-MXI Problem HP VME-MXI Driver Module : National I
nstruments Products FORCE PowerCore 6603 or 6750 CPU
Modules : Power PC Tandra Universe or Universe II Chips :
PCI-VME Bus Bridge Chips CPU Module Halts and Never Restarts
Responding to the Reset Signal without Power is switched off
Improvements in ARCNET A HUB Box contains 3 ARCNET HUBs 33 HUB Boxes were Installed for 97 AR
CNET Lines Decreases 102/Day/Line ARCNET “Rec
onfiguration” Errors down to Zero
Configuration of the Test Bench
CP
U m
od
ule
VM
E-M
XI m
od
ule
Sy
ste
m M
on
itor m
od
ule
VX
I-MX
I Co
ntro
ller
MP
X m
od
ule
MP
X m
od
ule
RF
Vo
ltme
ter
CPU Modules tested
FORCE CPU40: MC68040, 33 MHz
FORCE CPU64: MC68060, 66 MHz
FORCE PowerCore 6603: PPC 603, Universe Chip
FORCE PowerCore 6750: PPC 750, 266 MHz, Univ. II
FORCE PowerCore 6750: PPC 750, 400 MHz, Univ. IIB
Normal and Abnormal Bus CyclesNormal Bus Cycle Abnormal Bus Cycle
Abnormal Bus Cycle
Normal and Abnormal DTACK* Signals
PowerCore 6603, with Universe Chip
Normal Abnormal
Abnormal DTACK* Signal
PowerCore 6750, 266 MHz, with Universe II
Normal Abnormal
Realtime Linux
Linux Disk I/O causes 20 - 30 ms of Latency,
possibly Latency can be around 100 ms or more.
Cause:
– Non-preemptive Kernel
– Scheduling Algorithm
– Interrupt Disabling
Non-Preemptive Kernel
Interrupt
Low Priority
High Priority
KernelLatency
Preemptive Kernel
Low Priority
High Priority
Interrupt
Kernel
EPICS iocCore
Network
Channel AccessChannel Access
Run-time DatabaseRun-time Database
Device Access LayerDevice Access Layer
I/O Buses (VME/PCI/ISA)I/O Buses (VME/PCI/ISA)
EPICS under L4-Linux
L4 real-time micro-kernel
XLinuxServer MEDM iocCore
Linux System Calls
Linux Server
Process
Interrupt
L4 Kernel ( IPC )
Linux Kernel
Process
Interrupt
Standard Linux L4-Linux
RT-thread Preempts Linux
Linux Server
RT- thread
Interrupt
L4 Kernel
Process
RT-thread Calls Linux
Linux Server
RT- thread
Interrupt
L4 Kernel
Process
What Happens if RT-thread Calls Preempted Linux?
Linux Server
RT- thread
Interrupt
L4 Kernel
Process
Latency