NICA control system, beam diagnostics

V.Andreev, E.Gorbachev, A.Kirichenko, D. Monakhov, S. Romanov, G.Sedykh, T. Rukoyatkina, V.Volkov

VBLHEP, JINR, Dubna

MAC'2015 NICA Control System 2

Control System development goals

• Centralized control of distributed objects• Reliable operation and fast recovery• Provide safe operation, access restriction• Easy support, modification and scalability• Easy and fast development and deployment• Integration of third-party and existing systems

MAC'2015 NICA Control System 3

Plan• Control system layout– Middleware– Front-end layer– Client layer– Service layer

• Control System services• Nuclotron and NICA subsystems development• Several information about beam diagnostics• Conclusion.

MAC'2015 NICA Control System 4

Control System layout

PXI, VME, PC

Devices, PLCs

Front-end layer.PXI, VME, Industrial PCs.

PC

Client layerPC workstations.Windows, Linux

Service layer.Processing apps, services.Windows, Linux

VMsArchivingLinux

Tango DBLinux.

Storage

EthernetCS services

Ethernet

Field bus

Mid

dlew

are

MAC'2015 NICA Control System 5

MiddlewareBased on :• CORBA based distributed object-oriented control

system framework• Multiplatform• Hides network location and technical details• Provides unified interface for hardware equipment or

service:– Commands– Attributes– Properties

• Has services and tools for control implementation.

MAC'2015 NICA Control System 6

Front-end layerNational Instruments PXI – high performance modular platform for

measurements and automation systems

Digital signal processing modules FlexRIO High performance embedded systems

Compact RIO

MAC'2015 NICA Control System 7

Front-end layerTango drivers with runtime configurable properties for NI equipment were developed:• Digitizers and scopes• Analog and digital I/O• Timers and counters• Digital multimeters• RTD input modules• TANGO interface for FlexRIO and CompactRIO.Combination of tango devices can be used to quickly deploy a solution based on almost any NI acquisition equipment.

MAC'2015 NICA Control System 8

Client layer: Desktop & Web appsThe client layer is responsible for presentation of the accelerator complex state to the operator, visualize measurements and provide user interface to fulfill some control tasks.

MAC'2015 NICA Control System 9

Service layer

Contain high-level TANGO device servers, representing whole subsystem:• Collects and process data from low-level

devices• Performs control loop• Provides standard interface to client

applications: commands, attributes

MAC'2015 NICA Control System 10

Service layer: VirtualizationApplications running on service layer are hardware independent and can be virtualized (run on VM).Advantages of Virtualization: easier management, better tasks isolation, resources tuning and utilization, high availability.

Proxmox VE - complete open source virtualization management solution for servers. It manages virtual machines, storage, virtua-lized networks, HA clustering

Proxmox VE features:• Web GUI• Role-based administration• Integrated backup tool• Flexible Storage

MAC'2015 NICA Control System 11

Service layer: Shared storage• Crucial component of virtualization to provide high

availability of VMs• Need good performance , both transfer rate and IOPS

to manage read/write operations of number of VM images

• Have to be redundant to provide data consistency in case of hardware or network failure

• Need to be scalable – should be able to add more storage space without loosing performance

MAC'2015 NICA Control System 12

Service layer: CEPH storage is a distributed object store and file system. Ceph is highly reliable, easy to manage and open-source. It provides redundancy, excellent performance and scalability.We currently use 3 ceph storage nodes based on Supermicro servers:

• Dual socket E5-2600 Intel Xeon• 32Gb RAM• 5 x 600G SAS2 15k RPM drives• 1 enterprise class SSD Intel S3710• LSI 2128 RAID controller with 512Mb cache• 2x1Gb Ethernet card for public network• 2x10Gb Ethernet card for CEPH networks

MAC'2015 NICA Control System 13

Service layer: Scalability

The scalability achieved by adding nodes to cluster. CEPH retains most characteristics at scale or even improves. Adding more nodes to run more VMs:• Capacity increases• Throughput increases• IOPS increase• CPU cores and RAM increaseTemporary impact during re-balancing.

MAC'2015 NICA Control System 14

Service layer: AdministrationAstor/Starter –tools for remote control and monitoring of TANGO applications

MAC'2015 NICA Control System 15

Service layer: Monitoring TANGOSpecial TANGO device is running on every F-End computer. It presents information about computer metrics. Another device monitoring states of every TANGO device of certain subsystem.

MAC'2015 NICA Control System 16

Service layer: monitoringDesktop and web clients to represent states and statuses of TANGO

MAC'2015 NICA Control System 17

Service layer: Access controlAccess control and security are vital to safely run a distributed control system.

Tango provides AccessControl service that allows restricting tango connections to some specific devices only for certain users from certain locations.

Additional authorization checks are necessary on the server side for more secure and flexible access restrictions. We developed our own authorization system which use additional authorization device server.

MAC'2015 NICA Control System 18

NicaControls database

MAC'2015 NICA Control System 19

NicaControls databaseDatabase WEB-manager written on Java (JSF, JPA, PrimeFaces):• Tree view to display

subsystems hierarchy.• Plain tables output. Sorting,

filtration, navigation between tables are provided.

MAC'2015 NICA Control System 20

Nuclotron CS development• Injection control and diagnostics

MAC'2015 NICA Control System 21

Nuclotron CS development• Slow extraction control and diagnostics

MAC'2015 NICA Control System 22

Nuclotron CS development• Betatron tune measurement

MAC'2015 NICA Control System 23

NICA CS components prototypes• Booster thermo diagnostics – 80 channels PXIe based

measurements.• Booster magnetic cyсle control – FlexRIO + custom IO

modules.• Booster RF – integration of third-party CS. • Booster vacuum – integration of third-party Zenon

SCADA control.• NICA injection/extraction control – CompactRIO +

custom IO modules.

MAC'2015 NICA Control System 24

Booster BPM assembly drawing

MAC'2015 NICA Control System 25

Booster BPM prototype

MAC'2015 NICA Control System 26

ConclusionsDistributed, scalable control system infrastructure based on Tango has been designed to provide :• rapid development by using code generation and

drivers for NI equipment and other;• reliable operation by deployment on Proxmox cluster;• easy administration by using hardware and software

monitoring, server-based access control; • easy management of equipment and software by using

NicaControls database.Works on beam diagnostics are in progress now

MAC'2015 NICA Control System 27

Thank you for your attention!