Review of Daresbury Workshop MICE DAQ Workshop Fermilab February 10, 2006 A. Bross.

Review of Daresbury Workshop

MICE DAQ WorkshopFermilab

February 10, 2006A. Bross

Daresbury MICE DAQ WS

The first MICE DAQ Workshop was held at Daresbury Lab in September of 05

Focus was to give an overview of requirements of the experiment and discuss possible hardware and software implementations.

Included Front-end, monitoring and controls

paul drumm daq&c-ws august/september 2005 3

DAQ Concepts

Daresbury Aug 2005

Jean-Sébastien Graulich

IntroductionIntroduction

Normal Detector DAQ is Normal Detector DAQ is synchronised with MICE Beamsynchronised with MICE Beam

We want RF ON and RF Off DataWe want RF ON and RF Off Data (50/50 ?)

We need calibration dataWe need calibration data For each run

We want RF Noise dataWe want RF Noise data Dedicated Run

I SI S Beam MI CE Target

MI CE RF

Beam ON RF OFF

Beam ON RF ON

RF Noise Measurement

Other I SI S Users

Should not happen

Detector Calibration

Daresbury Aug 2005


DDAQ vs CMDDAQ vs CM

Detector DAQDetector DAQSynchronised with the beam

Very fast reaction time (~s)

High transfer rate (~50 MB/s)

Read and Store, no time for on-line processing

Limited User InterfaceRun Control only

(Slow) Control and (Slow) Control and MonitoringMonitoring

Continuous and permanent

Very reliable (Safety issue)

Deal with a lot of ≠ hardware

Read and CheckCalibration, manage alarms at ≠ levels, soft interlocks, take actions, log history, etc.

Extended UISet many parameters, manage complicate initialisation procedures, etc.

Why separate Particle Detector DAQ and Why separate Particle Detector DAQ and Control Sensor DAQ ?Control Sensor DAQ ?

Daresbury Aug 2005


Concept SchematicConcept Schematic

MICE Beam Line

MICE CoolingChannel

MICE

Detectors

Detector

DAQ

Slow Control

Data

Storage

Monitoring

Data Log

Run Control

UI

MICE

User Interface

Environment

RF Phase

The Parts of MICE

Parts is Parts


Beam Line Overview and Controls Needs


Beam Line Overview

• What’s in the beam line?– ISIS synchronisation pulses– Target– Beam Loss– Beam optics elements (power supplies)

• Quads, dipoles and muon decay solenoid• Trim coils, water flow, temperature

– Vacuum system – Warm ion pumps– Muon Decay Solenoid Cryogenics &

Vacuum– Diffuser – id-sensor– Diagnostics – Beam Scintillators (not at

same time as MICE Running


dipole

dipole

quads solenoid

quads quads

Power supplies - control & diagnosticAssume control feedback is done in the power supply“Control” – consists of setting I in each element“Diagnostic”

– output = alarms/mimics etc– input parameters:

Quads I,V – x4 each x9Dipole I,V – x1 each x2Solenoid I,V

Diffuserbar-code reader?

Environmental parameters (marshalled elsewhere?) cooling water temperature fault conditions (power supplies) fault conditions (solenoid cryogenics)

Vacuum – per pump setGauges – three: rough – turbo – systemIntegrated system control through PLC oroff-shelf controller:- pump on/off; valve open/closed- auto start-up & shutdown

v

vv

v

vv

V V

Target

ISIS:-BLM-Cycle information

Solenoid Cryogenics & control system

MICE

DiagnosticsDAQ Control System Hybrid


Target System

• Target motion is locked to ISIS cycle

R/O system

TargetMechanism

ControlSystem

Drive

G-MS

PC SystemLinux / EPICS

Event BusEthernet

Infrastructure

System implemented in MICE hall


• Mix of levels of monitoring & control– Dedicated hardware

• to drive the motors• Read the position• Control the current demand

– Control system• Used to set parameters in hardware

– Amplitude, delays, speed

• Used to read & analyse diagnostic info – position…

• GUI – simplified user interaction• Used to display diagnostics info


Cooling Channel


Physics Parameters• Monitoring system can record many parameters,

but:• What are the critical parameters?• Is the following sufficient?

– Beam Line & Target• Almost irrelevant• Dipole one = momentum of pions• Dipole two = momentum setting of muon beam• Diffuser = energy loss = momentum into MICE

– Poor measurement & p measured by TOF and Tracker• Use Table of settings

– record table-ID when loaded– “All ok” should be a gate condition

– Magnets/Absorber• Table of settings as above

– RF – more dynamic information is needed• High Q system – what does this mean for V & ?• Tuning mechanism – no details yet – pulse to pulse


Superconducting magnets• Current Setting

– Process control – turn on/off/adjust– Passive monitoring – status

• Power supply = lethargic parameters• Transducers = V&I

• Temperature monitoring• Magnetic field measurements

– 3d Hall probes – CANBus – PCI/USB/VME interfaces available

• Feedback system or…• Fault warnings

– Temperatures / operational features• MLS Record Settings, Report Faults, “All ok” in

gate


Absorber (and Focus Coils)

• Absorber & Focus Coil Module– Mike Courthold’s talk on Hydrogen &

Absorber Systems• Process control• Passive monitoring

– Suit of instruments:» Temperature» Fluid Level» Thickness (tricky)

• Fault warnings– MLS Record Settings, Report Faults,

“All ok” in gate


RF (& Coupling Coil)

• RF Cavity– Tuning– Cavity Power Level (Amplitude)– Cavity Phase

• RF Power– LL RF (~kW)– Driver RF (300kW)– Power RF (4MW)

MICE has separate tasks to develop cavity and power systemsbut they are closely linked in a closed loop

The Parts of MICE

Parts is Parts

MICE Safety System

DE Baynham

TW Bradshaw

MJD Courthold

Y Ivanyushenkov

MICE Layout

MICE Hazards

Beam

Radiation

FireExplosionOverpressureMaterial brittlenessSkin burn Radiation

HV=>Sparks

HV

RF RFLH2Tracker LH2 LH2 TrackerParticledetectors

High magnetic field =>High mechanical forcesMagnetic stray field

Photo detectorsand front-end

electronics

Optical fibres

Photo detectorsand front-end

electronics

Optical fibres

HV

TOF

Monitoring the MICE Safety Systems is Important Component of Monitoring System

Controls and Monitoring Architecture

EPICS Experience at Fermilab

Geoff SavageAugust 2005

Controls and Monitoring Group

What is EPICS? Experimental Physics and Industrial Control

System• Collaboration• Control System Architecture• Software Toolkit

Integrated set of software building blocks for implementing a distributed control system

www.aps.anl.gov/epics

Why EPICS? Availability of device interfaces that match or

are similar to our hardware Ease with which the system can be extended to

include our experiment-specific devices Existence of a large and enthusiastic user

community that understands our problems and are willing to offer advice and guidance.

EPICS Architecture

Input OutputController

(IOC)

OperatorInterface

(OPI)

Local Area Network (LAN)

OPI – a workstation running EPICS tools.Operating Systems: Linux and Windows XP.

IOC – platform supporting EPICS run-time database.Example: VME based PPC processor running vxWorks, Linux workstation.

LAN – communication path for Channel Access (CA), the EPICS communication protocol.

IOC Architecture

IOC Database Collection of records Each record represents a system parameter

(process variable, PV)• Unique name• Set of attributes• Attributes and value can be modified

Records must process to do something• An input record can read a value every 10 seconds• A CA write to an output record causes the record to

process• Either input or output, not both

EPICS Records Input

• Analog In (AI)• Binary In (BI)• String In (SI)

Algorithm/control• Calculation (CALC)• Subroutine (SUB)

Output• Analog Out (AO)• Binary Out (BO)

Custom – only needed when existing record types or a collection of existing record types are inadequate

Channel Access

IOC(CA Server)

LAN

EPICS Tool(CA Client)

• One client can connect to many servers.• A channel connects to the value or an attribute of a PV.• Each channel has a unique name = record name + attribute.• CA Services:

• Search – find a channel by name• Get – retrieve a channel value• Put – modify a channel value• Add monitor – notification of state change

MIL-STD-1553B Bus Restricted detector access while running Provides a robust and highly reliable

connection to electronics in the remote collision hall

Developed a queuing driver, device support, and a generic record

12 IOCs with ~70 1553 busses from the counting house to the detector and ~10 busses within the counting house

Describe a Device with Records

TemplateFile

GeneratorFile

EPICSDatabase

EPICStool

Multiple record definitions with substitutionparameters This defines a device.

Define instances of a template. Assign valuesto the substitution parameters in the template file.

DatabaseLoad

ASCII fileInstances of records read by IOC to create record database.

Centralized Database System

OracleHardwareDatabase

TemplateFile

DBCreation

EPICSDatabase

RecordExtract

GeneratorFile

EPICSDatabase

IOC Id

TemplateExtract

GeneratorFile

InstanceCreation

PythonScripts

WebAccess

Support for CFT Device support for the AFE board using the

MIL-1553 driver AFE and AFE power supply templates Record and device support for the SVX

sequencers Expert GUI for downloading

• Does not use COMICS Calibration software

• Runs on processor in VRB crate• Communicates with AFE

Analog Front End

Fiber

Wave guide

Cryostat

VLPC

Analog Front End

Stereo Board

Cassette

MIL-1553Communication

EPICS at Fermilab SMTF

• Superconducting module test facility• Cavity test in October• DOOCS for LLRF – speaks CA• EPICS for everything else• No ACNET – beams division controls system

ILC• Control system to be decided• Management structure now forming

EPICS Lessons Three Layers

• Tools EPICS tools OPI development - CA library

• Applications Build base Build application - combine existing pieces Develop device templates

• Development Records, device, and driver support

EPICS is not simple to use• Need expertise in each layer• Support is an email away

IOC operating system selection


Integration of DAQ & Control systems


Monitoring System


Data Stream?

Controls Header

MHz Data (DAQ)

Controls Footer

DB Pointer

MHz Data (DAQ)

DB Pointer

clock


System Concept?

equipment

ControlPCs

PCs

Master

Monitor & LoggingControlDisplay

Event (Trigger) bus

Ethernet bus

An idea of the DAQ architecture

Bit3

SA

Se

q#

1

SA

Se

q#

2

SA

Se

q#

3

SA

Se

q#

4

SE

RD

ES

#1

SE

RD

ES

#2

SE

RD

ES

#3

SE

RD

ES

#4

SE

RD

ES

#5

SE

RD

ES

#6

SE

RD

ES

#7

SE

RD

ES

#8

VL

PC

#1

L

VL

PC

#1

R

VL

PC

#2

L

VL

PC

#2

R

VL

PC

#3

L

VL

PC

#3

R

VL

PC

#4

L

VL

PC

#4

R

TrackerCollectorUpstream

TrackerCollectorDownstream

TrackerBuilder

PIDBuilder

BeamBuilder

MICEBuilder

MICEStorage

MICEControl

Bit3

SA

Se

q#

1

SA

Se

q#

2

SA

Se

q#

3

SA

Se

q#

4

SE

RD

ES

#1

SE

RD

ES

#2

SE

RD

ES

#3

SE

RD

ES

#4

SE

RD

ES

#5

SE

RD

ES

#6

SE

RD

ES

#7

SE

RD

ES

#8

TrackerControl

Bit3

15

53

15

53

TrackerSlow Ctrl

VL

PC

#1

L

VL

PC

#1

R

VL

PC

#2

L

VL

PC

#2

R

VL

PC

#3

L

VL

PC

#3

R

VL

PC

#4

L

VL

PC

#4

R

Upstream Tracker Downstream Tracker4096ch

4kBytes/event

8MBytes/spill

4kBytes/event

4MBytes/spill

CryostatCtrl/Monitor

Online System

UniDaq

Makoto described the KEK Unidaq system Used for Fiber Tracker test at KEK

Worked well, interface to tracker readout (AFEII-VLSB) would be the same as in MICE

Could be used as MICE online system Needs event builder

Significant issue

ALICE approach• HLT embedded in the DAQ• More than one hardware trigger not straightforward to change trigger

– But DAQ can be very flexible with standard technology. Easy to partition down to the level of the single front-end

• HLT / Monitoring

Alice HLT

Ready-to-use GUIs

Run control should be implemented as a state machine for proper handling of state change

Configure and partition

Set run parameters and connect

Select active processes and start them

Start/stop

Conclusions: never underestimate …

• Users are not experts: provide them the tools to work and to report problems effectively.

• Flexible partitioning.• Event-building with accurate fragment

alignment and validity checks, state reporting and reliability.

• Redundancy / fault tolerance• A proper run-control with state-machine

– And simplifying to the users the tasks of Partition, Configure, Trigger selection, Start, Stop

• A good monitoring framework, with clear-cut separation between DAQ services and monitoring clients

• Extensive and informative logging• GUIs

This represents a significant amount of work

Not yet started!

Conclusions Coming out of Daresbury

Made Decisions!: EPICS Linux VME/PC

Ethernet backbone Event Bus (maybe)

The Start of a Specifications document Has had one iteration

Next

Specify/evaluate as much of the hardware as possible Front-end electronics needs to be fully specified since it

can effect many aspects of the DAQ Crate specs

PCs Various interface cards

Assemble Test System Core system

Must make a decision here (UNIDAQ, ALICE-like TB system,?)– I think we need to make a decision here relatively soon and will

likely involve some arbitrariness.• Who does the work will present a particular “POV”

VME backbone Trigger architecture Front-end electronics EPICS interface

Need experts to be involved

Review of Daresbury Workshop MICE DAQ Workshop Fermilab February 10, 2006 A. Bross.

Documents

Transcript of Review of Daresbury Workshop MICE DAQ Workshop Fermilab February 10, 2006 A. Bross.