LUCID ROD – USER’S MANUAL (v. 0.2 30 sept. ’09)

I. IntroductionThis document describes the implementation of the ROD card, which has to receive data from the Front-end Driver boards (LDA) devoted to the MAPMT acquisition. The board was produced from the Weizmann Institute for the TGC muon system and described elsewhere [1]. Since the format of the Lucid Front-end data is different than the TGC’s one, some modifications of the firmware implemented on the FPGAs have been performed.Along this document, all changes with respect to [1] will be pointed out, with main focus on:

1. how to configure the ROD and prepare for run;2. which procedures could be useful for controlling both board and data during the run;3. format of output data;4. selected notes on lab tests.

There’s no hardware description in this document; for any missing information, refer to [1].

II. Description of VME Board register

VME interface is standard A32/D32 slave and interrupter. Board base address is set with two rotary switches. IRQ level is set in a programmable register (to check !!).

At the power-on of the VME crate, only the Service Cpld is already configured (programmed with Eprom). Registers are described in Appendix, Table 2.

The first step is to perform a cold-start (§ ), so configuring both FPGAs on the mezzanines (2 mezzanines with 2 FPGA each one) and then the main FPGA hosted on the main card. As described in the next section, when configuring the mezzanine FPGAs, a dedicated firmware is loaded in the main FPGA, with a restricted number of VME registers (Table 3). When the Main FPGA is configured, all its internal registers are accessible starting from offset 0x10’0000 with respect to the board base address. Registers are listed in Table 4; only the highlighted in green will be described in the following of this document. The others are reported only for debugging purposes or for future implementation.

a. SR1

The meaning of bits of the SR1 register is shown in Table 5. Bits 0 to 15 refer to Service Calls (i.e. actions to be taken by the VME monitoring software). Each bit should trigger a given action in the monitoring software. Service Calls Requests could also generate a VME interrupt (under test), whose IRQ level can be set in Register Set 0 IRQ register (Table 2). See Table 6 for a short description of the implemented Service Calls.

b. FFR

Front-End FIFOs status (implemented in the mezzanine FPGAs). Lower 16 bits indicate FIFO busy: the number of data has excess a given threshold (3/4 of the FIFO depth) ; events are still processed, but the RODBUSY signal is issued. Higher 16 bits indicate FIFO overflow: FIFO are full and no more data will be accepted.

Bit correspondence is shown in Table 8

c. ERRS

Error flags of the last processed event. See Table 7 for bit description.

d. FEMT

Lower 12 bits indicates empty flags of the Front-End FIFO (Table 8). Higher Bits are Select Link FIFO status (Table 9). The Select Link is the name of the high-speed bus connection between the mezzanine FPGAs and the main FPGA.

e. FWVER

Firmware version: higher 16 bits are the mezzanine FPGA version; lower ones are the main FPGA version. 16-bits groups are divided in two 8-bits subgroups (higher 8 bits are major version; lower one are minor revision).

f. BTIME

Counter of the accumulated RODBUSY time. Each tick is 216 * (1/FLHC), where FLHC is the bunch collision frequency (about 25 ns). Consequently, each tick is about 1.6 ms.

g. L1AR

It’s the last L1A received (the 32-bit extended version: lower 24 bits are L1A counter; 8 higher ones are the ECRa counter).

h. FEOUT

Front-End link status during data-taking. The main FPGA can assert a link-timeout when no data are received from the LDA after about 1.6 ms (more exactly: (216 – 1)* (1/FCK) where FCK is the clock frequency of the main FPGA, successfully tested up to 40 MHz). Moreover, if the main FPGA received Front-End data with severe errors could disable the corresponding link (flagged as dropped); this option is actually disabled but bits are left for future implementation (offending errors are also to be defined). See Table 10 for bits definition.

i. STALD

Stalled pipe flags. These flags are set when a firmware process is trying to read from an empty pipe (main FPGA). To be used as a diagnostic tool in case of unexpected device behaviour.See Table 11 for bits correspondence.

j. FERDY

Higher 16 bits are left for debugging purposes. Lower 12 bits are the Ready Flag, issued by the G-Link receivers hosted in the mezzanines. Bit definition is shown in Table 12.

k. Output Statistic Registers

a Event Counter Reset.

These 4 registers are read-only. L1AX is the L1A number of the last event transmitted to the ROS. VMFW is the total number of 32-bit words transmitted as formatted events to the VME. L1AP is the L1A number of the event currently in process. NEVS is the total number of events processed (eventually not transmitted yet).All words are formatted as 32-bit unsigned integer.

l. FCR1

This register hosts control bits useful for setting several different functionalities (Table 13); all of them are read/write. CR1_TTCENA enables processing of TTC signals; when it’s disabled no L1A are processed.

Four bits are implemented to control the S-Link: CR1_SLNK_SVC enables the corresponding Service calls when link is down; CR1_SLNK_TSTCONF enables a continuous S-Link test (when started from

CMR1_SLNK_TST bit in CMR1 register); if disabled the link test runs once; CR1_OUTLENA enables the output formatted events to be sent through S-Link to the ROS; CR1_SLINK_FORCE allow sending output events regardless the S-Link connection status.

CR1_FMTENA must be set in order to enable building of formatted events.

Recording of formatted events in the VME pipe is implemented with some logic described in the following together with the description of the related control bits.First of all the event must fulfil at least one of the two following conditions (evaluated in parallel):

CR1_ERRFMT is enabled and the event has no masked errors (set in EMUTE register);1. CR1_FLTFMT is enabled and the event is selected by firmware (in this first firmware

version all events are selected by default; in future development some kind of events could be selected through proper Front-End data elaboration.

If none of these conditions apply, then the event Trigger Type is evaluated: if CR1_TTYPE_INCL is disabled event is selected, otherwise it should have trigger type equal to the TTACC register setting (if TTACC = 0x0 all trigger types are considered).Finally if CR1_ALLFMT is set the event is put to the VME output pipe, otherwise it is put only if the pipe is empty. Such a way the VME output pipe never fills or block and only one event will be available until the pipe has been read from VME.

Few more bits control the flow of Front-End (i.e. raw) data: CR1_INCLRAW must be set in order to include raw data in the formatted event, while CR1_ONLYRAW forces to include only raw data. In this firmware version their behaviour is identical and only one of two could be set if raw data are wanted to be included. CR1_DISRAWCHK disables some basic checks on raw data format (if failed data are discarded) and they are included in the event as they are received in the Main FPGA.

CR1_INFOOFF is supposed to be enabled and in future versions it will be as default.

CR1_IGNORE_ID select consistency check of Bunch Counter in raw data: if set, only BCs from Front-End devices and confronted, otherwise they are checked also against the internal one (BCOF register should be used to put a given offset). In case of inconsistency the correspondent error flag is issued.

m. CMR1

This registers contains only-write bits. Once set (to 1) they execute a given action and then are reset to 0 when done. Register content is shown in Table 14. History buffer is a VME pipes that, when activated, collects all output events data until a significant error occurs; in this case writing on the pipe is stopped. It should be used mainly for debugging purposes.

n. SVCA

This register has to be used to acknowledge the Main FPGA that the Service Call correspondent to the written value has been satisfied.

o. BCOF

When the Main FPGA receives (from TTC) a Bunch Counter Reset, the internal Bunch Counter starts from the value of this register. It should be set in order to synchronize Bunch Counters of ROD and Front-End devices.

p. RUN

It sets the Run Number. When a Run Number is written all Output Statistic registers (§ k) are cleared as well as the Orbit Counter.

q. TGCC0/1

This two registers are used to configure the Front-End links. 4-bits are used for each link; the correspondence is shown in Table 15. 0xF value means that the link is disabled. Valid values range from 0 to 9. For the first LDA links, the numbers must be set equal to the GOL ID of the connected link minus 1 (so ranging from 0 to 4); for the second LDA connected the numbers should be increased by 5 (so ranging from 5 to 9).

r. EL1ID

This register will be used in future firmware revisions in order to handle stop and resume of the data acquisition (in case of loss of synchronization during the run). It will manually set the 8 most significant bits of the L1A counter.

s. SVCR

This register contains the code of the requested Service Call.

t. RODID

It’s the 5-bits unique ID code of the ROD. It is inserted in the formatted output event.

u. EMUTE

Register used to disable issuing of errors in data. Bit correspondence is referred to Table 7; each bit mutes the error with the correspondent code.

v. EMUTS

Register used to disable issuing of global errors in data transmission. Actually only one error is available, with code 4: the output formatted event exceeds the VME output pipe depth (4096 32-bit words), then it is skipped.

w. TTACC

It is used to select a given Trigger Type. Only events with this Trigger Type are sent to the VME output pipes (0x0 means “accept any type”).

x. Occupancy Counters

Several pipes are used in the firmware in order to transfer data between processing threads as well as between different clock domains. Occupancy counters of the main pipes are accessible from VME. They can be used both for monitor purposes and to know how many data are available for VME read-out.List of the counters and of the relative widths are in Table 4.

y. VME accessible Pipes

Seven pipes are accessible (read-only) from VME: 4 raw data (one each Mezzanine FPGA), the Output event FIFO (for debug or slow monitoring), the Exception FIFO (pipe with all the errors occurred), history FIFO (when enabled, all output event data are put in; in the meanwhile the oldest data are cleared so that FIFO never fills).

III. Start-up ROD configuration (a.k.a. Cold Start)

Sequence of operation from scratch:1. Disable TTC and Force RodBusy (CR0 reg - Table 2);2. Set Clocks frequency (CLK reg – use “LUCIDrod_V1_Clock_Set” );3. Enable Cloks (CLK reg – use “LUCIDrod_V1_Clock_On”);4. Load in the Main FPGA a special configuration, used to configure Mezzanine FPGAs (use

“LUCIDrod_V1_loadDesign”);5. Set to 0x1 CFGMEZZ reg (Table 2);6. Load for each Mezzanine the same FW in both FPGA (use

“LUCIDrod_V1_loadDAUGDesign”);7. Set to 0x0 CFGMEZZ reg (Table 2);8. Load in the Main FPGA the final working configuration (use

“LUCIDrod_V1_loadDesign”);9. Enable TTC and keep active RodBusy.

Notes:1. Clocks frequency are set by properly configuration of two programmable clock generators

through the CLK register [2]; the “LUCIDrod_V1_Clock_Set” function takes as main argument an unsigned integer value to be loaded in clock generator, correnspondant to a given clock frequency and a given maximum allowable jitter. Data to be loaded as a function of frequency could be calculated by looking at the datasheet, but, since it’s not trivial the IDT provides a web tool in order to make calculation [3]. Values used and successfully tested in laboratory are:

1st clock (Main FPGA clock) : 40 MHz

(configuration data: 0x000801) => generated with:o input frequency = 25 MHz;o accuracy = 0 ppm;o clock 2 output = REF;o output driver = TTL;o crystal load cpacitance = 00.

2nd clock (Chipscope clock) : 10 MHz(configuration data: 0x000204) => generated with:

o input frequency = 25 MHz;o accuracy = 0 ppm;o clock 2 output = REF;o output driver = TTL;o crystal load cpacitance = 00.

IV. Prepare for Run

Sequence of operation at the beginning of a new data acquisition run: Reset all FPGAs (write 1 in the CMR0 register); Configure TGCCs registers (link configuration); Configure RODID,EMUTEE,EMUTS,TTACC,BCOF registers; Set the run number (RUN register); Set the CR1 register; De-assert forced RODBUSY (CR0).

V. Output Data Format

The output data format is compliant with the Atlas DAQ format, version 3.1 [4]. Detailed ROD output data format is given in [5]. Since the document is not yet officially published, the format description is fully reported in the following of this document, pointing out all significant differences with respect to the one of the TGC system.

All output words are 32-bit wide and logically grouped as Frame, Header, Status, Data and Trailer.They are sent in the following order:

1st word (Frame): 0xB0F00000 sent in control mode; Header (9 words); Status (5 words); Data (variable number of words); Trailer (3 words); Last word (Frame): 0xE0F00000 sent in control mode.

Detailed description can be found in Table 16, where information on Header and Trailer fields are in the Comments column.Status and Data field are described hereafter.

a. StatusFirst Status word must be equal to zero if the event has no errors. Specific Error Bytes are put to 0 in this firmware version. Concerning the Generic Error Condition only two of them are eventually generated:

Bit 2 (i.e. word= 0x4): timeout occurred in at least one of the Front-end links; the fragment is incomplete;

Bit 3: Data may be incorrect; at least one of the errors listed in Table 7 occurred.

The second Status word reports which errors occurred in the event. Any bit corresponds to a given error code (see Table 7). For instance, if error 24 occurred in the event, the 24 bit (first is 0) of this word is raised (0x01000000).

The third Status word is divided in two sections: VME filter condition (27:16 bits) and Front-End timed-out (11:0). Bit i corresponds to Front-End link i in the TGCCs registers. In this firmware version no additional filtering applies on Front-End data different than data stream existence.

Fourth Status word is divided in two sections: Local Status (only higher bit, 31, is significant and set equal to CR1_IGNORE_ID in CR1 register) and Presence bits.Presence bits indicate which kinds of data fragment are present. In this version only Raw Data Fragment (Fragment ID = 1) are recorded to the output, so that only bit 1 can be set to 1. If presence bits are all zero, no Data words are in the event.

Last Status word is the orbit count of the formatted event.

b. Data

First Data words are counters correspondent to the fragment present and reported on the Status element.

Raw Data are formatted in the following way: Higher 16 bits of the first Raw Datum are the number of 16-bit words correspondent to a given Front-End link. The Lowest 16 bits are the first Raw Datum received. Then, Data should be read higher 16-bits first to reproduce the same order when sent from the Front-Link.Let’s see an example.Front-end data have the format shown in Figure 1 (a detailed description can be found in [6]). Header and Footer are removed from the stream.

Figure 1: Front-End Data format (each block is a 16-bit word; temporary sequence of the transmission is from left to right)

Data block has a programmable number of Frames; each of them is made of two 16-bit words called Stream. Suppose a Front-End link is programmed so that 3 frames are sent every L1A, then data are formatted in 32-bit words as shown in Table 1.

Table 1: example of formatted raw data

WORD Higher 16 bits Lower 16 bits

0 16-bit Counter ( = 9 in this example) Source ID1 FRAME 0 – Stream 0 FRAME 0 – Stream 12 FRAME 1 – Stream 0 FRAME 1 – Stream 13 FRAME 2 – Stream 0 FRAME 2 – Stream 14 GOL+L1BCN CHKSUM

VI. Firmware versionsActual firmware version:

Main FPGA:o file “lucidrod_top11_v3.bit” – created 1/07/2009 at time 11.10.35

Mezzanine FPGAso file “rx3_v5_02_v3.bit” – created 25/06/09 at time 12.03.09

Main features: 100 MHz Select Link bus between mezzanine and Main FPGA; Successfully tested with 40 MHz Main FPGA clock; S-Link output not tested yet (set to 40 MHz link clock).

VII. Tests in Bologna Lab

VIII. Various Notes

The programmable clock generator circuit has been designed in order to be automatically disabled when the temperature inside the VirtexII Main FPGA (measured with the current drawing through the factory temperature-sense diode), raise the +95 °C threshold.

The L1A led (on the front panel) blinks only when BCR are received, otherwise it remains stuck after the first L1A.

Maximum rate for the HOLA S-Link TX is higher than 32-bit x 40 MHz (tested with UCLK faster than 50 MHz), but the limitation is the maximum bandwidth of the S-link optical fibers: 32-bit x 50 MHz .

IX. References1. “Endcap Muon Trigger System: Read-out Driver Design Overview” L.Levinson, Weizmann

Institute of Science, Atlas TGC group, 2 january 2009 (http://cern.ch/atlas-tgc/doc/TGCROD_design.pdf )

2. “Serially Programmable Clock Source - ICS 307 datasheet” IDT (http://timing.idt.com/icscs/PartSummary.aspx?id=b5307451-82d2-4de0-803b-f23dc018f102&name=ICS307-01/02&mode=short)

3. “Using the ICS307 Programmable Clock”- IDT Web Form (http://timing.idt.com/calculators/ics307inputForm.html)

4. “The raw event format in the ATLAS trigger and DAQ” Atlas Trigger and DAQ, ATL-D-ES-0019, version 4.0c , 09 February 2009

5. “Endcap Muon trigger system: ROB input buffer format” ATLAS Israel TGC group, version 3.1, 19 August 2008 (note in preparation) (http://cern.ch/atlas-tgc/doc/ROBformat.pdf)

6. “Modifiche FPGA DBT per la trasmissione a 800 MHz”, DBR note in preparation, C.Baldanza, 11 Sept. 2008

A. Appendix: Internal Registers

Table 2: Register Set 0 (Service Cpld)

offset (hex) R/W name bit sub-range comment # bits00'0000 R SR0   Status register 0 8

      0 /INIT 1      1 FPGA config done 1      2 interrupt pending 1      3 RODBUSY 1      4 TTC ready 1      5 output S-Link ready 1      7:6 from FPGA 2

00'0004 RW CR0   Control Register 0 8      0 interrupt enable 1      1 force_RODBUSY 1      2 microwire port for unique ID 1      3 TTC_SDA 1      4 TTC_SCL 1      5 TTC_ENA 1      7:6 to FPGA 2

00'0008 W CMR0   Command register 0 3      0 reset board 1      1 start configuration 1      2 test interrupt 1

00'000C W CFG   Configuration Data Register 8      7:0 data[7:0] 8

00'0010 RW PHYS   Ethernet Phisical Config 4      0 /reset 1      1 coma (deep sleep) 1      6 maintenance port clock 1      7 maintenance port data 1

00'0014 W CLK   Clock Control Register 6      3:2 Enable Clock[1:0] 2      1:0 data[1:0] 2      5 strobe 1      4 programming clock 1

00'0018 R BVER   Board Version 400'001C RW IRQ   Interrupt request level 400'0020 R SW   Software INT  00'0024 RW IVR   Interrupt vector register 800'0028 R SERR   TTC Single Error 800'002C R DERR   TTC Double Error 800'0030 RW CFGMEZZ   Set mezzanine mux for configure 1

      0 control bit 1

Table 3: VME registers for the special Main FPGA firmware used for configuring the mezzanine FPGAs

Offset (hex) R/W Name # of Bit Decsription

00’0400 R RXCFGSR 4 RX mezzanine board configuration status register

00’0404 W RXCFGCMR 4 RX mezzanine board configuration command register

00’0408 W RXCFGA0 8 configuration register (daughter A0)

00’040C W RXCFGA1 8 configuration register (daughter A1)

00’0410 W RXCFGB0 8 configuration register (daughter B0)

00’0414 W RXCFGB1 8 configuration register (daughter B1)

Table 4: Register Set 1 (internal to main FPGA) ; only green flagged registers are used throughout this document

offset (hex) R/W name comment      Read Only

10'0000 R SR1 FPGA status Register 1 (Table 5)10'0004 R FFR FE Fifo full indicators: hi 16 ovfl, Lo 16 busy (continuosly monitored) B1[2:0] @ .. A0[2:0]10'0008 R ERRS Errors flag of last processed event (same as FormatOut Status second word)10’000C R FEMT Low 12 bits: FE FIFO empty flags (4 rx * 3 FEchan); HI : 4 SLMainReady @ 4 SL empty @ 4 SL full10’0010 R DBGR debug register (actual: Hi 4 bit rx_svc(inactive now) - 10 bit ldb_done - 4x4 ldb_active)10'0014 R FWVER Firmware Version: hi 16 RX ver; lo 16 main board version (hi8 : main version, low8 minor version)10'0018 R DIAG2 2nd set of diagnostic bits (do_RX_link_state 8x2 bit @ do_RX_link_irx 8x2 bit)10'001C R BTIME accumulated RODBUSY time (in microsecs) 32 bits=1.2 hours10'0020 R GBSR Gigabit Ethernet status register (actually disabled)10'0024 R L1AR counter with Last L1A received10'0028 R FEOUT high 16: FEchan_dropped[12] (never in this version); low 16: FEchan_timedout[12] (timed out are disabled!);10'002C R STALD stalled pipe write flags (i.e. trying to read an empty pipe)10'0030 R FERDY SL state (2 bit x SL for debugging) + FE link ready flags (from G-link chips): 12 bit10'0034 R DIAG diagnostic10'0040 R LINF0 last info msg issued from frag proc0, for debugging10'0044 R LINF1 last info msg issued from frag proc1, for debugging10'0048 R LINF2 last info msg issued from frag proc2, for debugging10'004C R LINF3 last info msg issued from frag proc3, for debugging10'0050 R LINF4 last info msg issued from fragment scheduler, for debugging10'0054 R LINF5 last info msg issued from output formatter, for debugging

      Output Statistics - Read Only 32 bit10'0100 R L1AX Last L1A transmitted to ROS10'0104 R VMFW number of formatted event words transfered to VME10'0108 R L1AP L1A being processed by frag processors10'010C R NGIG number of events transfered out via the gigabit ethernet link10'0110 R NEVS number of events processed

      Read/Write/Signals (valid bits ; 32 if not indicated)10'0200 RW FCR1 Control Register 1 (see FCR1 sheet)10'0204 WS16 CMR1 Command register 1 (see CMR1 sheet)10'0208 RW4 MXR Memory Extension Register (extra 4 bits of memory to/from SRAM)10'020C WS4 SVCA acknowledge that requested Service Call has been executed10'0210 RW12 BCOF BC offset10'0214 RW RUN run number to be included in formatted events (must be written and the beginning of the acquisition)10'0218 RW TSTN reserved10'021C RW GBCFG Gigabit Ethernet MAC config register, lo byte is lo byte of MAC address10'0220 RW TGCC0 FE connection config reg: 4bits each: ch7-GOL ... ch0-GOL 0xF means disable Felink (GOL range 0 to 9)10'0224 RW16 TGCC1 TGC FE connection config reg: 4bits each: ch12-GOL ... ch8GOL (GOL_CNTRL = 4 and 9)

10'0228 RW reserved  10'022C RW8 EL1ID Extended Level-1 ID10'0230 R8W0 SVCR Service Call request Register (accessed from CPLD?)10'0234 RW5 RODID ROD ID (MUST be decided by us!!)10'0238 RW EMUTE bits to mute error reporting for each event error check10'023C RW EMUTS bits to mute error reporting for each system error check10'0240 RW8 TTACC bit on indicates that events with a trigger type with this bit on will be sent to monitoring

      Occupancy Counters - Read Only 32 bit10'030N R IFWCN occupancy counter fro FE link N (N = 0,...,9,a,b) (16 bits)10'0330 R XFWC Exception log FIFO item counter (12 bits) items are 32-bit words10'0334 R EFWC Event ID FIFO item counter (8 bits) 10'0338 R YFWC HistorY FIFO item counter (12 bits) items are 32-bit words10'033C R reserved  10'0340 R OFWC output event FIFO item counter (12 bits) items are 32-bit words10'0344 R reserved  10'0348 R reserved  10'034C R reserved  10'0350 R EHFWC event header FIFO item counter (12 bits) items are 32-bit words10'0354 R R0FWC Raw event FIFO A0 item counter (12 bits) items are 16-bit words10'0358 R R1FWC Raw event FIFO A1 item counter (12 bits) items are 16-bit words10'035C R R2FWC Raw event FIFO B0 item counter (12 bits) items are 16-bit words10'0360 R R3FWC Raw event FIFO B1 item counter (12 bits) items are 16-bit words

      Fifo window address start - 8K word each (allow use of DMA) - Read Only 32 bit20'0000 R XFIFO Exception log FIFO21'0000 R YFIFO HistorY FIFO22'0000 R reserved  23'0000 R OFIFO Output event FIFO for debugging24'0000 R reserved  25'0000 R reserved  26'0000 R reserved  27'0000 R reserved  28'0000 R reserved  29'0000 R reserved  2A'0000 R reserved  2B'0000 R R0FIFO Raw FIFO A02C'0000 R R1FIFO Raw FIFO A12D'0000 R R2FIFO Raw FIFO B02E'0000 R R3FIFO Raw FIFO B1

Table 5: SR1 Register description

bit width name meaning15:0 16 SR1_PENDING flag of pending SVC (see SVC sheet)19:16 4 SR1_RSRVD1 reserved20 1 SR1_SLNK_LFF Output S-link full flag21 1 SR1_SVCPEND SVC request waiting for acknowledge22 1 SR1_SLNK_LDOWN Output S-link is down (1=down)23 1 SR1_SLNK_TST S-link test in progress (1= test running)24 1 SR1_SLNK_RST S-link reset in progress (1 = resetting)25 1 SR1_FRAGWAIT waiting for first (non-dropped) fragment26 1 SR1_TTYPEWAIT waiting for trigger type27 1 SR1_TTCrxREADY TTCrx on TTCrm mezz board is ready28 1 SR1_TRIGWAIT waiting for trigger (L1A)

29 1 SR1_DONEWAIT waiting for all Fron-end RxLink to be processed30 1 SR1_HSTRY_ENA recording in history buffer is enabled;set inactive by significant event in ROD logic31 1 SR1_RSRVD2 reserved

Table 6: List of implemented Service Calls (bits 0 to 15 of SR1 register)

bit name description0 SVC_rsrvd0 reserved1 SVC_rsrvd1 reserved2 SVC_rsrvd2 reserved3 SVC_rsrvd3 reserved4 SVC_ldown Unexpected link down from S-link5 SVC_L1Atime Requesting to log the time-of-day of first Level-1 Accept6 SVC_rsrvd6 reserved7 SVC_rsrvd7 reserved8 SVC_rsrvd8 reserved9 SVC_rsrvd9 reserved

10 SVC_rsrvd10 reserved11 SVC_rsrvd11 reserved12 SVC_hstry Stop history buffer, generated on event significant error13 SVC_rsrvd13 reserved14 SVC_outpipe Data ready in output pipe15 SVC_errpipe Data ready in error pipe

Table 7: Error Codes (in the error flags registers data the correspondant bit is set to 1)

codes error related data specific data0 Bad Checksum (as checked in the mezzanine) 4 bit FE flags + word count1 Front End link G-link error 4 bit FE flags + word count2 Error in control FE word sequence 4 bit FE flags + word count3 Input FE event is too long or FE FIFO overflow 4 bit FE flags + word count4 Unexpected null in FE input first word 4 bit FE flags + word count5 Number of FE frames equal to 0 SRC_ID FE word6 Invalid or unexpected received GOL ID 3 bit GOL_ID7 Expected more FE frames than received 4 bit : number of missing frames8 Unexpected even number of FE words last received FE word 9 A Front End link has timed out - abandoned rx id (2bit) @ FE channel (2 bit)10 Bad BC parity check caclulated(1bit) @received(1 bit)11 FE board BC counter mismatch received BC counter12 reserved  13 reserved  14 reserved  15 Timeout expired for at least one FE link 12 bit FE channel timeout flags16 reserved  17 reserved  18 reserved  19 reserved  20 WC not 0 after last frame + l1bcn + checksum 4 bit FE flags + actual word count21 reserved  22 reserved  23 reserved  24 Bad checksum from mezz board Final Checksum result25 reserved  26 reserved  

27 reserved  28 reserved  

29Event has WC=0 or WC > max WC (640 x FE chan)  

30 Error in request to send an event via RXlink 4 bit FE flags + word count31 GOL ID greater than 5 SRC_ID FE word

Table 8: Front-End FIFO status (B1,B0,A1,A0 are mezzanine FPGAs, having 3 link each one)

Register Bit DescriptionFFR 27 FIFO overflow B1[2]FFR 26 FIFO overflow B1[1]FFR 25 FIFO overflow B1[0]FFR 24 FIFO overflow B0[2]FFR 23 FIFO overflow B0[1]FFR 22 FIFO overflow B0[0]FFR 21 FIFO overflow A1[2]FFR 20 FIFO overflow A1[1]FFR 19 FIFO overflow A1[0]FFR 18 FIFO overflow A0[2]FFR 17 FIFO overflow A0[1]FFR 16 FIFO overflow A0[0]FFR 11 FIFO busy B1[2]FFR 10 FIFO busy B1[1]FFR 9 FIFO busy B1[0]FFR 8 FIFO busy B0[2]FFR 7 FIFO busy B0[1]FFR 6 FIFO busy B0[0]FFR 5 FIFO busy A1[2]FFR 4 FIFO busy A1[1]FFR 3 FIFO busy A1[0]FFR 2 FIFO busy A0[2]FFR 1 FIFO busy A0[1]FFR 0 FIFO busy A0[0]FEMT 11 FIFO empty B1[2]FEMT 10 FIFO empty B1[1]FEMT 9 FIFO empty B1[0]FEMT 8 FIFO empty B0[2]FEMT 7 FIFO empty B0[1]FEMT 6 FIFO empty B0[0]FEMT 5 FIFO empty A1[2]FEMT 4 FIFO empty A1[1]FEMT 3 FIFO empty A1[0]FEMT 2 FIFO empty A0[2]FEMT 1 FIFO empty A0[1]FEMT 0 FIFO empty A0[0]

Table 9: FEMT register: bits showing the status of the Select Link FIFOs

bit name description11 SLMainReady B1 Non almost full (i.e. data less than 2/3 fifo depth

10 SLMainReady B0 “9 SLMainReady A1 “8 SLMainReady A0 “7 SL Empty B1 Empty FIFO flag6 SL Empty B0 “5 SL Empty A1 “4 SL Empty A0 “3 SL Full B1 Ful FIFO flag2 SL Full B0 “1 SL Full A1 “0 SL Full A0 “

Table 10: FEOUT register description

bit Name/description27 FE link Dropped B1[2]26 FE link Dropped B1[1]25 FE link Dropped B1[0]24 FE link Dropped B0[2]23 FE link Dropped B0[1]22 FE link Dropped B0[0]21 FE link Dropped A1[2]20 FE link Dropped A1[1]19 FE link Dropped A1[0]18 FE link Dropped A0[2]17 FE link Dropped A0[1]16 FE link Dropped A0[0]11 FE link Timed-Out B1[2]10 FE link Timed-Out B1[1]9 FE link Timed-Out B1[0]8 FE link Timed-Out B0[2]7 FE link Timed-Out B0[1]6 FE link Timed-Out B0[0]5 FE link Timed-Out A1[2]4 FE link Timed-Out A1[1]3 FE link Timed-Out A1[0]2 FE link Timed-Out A0[2]1 FE link Timed-Out A0[1]0 FE link Timed-Out A0[0]

Table 11: Stalled pipe flags (STALD register)

bit width name0 1 raw0 (A0) control pipe1 1 raw0 (A0) pipe2 1 raw1 (A1) control pipe3 1 raw1 (A1) pipe4 1 raw2 (B0) control pipe5 1 raw2 (B0) pipe6 1 raw3 (B1) control pipe7 1 raw3 (B1) pipe

20:8 13 Reserved21 1 Event header pipe22 1 Reserved23 1 Event out pipe27:24 4 reserved28 1 rx link 0 (A0) pipe29 1 rx link 1 (A1) pipe30 1 rx link 2 (B0) pipe31 1 rx link 3 (B1) pipe

Table 12: FERDY register description.

bit Name/description11 FE G-link Ready Flag B1[2]10 FE G-link Ready Flag B1[1]9 FE G-link Ready Flag B1[0]8 FE G-link Ready Flag B0[2]7 FE G-link Ready Flag B0[1]6 FE G-link Ready Flag B0[0]5 FE G-link Ready Flag A1[2]4 FE G-link Ready Flag A1[1]3 FE G-link Ready Flag A1[0]2 FE G-link Ready Flag A0[2]1 FE G-link Ready Flag A0[1]0 FE G-link Ready Flag A0[0]

Table 13: FCR1 register description.

bit name Short description0 CR1_ALLFMT if=1 all formatted candidates are sent to VME; if=0 only 1st event and

wait to be read from VME1 CR1_ERRFMT Formatted events with any *unmuted* error bits on are sent to VME (*)

(see EMUTE and EMUTS regs)2 CR1_FLTFMT filtered formatted events are sent to VME (*) (= fragment timeouts?)3 CR1_INCLRAW Include copy of ALL fragments received from FE links in the formatted

event4 CR1_ONLYRAW only raw data output is forced (FWver1 = equivalent to

CR1_INCLRAW)5 CR1_INFOOFF suppress writing all info msgs to exception pipe6 CR1_IGNORE_ID Ignore the ROD TTCrx data for checking BCID consistency; use that

from FE board received instead7 CR1_TTYPE_INCL Include Trigger type from TTCrx8 CR1_SLNK_SVC enable SVC on S-link down9 CR1_SLNK_TSTCONT if set, S-link test runs continuously, else once

10 CR1_GIGA_SAMPLE sampled formatted events are sent to gigabit ethernet11 CR1_FLTFMT_GIGA only sampled FILTERED formatted events are sent to gigabit ethernet12 CR1_FMTENA Enable building formatted events for output13 CR1_SLNK_FORCE if set, write to S-link without checking link ready14 CR1_DIS_RAWCHK disable consistency checks on raw data

15 CR1_JUMBO_PKTS enable jumbo packets for gigabit interface16 reserved  17 CR1_DIAGNOSE allow diagnostic state, which forces flushed (1) output to be written to

raw data pipe18 CR1_TSTEVS Send test event, reset by ROD when done19 CR1_FAKE_L1A fake L1A internally to ROD FPGA, at 40MHz/512 = 78kHz20 reserved  21 reserved  22 reserved  23 CR1_OUTLENA Enable output S-link24 CR1_TTCENA Enable TTC and Event ID FIFO

Table 14: Command register CMR1 description

bit width name meaning0 1 reserved  1 1 CMR1_ENA_HSTRY Enable writing to history buffer (stopped on event significant error! )

2 :11 10 reserved  12   CMR1_SLNK_RST do S-link reset

13   CMR1_SLNK_TSTdo S-link test, reset by FPGA at end of test, if test once (CR1_SLNK_TSTCONT=0)

14   CMR1_ORBITCNT clear orbit counter15   CMR1_FAKE_L1A generate a fake L1 trigger

Table 15: Front-End Link configuration

register bits mezzanine linkTGCC0 3:0 A0 0TGCC0 7:4 A0 1TGCC0 11:8 A0 2TGCC0 15:12 A1 0TGCC0 19:16 A1 1TGCC0 23:20 A1 2TGCC0 27:24 B0 0GCC0 31:28 B0 1TGCC1 3:0 B0 2TGCC1 7:4 B1 0TGCC1 11:8 B1 1TGCC1 15:12 B1 2

Table 16: ROD output data format to the ROB

Data Word Comments31..24 23..16 15..8 7..0

Frame 0xB0F00000 Control modeHeader 0 0xEE1234EE Start of header marker for ROD dataHeader 1 reserved reserved reserved Header size = 9 # of header words (excluding 1st frame)Header 2 Atlas format version =3.0 Lucid format version =1.0 i.e.: 0x0301 0x0100Header 3 0 0x82 0 4-bit RODID Source ID: Lucid=0x82–RODID to be configuredHeader 4 Run Type Run Number

Header 5 Level-1 ID High byte is the Extended Level-1 IDHeader 6 reserved reserved Bunch Crossing ID [11:0]Header 7 reserved reserved reserved Trigger typeHeader 8 Detector Event Type Not used yet (set to 0xAFFEC0C0)Status Specific Error Generic Error If not 0 => event not OKStatus Lucid Rod Event StatusStatus ROD VME filter bits Front-End filters and timeoutsStatus Local Status Word Presence BitsStatus Orbit Count Zero for first L1AIDData Fragment ID Raw data word count Fragment ID = 1 , length in wordsData Raw dataData ……..Data Last Raw DatumTrailer 0 Number of status element = 5Trailer 1 Number of data elementTrailer 2 Status block position = 0 ; => data follow statusFrame 0xE0F00000 Control Mode