EVIO, Databases, and All That Elliott Wolin CODA Fest 7-Jun-2004.

EVIO, Databases, and EVIO, Databases, and All ThatAll That

Elliott WolinElliott Wolin

CODA FestCODA Fest

7-Jun-20047-Jun-2004

PlanPlan

Brief presentation on many topicsBrief presentation on many topics Work of many peopleWork of many people Consider the following as Consider the following as

advertisementsadvertisements Contact me if you are interestedContact me if you are interested

– Not everything mentioned is in CODA distNot everything mentioned is in CODA dist There are others utilitiesThere are others utilities

– database, etc.database, etc.– ASK!ASK!

OutlineOutline EVIO (CODA format) - Version 2EVIO (CODA format) - Version 2

– New vsn, gzip, utilities, xml, java, Root & JAS histNew vsn, gzip, utilities, xml, java, Root & JAS hist VXWorksVXWorks

– tcpServer, envGetRemote()tcpServer, envGetRemote() ETET

– et2xml, et2fileet2xml, et2file Database proxy server Database proxy server Run controlRun control

– xml configurations, CODA Masterxml configurations, CODA Master Fastbus utilitiesFastbus utilities Portable Channel Access (epics) serverPortable Channel Access (epics) server Timeline graphing utilityTimeline graphing utility CODALOG elogbook – Maybe coming soon?CODALOG elogbook – Maybe coming soon?

EVIO – Version 2EVIO – Version 2 Full support for 8-byte data typesFull support for 8-byte data types New tagsegment bank typeNew tagsegment bank type New swapper fixes byte-swap bugsNew swapper fixes byte-swap bugs Built-in gunzip on input, perhaps gzip on Built-in gunzip on input, perhaps gzip on

outputoutput UtilitiesUtilities

– evio2xml – dump binary to xml, gzip optionevio2xml – dump binary to xml, gzip option– xml2evio – convert xml to binary, auto-gunzipxml2evio – convert xml to binary, auto-gunzip– eviocopy – selective event copyeviocopy – selective event copy

Full Java support – JevioFull Java support – Jevio Root and JAS access and histogrammingRoot and JAS access and histogramming

evio2xml, xml2evioevio2xml, xml2evio

<event format="evio" count="31" content="bank" data_type="0x10" tag="4" num="204"> <uint32 data_type="0x1" tag="49152" num="0"> 0xb 0x4 0 </uint32> <uint32 data_type="0x1" tag="1" num="11"> 0x8008541 0xe0001e7 0xa0201f9 0xa04019a 0xa060201 0xa080224 0xa0a0158 0xa0c01b6 0xe0e01a8 0xe1001c1 0xe12016a 0xe1401f6 0xa160103 0xa18023d 0xa1a0230 ........... </uint32> <uint32 data_type="0x1" tag="2" num="11"> 0x10001005 0x122a148f 0x162c0d2f 0x122e0e20 0x16300d17 0x18001002 0x1e440dd3 0x28001004 0x2a420e06 0x2a780d9c ………….

</uint32> <uint32 data_type="0x1" tag="5" num="11"> 0x1 0x4080 </uint32> ...........

<uint32 data_type="0x1" tag="9" num="11"> 0x4 </uint32> <uint32 data_type="0x1" tag="20" num="11"> 0x4 </uint32></event>

evio2xml, xml2evioevio2xml, xml2evioollie:wolin> evio2xml -h

usage:

evio2xml [-max max_event] [-pause] [-skip skip_event] [-dict dictfilename] [-ev evtag] [-noev evtag] [-frag frag] [-nofrag frag] [-max_depth max_depth] [-n8 n8] [-n16 n16] [-n32 n32] [-n64 n64] [-w8 w8] [-w16 w16] [-w32 w32] [-w64 w64] [-verbose] [-xtod] [-m main_tag] [-e event_tag] [-indent indent_size] [-no_typename] [-debug] filename

ollie:wolin> xml2evio -h

usage:

xml2evio [-xml xmlfilename] [-max max_event] [-skip nskip] [-evio eviofilename] [-dict dictfilename] [-m main_tag] [-e event_tag]

eviocopyeviocopy

ollie:wolin> eviocopy -h

usage:

eviocopy [-max max_event] [-skip skip_event] [-ev evtag] [-noev evtag] [-debug] input_filename output_filename

JevioJevio Jevio evio = new Jevio(); JevioEvent evt = null; int[] buf = null;

evio.openFile(“myfile.dat”,”r”);

// loop over events while(!evio.endEvent) { try { evt=evio.readEvent(); buf = evt.getEvtArray(); … } catch (Exception e) { System.out.println(e); } } evio.closeFile(); }

EVIO and Root, JASEVIO and Root, JAS

EVIO plus C++/Root or Java/JASEVIO plus C++/Root or Java/JAS See me for example analysis routinesSee me for example analysis routines

VXWorks - tcpServerVXWorks - tcpServer From Unix, execute command on remote From Unix, execute command on remote

VXWorks system and get resultVXWorks system and get result Like dpsh/DP_ask and coda_roc but w/o Like dpsh/DP_ask and coda_roc but w/o

TCL/DPTCL/DP Lightweight compared to TCL/DP mechanismLightweight compared to TCL/DP mechanism On VXWorksOn VXWorks

– ld < tcpServer.old < tcpServer.o– sp tcpServersp tcpServer

On UnixOn Unix– $ vxcmd myNode ifshow()$ vxcmd myNode ifshow()– C-callable libraryC-callable library

VXWorks – envGetRemote()VXWorks – envGetRemote()

Get env vars from a remote hostGet env vars from a remote host Analogous to setenvAnalogous to setenv

– envGetRemote(“MY_ENV_VAR”,”host”,”uenvGetRemote(“MY_ENV_VAR”,”host”,”user”)ser”)

– NULL host,user NULL host,user use boot host,user use boot host,user

ET utilitiesET utilities

Utilities connect to ET system and:Utilities connect to ET system and:– et2xml – dumps events to stdout in xmlet2xml – dumps events to stdout in xml– et2file – dumps events to file in evio et2file – dumps events to file in evio

formatformat

et2xmlet2xmlwolin@lucy:analysis> et2xml -h

usage:

et2xml [-f et_file_name] [-host et_host_name] [-station et_station_name] [-queue qsize] [-block] [-max max_buffer] [-pause] [-skip skip_buffer] [-dict dictfilename] [-ev evtag] [-noev evtag] [-frag frag] [-nofrag frag] [-max_depth max_depth] [-n8 n8] [-n16 n16] [-n32 n32] [-n64 n64] [-w8 w8] [-w16 w16] [-w32 w32] [-w64 w64] [-noswap] [-nocontrol] [-verbose] [-xtod] [-m main_tag] [-e buffer_tag] [-b bank2_tag] [-indent indent_size] [-no_typename] [-debug] [-raw] [-ndump ndumpmax]

et2fileet2file

wolin@lucy:analysis> et2file -h

usage:

et2file [-f et_file_name] [-host et_host_name] [-o output_file] [-station et_station_name] [-queue qsize] [-block] [-max max_buffer] [-skip skip_buffer] [-noev evtag] [-frag frag] [-nofrag frag] [-noswap] [-debug]

Database Proxy ServerDatabase Proxy Server

Can use any JDBC-compatible Can use any JDBC-compatible database with CODAdatabase with CODA

MySQL, Postgresql, any vsn of mSQL, MySQL, Postgresql, any vsn of mSQL, etc.etc.

To Use Proxy ServerTo Use Proxy Server

#!/bin/tcsh -f

# E.Wolin, 5-dec-2003

setenv CLASSPATH $CODA/common/lib/java:"$CLASSPATH" java CodaDatabaseProxyServer >>& proxy_server.log &

First must rebuild CODA to use correct shared libs, then:

Run controlRun control

XML configurationsXML configurations– CODA configurations in XML filesCODA configurations in XML files

CODA Master CODA Master – DAQ process controlDAQ process control

XML CODA ConfigurationsXML CODA Configurations

For backup, can delete and reinsertFor backup, can delete and reinsert Can copy, modify, then re-insert with Can copy, modify, then re-insert with

same or new name, with no loss of same or new name, with no loss of information (scripts, options, etc.)information (scripts, options, etc.)

codadb2xml.pl, xml2codadb.pl, codadb2xml.pl, xml2codadb.pl, initcodadb.plinitcodadb.pl

Can skip cedit, dbeditCan skip cedit, dbedit

XML configurationsXML configurations<config name="E_PROD">

<component name = "dc1" type = "ROC" code = "{$CLON_ROL/fbrol1.o usr} {$CLON_ROL/trans_rol2tt.ppco usr}" inputs = "" outputs = "EB1:clon10-daq1" first = "yes" next = "dc2" />

<component name = "EB1" type = "EB" code = "{BOS} {BOS}" inputs = "ec1:ec1 ec2:ec2 cc1:cc1 sc1:sc1 lac1:lac1 tage:tage dc1:dc1 dc2:dc2 dc5:dc5 dc6:dc6 dc11:dc11 scaler1:scaler1 scaler2:scaler2 polar:polar scaler3:scaler3 scaler4:scaler4 cc2:cc2 dc3:dc3 dc4:dc4 dc7:dc7 dc8:dc8 dc9:dc9 dc10:dc10" outputs = "" first = "yes" next = "" />

CODA MasterCODA Master

Uses a CODA configurationUses a CODA configuration Launches CODA componentsLaunches CODA components

– EB, ER, ET, RunControl, CMLOG browserEB, ER, ET, RunControl, CMLOG browser Monitors CODA componentsMonitors CODA components

– Rocs, EB, ER, ETRocs, EB, ER, ET You supply “kill” scriptYou supply “kill” script Tcl/Tk, easy to modifyTcl/Tk, easy to modify

Fastbus utilitity - fbtestFastbus utilitity - fbtest

Tests LeCroy fastbus modules in Tests LeCroy fastbus modules in cratecrate

fbtest connects to VXWorks rocfbtest connects to VXWorks roc– Gets system status Gets system status

(i,ifshow,memshow,etc.)(i,ifshow,memshow,etc.)– Finds all Fastbus modulesFinds all Fastbus modules

Gets module CSR statusGets module CSR status Performs internal testsPerforms internal tests

Portable CA ServerPortable CA Server

Can serve ANY information as EPICS “channel”Can serve ANY information as EPICS “channel” Alarms, monitors, read/write etc.Alarms, monitors, read/write etc. Full CA support, but NOT full EPICS channel Full CA support, but NOT full EPICS channel

support (use IOC instead)support (use IOC instead) Only one PCAS per node allowedOnly one PCAS per node allowed I have templates for different CA support I have templates for different CA support

levelslevels Not the simplest C++ library to deal with…Not the simplest C++ library to deal with… caMon utility monitors epics broadcast trafficcaMon utility monitors epics broadcast traffic

// caMon.cc

// lists ca broadcasts

#include <iostream.h>#include <iomanip.h>#include <casdef.h>#include <fdManager.h>

//---------------------------------------------------------------------------

class caMon : public caServer {

public: pvExistReturn caMon::pvExistTest(const casCtx &ctx, const char *pPVName) { cout << pPVName << endl; return pverDoesNotExistHere; } ~caMon() { return; }};

//---------------------------------------------------------------------------

int main() { caMon *c = new caMon(); if(!c) { cout << "can't create server object" << endl; exit(EXIT_FAILURE); } for(;;) { fileDescriptorManager.process(10.0); }}

Timeline Graphing UtilityTimeline Graphing Utility

Creates timeline plots out of text filesCreates timeline plots out of text files Define graphs/plots in config filesDefine graphs/plots in config files You provide updating text data filesYou provide updating text data files TCL utility periodically gets data from TCL utility periodically gets data from

files, then graphs datafiles, then graphs data– multiple graphs on one plotmultiple graphs on one plot– alarm limits (plot turns red)alarm limits (plot turns red)– multiple plotsmultiple plots

CODALOGCODALOG Expanded/upgraded vsn of CLAS elogbookExpanded/upgraded vsn of CLAS elogbook Web based, relational DBMS underneathWeb based, relational DBMS underneath Combines best features of control room Combines best features of control room

logbook, analysis notebook, logbook, analysis notebook, helpdesk/request tracker helpdesk/request tracker – Threads, topics, subtopics, references, Threads, topics, subtopics, references,

document/image uploads, bug/request tracking, document/image uploads, bug/request tracking, xml configuration, searching etc.xml configuration, searching etc.

Database design done, needs guiDatabase design done, needs gui Velocity/Servlet basedVelocity/Servlet based

– Allows for easy user customizationAllows for easy user customization See See http://http://

www.jlab.org/Hall-D/presentations/codalog_for_halld.pptwww.jlab.org/Hall-D/presentations/codalog_for_halld.ppt

SummarySummary

Many utilities developed over the Many utilities developed over the yearsyears– ET, run control, databases, CA, elog, etc.ET, run control, databases, CA, elog, etc.

Most not very well knownMost not very well known See me if you are interestedSee me if you are interested

EVIO, Databases, and All That Elliott Wolin CODA Fest 7-Jun-2004.

Documents

Transcript of EVIO, Databases, and All That Elliott Wolin CODA Fest 7-Jun-2004.