CMW infrastructure Status report

P.Charrue – LBCM 14 Sept 2010For the CMW team

Current CMW issues (3)

CMW middle and long term plans

How to report issues to the Controls Group

4th May 2010 P.Charrue - LBCM 2

Description : JAVA clients blocked (XPOC project) and not getting data anymore from

the devices Cause :

Socket blocking situation in the JacORB CORBA library (part of the CMW infrastructure) – known bug in JacORB

Occurence : Once to the XPOC client Often for the Logging infrastructure

Immediate cure : Restart the client application as the blocking situation cannot be

resolved CMW proposal :

Today: We provide a callback to the client application which detect such blocking situation and take take action (mail, sms, alarm, restart, log, …)

In 2 weeks: We will deliver a patch to this external Jacorb library to solve this blocking situation;currently tested.

4th May 2010 3P.Charrue - LBCM

Description : CMW Proxy is blocked due slow consuming clients

Cause : ‘Slow clients’ subscribed to Proxy are not consuming the data quick enough

and block many notification threads (in Proxy) resulting in a complete blocking of the Proxy

Occurrence : BBQ, Hump Buster

Immediate cure : Kill the ‘slow client’ application as the blocking situation cannot be resolved

automatically CMW proposal :

A new version of the Proxy has been developed that handles correctly slow clients (by reserving processing resources for every subscribed client) and minimizes impact of slow consumers on the well behaving clients

Currently tested for the CMW-Proxy-BQ When the test are completed the upgraded Proxy will be deployed in close

collaboration with Operations – end this week


Description : Client/Server communication is lost inside the JAVA client application: busy

CMW notification thread inside the JAVA client prevents any subsequent communication (idle socket in FIN_WAIT1 left in the FrontEnd)

Cause : JAVA client CMW thread responsible for the socket operation is too busy by

doing data processing and therefore cannot cleanly close the communication Occurrence :

Collimators Immediate cure :

Restart the JAVA application as the blocking situation cannot be resolved CMW proposal :

Get more data from blocked JAVA application to confirm our hypothesis Organise code review with the authors of these JAVA clients to understand

why the communication threads are blocked Help the developers of the Java Clients to move to JAPC (as this issue is solved

using JAPC)


Medium term plans Deploy Proxies with support to slow clients Deploy patched Jacorb library to solve the JAVA client blocking

situation Push the usage of JAPC to avoid the loss of communication from

certain JAVA client applications Long term plans

The CMW team is currently preparing a complete technical review of the Communication Infrastructure▪ Several clients have already been interviewed▪ The issues of the present infrastructure have been captured and prioritised

along with the new functionality requested▪ Several solutions have been evaluated▪ External middleware experts have been contacted to help us confirm our

choices The actual review will take place in October 2010 https://wikis.cern.ch/display/MW/CMW+Review


From the e-logbook, a simple right-clic on an entry will create a JIRA issue Each JIRA issue is then assigned and is closely followed-up http://issues/browse/APS PS and SPS operators are making good use of this

From your browser, go to http://issues and fill in a new JIRA issue

As a last solution: Avoid direct email to individuals (they might be on vacation, not

reading their mail, sick, on leave, ….) Instead opt for the support mailing lists (e.g. cmw-

[email protected], [email protected], …)


CMW infrastructure Status report

Documents

Transcript of CMW infrastructure Status report