Post on 14-Jan-2016
JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
JBQS presentation IN2P3 Computer Centerhttp://www.in2p3.fr/CC
Campus de la DOUA27, Boulevard du 11 Novembre 1918
69622 VILLEURBANNEFRANCE
JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
JBQS : JBQS :
Graphical InterfaceGraphical Interface
for BQSfor BQS
Slide No 3JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
BQS Summary
• Software component for batch management
• Allows– The submission of jobs;
– The scheduling and execution of jobs on workers;
– The monitoring of a job.
• Developed and used at the IN2P3 Computer Centre (France)In October 2000, BQS drives
– 900 workpoints on;
– 250 workers and;
– 10 classes.
Slide No 4JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
• Worker : A physical machine where jobs can run.
• Class : – Description of resources such as
Amount of CPU, memory, scratch, spool size, etc.
– Other characteristics such asprivileged access, AFS token information.
– Example :
Workers, Classes, Workpoints
225000 300000 256 512 1000 1000 5 50
R class exampleCPU (IU) Virt. Mem (MB) Scratch (MB) Spool (MB)
Slide No 5JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
• Workpoint : – Used to get a better control over job production;
– Defined as a set of ordered classes allocated to a worker;
– Runs only one job at a time, but
– Multiple workpoints on a worker.
– Example : workpoints definition for ccwasn01 (two processors SUN worker) 1@ccwasn01.in2p3.fr : T,S
2@ccwasn01.in2p3.fr : G,C,A
3@ccwasn01.in2p3.fr : I,R,J,P
4@ccwasn01.in2p3.fr : S,T,I,R
Up to 4 jobs can run at the same time for the ccwasn01 worker.
– About 900 workpoints at the IN2P3 Computer Center.
Workers, Classes, Workpoints
Slide No 6JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
JBQS design goals• A graphical interface for
– production manager;
– BQS administrator;
– on duty staff.
• Avoid knowledge of BQS Unix command syntax.
• Allows – Graphical query;
– Results display with• Editing abilities (finding, sorting, selecting);
• Interactions facilities on single or multiple selections.
Slide No 7JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Structure of JBQS
• Batch management Production manager Available
• BQS administration BQS staff Available
• Batch status, On duty staff Under development
Resources optimisation
Slide No 8JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Batch management : Description
• Batch management allows query & control over– jobs;– workers;– workpoints;– dispatching authorisation.
• Queries are made through various criteria from menus.
• Results interaction with– Popup menu allows actions on single or multiples selected cell.
– Editing abilities allows finding, extracting, sorting.
• Predefined options such as job type or frequently used command options can be set.
Slide No 9JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Batch management : Diagram
Workers
Jobs Workpoints
Query by:•user•group•platform•worker•class•workpoint
Query by:•platform•name
Query by:•platform•worker•name
Switchingability
Slide No 10JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
• First example :– CPU limit modification on a running job.
• Second example– Draining a set of workers.
Screenshots examples
Slide No 11JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Job query by account
Account choice
Job menu selection
Slide No 12JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Job results
Job results panel
Text-area panel
Slide No 13JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Action on a job
Popup menufor the selected cell
Job action choice panel
Slide No 14JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Multiple workers drain action
Worker menu selection
Platform choice
Slide No 15JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Worker results
worker results panel
Action status
Slide No 16JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Multiple workers drain action
3) Worker action panel
1) Selection of multiple workers
2) Popup menu for selected cells
Slide No 17JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
BQS administration description
• Allows modification on all BQS objects i.e. :Farm, platform, machine, workpoint, class and account.
• Objects can be created or deleted
• Parameters can be modified and then– applied to the object itself;
– propagated to other selected object.
Slide No 18JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
• First example :– Default CPU modification on one class (G).
• Second example– Swap space modification on multiple workers.
Screenshots examples
Slide No 19JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Default CPU modification on class G
New default AND max CPU values
(previous 30,000)
Apply new values(default and max CPU)
to the class G
Slide No 20JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
New swap space for multiple workers
2) New Swap Space value(previous 4,000)
1) Selection of multiple workers
3) Button for (only) Swap Space modification
Slide No 21JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Implementation features
• A Java standalone application with BQS calls made through an C API(uses of JNI *).
• Available on Solaris 2.7 and Linux RH 6.1.On any platform as soon as RMI * functionality is implemented.
• Multithreaded application for BQS calls,in order to get a better response time.
• Dynamically dialog boxes generation (thanks to Java).
• Multiple windows with menus.
• Text area panel with file Open/Save et Cut/Copy/Paste facilities.
* JNI : Java Native Interface. * RMI : Remote Method Invocation.
Slide No 22JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Next implementation
• Editing functions for results– Finding,
– Sorting,
– Selecting
• RMI * functionality
* RMI : Remote Method Invocation.
Slide No 23JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Future development
• BQS status functionality– Worker status;
– Job running status (dead lock, infinity loops).
• Batch production flow (in order to optimise)– Estimated ending time of running and queued jobs;
– Workpoints and classes occupation rate.
• JDBC * implementation in JBQS
(After a data-base migration for BQS software)
* JDBC : Java Data Base Connectivity
Slide No 24JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
More information
• About BQS– http://www.in2p3.fr/ CC/interne/pub/bqs/index.html
– email : fouilhe@cc.in2p3.fr (Yves FOUILHE)
chambon@cc.in2p3.fr (Bernard CHAMBON)
• About CC-IN2P3– http://www.in2p3.fr/CC/
• About Java– http://www.javasoft.com