US CMS Software and Computing Project Management Plan

Project Management Plan for the

Draft

June 24, 2000

1

US CMS SOFTWARE AND COMPUTING PROJECT MANAGEMENT PLAN.....................................................................1

1. GOALS OF THE US CMS SOFTWARE AND COMPUTING PROJECT........................5

2. PROJECT ORGANIZATION...........................................................................................7

2.1 The Core Applications Software Subproject............................................................................................................7

2.2 The User Facilities Subproject...................................................................................................................................8

3. UPPER LEVEL PROJECT MANAGEMENT.................................................................10

3.1 The Level 1 Project Manager...................................................................................................................................12

3.2 The Level 2 Project Managers..................................................................................................................................133.2.1 The Core Applications Software Subproject........................................................................................................133.2.2 The User Facilities Subproject..............................................................................................................................13

3.3 The US CMS Advisory Software and Computing Board (USASCB)..................................................................143.3.1 Development of the Project Plan..........................................................................................................................153.3.2 Scientific and Technical Policy............................................................................................................................153.3.3 Project Input and Feedback..................................................................................................................................15

3.4 Fermilab Oversight and the Project Management Group.....................................................................................153.4.1 Project Management Group (PMG)......................................................................................................................163.4.2 External Review Committee.................................................................................................................................163.4.3 Change Control.....................................................................................................................................................16

4. INTERRELATIONSHIP WITH OTHER ENTITIES........................................................17

4.1 Relation to CMS.........................................................................................................................................................17

4.2 Relation to US CMS and to the US CMS Construction Project............................................................................17

4.3 Relationship to the US Funding Agencies...............................................................................................................18

4.4 Relation to the Fermilab Computing Division........................................................................................................18

5. EVOLUTION OF THE US CMS SOFTWARE AND COMPUTING PROJECT..............18

6. HIGH LEVEL MILESTONES, WORK BREAKDOWN STRUCTURE AND BUDGET FOR THE USER FACILITIES SUBPROJECT..................................................................19

6.1 User Facilities High Level Milestones......................................................................................................................19

6.2 User Facilities Work Breakdown Structure...........................................................................................................191.1 Tier 1 Regional Center at FNAL.............................................................................................................................19

2

1.2 System and User Support.........................................................................................................................................231.3 Maintenance and Operation.....................................................................................................................................251.4 Support of Tier 2 Regional Centers.........................................................................................................................251.5 Networking..............................................................................................................................................................251.6 R&D Phase Activities..............................................................................................................................................27

6.3 User Facilities Budget and Personnel Requirements.............................................................................................29

7. HIGH LEVEL MILESTONES, WORK BREAKDOWN STRUCTURE AND BUDGET FOR THE CORE APPLICATIONS SOFTWARE SUBPROJECT.....................................33

7.1 Core Applications Software High Level Milestones...............................................................................................337.1.1 Schedule Considerations.......................................................................................................................................337.1.2 Software Milestones.............................................................................................................................................33

7.2 Core Application Software Work Breakdown Structure......................................................................................342.1 ORCA – Detector Reconstruction (CMS WBS 1.3.10)..........................................................................................342.2 User Analysis Environment (CMS WBS 1.3.12)....................................................................................................352.3 ODBMS -- Object Database Management System (CMS WBS 1.3.15)................................................................362.4 Software Support.....................................................................................................................................................37

7.3 Core Applications Software Budget and Personnel Requirements......................................................................38

8. HIGH LEVEL MILESTONES, WORK BREAKDOWN STRUCTURE AND BUDGET FOR OVERALL PROJECT...............................................................................................40

8.1 High Level Project Milestones..................................................................................................................................40

8.2 High Level Work Breakdown Structure.................................................................................................................408.2.1 The User Facilities Subproject..............................................................................................................................408.2.2 The Core Applications Software Project..............................................................................................................408.2.3. Project Office.......................................................................................................................................................40

8.3 Budget Summary.......................................................................................................................................................40

REFERENCES..................................................................................................................41

APPENDIX: CMS SOFTWARE SCHEDULE AND WORK BREAKDOWN STRUCTURE...........................................................................................................................................42

CMS Schedule and Milestones.......................................................................................................................................42

CMS Work Breakdown Structure.................................................................................................................................42

REFERENCES FOR APPENDIX......................................................................................45

3

LIST OF FIGURES AND TABLES

Figure 1: Abbreviated version of organization of the US CMS Software and Computing Project....................................8Figure 2: Organization Chart for US CMS Software and Computing Project. This shows how the project components

interact among themselves, with Fermilab, the US CMS Construction Project, and US CMS software efforts in reconstruction and physics analysis...........................................................................................................................11

Table 1: Personnel Requirements for the User Facilities Subproject, summarized for the higher level WBS items. …..31Table 2: Total Installed capability by year, for Implementation and CMS Operations Phases of the Tier 1 Regional

Center.........................................................................................................................................................................31Table 3: User Facilities Subproject costs by year, for the R&D, implementation, and operations Phases.......................32Table 4: Milestones for CMS Computing and Software Project.......................................................................................34Table 5: Estimated requirements of software personnel (FTE) for the Core Application Software subproject…………39Table 6: Budget Summary of US CMS Software and Computing Project........................................................................41

4

1. Goals of the US CMS Software and Computing Project

The software and computing effort for CMS exceeds in scale and complexity anything that has so far been achieved in High Energy Physics. Even the new generation of experiments that will be coming on between 1999 and 2001 will not approach this scale. Because of the large number of participants and their wide geographical distribution CMS will need to employ what is essentially a new model of distributed computing and data analysis, which is without precedent in HEP. It will do so during a period of rapid change in software practices and hardware technologies. The US CMS Software and Computing Project is the response of US physicists in CMS to this challenge.

The goal of the US CMS Software and Computing Project is to provide the software and computing resources needed to enable US physicists to fully participate in the physics program of CMS. Additionally it should allow US physicists to play key roles and exert an appropriate level of leadership in all stages of the computing-related activities, from development of the reconstruction programs and software infrastructure to the extraction of physics results. This capability should extend to physicists working at their home institutions.

A key element in achieving this goal is to develop the software and to construct the facilities to provide an integrated environment for remote collaboration that would make possible central US roles in the data analysis. This includes:

1. providing the resources to support participation in the development of software associated with the design, calibration, commissioning, and analysis of the detectors in which US CMS members are involved;

2. providing the resources to support the participation in the development of reconstruction, simulation, and analysis frameworks and other physics applications infrastructure at a level appropriate to the number, capabilities, and experience of US CMS physicists; and

3. providing the resources and facilities for participation by US CMS physicists, especially those who wish to remain based in the US, in all analysis efforts and activities of interest to them.

The word ‘resources’ is meant to include personnel -- for development, operations, and support, as well as the hardware, commercial software purchases, and contracts for other services required to achieve these goals. US CMS functions within the context of the full CMS experiment which in turn functions as an experiment of CERN. It is essential that this project stay well aligned with both the scientific goals of US CMS and with the policies and approaches of CMS and CERN. This project provides services and facilities that couple smoothly to CERN central computing, and to CMS facilities worldwide, so that US physicists can work productively whether at their home institutions or at CERN.The US CMS Project is closely coordinated with the international CMS Software and Computing project1 that covers the computing aspects of the design, construction, evaluation and calibration of the CMS detector

(1) the storage, access and processing of event data

(2) event reconstruction and analysis, and

(3) the computing and remote collaborative infrastructure for the above.

This project supports US CMS efforts to do its appropriate share of the CMS work and also to solve special problems related to the geographical separation of US physicists from the site of the experiment.

CERN has stated clearly its policy that significant resources to support data analysis must come from sources external to CERN. This project responds to that policy by marshaling US national resources to support the analysis activities of US physicists on CMS. The US expects to do this in a cost effective way by leveraging the knowledge, talent, and experience with HEP computing that exists within US universities and Fermilab, which is the US CMS host institution

The US also has particular issues to deal with as a result of being separated from the experimental data and the central data repositories by an ocean and by six to nine time zones. The US CMS Collaboration is itself widely spread out, as a

5

result of the geographical expanse of the United States. US physicists are thus particularly dependent on a well-coordinated distributed data analysis system, able to deliver data and/or analysis results reliably and with acceptably short delays across transoceanic as well as national networks. These issues coupled to the unique position of the US in computing and networking technologies make it unavoidable that the US takes a lead role in these efforts, and bears the brunt of much of the R&D work and associated costs.

The long distance to the experiment means that US physicists are particularly reliant on videoconferencing for meetings and participation in the daily activities of the experiment. A high quality remote collaborative environment is required for collaborative work on the software and data analysis, and will be an important part of this project. An extension of this work will result in the ability to operate a remote “shift taking” facility in the US, including in-depth monitoring of the detector and the data acquisition system.

No remote collaborative technology can fully compensate for the out-of-phase work cycles resulting from the six to nine hour time difference between the US and Europe. The US is thus obligated to focus some of its activities nationally and sometimes regionally within the US, in order to allow the physicists of US CMS to work efficiently. This problem will be partly solved by the presence of a Tier 1 Regional Center at Fermilab in the US Central Time zone, use of remote collaborative technologies within the US, and the use of an hierarchical “computational data grid” that places a “Tier 2” Regional Center in each of several US regions.

The remainder of this document describes the management structure of the project and presents the Work Breakdown Structure (WBS), budget, schedule and key milestones.

6

2. Project Organization

The project organization chart is presented in Figure 1.

The Software and Computing project has two major subprojects:

Core Applications Software (CAS)2

User Facilities (UF)3

This project will provide the core framework and infrastructure software, and hardware to support the reconstruction, simulation, and physics analysis. The project does not include the development of the actual reconstruction software or software for specific physics analyses, much of which will be written by physicists. These activities appear on the organization chart as two "boxes" called "reconstruction and detector software", and "physics analysis"4. While the funding of the project described here is separate from the reconstruction and physics analysis software, they are obviously closely related and must be coordinated. This is discussed below.

The remainder of this section provides a brief characterization of the two major subprojects.

2.1 The Core Applications Software Subproject

The Core Applications Software Subproject will develop software

to support the design, development, modeling, optimization, and commissioning of software related to detectors being constructed by US CMS;

to provide its share of the framework and infrastructure software required to support data analysis and simulation for CMS;

for remote collaborative tools and services for the distributed model of computing that will enable members of US CMS to carry out data analysis while they are at home in the US or resident at CERN; and

to satisfy any specialized needs required to carry out data analysis activities of interest to members of US CMS.

US CMS' core software efforts have focussed on three main areas: ORCA: the object-oriented event reconstruction The User Analysis Environment Distributed System Development

The US will also, by virtue of its interests and its expertise, participate through the Core Applications Software Project in various R&D and common projects.

In addition to developing software, this subproject will also provide expert programming personnel to assist the physicists in developing reconstruction and physics analysis programs by serving as mentors, reviewers, advisers and, where appropriate as software tool-writers. This will ensure that the software produced will be easy to integrate into the whole system, will be efficient in its use of hardware resources, and will be maintainable and adaptable for the full life of the CMS data analysis.

7

Figure 1: Abbreviated version of organization of the US CMS Software and Computing Project showing the main subprojects and the key links to US CMS, Fermilab, and the US funding agencies. More detail on the relation to the US CMS construction project is shown in Fig 2.

2.2 The User Facilities Subproject

The mission of the "User Facilities" subproject of the US CMS Software and Computing Project is to provide the enabling infrastructure to permit US CMS collaborators to fully participate in the physics program of CMS, and to do so from their home sites if they choose. This enabling infrastructure will consist of the software and hardware needed to access, analyze and understand data as well as to support software development efforts by US CMS physicists and computing professionals. The major cost and personnel-requirements driver for the subproject is a regional computing center at Fermilab to support US physicists working on CMS. It is appropriately sized to support this community which comprises 20% of the full CMS collaboration.

The Fermilab regional center will include substantial CPU, data storage and data access facilities. It will be able to deliver portions of the data as required to other US CMS institutions through high-speed networks and will have a high bandwidth network connection to CERN. It will also include user support personnel, personnel to manage licenses and

8

license acquisition, and personnel to contract for needed services. It will have the responsibility and personnel to develop or acquire any software that is required to carry out its production and operation activities. It will provide a variety of training and consulting services to help university physicists carry out their computing activities at their home institutions and at Fermilab. It will also provide support for many development activities during the detector construction period before data taking begins.

In addition to developing software required to carry out its mission, this subproject will also provide expert programming personnel to serve as liaisons to programmers developing software by serving as reviewers and advisers. This will ensure that the software produced elsewhere in US CMS will be easy to integrate into the whole system, will be efficient in its use of hardware resources, and will be maintainable and adaptable for the full life of the CMS data analysis.

The User Facility subproject has as part of its mandate supporting Tier 2 regional centers. A Tier 2 regional center, as defined by the joint ATLAS/CMS MONARC project, is expected to be sized at about 2-5% of the main CMS CERN facility (and therefore 10-20% of the capacity of a ‘Tier 1’ center). The US Tier 2 regional center sites are not yet identified, but Fermilab plans to support data import/export, documentation and software distribution to these centers.

9

3. Upper Level Project Management

The roles and responsibilities of the upper level project management are described in this document. These entities are:

the Level 1 Project Manager (L1PM); the Level 2 Project Manager (L2PM) for the Core Applications Software subproject; the Level 2 Project Manager (L2PM) for the User Facilities subproject; the Fermilab Computing Division (CD) and its Division Head; the US CMS Advisory Software and Computing Board (USASCB); and the Fermilab Director or designee advised by the US CMS Project Management Group (PMG).

Fig. 2 shows the organization chart again but this time with a more complete view of how the project components are tied to Fermilab, the US funding agencies, the US CMS construction project, and US CMS software efforts in reconstruction and physics analysis (These software efforts are shown schematically by two boxes which are meant only to indicate the activities. They do not imply any specific organizational structure since they are still evolving.)Figure 2: Organization Chart for US CMS Software and Computing Project. This shows how the project components interact among themselves, with Fermilab, the US funding agencies, the US CMS Construction Project and US CMS software efforts in reconstruction and physics analysis. 3.1 The Level 1 Project ManagerThe Level 1 Project Manager is responsible for completing the project by achieving the approved scope within budget and on schedule. He/she must also ensure that the deliverables of the project conform to the technical specifications that are set for them. Finally, he/she is responsible for doing all this in manner consistent with CMS scientific policy.The L1PM will be appointed by Fermilab in consultation with the USASCB and will receive the concurrence of the DOE and the NSF. During his/her tenure, the L1PM will be a member of Fermilab’s staff. Administratively, the L1PM will be a member of the Fermilab Computing Division and his/her staff organization will reside in the CD. It is expected that the L1PM will have experience and background in HEP experiment, software development, management, and operations issues and skills that would predict success as a project manager in this role.The specific responsibilities of the L1PM include

1. developing the baseline project plan especially with respect to budget, personnel requirements, schedule, Level 1 (L1) milestones, and L1 deliverables;

2. executing the approved project plan in a manner consistent with the technical and scientific policies of US CMS;

3. developing an integrated Cost and Schedule Plan; 4. establishing and maintaining the project organization, within the Fermilab Computing Division, required

to manage procurements, maintain schedules, submit reports, develop budgets, carry out quality assurance, maintain the project plan and the record of all revisions to it, and maintain safety standards and records;

5. developing the annual budget request to the DOE and NSF. This request is reviewed by the USASCB and must be approved by the PMG;

6. providing liaison between the project and the CMS Software and Computing Technical Board (SCTB);7. at his/her discretion, in consultation with the USASCB and with the concurrence of the PMG , appointing

a deputy or other supporting staff to assure management continuity during periods when he/she is absent or unavailable;

8. appointing, the two Level 2 (L2) managers in consultation with the USASCB and with the concurrence of the PMG. Consulting on and concurring with the appointment of Level 3 (L3) managers by the L2 managers;

9. developing or adopting general technical and quality assurance standards to which deliverables must conform. This includes making sure that US CMS software and facilities conform to applicable CMS and CERN standards and practices, and can operate with them without unnecessary additional integration effort;

10. providing coordination and oversight to the Level 1 and Level 2 projects. Oversight may be accomplished by requiring appropriate regular reports, following and tracking the results of technical reviews, and conducting reviews using both internal and external committees or experts. Coordination may involve

10

making sure that the User Facilities Subproject can supply test beds to the Core Applications Subproject for major development activities;

11. making adjustments to Memoranda of Understanding, MOUs, and Statements of Work, SOWs, with collaborating universities and laboratories. These MOUs and SOWs are initially formulated as part of the project plan but adjustments may be necessary in the course of the project;

12. allocating resources within the project. These allocations are largely set by the project plan and the annual budget request. However, adjustments are often necessary. The project plan includes a contingency that will be allocated subject to the formal change control procedure;

13. reporting variances from the scope, schedule, or cost estimates to the PMG and developing action plans for dealing with them. The L1PM informs the USASCB of such variances;

14. exercising change control authority as described in this plan and bringing change issues that exceed his/her authority to the attention of the PMG;

15. establishing technical advisory committees where appropriate; 16. providing reports and organizing reviews in conjunction with the funding agencies; responding to requests

for information from US CMS, Fermilab, and the funding agencies; and17. developing a technology-tracking plan, in conjunction with the Level 2 managers and with similar efforts

at CERN, with the advice of the USASCB. The tracking plan should allow the project to take advantage of new, more cost-effective technologies that may arise during the period of the project's execution.

3.2 The Level 2 Project ManagersThe Level 2 Project managers are appointed by the Level 1 Project manager in consultation with the USASCB and with the concurrence of the PMG. The main responsibility of the L2 Project managers is to manage their subproject so that they successfully produce the deliverables of the project on schedule, within budget, and within their technical specifications.3.2.1 The Core Applications Software SubprojectThe L2 manager of the Core Applications Software Subproject is appointed by the L1PM in consultation with the USASCB and with the concurrence of the PMG. This position will normally be held by a physicist who is a member of US CMS and is expert in software development and HEP data analysis. The subproject will be responsible for developing software components, carrying out evaluations and feasibility studies, and performing various tests.Responsibilities of the L2 Project Manager for Core Applications Software include:

1. defining, in consultation with and subject to the concurrence of the L1PM, the milestones and deliverables of the subproject;

2. developing the technical specifications of each component and deliverable of the subproject;3. defining, in consultation with and subject to the concurrence of the L1PM, the organizational substructure

of the subproject. This includes defining the L3 projects and proposing, subject to the approval of the L1PM, the L3 project leaders;

4. developing, under the guidelines provided by the L1PM, the annual budget proposal for the subproject;5. allocating resources and identifying resource imbalances or deficiencies within their subprojects.

Allocations are largely set by the project plan and the annual budget request.

Making adjustments within prescribed limits of resources within their subprojects. Proposing adjustments to SOWs with collaborating universities and laboratories to the L1PM. Changes are subject to the change control procedures outlined in this document;

6. delivering the scope of the subproject on schedule, within budget, and in conformance with the technical specifications and quality assurance guidelines of the project;

7. organizing (and documenting) internal reviews to make sure that deliverables will meet the technical specifications and the quality assurance standards of the project;

8. being accountable for all funds allocated to the subproject;9. maintaining the Cost and Schedule Plan for this subproject; 10. providing reports as required to the L1PM; and

11

11. carrying out any other duties assigned by the L1PM., the SCB, and the PMG.

3.2.2 The User Facilities Subproject The Level 2 manager of User Facilities Project is appointed by the L1PM in consultation with the USASCB and with the concurrence of the PMG. He/she will be experienced in and knowledgeable about HEP physics analysis production issues, hardware acquisition, operations issues, and the development of software components as relates to operations and production. The Level 2 manager will have significant responsibilities connected with the US CMS Regional Computing Center at Fermilab and will normally be a member of Fermilab’s staff. Responsibilities of the L2 Project Manager for the User Facilities Project include:1. developing the definition of the milestones and deliverables of the subproject;

2. developing, subject to review by the L1PM and with the input from the USASCB, the technical specifications of each component and deliverable of the subproject which will include the acquisition of hardware components, contracting for services (such as network connectivity), acquiring software for operations or monitoring, and developing software to integrate components and to manage production or operations activities where required;

3. defining the organizational structure of the subproject. This includes defining the L3 projects and proposing, subject to the approval of the L1PM, the L3 project leaders. Each Tier 2 and special purpose center will be a Level 3 project and have its own Level 3 manager, who will report to this L2 manager. For Tier 2 and special purpose centers, each center will appoint, with the concurrence of the Level 2 manager for User Facilities, a manager who will be responsible for CMS activities at the Tier 2 or special purpose center and this manager will be the Level 3 project leader for the center for the US CMS Software and Computing Project;

4. developing, under the guidelines provided by the L1PM, the annual budget proposal;5. negotiating adjustments to the MOU with Fermilab, in particular with the Computing Division, for

support of US CMS Regional Center;6. negotiating annual MOUs with Tier 2 or special purpose centers, and with their managing organizations

as appropriate, should they become part of US CMS computing. The purpose of the MOU is to define what specific deliverables the center will provide and what resources and services are needed from the Tier 1 center;

7. delivering the scope of the subproject on schedule, within budget, and in conformance with the technical specifications and quality assurance guidelines of the project. This will include developing Requests for Information (RFIs), Requests for Proposals (RFPs), acquisition plans, implementation plans, technical specifications, acceptance tests and criteria, etc;

8. organizing (and documenting) internal reviews to make sure that deliverables will meet the technical specifications and the quality assurance standards of the project;

9. being accountable for all funds allocated to the subproject;10. maintaining the Cost and Schedule Plan for this subproject;11. providing reports to the L1PM;

12. working with potential candidates for Tier 2 and special purpose centers to prepare their proposals. The parts of proposals for such centers , which relate to CMS, must be approved by the management chain described in this document; and

13.carrying out any other duties assigned by the L1PM.

3.3 The US CMS Advisory Software and Computing Board (USASCB)

The US CMS Advisory Software and Computing Board provides input and feedback for the US CMS Software and Computing project. The composition of the board and its relationship to US CMS and CMS are partially described in this document5. It is composed of:

12

six at large members elected from the US CMS collaboration;

the US CMS Physics Coordinator (also elected);

the Chair of the US CMS Collaboration Board (ex-officio);

the Head of the Fermilab Computing Division (ex-officio);

the CMS Project Manager for Software (ex-officio);

the Project Manager of the US CMS Construction Project (ex-officio);

the Level 1 Project Manager for the US CMS Software and Computing Project (ex-officio); and

the Level 2 Project Managers for the User Facilities subproject and for the Core Applications subproject (ex-officio).

The 7 elected members of the ASCB will select a chair who shall be one of the 6 at large members.

3.3.1 Development of the Project Plan

Working with the USASCB, the Level 1 Project Manager, in consultation with US CMS, creates a plan for the Software and Computing Project, subject to approval by the PMG. The USASCB has the specific responsibility of providing the interface to US CMS in setting the requirements for this plan and for helping to develop any overall policies associated with it.

The Level 1 Project Manager, with advice from the USASCB, will develop initial funding profiles, resource loaded schedules, budgets, Memoranda of Understanding (MOUs), and Statements of Work (SOWs) for the "baseline" project. These will be presented by the Level 1 Project Manager to the PMG for approval and eventually to the funding agencies.

3.3.2 Scientific and Technical PolicyThe USASCB advises the Level 1 Project Manger on scientific and technical policy for the US Software and Computing Project consistent with the scientific direction and goals of US CMS and CMS. Technical, scientific, operations, and resource allocation issues are to be discussed frequently by the board with the goal of providing continuous input and feedback as required for effective execution of the Software and Computing project. Recommendations are made to the Level 1 Project Manager. Scientific issues with a major potential impact on the physics results to be obtained by US CMS will be brought before the US CMS Advisory Board, and to the US CMS Collaboration Board, if needed, and resolved in a manner consistent with International CMS.

13

3.3.3 Project Input and Feedback The USASCB provides scientific and technical input and feedback to assure that the project is indeed carrying out the policies of US CMS and International CMS , which will certainly evolve over time, and is faithfully executing the project plan.3.4 Fermilab Oversight and the Project Management GroupThe Department of Energy and the National Science Foundation have requested that Fermilab, as CMS Host Institution, exercise management oversight for the US CMS Software and Computing Project. Oversight responsibility is vested in the Fermilab Director or designee, who is advised by the US CMS Project Management Group. The Fermilab Director or designee reports to the Joint Oversight Group.3.4.1 Project Management Group (PMG)To provide oversight of the US CMS Software and Computing Project, the mandate of the Project Management Group for the US CMS Construction Project, will be extended. Specific members will be added to the PMG for their expertise in software, computing, and data analysis issues. These will includethe Level 1 and Level 2 Project Managers; and

the chair of the USASCB.The US CMS Collaboration Board Chair , the Fermilab Associate Director for Research, and the Head of the Fermilab Computing Division are members of the PMG with a particular interest in the Software and Computing project. Other members or observers with specific expertise or interest in this area and who represent US CMS, Fermilab, or the funding agencies may be added.These members will comprise the PMG subgroup for the US CMS Software and Computing Project. The chair of the PMG will appoint a chairperson for this subgroup. The subgroup chair in consultation with the L1PM and with input from the chair of the USASCB prepares the agenda for these meetings. The PMG Chairperson may also choose to hold meetings with this subgroup or to have joint meetings of the group associated with the Construction Project and this subgroup based on the issues to be addressed. The chair of the PMG prepares the agenda for these meetings. The PMG receives the reports of the L1PM for the US CMS Software and Computing Project.Oversight of the project is implemented in part through reviews. Along with providing routine interactions with the project management, the PMG will identify actions and initiatives to be undertaken to achieve the goals of the project including the allocation of both financial and human resources. The PMG also functions as the Baseline Change Control Board for the project.3.4.2 External Review CommitteeThe chair of the PMG will establish a standing external review committee that will periodically examine and evaluate all aspects of the US CMS Software and Computing Project. It is highly desirable that this committee be chaired by a recognized expert in HEP computing and that its membership include computing experts from CERN and strong representation from CMS software management.3.4.3 Change ControlDetailed change control thresholds are established in three areas: technical changes, schedule changes, and cost changes. The values of these thresholds and the authority that approves a proposed change in each area at each threshold will be set when the project baseline is established.4. Interrelationship with Other entities4.1 Relation to CMSThe CMS collaboration has organized the software and computing efforts as a subsystem just like any other. The “L2 Manager” for software and computing represents this subsystem on the CMS Steering Committee. The technical activities of subsystems in CMS are managed in a “Subsystem Project Office”. For this particular subsystem, this entity is called the Software and Computing Technical Board (SCTB) which is chaired by the L2 manager. The CMS L2 manager has overall responsibility for and direction of the CMS Software Project including all tasks carried out for that project by the US CMS Software and Computing Project. The collaborative aspects of a subsystem in CMS are handled by an “Institutional Board”. To create a strong linkage between the US CMS Software and Computing Project and the overall CMS Project, the CMS L2 manager will be an ex officio member of the USASCB. The USASCB will have the responsibility for providing liaison between the US CMS Computing Project and the CMS Software and Computing Board, and the US CMS Software and Computing Project Level 1 Project Manager will act as liaison to the CMS Software and Computing Technical Board.The US CMS collaboration will be contributing to CMS computing in a variety of ways, each of which will have an appropriate formal mechanism for establishing milestones, deliverables, and specifications. Levels of support for production activities, including those required to support the design, testing, simulation, and commissioning of the detector should be supported by MOUs negotiated with CMS by the L1PM with input from the USASCB and with the approval of the PMG and funding agencies. The software development that directly relates to the international CMS effort should be developed as part of the CMS software and computing plan and approved, presumably as part of the project plan for the US CMS Software and Computing Project, by the PMG and the funding agencies. Software efforts specifically in support of US physicists or intended to solve particular problems specific to the US, should be developed as part of the project plan

14

with substantial input from USASCB and approved by the PMG and, if required, by the funding agencies.4.2 Relation to US CMS and to the US CMS Construction Project The US CMS Software and Computing Project provides suitable computing hardware and framework and infrastructure software to facilitate a broad range of offline software tasks by the US CMS Collaboration. At present, major US CMS activities are R&D on the network-distributed LHC computing models, building the reconstruction, detector and analysis software, study and verification of the CMS detector design, high level trigger and data acquisition system performance, and study of CMS' physics capabilities using fully simulated and reconstructed events.The work by US physicists on these tasks is fully integrated with the work of the CMS collaboration as a whole. US CMS has several areas of leadership, including the computing model R&D, reconstruction and detector software, the user analysis environment, and high level triggers. In the LHC running phase a more unified US CMS structure managing the software and data analysis will be formed as appropriate (see section 5). The US CMS Construction Project is responsible for the construction of specified elements of the CMS detector as designed by US physicists. The scope of the Project extends from the detector elements to transducers to front end electronics, through L1 pipelined triggers, L2 and L3 triggers and data acquisition including the L3 computing farm and data logger. The US Software and Computing Project concerns the steps needed to facilitate these tasks all the way to physics analysis by providing suitable hardware and framework and infrastructure software. The CMS experiment is a unified enterprise, in which the Construction Project and the Software and Computing Project will be well coordinated to meet the needs of US CMS physicists seamlessly as they work on tasks from the building and commissioning of detector elements, to building the software and production systems, to the physics analysis. 4.3 Relationship to the US Funding AgenciesThe US CMS Project Management Plan for the construction of the detector contains a detailed description of the interaction between the project staff and the funding agency (DOE and NSF) personnel. Key elements of this are the Joint Oversight Group, the U.S. LHC Program Office, the Chicago Operations Office of the DOE, the US LHC Project Office, and the Fermilab Director or designee . The oversight by the agencies is similar to the plan that is in place in the construction project. In particular, the organization chart (Figure 2) shows that the PMG advises the Fermilab Director or designee who reports to the Joint Oversight Group.

15

4.4 Relation to the Fermilab Computing DivisionThe Computing Division is home to the US CMS Software and Computing Project within Fermilab. The Division supports the project in many ways. The L1PM and the L2PM for the User Facilities subproject are members of CD. The ‘project office’, which provides administrative resources to the project management, resides within CD. The Head of the Computing Division is a member of the PMG and is an ex officio member of the USASCB. The staff for the Tier 1 Regional Center and much of the staff for the User Facilities Project are members of Computing Division. 5. Evolution of the US CMS Software and Computing Project This plan applies to the ‘initial development and deployment phase’ for the software and facilities. Once that phase is complete, the software and the facilities go into the ‘operation, support, and further development phase’. This should occur a few years after CMS starts taking data. At that point, there should be a new ‘operations plan’ which would replace this plan. It is very important to recognize that software development and hardware evolution will continue throughout the life of CMS. The resources required for the ongoing operation of the facilities and evolution of the software and hardware are quite significant. The operation of the Regional Center is at least a 15-20 year commitment. For at least 2/3 of its lifetime, it will be in an ‘operations’ rather than ‘construction’ phase. Continual investment in software development, technology tracking, and R&D throughout this period will be essential if the facilities are to continue to serve the interests of US CMS as physics and computing technology move forward. Similarly, the scientific software will be in a state of continual development and evolution throughout the operations period. This will be driven both by changes in physics goals and analysis techniques and by changes in underlying software and hardware technologies.

16

Table

6. High Level Milestones, Work Breakdown Structure and Budget for the User Facilities Subproject

The Tier 1 Regional Center will be managed within the Fermilab Computing Division, and falls under the responsibility of the Level 2 Project Manager for the User Facilities Subproject. The necessary R&D, prototyping, establishment and operations of the Regional Center are all part of the WBS for the US Software and Computing Project, which is described below. The Regional Center needs to be fully responsive to the needs of its constituencies. Sections 3-5 of this plan describe the points of contact with the US CMS and International CMS collaborations.

6.1 User Facilities High Level Milestones

A timeline for the establishment of the Tier 1 Regional Center, together with some relevant CMS milestones, are listed below:

Year USCMS activity CMS Milestone (by US fiscal year) (calendar year)1999-2003 R&D Phase

17

2000 select Regional Centers2002 "Turn on" a prototype functional Regional Center2004-2006 Implementation Phase2005 Fully operational centers2006 Begin operation 2007 Operations phase

6.2 User Facilities Work Breakdown Structure

The numbers at the beginning of each of the following items are the Work Breakdown Structure levels. Level 1.0 is the User Facilities Subproject.

1.1 Tier 1 Regional Center at FNAL

This item covers the capital equipment and ongoing operational requirements of the Tier 1 Regional Center. Hardware will be acquired over a three year period from 2004 to 2006, in order to spread the cost over several fiscal years, to provide some reduced capacity capability before 2006 for mock data challenges and test beam analysis, and to provide an adequate time period for full system integration. Consistent with these aims, the hardware will be acquired as late as possible to maximize the performance/price ratio of the systems purchased. There will also be ongoing expenses for the required hardware support to ensure full availability of the services and systems provided. Software systems do not share the falling prices expected of hardware, and in some cases have longer lead times for development, and so software system expenses here will start in earlier years.

1.1.1 Hardware

1.1.1.1 Development and Test Systems

These systems support software development and testing of both system and application software, including testing of user jobs on event samples which are too large for the desktop but do not require full production systems. Small-scale systems of each architecture used in production servers will be required.

1.1.1.2 Simulation Systems

These systems support large scale production of simulated events, including event generation and full detector simulation. This activity will almost entirely be carried out at Regional Centers and local CMS institutions. This is an extremely CPU intensive application, and will make use of inexpensive commodity computers with limited I/O performance.

1.1.1.3 Reconstruction Systems

These systems support the reconstruction of both data and simulated events. Primary reconstruction of data will take place at CERN in quasi real time, but the regional center will perform primary reconstruction of simulated data, and secondary reconstruction (with improved algorithms or calibrations) of samples of events, on both a scheduled and on-

18

demand basis. Special reconstruction of calibration data samples of interest to US subdetector groups will also be carried out here. This is again a CPU intensive application and will use inexpensive commodity computers.

1.1.1.4 Analysis Systems

These systems provide the computing hardware needed for data analysis, the primary function of the Regional Center. These systems must support large numbers of simultaneous users, provide a single system image no matter what physical hardware the user is logged on to that day, and provide high bandwidth input/output and network connections. Currently UNIX SMP (symmetric multi processor) machines are used for this function. It is expected that the I/O and network requirements will result in somewhat higher cost CPU for this function, but as much use as possible will be made of inexpensive commodity computers.

1.1.1.5 Data Access and Distribution Servers

These systems provide the computing, disk caching and network facilities required to provide access to the event data stored in databases, and to supply this data over the network to both local and remote users.

1.1.1.6 Data Storage Systems (Disk, Tape, Robotic Storage)

These systems provide the hierarchical data storage for event data, calibration data, and non-event data necessary for physics analysis. They will include on-line disk, secondary robotic tape storage, secondary human mounted tape storage, and import/export facilities to move data into and out of the various levels of the data storage hierarchy.

1.1.1.7 Database Servers

These systems provide the computing, disk caching and network facilities required to provide access to the non-event data stored in databases, including geometry, calibration, trigger, run condition, and other data bases.

1.1.1.8 Serial Media and Media Copy and Distribution

This system provides the hardware required to copy and distribute data on physical media (not over the network).

1.1.1.9 Information Servers

These systems provide the computing, disk and network resources required to serve information to the collaboration, primarily using the World Wide Web. This information includes documentation, publications, talks, engineering drawings, and other collaboration notices.

1.1.1.10 Desktop Systems

19

These systems provide desktop computing resources to local users and visitors at the Regional Center.

1.1.1.11 Monitoring and Problem Response

These systems provide the hardware tools to monitor and optimize performance of regional center hardware, and to detect bottlenecks or other problems at an early stage to allow rapid response.

1.1.2 Software Systems

These systems provide the critical software infrastructure for performing CMS computing tasks, other than the infrastructure for physics code provided by the Core Application Software Subproject. In most cases there is some overlap with the CAS Subproject. In general this subproject concentrates on those services related to operations and support of production systems, while the Core Applications Software (CAS) Subproject concentrates on software design and development, initial or non-production software deployment and end user interface aspects. Software development within the User Facilities Subproject addresses those systems needed to operate and generally deploy CMS software. These include enhancements to commercial packages or tools that increase the efficiency and effectiveness of the Regional Center. User Facilities expenses include the commercial software licensing and maintenance fees, the costs associated with developing and supporting the needed extensions to available software to meet the CMS collaboration needs, and the personnel costs associated with running and supporting the systems and their services. In all cases these projects are tightly connected with the overall CMS software efforts.

In some cases the US has taken a leadership role in these software projects, in others we are one of several collaborators, but in all cases it is critical that we ensure the chosen tools allow for efficient and effective participation by US physicists in collaboration activities.

1.1.2.1 Database and Database Management

These systems provide the database and database management software and services to store, access, and manage CMS event, calibration, and other non-event data. The User Facilities subproject will work as part of the overall CMS collaboration in the selection and/or development of a CMS standard database system. Extensions and enhancements to the base tools will be developed or acquired as needed for effective access to and management of the data. The size of these databases could be very significant and may need to provide a local central repository for data from and stored on behalf of other local centers. Special techniques are needed to ensure efficient query access, update and synchronization of the databases of the Regional Center with the central site at CERN and other local centers. Complete bookkeeping will be part of the services of the databases provided by the Regional Center. Experience gained from the development and support of such systems during CDF and DØ Run II data taking will be valuable input to the development of the services needed by CMS users. Further development to provide a distributed integrated database system that functions across wide area networks will be required. These developments will benefit from the 'Gird' developments in the US and Europe.

1.1.2.2 Data Access Software

These systems allow for optimized access by CMS physicists to the object-collections of interest, drawn from the large quantities of data stored at the Regional Center. The data access software must provide for and schedule the transfer of data between central site at CERN and the Regional Center, and between the Tier 1 Center and other local Tier 2 centers. It is expected that both network and “Federal Express” shipping options will be required to provide the

20

necessary guaranteed bandwidth to the users of the data. These facilities will be developed taking into account the experience gained from operation and support of the CDF and DØ Run II data access systems and the provision of Run II data to the offsite institutions. The data access software must handle prioritization and allocation of resources among competing users and physics groups, and combining or otherwise scheduling data requests for maximum performance. The systems must provide event and run catalogs and support creation of streams of data and selections of events, or object-collections drawn from events.

1.1.2.3 Data Distribution Software

This system concentrates on the problem of providing data access to those CMS users without good network connections, or who require larger amounts of data than can be conveniently delivered over the network. One solution would be for requested datasets to be automatically extracted and written to serial media, and then physically shipped. Again, these systems must provide resource allocation and optimization of requests.

1.1.2.4 Simulation Software

These systems provide the infrastructure for performing event generation and detector simulation on a production basis. Most of this software is application specific, but support must be provided for the generic detector simulation toolkits (presently GEANT4) that are used, as well as for the physics event generator software.

1.1.2.5 Data and Physics Analysis Software

This system provides tools and environments for analyzing and visualizing the event and non-event data. A major component of this system will be the tools selected as overall CMS standards, LHC++ and other such packages. A sufficient set of these systems must be easily and effectively usable and accessible to the Regional Center users and US physicists. These tools must be available on the typical inexpensive commodity desktop. These systems may require extension to support analysis of the data using multiple complementary tools.

1.2 System and User Support

These services and software provide the fundamental infrastructure for the support and use of the Regional Center Facilities. Because we are supporting physicists doing software development, detector design and test beam work during the R&D phase, system and user support services are necessary during all phases of the project. In addition to supporting physicists during the R&D phase, they provide for the efficient and effective use of the Regional Center facilities by the US CMS physicists, other local Tier 2 regional centers, and by CMS collaborators located throughout the world. In each case, the expenses are a mixture of commercial software acquisition and licensing costs, and personnel costs for software development and ongoing support and maintenance.

1.2.1 Document Generation and Access Tools.

Software to support preparation, access to, management of, and distribution of documentation, both of software systems and of CMS hardware and detector systems. Automatic generation of documentation from the code repository is already being used for Run II. Local system and software documentation and distribution is included in here.

21

1.2.2 Collaborative Tools

The key goal of facilitating physics research by CMS collaborators independent of their geographic location requires development, widespread deployment and systematic support of new tools to facilitate collaboration at a distance. This includes relatively conventional areas such as distributed code management systems and ordinary videoconferencing, to more advanced approaches such as extended virtual and tele-presence products and innovative distant sharing of physics analysis. Work is ongoing in these areas and further R&D is needed.

The UF subproject focuses on the deployment and general support in the US for these tools. Initial and ongoing R&D is being undertaken at Caltech in collaboration with CERN on remote collaboration systems as part of the CAS subproject.

1.2.3 Code Management and Code Distribution Tools

Tools to provide code versioning, code building, code release and code distribution. These tools must support worldwide physicist-developers, and must allow stable reliable software versions coexisting with rapid development of new code. Here again there will be an interface with the CAS subproject, which will cover the initial design, development and packaging of software releases, while the UF deals with their widespread deployment and support for their use in production-mode.

1.2.4 Software Development Environment

Development and installation of the software, which allows US CMS collaborators to develop software on the Fermilab User Facility employing standard software components and practices established for use throughout CMS.

1.2.4.1 Code Installation

This covers the actual CMS code installations from CERN CVS repositories on user facility machines at Fermilab.

1.2.4.2 Code Verification

This covers software development of code used to verify that the Fermilab and Tier2 center installations are correct and are consistent with the CERN installation.

1.2.5 Software Development Tools

Software systems used by physicists in designing, developing and testing their own software, whether it is to be used privately or included in official CMS software packages.

1.2.6 System Administration, Monitoring and Problem Resolution Software

Tools to allow system administration (account creation and deletion, disk quota and allocation, system software installation and update, etc) for the very large number of individual systems required at the Regional Center, and to allow monitoring and problem detection of the systems and automated and efficient response.

22

1.2.7 User Help Desk and Information Systems

Personnel and tools to allow for efficient consulting and help services at the Regional Center, providing for needs of both local and remote users.

1.2.8 Training Office

1.2.8.1 Coding classes and instruction

One of the roles of the Regional Center will also be to provide training for US CMS collaborators. We have already initiated training courses at Fermilab in the use of the C++ programming language, based on the successful courses already widely used by CDF and DØ, as well as Object Oriented Analysis and Design. In addition to training in C++, Java, software design and documentation and (later) the software development process for large projects, we intend to provide tutorials on the use of CMS-specific software and computing systems.

Training will take the form of packages for self education as well as formal training classes, and will provide for both local and remote users. Remote collaborative systems being developed will include support for multi-site interactive training classes, using packet videoconferencing and associated tools.

1.2.8.2 Software and training workshops

In addition to more formal classes in coding and design, a series of CMS software workshops is envisioned to instruct US CMS physicists in running and developing software at supported systems at Fermilab.

1.2.9 Support for Tier 2 Centers

This item covers support in the design, implementation, installation and operations phase for Tier 2 Centers. Once these centers have been selected and scoped, the WBS for this item will be fully developed.

1.3 Maintenance and Operation

1.3.1 Hardware Maintenance

This item covers licensing and maintenance of the Regional Center and R&D hardware.

1.3.2 Software Maintenance

This item covers installation, licensing and maintenance of theRegional Center supported software.

1.3.3 Systems Operation

This item covers the Regional Center operation and includes media costs, operations staff, and other such expenses.

1.3.4 Infrastructure Support

23

This item covers management staff, building and site expenses, power and cooling costs, etc.

1.4 Support of Tier 2 Regional Centers

This item covers support provided by the User Facilities Subproject, and in particular by the Tier 1 Regional Center, for US CMS Tier 2 centers. This includes coordinating with Tier 2 Centers in sharing computing tasks, keeping regional copies of datasets, providing software and keeping it up to date, supplying various consulting and training services, and providing communication and coordination with CMS and CERN.

1.4.1 Support of US CMS University Tier 2 Centers

Several US CMS universities have proposed building modest computing centers at their home institutions for regional support of US CMS physicists. Both hardware and software support for those centers are covered in this item.

1.5 Networking

1.5.1 Onsite Networking

This item covers the network design and procurement, installation and commissioning of all necessary networking hardware within the Feynman Computing Center at Fermilab as part of the Analysis Systems and the Data Management and Access Systems, and other networking specifically for US CMS physics analysis. Additional networking may be required as part of Storage Area Networks providing data for analysis. Security measures to guarantee authorized access only will be implemented.

1.5.1.1 Networking Technology Selection and Prototyping

The networking technology will be evaluated and prototype installations will be performed together with R&D work and test installations. Performance and cost results will be assessed. The hardware choices will be reviewed on an ongoing basis to optimize performance and minimize maintenance and replacement cost.

1.5.1.2 Networking Equipment Procurement and Installation

This item covers installations of Local Area Networks (LANs), and the updating and replacement of obsolete equipment LAN equipment.

1.5.1.3 Operation of Local Area Network

This item covers the operation and ongoing maintenance and tuning of the Local Area Network.

24

1.5.2 Domestic Networking

This item covers US CMS needs for interfacing to the Wide Area Network for remote access to the data storage of the regional center from all US CMS groups doing analysis on CMS data. The resources required are a sizeable fraction of Fermilab's total needs for domestic Wide Area Network connectivity. Security measures to guarantee authorized access only will be implemented.

1.5.2.1 Connection of Wide Area Network to Local Area Network

This item covers possible US CMS contributions to the procurement, installation or commissioning of the necessary hardware to connect the local area networks to the domestic wide area networks, as required for collaborating groups doing physics analysis.

1.5.2.2 Domestic Wide Area Network Connection

This item covers the operation and connection cost.

1.5.3 International Networking: US to CERN (dedicated CMS part)

The network requirements in the User Facilities Subproject are presently confined to the CMS-specific network facilities at the Fermilab site, and interfacing equipment of sufficient speed and functionality to connect the Regional Center to ESNet and other US national networks serving US CMS collaborators. US-CERN networking is assumed to be provided through common facilities provided for the LHC and other major DOE/NSF high energy physics programs, as is currently the case, in order to exploit economies of scale.

This item may be adapted as needed to ensure that US CMS collaborators at their home institutions, as well as at CERN, have adequate connectivity with the Regional Center.

1.5.4 Networking R&D

There are many candidate networking solutions for onsite intra-system communication, and this item covers the researching, prototyping and testing of different commercial networking systems to see which are most suitable for LAN/SAN operation. It also covers participation in the R&D required to ensure high throughput, transparent access to data distributed among CERN and the regional centers.

1.6 R&D Phase Activities

This item covers both hardware and software activities of the User Facilities Subproject up to and including commissioning of the initial prototype regional center in 2002. These include activities carried out primarily at the site of the Tier 1 Regional Center (FNAL), as well as R&D efforts at other institutes important to the design and operation of the Regional Center itself and the overall distributed data access and analysis architecture. There is some overlap of these activities with the Core Applications Software subproject, especially in the software development and testing area. In general this subproject concentrates on those aspects related to deployment, operations and support, while the

25

CAS subproject concentrates on software design and development, preparation of software releases, and end user interface aspects.

1.6.1 Hardware

1.6.1.1 Distributed Data Management and Distribution Test Beds

A key concept in the CMS computing plan is the use of interconnected centers located throughout the world to perform CMS reconstruction and analysis. The MONARC (Models Of Network Analysis at Regional Centers) R&D project 10, a common project endorsed and monitored by the LCB (LHC Computing Board) is actively engaged in studying this issue. US CMS has assumed an important leadership role in MONARC and a number of US CMS scientists are collaborating on this project.

Hardware will be needed to perform tests of MONARC concepts and to measure performance. These tests will provide important inputs to the actual design of regional centers and their interconnections, and to the proposed strategies to be used for distributed data analysis. This WBS item provides the funding for MONARC associated test-bed systems that will be used to measure fundamental parameters of distributed data analysis using various databases (as input for modeling activities described below), and to create small scale distributed data analysis test-bed systems as described in the MONARC project execution plan. The test-bed systems at FNAL will interact with other systems in the US and in Europe over wide area networks, with an emphasis on the next generation networks that are becoming available in the US.

1.6.1.2 Prototype Fully Functional Regional Center by 2002

It is expected that the hardware for the initial Regional Center operation (which will start at initial data taking in 2005) will be acquired over a three-year period, as described above. However, it is necessary to assemble a lower capacity but fully functional regional center at an earlier date to provide proof of concept at an early enough date to allow design modifications in potential problem areas. CMS has a milestone for such prototyping of Regional Center operation in 2002. This WBS item covers the hardware required for these initial tests.

1.6.1.3 Systems to Support Construction Phase Activities

Much of the computing power needed during the construction phase can be supplied by leveraging the use of shared FNAL computing resources. There will be a need for some modest dedicated CMS systems, including support for test beam data acquisition and analysis, certain compute intensive simulation projects, and software development and distribution servers. Some of this equipment will logically be located at other sites (Tier 2 centers and local CMS institutions) to maximize the efficiency of utilization, and to take advantage of existing facilities that may be available at US CMS universities.

1.6.1.4 Import/Export Facility

In order to support the construction phase of the experiment, and in order to research ideas in data handling and management during the R&D phase, the ability to import and export simulated CMS event data is needed as well as systems for archiving terabytes of data. This WBS item covers these three functions and will serve as both a production

26

system for CMS physicists during the construction phase and as an R&D project for data retrieval and archiving for the Regional Center.

1.6.2 Software

These items cover software activities during the R&D phase prior to the commissioning of fully functional prototype Regional Centers in 2002. Costs are primarily for personnel, located both at FNAL and at other CMS institutes, with some costs for commercial software licensing.

1.6.2.1 Distributed Data Management, Access and Monitoring Software

One of the major responsibilities of the Regional Center is the efficient delivery of data of various types to collaboration physicists. Software must be available to access and deliver the data, taking account of experiment assigned priorities and optimizing use of resources; to manage the data stores, allocating space in the various levels of the storage hierarchy; and to monitor the operation and performance of these systems. This software will most likely be based on a commercial object database management system (ODBMS) such as Objectivity/DB, but additional layers of software to provide user access, monitoring and robust operation across wide area networks that are not provided by the commercial packages will also be needed.

This item covers the deployment and use in production-prototypes of the ODBMS and additional software layers. This will leverage work for FNAL Run II that deals with similar problems on a scale that will approach that needed for CMS. The CAS subproject covers the complementary aspects of the early R&D work on the design and conceptual development of the use of the ODBMS for CMS applications, as well as the initial design and development of any software “superstructure” needed for robust operation over networks.

1.6.2.2 Distributed Computing Model Prototyping Software

As described above, the MONARC project is studying the parameters of distributed data analysis architectures with full participation by US CMS physicists. One important component of this study is to model and simulate the CMS analysis process, including a full complement of (modeled) regional and local centers. This item covers the installation, support and use of prototype software, including a selected set of analysis and database applications, in production-prototype systems to test the MONARC concepts and measure key parameters that govern the performance of the distributed systems.

This complements, and must be coordinated with, the efforts to understand and characterize the physics analysis process, and to develop, run and analyze simulations of these systems, which are described in Section 8.2, under CAS Work Breakdown Structure item 2.3.

1.6.2.3 Production System infrastructure software

This software covers fully integrating the CMS software into a production system. Examples are batch systems, archiving system, automated job submissions, etc.

27

1.6.2.4 Data import/export software

In order to support US CMS during the construction phase, and as R&D for a Tier 1 Regional Center, the problem of media exchange and non-networked data import/export must be attacked. This WBS covers development of software for importing/exporting data from/to US CMS institutions and from/to CERN.

1.6.2.5 Data archival/retrieval software

This covers software necessary to archive and retrieve data and databases in a mass storage system.

6.3 User Facilities Budget and Personnel Requirements

The scope of the Regional Center is taken from the results of MONARC and is consistent with CMS estimates. It represents 20% of CERN’s projected capability for a single experiment. This coincides nicely with the US fraction of CMS and hence represents a fair contribution. The hardware costs are based on the CMS Computing Technical proposal1, Fermilab Run II experience, CMS estimates, and present Fermilab expenditures. The CPU requirements include an estimate of the overhead engendered by the use of an object database to store data (network access and database processing).

It is difficult to set the personnel needs of the UF Subproject since there is as yet no CMS Technical Design Report for Computing. As a first estimate, the profile and FTE requirements have been based on experience with the Fermilab Run I and Run II computing and software projects, scaled appropriately. We will continue to refine these estimates as we increase our understanding of the needs of CMS, but we do not expect the final results to be significantly different than those shown.

The cost and personnel for the fiscal years 1999-2006, and for operations in the years 2006 and onward, are listed below. To summarize:

During the R&D phase of the project (roughly 1999-2003), we will purchase test bed and prototype systems for studying Regional Center architectures while also supporting ongoing US CMS simulation, software development and test beam analysis activities. The total hardware cost for R&D activities is $2.22M. During this period, the number of Fermilab CD personnel working on user facilities activities will increase from 3 (1999) to 13 (2003). The total anticipated new personnel cost for the R&D phase is $3.4M.

The total hardware cost for the Regional Center itself is $9.2M. It will be purchased during 2004-6. The three-year purchase period is the same length as that for the Run II systems. We have estimated that an additional $100k per year will need to be spent on software licenses, and $1.8M for networking, during the implementation phase. During this time the number of Fermilab CD personnel working on user facilities activities will rise from 13 (2003) to 35 (2006). The latter number is estimated from the Run II experience of the staff required to support a similar number of physicists in CDF or DØ. The total cost of new personnel for the Regional Center itself during 2004-2006 is expected to be $12.4M.

During the operations phase, $3.1M will need to be spent on hardware acquisitions per year (to replace and augment the facility). Networking will cost $1.2M per year and licenses $120k. Continuing costs for the staff supported by User Facilities subproject funds will be $5.0M/year.

We note that our CPU cost estimates are higher than some others are; this reflects the choice of relatively expensive SMP machines in the Run II plan from which the costing is derived. If the computing model permits the use of mostly commodity CPU (as was not the case for Run II), the CPU cost might be reduced. It should be emphasized however that FNAL (as well as SLAC) experience indicates that a configuration that includes a core of facilities with higher I/O performance is more cost-effective than an architecture based wholly on loosely coupled computing "farms", once the manpower requirements and costs are properly taken into account.

28

The detailed resource estimates are given in three tables, which follow.

Table 1 shows the FTE profile for the Level 2 WBS categories in the User Facilities Subproject and their distributions at Level 3. In the near term, we anticipate a need for 7 FTEs in 2000, 9.5 in 2001, 11.5 in 2002, and 13 in 2003. We intend to leverage 3, 5,6 and 6 from existing Fermilab staff, and hence expect four new positions will be needed in 2000, 0.5 additional position in 2001, one additional position in 2002, etc. The numbers include a contingency factor of 25% for the years 2004 onwards to handle difficulties and guarantee the completion of the User Facilities Subproject by the start of data taking in 2006 and a successful startup. For 1999-2000 no contingency is taken into account, while for 2001-2003 a contingency factor of 10% is included

Table 1 also shows the anticipated costs of new personnel. The average market base salary for suitably skilled Computing Professionals in the Chicago area at Fermilab, at the end of 1999, is about $78k/year. Given 28.5% for benefits, 30% for overhead, and adding travel costs, infrastructure and software licensing costs, the fully encumbered average salary for a Computing Professional is approximately $154k/year. The cost in the table is corrected for inflation starting in 2001.

29

Table 1: Personnel Requirements for the User Facilities Subproject, summarized for the higher level WBS items. The numbers include a contingency factor of 25% from years 2004 onwards to guarantee the completion by start of data taking in 2006 and a successful startup period. For the years 1999-2000 no contingency is taken into account, for the other years a factor of 10% is included. The cost is corrected for inflation starting in 2001; the official DOE escalation index is used.

Table 2 shows the evolution of installed capacity for CPU, disks, tape, and robotic mass storage for the implementation and operation phases of the Tier 1 Regional Center.

FY 2000 2001 2002 2003 2004 2005 2006 2007 (CMS operations)

CPU (SI95) - - - - 7k 30k 70k 110kDisk (TB) - - - - 10 40 100 150Tape (PB) - - - - 0.06 0.25 0.6 0.9Robots - - - - 1 1 1 1.33

Table 2: Total Installed capability by year, for Implementation and CMS Operations Phases of the Tier 1 Regional Center. In order that spending is roughly equal in each year, we plan to acquire 10% of the disk and CPU in 2004, 30% in 2005, and 60% in 2006. We will buy one tape robot in 2004, and then budget to buy an additional one every three years during operations, hence the 0.33 robots apparently purchased in 2007. Test-bed and prototype systems for 1999-2003 are not included.

30

Finally, Table 3 shows a summary by year of all hardware, R&D, networking, and software licensing costs.

FY 1999 2000 2001 2002 2003 2004 2005 2006 Total 2007 (CMS operations)

CPU ($M) - - - - - 0.91 2.46 3.08 6.45 2.13Disk ($M) - - - - - 0.20 0.40 0.54 1.14 0.44Tape and robot ($M) - - - - - 0.90 0.08 0.02 1.00 0.56

Total Regional Center 2.01 2.94 3.64 8.59 3.13

R&D systems:at Fermilab 0.00 0.28 0.51 0.75 0.75 2.29elsewhere 0.00 0.10 0.07 0.05 0.05 0.06 0.33

Networking within the US 0.52 0.57 0.73 1.82 1.24Licenses 0.11 0.11 0.12 0.34 0.12

User Facility cost(except personnel) 0.00 0.38 0.58 0.80 0.80 2.7 3.62 4.49 13.37 4.49

Table 3: User Facilities Subproject costs for the R&D, implementation, and operations Phases. The hardware costs for R&D systems cover testbed and prototype systems for the Tier 1 regional center, support of ongoing US CMS simulation, software development and test beam analysis activities at Fermilab, and the provision of a disk pool (1TB/year) for ODBMS R&D and a DLT tape system for Monte Carlo event distribution at Caltech. The cost is corrected for inflation starting in 2001; the official DOE escalation index is used.

31

7. High Level Milestones, Work Breakdown Structure and Budget for the Core Applications Software Subproject

7.1 Core Applications Software High Level Milestones

7.1.1 Schedule ConsiderationsThe schedule is defined taking into account a number of constraints: the ongoing needs of the physicists involved in the detector and trigger design optimization and construction; the research and development required, particularly for large network-distributed storage and computing systems; and the availability of resources, especially manpower. The schedule reflects an iterative development strategy, which aims to provide continuously working software while simultaneously evolving into software systems with the functionality and performance that is ultimately required.

7.1.2 Software MilestonesThe first key element of the plan is the management of the transition from a procedural software paradigm, based on Fortran and C, to an Object Oriented paradigm based on C++. This is necessary in order to manage the complexity of the task and to maintain consistency with prevalent software practices elsewhere in the research and commercial sectors. The transition will need to be carefully managed as detector design, simulation, and test beam activities continue to require continuous computing support. The software following the transition will provide a powerful and user-friendly environment for physicists to carry out physics studies related to the optimization of the detector and trigger. It should be emphasized that the additional effort initially required to make this transition will be rapidly compensated by the benefits of using more modular and manageable OO software.

Table 4 shows the major milestones for the CMS Software sub-project, which have been approved by the CERN LCB committee. They reflect the iterative software development process, which has four phases. The first phase, which was completed at the end of 1998, is the proof of concept for the key elements of the software architecture proposed in the CMS Computing Technical Proposal 1. The second phase, which is currently in progress, covers the development of functional prototypes of the various software modules, which are now being exercised using large samples of simulated events and test beam data. This software has been, and is being, used for detailed studies of the detector and its sensitivity to various physics processes. The third phase involves the development of the prototypes into fully functional software while the fourth phase is the preparation of the software for production and the exercising of the complete system, in conjunction with the pre-production hardware systems. The milestones associated to the database reflect its central importance in the CMS Computing Model.

32

CORE SOFTWARE 1997 1998 1999 2000 2001 2002 2003 2004 2005End of Fortran DevelopmentGEANT4 Simulation of CMSReconstruction/Analysis frameworkDetector reconstructionPhysics Object ReconstructionUser Analysis Environment

DATABASE 1997 1998 1999 2000 2001 2002 2003 2004 2005Use of ODBMS for test-beam dataEvent storage/retrieval in ODBMSData organization/access strategyFilling at 100MB/sSimulation of data access patternIntegration of ODBMS and MSSChoice of vendor for ODBMSInstallation of ODBMS and MSS

Table 4: Milestones for CMS Computing and Software Project ( 1- proof of concept; 2-functional prototype; 3-fully functional prototype; 4- production software)

7.2 Core Application Software Work Breakdown Structure

Since this subproject is intimately connected to the comprehensive CMS software project, we include the Work Breakdown Structure for that project in Appendix A. Here, as we present the WBS for the Core Applications Software Project, we provide pointers to the CMS Software WBS where appropriate.

2.1 ORCA – Detector Reconstruction (CMS WBS 1.3.10)

This item covers software engineering and professional programming support for the US activities connected to the ORCA (Object Reconstruction for CMS Analysis) project. As a principle CMS tries to ensure that software responsibilities are assumed or at least shared by the groups responsible for the sub-detector construction. This is reflected in the WBS for this task, described below.

2.1.1 Calorimetry

This item covers software engineering and professional programming support for the US activities connected to ECAL and HCAL. The ECAL and HCAL software are both part of a unified “Calorimetry” subsystem with much of the software and interfaces being common for the two systems.

2.1.2 Endcap Muons

This item covers software engineering and professional programming support for the US activities connected to the endcap muon systems. It is part of the overall “Muon” software sub-system, which encompasses both the barrel and

33

?? ? ?

1

1

11

1

1

2 3 4

222

22

33

333

44

444

endcap systems. Specific responsibilities include the muon track fitting in this detector and the implementation of the mechanisms for creating and processing digitizations.

2.1.3 Tracker

This item covers software engineering and professional programming support for the US activities connected to endcap pixel software, detector description and simulation studies. Where the very different geometries permit, software is shared or reused for both barrel and endcap pixels.

2.1.4 Trigger

This item covers software engineering and professional programming support for the US activities connected to the simulation of the level-1 trigger, which is vital for an understanding of the physics capabilities of the detector. This includes responsibility for simulating the regional calorimeter trigger in ORCA, for implementing the simulation of the calorimeter trigger signals required by the level-1 trigger simulation, and for the simulation of the Forward Muon CSC (Cathode Strip Chamber) trigger.

2.1.5 Detector and Event Visualization

This item covers software engineering and professional programming support for the US activities connected to the provision of interactive 3D detector and event visualization software. Much of this software is generic to CMS, and indeed HEP, and is therefore part of the “User Analysis Environment” task described below. In addition, a significant amount of ORCA-specific software is required to produce graphical objects from reconstructed objects and to support their interactive manipulation.

2.1.6 Examples

This item covers software engineering and professional programming support for the US activities connected to the provision of a set of documented and working examples of simple analyses based on ORCA software. This work is related to the “User Analysis Environment” task described below.

2.1.7 Persistency

This item covers software engineering and professional programming support for the US activities connected to support within ORCA of persistent storage in an ODBMS for objects of subsequent interest to physicists. These include hits and digitizations, reconstructed tracks and clusters, etc. This provides crucial input to the “Physics Reconstruction and Selection and Higher Level Triggers” groups. By avoiding the need to repeat the time-consuming ORCA reconstruction step the process of optimizing higher-level algorithms will be speeded up significantly. The general mechanisms of persistency depend on the ODBMS and CARF subtasks. This work combines the software aspects of implementing persistency with studies of the computing and distributed system aspects (described below), thereby ensuring that the solutions will work well in practice.

2.2 User Analysis Environment (CMS WBS 1.3.12)

This item covers software infrastructure and tools for performing user analysis of real and simulated CMS data. It includes the following tasks: interactive data analysis and presentation including histogramming; interactive detector and event visualization; (graphical) user interfaces, and statistical and numerical analysis.

34

2.2.1 Interactive Visualization

This item covers software engineering and professional programming support for the US activities connected to developing the ability to interactively visualize the CMS detector elements and their response to particle interactions. This is of paramount importance in the design, construction, and data analysis phases of the experiment. Detector and event visualization programs are used for the design and debugging of the software, the detector, and the readout, in the evaluation of event reconstruction algorithms, and in producing understandable images for communicating complex concepts to the physics community and the general public. This software will be deployed for simulated, real, and test-beam events in the context of ORCA and physics analysis programs.

2.2.2 Physics Analysis Tools

This item covers software engineering and professional programming support for the US activities connected to developing “Physics Analysis Tools”. These include software for interactive data analysis and presentation, statistical and numerical analysis, and histogramming. It must initially provide at least the equivalent functionality of PAW but be newly implemented using maintainable and extensible OO software. Then it must subsequently evolve to fully exploit the much higher flexibility of the OO data model compared to the traditional solutions.

This task is related to the LHC++ project which is replacing the procedural CERNLIB FORTRAN/C libraries with a suite of OO software (predominantly C++) with roughly equivalent functionality. The scope of the LHC++ project covers the following: foundation level class libraries; mathematical libraries; graphical libraries; visualization tool-kits, data analysis, and histograms; event generators; detector simulation (GEANT4); and object persistency (RD45). The project exploits solutions based on commercial software where appropriate and affordable.

2.3 ODBMS -- Object Database Management System (CMS WBS 1.3.15)

2.3.1 Persistent Object Management

This item covers the development, testing, commissioning and operation of the Object Database Management system (ODBMS) to be used by CMS and the other LHC experiments to manage persistent objects, as well as some of the associated architectural issues in the experiments’ object oriented software. The database management system, which will be chosen by the end of 2001, will provide the mechanisms for data persistence, internal relationships, wide-area heterogeneous' distribution, and concurrent read-write control. It is intended that the ODBMS, with enhancements for the robust operation across wide area networks, also provide transparent access (independent of data location and the storage medium as seen from the user’s code) to persistent objects stored in a network-distributed database federation.

The “Objectivity/ DB” product has been chosen as the current focus of work by CERN, CMS and ATLAS, as well as BaBar and other large experiments. Much of the work on object persistency has been carried out in the context of the RD45 joint project6 at CERN and in the Caltech/CERN/HP/FNAL GIOD project7. RD45 and Caltech (HEP and the Center for Advanced Computing Research) have also surveyed, and investigated other competing database systems, beginning sufficiently in advance of the final ODBMS choice to provide useful input. The storage in robotic tape systems (tertiary storage) of infrequently accessed data will be accomplished using the High Performance Storage System (HPSS)8 .

2.3.2 Networked Object Databases

This item covers software engineering and professional programming support for the US activities which address specific technical issues associated to the operation of Petabyte-scale object databases over local- and wide-area networks. Currently, this is accomplished by participation in GIOD and the DOE/NGI and HEP-funded Particle Physics Data Grid9, which are complementary to and compatible with the aims of the RD45 and MONARC10 projects.

35

2.3.3 MONARC

This item covers software engineering and professional programming support for the US participation in the MONARC Project (Models of Networked Analysis at Regional Centres). This project studies network-distributed computing architectures, data access and analysis strategies, and data management systems that are the major components of the LHC Computing Models and the ways in which the components interact across networks. The approach taken by MONARC is to develop discrete-event and other simulations of the computing facilities at each site, the networks and their handling of the traffic flow, the users’ demands on the network, and high-level representations of key components such as the database system and hierarchical storage manager. While the software-development aspects of this work are part of the Core Applications Software subproject, other activities associated with MONARC are being carried out within the User Facilities subproject 3.

2.4 Software Support

This activity supports physicists in the US who wish to develop and use CMS software in their home institutes. This task includes the development, deployment and subsequent support of the CMS software sub-systems, associated tools, and the developers’ and users’ environment at US institutes, together with associated training in their use. This task also covers software engineering and professional programming support to US physicists participating in the detailed evaluation of sub-detector and trigger systems and the overall CMS physics capabilities. These activities include the assessment of the input to the higher level triggers coming from the Level 1 hardware trigger as well as the evaluation of the Level 2 and Level 3 selection algorithms. Special requirements on the software arise from CPU and bandwidth restrictions in the Level 2 /Level 3 trigger.

Whereas the Core Applications Software Subproject concentrates on development and initial deployment and support, the User Facilities Subproject is responsible for general support of stable production software. For example, the Core Applications Software Subproject covers the design, development, and test deployment of systems for collaboration over networks, while the User Facilities Subproject includes the general deployment and support for such systems once they are reasonably stable. Similar categorization of responsibilities will occur whenever new types of software systems are developed (within the Core Applications Software Subproject) and then brought into production use (in the User Facilities Subproject).

2.4.1 Software Support Tools (CMS WBS 1.3.3)

This sub-task covers the US CMS contributions to the development of generic tools for use not only by US CMS but also throughout the wider CMS community.

2.4.1.1 System Configuration Management

This item covers software engineering and professional programming support to help develop the process and supporting programs for software configuration management, build procedures, and release mechanisms in the context of various compilers, platforms, operating systems, and applications. This item also covers support of the maintenance load for various parts of the CMS software on specific operating systems based on a division of labor among all CMS collaborating institutions.

2.4.1.2 Collaborative Working Tools

This item will provide software engineering and professional programming support for the development of network-based systems for videoconferencing and other tools for computer-supported collaborative work, which are required to

36

support daily communications within CMS, collaborative development of the software, as well as the execution of the data analysis. These tools must be flexible enough to take advantage of new networks as the situation evolves.

2.4.2 Tools to Address Specific Issues for US CMS (no overall CMS WBS)

Despite the widespread use of standards and common solutions, it is expected that there will be some limited areas where the US CMS software community requires some additional software support than that provided by the collaboration at large. Examples of areas needing such support could be support of particular operating systems or hardware platforms used widely in the US but not elsewhere in the collaboration, specific requirements imposed by computer security considerations, or other software requirements due to particular hardware and network configurations present in the US. Moreover, some general-purpose tools could give better performance when optimized for dedicated US use. This subtask will provide the software and engineering and development resources to meet these needs.

2.4.3 Professional Software Engineering Support (no overall CMS WBS)

This subtask provides professional software expertise to support US CMS physicists developing simulation, reconstruction and physics analysis software (which is not already covered by the ORCA, IGUANA, and ODBMS activities supported by WBS elements 2.1, 2.2, and 2.3). Even though this software is not a formal part of the US CMS Software and Computing Project, it will be highly cost-effective to have a single pool of software experts who can, by being shared among different tasks, communicate CMS software standards throughout the US CMS community. Tasks to be performed include mentoring, code reviews, and provision of examples and templates, and software tools.

7.3 Core Applications Software Budget and Personnel Requirements

The resources required for the CMS Software subproject and for the US CMS Core Applications Software subproject are estimated using the resource-loaded WBS, described in the Appendix. This WBS will continuously evolve to meet the changing needs, schedule, and availability of resources. The dominant resource required is professional software manpower, which is obtained by filtering the resource-loaded schedule for such personnel (i.e. no physicist manpower is included). The CMS Software sub-project requires a total of 21.5 FTE’s in 1999 of which 14.5 are currently deployed. The total for CMS will plateau at 33.3 FTE’s in 2003 in preparation for the operation of production software, in conjunction with the production hardware systems, in 2005. The incremental requirement is estimated to remain approximately flat for several years after 2005. This is to permit the shakedown and optimization of the operational system with a real and increasing data set and significantly increased use activities. The personnel requirements of the US CMS Core Applications Software sub-project that are directly associated to activities in the CMS Software sub-project are estimated assuming a 25% US contribution to the total. In addition, the US CMS Core Applications Software sub-project includes personnel dedicated to supporting software for US physicists.

The personnel requirements for the US CMS Core Applications Software sub-project are shown in Table 5. The support personnel have been rounded to yield integer FTE’s in the total requirement. This procedure is consistent with the US CMS strategy of dedicating approximately 25% of each FTE to an appropriate support task to ensure their contact with the physicist developers and users (rather than assigning dedicated personnel full-time to the support task). Although the overall scope of the US CMS Applications Software subproject is bounded by this canonical scaling law, specific tasks are assigned to US CMS individuals in the WBS. By filtering explicitly on US-tagged personnel, consistency is maintained with the over all US CMS profile and responsibilities are adjusted accordingly as a function of time. The US CMS Core Applications Software subproject cost profile is obtained assuming:

The annual salary cost, including overhead and fringe benefits, is $154k / FTE. This is the average of the actual 2000 US CMS requests. The base salaries for these are based on median salaries for such personnel based on

37

previous experience and industry surveys, with allowances for geographical location and skills. The rates of overhead and fringe assumed are those currently in effect at the institutions involved.

Each FTE requires a desktop system costing 6k$, with a lifetime of 3 years. Each FTE requires 10k$ / year to cover associated items such as travel, training and documentation, and software

licenses (not covered elsewhere by CMS or US CMS). The contingency requested is 0% for the years 1999 and 2000, 25% for the years 2004 onwards and 10%

elsewhere. The cost is corrected for inflation starting in 2001; the official DOE escalation index is used.

The costs for the US CMS Core Applications Software subproject are shown in Table 5. The cost requirement rises from $1.00 M in 1999 to a plateau of $2.8M in 2005, yielding a total cost for 1999–2005 of $14.1M. The year 2006 is shown to indicate the required resources during each year of operation of the CMS experiment. The numbers for contingency are 25% from years 2003 onwards to guarantee the completion by start of data taking in 2005 and a successful startup period. For the years 1999-2000 no contingency is taken into account, for the other years a factor of 10% is assumed to be adequate.

Table 5: Estimated requirements of software personnel (FTE) for the US CMS Core Application Software subproject, including US-specific support and the cost for the sub-project, including personnel cost and associated support, such as travel, desktops, training and software licenses. The numbers for contingency are 25% from years 2004 onwards to guarantee the completion by start of data taking in 2006 and a successful startup period. For the years 1999-2000 no contingency is taken into account, for the other years a factor of 10% is assumed to be adequate. Contingency numbers for FTEs are rounded to integer values.

38

8. High Level Milestones, Work Breakdown Structure and Budget for Overall Project

8.1 High Level Project Milestones

The high level milestones for the US CMS Software and Computing Project are chosen to be consistent with the overall high level milestones of the CMS Software Plan and the LHC/CMS schedule. These are shown in sections 6.1 and 7.1 for the User Facilities Subprojects and the Core Applications Software, respectively. The overall CMS milestones for software are shown in Table 4.

8.2 High Level Work Breakdown Structure

In addition to the two main elements of the project, the Core Applications Software Subproject and the User Facilities Subproject, there is a subproject which supports the Level 1 Project Manager's activities.

8.2.1 The User Facilities Subproject

This subproject is described in section 2.2. Section 6 gives the WBS, milestones, and budget for this subproject.

8.2.2 The Core Applications Software Project

This subproject is described in section 2.1. Section 7 gives the WBS, milestones, and budget for this subproject.

8.2.3. Project OfficeThis item provides personnel and operating funds to support the management of the project. It includes a staff of 1/2 FTE administrative assistant, 1/2 FTE Budget officer/planner, and 1 FTE assistant project manager. The budget for travel for the L1PM and for support of reviewers, special consultants, etc is also under this item. The budget is currently absorbed in the User Facilities Subproject budget.

8.3 Budget Summary

Table 6 below lists the budget for the whole US CMS Software and Computing Project. This table summarizes the resources as described in Table 3 and Table 5 and the cost of the project office.

The numbers for contingency for CAS and UF FTE requests are 25% from years 2004 onwards to guarantee the completion by start of data taking in 2006 and a successful startup period. For the years 1999-2000 no contingency is taken into account, for the other years a factor of 10% is assumed to be adequate. For the UF hardware request the contingency is included in the numbers quoted.

39

Table 6: Budget Summary of US CMS Software and Computing Project. Amounts are given in units of $M. The costs shown for Tier2 centers in this table are for staff and hardware located at the Tier 2 centers themselves. A more detailed description of Tier 2 centers is given in section 2.2.2 (see also ref. 16).

REFERENCES

1 The CMS Collaboration, Computing Technical Proposal. CERN/LHCC 96-45, December 1996. The full CTP text is available at http://cmsdoc.cern.ch/cms/CMG/CTP/index.html2 US-CMS Applications Software Sub-project, L. Taylor (on behalf of US-CMS), 23 April 1999. http://cmsdoc.canr.ch/~cmscan/uscmssw/review_may_99/papers/app_sw.pdf3 US-CMS User Facilities Sub-project, J. Womersley (on behalf of US-CMS), 23 April 1999. http://cmsdoc.canr.ch/~cmscan/uscmssw/review_may_99/papers/RegionalCenter.pdf4 These two boxes represent the current de facto division of labor in US CMS and not a formal organizational structure. The organization of these activities is tightly coupled to the whole CMS experiment and is currently under active discussion.5 The membership of the USSCB requires an amendment to the CMS constitution which is currently being revised. The proposed membership will include broad representation from the US CMS collaboration.6 RD45 - A Persistent Object Manager for HEP. See http://wwwinfo.cern.ch/asd/rd457 J. Bunn, Globally Interconnected Object Databases (GIOD) Project Summary Report. Internal CMS Note IN/1999-044, October 1999. See also http://pcbunn.cacr.caltech.edu8 A. Shoshani et al., Storage Management for High Energy Physics Applications, Computing in High Energy Physics 1998 (CHEP 98). See http://www.lbl.gov/~arie/papers/proc-CHEP98.ps and http://gizmo.lbl.gov/sm/.9 The Particle Physics Data Grid involves all of the US HEP laboratories, JLAB, Caltech, Wisconsin (Computer Science) and the San Diego Supercomputer Center. See http://www.cacr.caltech.edu/ppdg.10 M. Aderholz et al., Models of Networked Analysis at Regional Centres for LHC Experiments (MONARC) Project Execution Plan, LCB 99-5, June 1999; see also http://www.cern.ch/MONARC

40

Appendix: CMS Software Schedule and Work Breakdown Structure

The Core Applications Software is closely aligned with the overall CMS software. In order to understand how the US project relates to all of CMS software development, we present here the current CMS Software WBS at Level 3. The WBS for the Core Applications Software project references the CMS WBS numbers shown here wherever appropriate.

The CMS WBS has the CMS Software Project as a Level 2 Project designated as 1.3. The Level 3 software subtasks are described in detail below.

CMS Schedule and Milestones

Table 4 above shows the major milestones for the CMS Software subproject, which have been approved by the CERN LHCC. They reflect the iterative software development process, which has four phases. The first phase, which was completed at the end of 1998, is the proof of concept for the key elements of the OO software architecture proposed in the CMS Computing Technical Proposal. The second phase, which is currently well underway, covers the development of functional prototypes of the various software modules that are being exercised using large samples of simulated events and test beam data. This software will be used for detailed studies of the detector and its sensitivity to various physics processes. The third phase involves the development of the prototypes into fully functional software while the fourth phase is the preparation of the software for production and the exercising of the complete system, in conjunction with the pre-production hardware systems. The milestones associated to the database reflect its central importance in the CMS Computing Model.

CMS Work Breakdown Structure

WBS 1.3.1 CMSIM

CMSIM is the Fortran-based CMS simulation package. Since new development has essentially ceased, CMSIM will need to be supported for physics studies until GEANT4 and the corresponding OSCAR simulation software have comparable functionality.

WBS 1.3.2 Software Process

This item covers the process associated with the definition, design, development, documentation, integration, verification, deployment, and maintenance of the CMS Software with the aim of improving efficiency and quality1. The pragmatic "Cyclic Life Cycle'' model which emphasizes continuous improvement following ISO/IEC 15504 (SPICE) 2 has been adopted.

Items which have been implemented include the structure of the ORCA software repository and the strategy for ORCA development, release, and testing 2 and the CMS coding rules3, guidelines4, and style-guide5. Processes currently under construction include the more comprehensive assessment of user requirements, checking of software dependencies between software subsystems and packages, automated checking of coding rule and style conformance, and more comprehensive integration of test suites and examples. Processes to be implemented more rigorously in future include: more formal and uniform design procedures; documentation of code design, implementation, and usage; and problem reporting, tracking and resolution mechanisms.

WBS 1.3.3 Software support

41

This item covers support for the CMS software and environment including the associated code management systems, version control mechanisms, and the code release, distribution, and build procedures 1. The current solution for CMS code is to organize the code into major tasks (ORCA, OSCAR, User Analysis Environment, etc.), which are subsequently divided hierarchically into sub-systems (Tracker, Calorimetry, etc.), each of which contains a number of smaller software "packages'' (each of which corresponds to a software library). The CVS system, which is prevalent in HEP and elsewhere, is used to manage the access, change, and version control of the software repositories. The build and distribution system uses SCRAM, a product developed by CMS. An important aspect of this task is the porting, technical verification, and support of CMS and external code on multiple systems. This task also includes the development and support of network-based collaborative working tools such as videoconferencing, which are invaluable to the highly distributed members of any HEP collaboration.

WBS 1.3.4 Information Systems

This item covers the construction, operation, and maintenance of the collaboration information systems. These are centered around the CMS WWW server and include: management of news, agendas, and minutes in a uniform fashion for the complete hierarchy of CMS groups; the CMS collaboration members database; and storage, access, and archival of documents, including internal working documents, technical notes, and collaboration official documents.

WBS 1.3.5 CARF -- CMS Analysis and Reconstruction Framework

The foundation of the CMS software is a professionally engineered framework into which physics modules may be inserted. The "CMS Analysis and Reconstruction Framework'' known as CARF 6, aims to provide the user with an intuitive and flexible means of performing analysis and reconstruction tasks. CARF should support standard operations performed by physicists, including: accessing the CMS data store; selection and classification of events; the ability to apply calibration algorithms; invocation of standard or custom-built reconstruction algorithms; interactive visualization and physics analysis systems with presentation-quality output; and creation of user-defined transient or persistent objects. The user of CARF will be able to choose what specific actions to perform and which specific algorithms to use even at run-time or interactively. CARF will use an ODBMS to store objects persistently and will provide classes and methods to shield end-users from details of the ODBMS storage and access mechanisms, mostly in terms of C++ smart references and smart iterators. The underpinnings to ensure that the ODBMS operations of CARF function as planned are being addressed through the ODBMS task described below.

WBS 1.3.6 Event Generators

This item covers the software required to support external physics event generator programs in the CMS environment. In particular it includes software for the management of input parameters in a consistent fashion and procedures for storing output events with a standard format for subsequent use by, for example, OSCAR and the Fast Simulation software. This task does not cover the software of the generator programs themselves.

WBS 1.3.7 Detector Description

This task will provide an environment for creating, manipulating, and using the parameters describing the CMS detector in a consistent manner 7. In particular, it covers the geometrical description of the detector elements at various levels (full engineering detail, full GEANT detail, fast simulation, trigger tower geometries, etc.), associated material properties, magnetic field map, etc. The Detector Description sub-system will serve a number of clients, including OSCAR, the Fast Simulation, ORCA, the Calibration, and the User Analysis Environment.

WBS 1.3.8 Fast Simulation

42

This item covers fast simulation of particles traversing the CMS detector and the response of the detector elements, readout electronics, and triggers 8. Parameterizations will be verified with OSCAR simulations, test-beam analyses, and ultimately real CMS data. The Fast Simulation should use the Common Detector Description and be consistent with OSCAR, ORCA, and the Physics Object Reconstruction software to permit flexible the use of varying degrees of simulation detail according to the requirements of the user.

WBS 1.3.9 OSCAR -- Detector Simulation

This item covers the full simulation of particles traversing the CMS detector using the GEANT4 Toolkit, including the use of an appropriate detector description, tuning and verification of physics processes, and simulation of neutron and other backgrounds 9. OSCAR will provide simulated signal and background events for ORCA and will be used to develop and tune parameters and algorithms of the Fast Simulation software sub-system.

WBS 1.3.10 ORCA -- Detector Reconstruction

There is an immediate need to perform detailed studies of the detector and trigger performance such that the design can be optimized. CMS has committed to perform all such studies using OO software. As a result, a high priority "Reconstruction Task Force'' was set up in the summer of 1998 under the leadership of D. Stickland of Princeton. This task force had the charge of rapidly building functional OO reconstruction code. The first tangible result of this effort was the successful release of the ORCA (Object-Oriented Reconstruction for CMS Analysis) Version 1 in December 1998. Since then the project has continued to grow and improve the functionality of ORCA10.

The ORCA task covers software for the reconstruction of simulated (and ultimately real) events. In the case of simulated events, it also includes the simulation of the detector and electronics response to the GEANT hits, the creation of digitized hits and L1 trigger primitives, and the simulation of the L1 trigger response. For both simulated and real data, it provides software for the reconstruction of tracks segments, clusters, etc. within individual subdetectors and certain entities which combine information from more than one subdetector, such as tracks spanning the whole of the tracker.

WBS 1.3.11 OBSELETE

This WBS item previously covered the task of "Physics Object Reconstruction and Higher Level Triggers''. Since the task has Software, Physics, and Higher Level Trigger aspects it was decided to set up distinct but closely-linked "Physics Reconstruction and Selection'' (PRS) activity. For backwards compatibility, this WBS number is retained but it has no associated tasks or resources.

The PRS task covers software to create physics classes describing reconstructed particle candidates, perform global analyses of complete events, and develop Level 2 and Level 3 trigger algorithms using both localised subsets of data and full event information. The L2 and L3 selection algorithms then need to be evaluated in terms of background rejection power and signal efficiency, taking into account the CPU and bandwidth available in the L2/L3 trigger systems. Four subtasks cover the reconstruction of: electrons and photons; muons; jets and missing energy; and b's and taus.

WBS 1.3.12 IGUANA -- Interactive Graphical User Analysis

This item covers software infrastructure and tools for performing user analysis of real and simulated CMS data 11. In particular, it covers: user interfaces (UI's); graphical user interfaces (GUI's); statistical and numerical analysis; interactive data analysis and presentation including histogramming; and interactive detector and event visualisation.

43

For some of these items, toolkits already exist either within HEP or the commercial sector. Therefore, a significant component of this task will be the assessment of available tools and their adaptation, extension, deployment, and support in the CMS environment. In some case, new tools may need to be developed although pre-existing components will be exploited wherever possible. A crucial aspect of this task, which is closely linked with the CARF development, is the provision of user interfaces and the corresponding graphical user interfaces. These will be needed not only for final physics analysis software but also for use by applications based on other CMS software sub-systems, such as ORCA and OSCAR.

WBS 1.3.13 Test Beam

This item covers generic software required for the acquisition, storage, processing, and analysis of test beam data12. An important aspect of this work is the deployment and evaluation of the ODBMS and mass storage systems for the storage of test-beam data in a real-time environment.

WBS 1.3.14 Calibration

This item covers software associated to the calibration of the CMS sub-detectors including alignments, energy calibrations, etc. using data from: test beams; laboratory measurements; survey data; in-situ dedicated systems; slow control systems; and physics event samples. The Calibration clients include ORCA and potentially the OSCAR and Fast Simulation projects, which may ultimately incorporate realistic detector performance data into the simulations.

WBS 1.3.15 ODBMS -- Object Database Management System

This item covers generic issues pertaining to the use of an ODBMS for the storage of CMS non-event and event data, including those associated to the operation of a database federation in a heterogeneous distributed environment 13. In addition, a construction database has been developed and deployed as part of the CRISTAL project.

Objects are stored persistently, reusing as much as possible what is provided by the ODBMS itself to store, organize and retrieve data, and to administer both data and metadata including schema declaration, schema evolution, object versioning and database replication. Optimization mechanisms such as caching, compression, and physical clustering of objects will be implemented such that they are transparent to the user. A large fraction of the ODBMS activities are carried out in the context of the CRISTAL, GIOD, MONARC, RD45, and WISDOM projects.

REFERENCES FOR APPENDIX

1 J-P.Wellisch, The Status of Software Process Improvement in CMS. CMS IN/1999-033, http://cmsdoc.cern.ch/cms/software/reviews/papers99/SoftwareProcess.ps

2 ISO/IEC DTR 15504 Software Process Assessment, the ISO/IEC Joint Technical Committee 1 (JTC1), Sub-committee 7 (SC7) Working Group 10 (WG10) Project Editor Terry Rout

3 J-P.Wellisch, CMS NOTE 1998/0714 J-P.Wellisch, CMS NOTE 1998/0705 J-P.Wellisch, CMS NOTE 1998/0726 V.Innocente, CMS Software Architecture: Software framework, services and persistency in high level trigger,

reconstruction and analysis. CMS IN/1999-034, http://cmsdoc.cern.ch/cms/software/reviews/papers99/cmsArchitecture1099.ps

7 S.Banerjee, A Multi-representations Detector Description for CMS Offline Physics Applications. CMS IN/1999-038, http://cmsdoc.cern.ch/cms/software/reviews/papers99/cms_detector.ps

8 S.Wynhoff, Fast Monte-Carlo Simulation in CMS. CMS IN/1999-037. http://cmsdoc.cern.ch/cms/software/reviews/papers99/famos.ps

44

9 M. Schröder, CMS Detector Simulation Project OSCAR. CMS IN/1999-036. http://cmsdoc.cern.ch/cms/software/reviews/papers99/oscar.ps

10 D.Stickland, CMS Reconstruction Software: The ORCA project. CMS IN/1999-035. http://cmsdoc.cern.ch/cms/software/reviews/papers99/cmsORCA.ps

11 G.Alverson, I. Gaponenko and L.Taylor, IGUANA - Interactive Graphical User Analysis. CMS IN/1999-042. http://cmsdoc.cern.ch/cms/software/reviews/papers99/iguana.pdf

12 L.Silvestris, CMS Test Beam Software. CMS IN/1999-043. http://cmsdoc.cern.ch/cms/software/reviews/papers99/TestBeam.ps

13 I.Willers, DataBase Activities in CMS. CMS IN/1999-039. http://cmsdoc.cern.ch/cms/software/reviews/papers99/Database_activities.ps

45

******* DRAFT ******* DRAFT ******* DRAFT ******* 04/10/23 2:01 PM 46 of 46

46