Post on 11-Jan-2016
DANSE Review – SNS/HFIR Update
Steve Miller
Scientific Computing Group Leader
Neutron Scattering Science Division (NSSD)
Spallation Neutron Source
May 31, 2007
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
2DANSE Review
Overview
SNS/HFIR (NSSD) Facility and Software Updates
Feedback from Instrument Scientists
DANSE – SNS integration
Summary
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
3DANSE Review
SNS Neutron Scattering User Facility
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
4DANSE Review
SNS Target and Instrument Hall
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
5DANSE Review
Instrument Hutch
• Workplace for users• Data acquisition system• Analysis computer
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
7DANSE Review
FY07 Operations Schedule
1 1 1 1 1 1 1 1 1 1 1 1
2 2 2 2 2 2 2 2 2 2 2 2
3 3 3 3 3 3 3 3 3 3 3 3
4 4 4 4 4 4 4 4 4 4 4 4
5 5 5 5 5 5 5 5 5 5 5 5
6 6 6 6 6 6 6 6 6 6 6 6
7 7 7 7 7 7 7 7 7 7 7 7
8 8 8 8 8 8 8 8 8 8 8 8
9 9 9 9 9 9 9 9 9 9 9 9
10 10 10 10 10 10 10 10 10 10 10 10
11 11 11 11 11 11 11 11 11 11 11 11
12 12 12 12 12 12 12 12 12 12 12 12
13 13 13 13 13 13 13 13 13 13 13 13
14 14 14 14 14 14 14 14 14 14 14 14
15 15 15 15 15 15 15 15 15 15 15 15
16 16 16 16 16 16 16 16 16 16 16 16
17 17 17 17 17 17 17 17 17 17 17 17
18 18 18 18 18 18 18 18 18 18 18 18
19 19 19 19 19 19 19 19 19 19 19 19
20 20 20 20 20 20 20 20 20 20 20 20
21 21 21 21 21 21 21 21 21 21 21 21
22 22 22 22 22 22 22 22 22 22 22 22
23 23 23 23 23 23 23 23 23 23 23 23
24 24 24 24 24 24 24 24 24 24 24 24
25 25 25 25 25 25 25 25 25 25 25 25
26 26 26 26 26 26 26 26 26 26 26 26
27 27 27 27 27 27 27 27 27 27 27 27
28 28 28 28 28 28 28 28 28 28 28 28
29 29 29 29 29 29 29 29 29 29 29
30 30 30 30 30 30 30 30 30 30 30
31 31 31 31 31 31 31
Accelerator Physics Machine Downtime Major Periods(Maintenance/Upgrades)
Accelerator Physics Option Machine Downtime Periods Weekly Maintenance Option
Accelerator Startup (RF) Neutron Production
Run 2007-1 Run 2007-2 Run 2007-3
Apr Aug
SeptJulyApr May June Aug
June JulyFeb Mar Sept
Oct Nov Dec Jan Feb Mar
MayOct Nov Dec Jan
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
8DANSE Review
HIFR will run 3 cycles in FY07
• Run cycles 408 and 409 are internal run cycles with collaborative users
• Run cycle 410 supports the full user program
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
9DANSE Review
General User Proposal Call Scheduled Announce June 1, 2007
Use Integrated Proposal Management System
Accept proposals for HFIR instruments
Ames Lab Triple Axis (HB 1a) Triple-Axis Spectrometers (HB 1 and HB 3) Residual Stress (HB 2B) SANS (CG 2 and CG 3)
SNS instruments Backscattering Spectrometer (BL 2) Magnetism Reflectometer (BL 4a) Liquids Reflectometer (BL 4b)
Accommodate requests for CNMS capabilities
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
10DANSE Review
Scientific Computing Scope
Data lifecycle management – “packaging” of raw data, metadata cataloging, data storage, long-term data archive, data retrieval
Experiment support – live histogramming and data reduction
Data reduction and visualization (via desktop or portal)
Analysis and simulation software – DANSE plays a key role here
Support instrument commissioning
Provide support for all instruments at SNS and HFIR
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
11DANSE Review
NSSD Scientific Computing Software Team
NSSD Scientific Computing Group: Steve Miller, Pete Peterson, Michael Reuter, Jean Bilheux, Shelly Ren
CSMD Network and Cluster Computing Group: Jim Kohl, Sudharshan Vazhkudai, Jens Schwidder – 1.5 FTE total
UT Software Quality Research Lab: Tom Swain, Bradford Smith – 0.5 FTE
Tech-X: Mark Green – 0.25 FTE
Neutron Science TeraGrid Gateway: John Cobb, Greg Pike, Vickie Lynch, Meili Chen – NSF Funded
Room to grow? YES!!!
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
12DANSE Review
Overall Software Architecture
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
13DANSE Review
Computer Network Diagram
Drawing Courtesy of Tom Swain
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
14DANSE Review
SNS Computing Resources Available for DANSE
Instrument Analysis computers
SNS compute and application servers
ARCS cluster now resident at SNS
Neutron Science TeraGrid Gateway (NSTG)
ORNL OIC clusters (future)
ORNL Super Computers (future)
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
15DANSE Review
Live Data Processing – Design Underway
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
16DANSE Review
Instrument Data Statistics 2007 YTD
Magnetics Reflectometer
Liquids Reflectometer
Backscattering Spectrometer
Number of Files Archived in 2007
131 827 34
Average Event File Size in MB
14.3 3.5 297.0
Average NeXus File Size in MB
7.9 7.3 12.0
Histogram Size in MB
148.4 222.7 703.1
Histogram compression ratio
94.9% 96.7% 98.3%
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
17DANSE Review
Protection Zone Overview (target)
ORNL
AdministrativeEnclave
(Desktops)
Follows ORNL C&A Requirements
NCCSEnclave
(servers only) Open ResearchEnclave
(Desktops and Servers)
Visitor/CollaboratorNetwork
(Desktops)
InfrastructureEnclave
(Servers only)Open/Public
Enclave(Internet facing
servers)
Moderate withEnhanced Controls
(Desktops and Servers)
Controlled ResearchEnclave
(Desktops and Servers)
Shared firewall
M/EC firewall
Slide Courtesy of
Victor Hazelwood
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
18DANSE Review
Benefits of an Open Research Enclave
Straight forward and centralized means for providing users computer accounts via the XCAMS system in place at ORNL.
XCAMS accounts can be used for both web applications AND for logging in to instrument analysis computers.
Remote desktop applications such as NX (VNC-like) can be supported.
Maintain the ability to centrally manage both user accounts and computer resources.
Users will have the same “look and feel” computing experience from instrument to instrument.
Data can be centrally located utilizing uniform access policies.
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
19DANSE Review
Using NX Remote Desktop
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
20DANSE Review
Portal Based Data Reduction Operational
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
21DANSE Review
Data Portal for Browsing and Searching Data
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
22DANSE Review
Citrix – Windows Remote Application Server
http://uchi.ornl.gov
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
23DANSE Review
Feedback from SNS Instrument Scientists
Instrument teams for next commissioning instruments desire to have DANSE software in place prior to first neutrons (the earlier the better for them). DANSE diffraction software coming along well To start with for SANS, SNS plans to produce data reduction software and
utilize existing IGOR Pro software CNCS can utilize similar software as ARCS Analysis software for SNAP high pressure single crystal diffractometer less
certain what is/will be available
Scientists keep an eye on software performance: Execution speed Handling of large data sets Easy to install and use (many prefer windows platforms) Easy to program, configure, or adapt Reliable – both in stability and accuracy of results
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
24DANSE Review
Feedback - continued
NSSD Science Groups: Low Q: SANS and Reflectometry TOF Inelastic Powder Diffraction Single Crystal Diffraction Triple Axis (HFIR instruments only)
Science group leaders are beginning to understand the DANSE project scope and are building their software plans based upon the level of support from DANSE. Note that DANSE supports 3 of 5 science groups at SNS/HFIR,
but provides less support for the single crystal diffraction and triple axis groups.
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
25DANSE Review
Some Metrics for DANSE interactions SNS
Number of UCAMS (ORNL) computer accounts: Currently 8 DANSE team members possess UCAMS (not XCAMS)
accounts. These UCAMS accounts enable users to access SNS resources such as computers and repositories.
Eight is a reasonably good number of users given the current number of instruments on-line, and this number is likely to increase as more instruments approach commissioning.
It takes time to get a UCAMS account – especially for foreign nationals.
Number of DANSE applications in use at SNS: Currently count two: ReflPak and PDFgui. Note that we are still understanding the role of the ARCS cluster recently
installed at SNS and DANSE software which may be available via that machine.
Number of visits between facilities: DANSE developer’s meeting held at ORNL in January ‘07 Jiao Lin at SNS in November ‘06, January, and May ’07 to work at SNS SNS personnel attended DANSE meeting at Caltech in August ‘06
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
26DANSE Review
Summary and Observations
SNS has developed the capability to produce software up thru data reduction and is in need of advanced analysis software produced by the DANSE project.
DANSE awareness at NSSD is rising and thought processes have transitioned from “if” to “what and how” – DANSE is taking root and we have some metrics to show this. Currently the most important metric for SNS is DANSE software in-use on-site.
SNS instrument teams have strong wishes to have DANSE software available before their users come. Some teams are already using DANSE software, and additional teams are actively planning and depending on using DANSE software.