Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute...
-
date post
21-Dec-2015 -
Category
Documents
-
view
219 -
download
3
Transcript of Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute...
![Page 1: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/1.jpg)
Introduction to DANSE Brent Fultz
Prof. Materials Science and Applied PhysicsCalifornia Institute of Technology
• Distributed Data Analysis Architecture for Neutron Scattering Experiments• Technical Issues, Scientific Organization• From a Concept to a National Project
![Page 2: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/2.jpg)
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
$ 1,411,000,000 hardware10,000,000 software
![Page 3: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/3.jpg)
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
ARCS
![Page 4: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/4.jpg)
![Page 5: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/5.jpg)
Analysis Software for Neutron Scattering • Enables science
• Helps new users get up to speed
• Better science with better software optimize beamtime usage experimental procedures tuned on-the-fly sample knowledge can constrain data interpretations
• Some experiments are impossible with today’s software
• New opportunities to connect to materials theory
![Page 6: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/6.jpg)
Logo for ARCS SpectrometerHeadstock
Neck
Frets
Pickups
Volume/ToneSolidBody
Tailpiece(Whammy Optional)
Logo for DANSE??
![Page 7: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/7.jpg)
Software with Power and Beauty
$?
![Page 8: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/8.jpg)
Software with Coherence
DANSE architecture is GeneralbutAll developers must know itHence this Workshop
![Page 9: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/9.jpg)
Data Analysis as a Web Service• Data analysis is a service controlled by the user • User’s laptop issues commands and receives results • Computation is arranged by the web server
![Page 10: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/10.jpg)
Present Example from inelastic scattering: Born - von Kármán Lattice Dynamics
Simplicity: Complexity:
1. Undergrowth of indices for tensor quantities:
2. Crystal structure:
€
mκ ˙ ̇ u κl = − Vκ ,l
∑ κl,κ ' l 'uκ ' l '
€
uκl
€
V κl ,κ ' l '
lattice, l, basis, allowed elements depend on symmetry
![Page 11: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/11.jpg)
The Bigger Concept Underneath
Components Pre-compiled Python objects called and re-arranged
by the Python Interpreter
Data Streams Standard communication protocol between
components (standard streams can connect components located anywhere)
![Page 12: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/12.jpg)
DANSE Project: Develop Tools for Programmers
Component Templates Standard Data Streams
![Page 13: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/13.jpg)
Levels of User Expertise• For using existing scripts, entry barrier nearly zero.• For altering existing Python scripts, entry barrier is very
low. • For writing new Python code, entry barrier is modest.
Performance may be comparable to IDL or Matlab. • Transition to high performance compiled code
ARCS: Writing Python bindings for C++DANSE: Component templates for C++, FORTRAN,
Java? Focus of this Workshop
![Page 14: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/14.jpg)
• Importance of remote control of neutron instruments• Remote collaboratory capability• Remote data analysis capability
• No strong support for remote control -- too risky
(sample temperature and saving data okay)• Strong support for remote data analysis capability• Overwhelming support for standardization of data analysis
(across different instruments and different facilities)
NSSA Neutron Software Poll
![Page 15: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/15.jpg)
Making a Software Project Plan
• Define and justify scientific needs
• Define and justify the software architecture
• Define the scope of the project
• Accurately estimate the effort
Proposal -- what, why, who, where, when, how?
![Page 16: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/16.jpg)
A DANSE Project Would Have 2 Parts
• Build Tools (hardware and software) for Developers
• Extend Software Beyond ARCSDiffractionEngineering diffractionReflectometrySANSInelastic scattering (ARCS+chemistry)
![Page 17: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/17.jpg)
Money
• Software tools development
• Hardware for multiple users
• Science-based subprojects
Four-Year Budget (salaries mostly)
10 M$ 4-5 subprojects
10 M$ Central Resources 7 FTE + hardware
![Page 18: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/18.jpg)
Software Development
![Page 19: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/19.jpg)
ARCS Software Project
![Page 20: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/20.jpg)
Recent Developments• DANSE is “the SNS standard” architecture
• NIST reflectometry sub-proposal
• TeraGrid Extension to ORNL (DANSE is core)
• SNS software developers at HFIR(?)
• SNS to write specifications for data analysis software• censored
• NSF intermediate-scale instrumentation program (December 2003)
• NSF ITR (Spring 2004)
![Page 21: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/21.jpg)
Reasons for Optimism about DANSE
• Upsurge in interest from DOE and beyond
• Alternative architectures are not full solutions
• Technically possible within the scale discussed
• No better group to do it
![Page 22: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/22.jpg)
DANSE Software Workshop
Wednesday Sept. 3, 2003 Introductions Lunch Python and OOP Dinner Evening tutorials
Thursday Sept. 4, 2003 The Python/C API, NeXus Lunch Subgroup Talks/Funding Evening tutorials
Friday Sept. 5, 2003 Architectural Framework Lunch Programmers' Tools Dinner Evening tutorials
Saturday Sept. 6, 2003 Software design for DANSE Lunch "Edward Weston: A Legacy"
Sunday
Monday Sept. 8, 2003 Python Extensions in C++ Lunch FORTRAN, Operating systems. DANSE Project integration. Dinner
![Page 23: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/23.jpg)
Welcome toCalifornia
![Page 24: Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture.](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d615503460f94a43b54/html5/thumbnails/24.jpg)
End of Presentation
Michael is Next