WPEC Subgroup 43 -GNDS Moving towards API …...|This work was funded by the RCUK Energy Programme...
Transcript of WPEC Subgroup 43 -GNDS Moving towards API …...|This work was funded by the RCUK Energy Programme...
| This work was funded by the RCUK Energy Programme[Grant number EP/P012450/1]
WPEC Subgroup 43 - GNDSMoving towards API driven codes – A FISPACT-II perspectiveThomas Stainer, Mark Gilbert, Lee Packer, Tim Eade, Greg BaileyJune 2019
|
• FISPACT-II API developments
• FISPACT-II as a nuclear data consumer
• Lessons learned
• GNDS design ideas + thoughts
• Summary
Overview
Thomas Stainer - WPEC Subgroup 43 - June 20192
|
FISPACT-II
• Generic inventory, transmutation code
• Consumes vast number of nuclear data libraries: TENDL 17, 15, 14, JEFF 3.3, ENDF/B-VII,…
• Supports ENDF-6 and EAF formats• We are heavy consumer of
nuclear data• Estimate at least 50% of users
issues/errors relate to nuclear data processing
• Large amount of development time wasted due to poor, inconsistent, missing nuclear data• Only caught later upstream
Thomas Stainer - WPEC Subgroup 43 - June 20193
|
• February 2019 – API version of FISPACT-II developed (internal only)• Significant refactor:
• Removed input/output file reading/writing• Purely API driven• Decoupled nuclear data from calculations• Nuclear data abstracted through use of APIs• New language bindings in Fortran, C, C++, and Python
• Large validation effort conducted • Automated testing in place – 49 test suites consisting of over 470 system tests + 403
unit tests• Regression analysis • FNS decay heat validation
• Ready to consume nuclear data in a better way:• Database connection, REST APIs, • Still maintains ENDF file support.
• Still not abstract enough, ENDF hangovers:• Requires knowledge of MT numbers• Internal data structures not optimized for common queries (linked lists overused)
FISPACT-II API
Thomas Stainer - WPEC Subgroup 43 - June 20194
|
Architecture
FispactProcess
FispactInputData
FispactOutputData FispactNuclearData
FispactNuclearDataReader
Monitor
Inputs
Computation Outputs
Error handling
File
processing
ENDF files
Thomas Stainer - WPEC Subgroup 43 - June 20195
Decoupled concrete ENDF files from code – Nuclear data accessed via getters/setters
Need consistent error handling for all language bindings
|
Architecture
Thomas Stainer - WPEC Subgroup 43 - June 20196
Pros:• Decoupled from ENDF files. • Nuclear data can be stored in any fashion.
Cons:• Need to write new “reader” for each concrete data format
FispactProcess
FispactInputData
FispactOutputDataFispactNuclearData
FispactNuclearDataReader
Monitor
Inputs
Computation Outputs
Error handling
File
processing
ENDF files
XMLReader
HDF5Reader
Concrete files
Database DatabaseConnection
|
FISPACT-II uses group wise data, and requires NJOY and PREPRO to produce libraries for end use.
• Natural elemental abundances and atomic weights (mainly NIST)• Decay data (MF8)
• Half lives • Decay modes + probabilities + secondaries• Decay spectra (discrete)• Spontaneous fission yields
• Cross section data (MF3, MF10)• Group wise data• Variances + covariances
• Resonance parameters (MF2)• Gamma attenuation data (legacy)• Fission yields• Probability tables (Self Shielding) – use CALENDF
FISPACT-II nuclear data
Thomas Stainer - WPEC Subgroup 43 - June 20197
ENDF-6
NIST
Legacy
|
• Large nuclear data libraries e.g. TENDL 2017 take a long time to read/process in comparison to inventory calculation • Typical ~80 seconds to read all
necessary data for 3850 nuclides• ~1-2 seconds to solve Bateman
equation• For large scale calculations, very
slow• API approach using in-memory
data• Much quicker – no processing• No serialization
Processing and Performance
Thomas Stainer - WPEC Subgroup 43 - June 20198
No large output files to sift through!
|
Nuclear data interrogation using F-II
Thomas Stainer - WPEC Subgroup 43 - June 20199
• Internal nuclear data structures are almost flat• Thin hierarchy – 3 nested structures at most• Problem is refactored Fortran still using Linked Lists in most places• Some effort to replace with more optimal data structures:
• hash tables for quick lookup of ZAIs to index• Contiguous data structures for iteration
GetReactionAvailableChannels(zai, error)
GetReactionXS(zai, mt, error)
|
Layers of abstraction
Thomas Stainer - WPEC Subgroup 43 - June 201910
getxs(922380, MT=102)
No accuracy issues due to file precision
Need to interrogate and visualize nuclear data easily
Database(PostGres)
REST-API
Data access layer
FISPACT-II API
Plotting library
Abst
ract
ion
|
• FISPACT-II is not a big code but it was a large effort to refactor and validate• Hard to motivate community to move to alternative methods and formats• Talk with users – let them define what APIs should look like• Large validation effort required, automation helps• Backwards compatibility is a must• Fixing bugs can look like introducing bugs in new/refactored code• Whilst providing bindings for many languages keeps clients happy, a lot of
maintenance for developers• Good documentation is essential – 700+ page reports and manuals are not ideal
Lessons learned
Thomas Stainer - WPEC Subgroup 43 - June 201911
Doxygen
FNS Validation
|
Our current software stack• NJOY, PREPRO, Talys, MCNP, FISPACT-II, …• We cannot have codes with file based inputs/outputs anymore
• Must avoid things like:“broadr -21 -22 -23”
• Our typical software stack for processing nuclear data is:• Single monolithic Fortran programs with input files containing magic numbers• Schema less input and output files• No automated testing in placeÞ Difficult to interface together, error prone, precision issues,…
• Version control – for codes and data => git is not the answer to everything• We need to get full stack improved
• NJOY21 and ENDFtk - but what about the rest of the pack?
Back to GNDS
Thomas Stainer - WPEC Subgroup 43 - June 201912
|
• ENDF-6 format is not the only problem• Lack of automated testing is a worry for our nuclear data
• All codes (processing, transport, inventory, …) need ability to mock and test easily
• Regression tests with every code change• Serialization tests – Read ENDF -> Write HDF5 -> Read HDF5 ->
Write ENDF• Minimal human verification of functionality – automate as much as
possible• Focus on interface
• Usability and maintenance should trump performance• Do not neglect performance, but API must work for users (both
evaluators + developers)• Must be abstract enough – decoupled from concrete file formats• Every API have an equal and opposite API – get/set, read/write• Who should drive this development – evaluators, transport codes?
• C API should be base line• Well supported• Longevity• C++ is nice but C is better for interoperability
• bind to Fortran, C++, Python, Java, …
Design principles
Thomas Stainer - WPEC Subgroup 43 - June 201913
Simple, high level interfaces
Testable, modular code
Interoperability
|
• Data provenience issues• Versioning – how can we reliably version nuclear data, and perform regression on
data only• Git cannot be used for everything!
• Full provenience tree - Who made changes that broke the tests? Was it a Friday evening when the last changes where committed? Etc..
• Validation requires well tested and validated ENDF reader – is ENDFtk the way?• How do we incorporate EAF and ACE formats?• ENDSF, EXFOR data – how do we incorporate this as well?• How can we automate as much as possible?
• CI is a must• Monolithic unmaintainable scripts should be abandoned
• Whilst performance is key, we should think from a user point of view.• It should be trivial to do simple queries for users without any notion of a “file”
• Get all nuclides with half life > 1e8 and have one stable child….• Get the (n,g) cross section for Ag108 for JEFF3.3..
• Should we engage with HEP?• ROOT - work ongoing to develop new database TNudy (Korea)
GNDS final thoughts
Thomas Stainer - WPEC Subgroup 43 - June 201914
|
• FISPACT-II is ready for GNDS• We can help with validation effort• API approach allows quicker analysis, in-memory nuclear data and no file
serialization• Adoption is starting to increase UKAEA, many new users requesting API version
• Next steps:• Development of REST APIs for FISPACT-II in development.
• Users will not require local version of nuclear data• Nuclear data as a service
fetch('https://fispact.com/api/v0.1/getzais&token=adsj67ads8d8').then(response => response.json()).then(json => console.log(json))
• Looking into alternative database options for FISPACT-II nuclear data• SQL, NON-SQL, in-memory(reddis), graph• Not clear what is the best representation for nuclear data
Summary
Thomas Stainer - WPEC Subgroup 43 - June 201915
|
Backup
Thomas Stainer - WPEC Subgroup 43 - June 201916
|
Code coverage
Thomas Stainer - WPEC Subgroup 43 - June 201917
• Unit + Integration + System tests• Covering Fortran, C, C++ and Python layers• 45 system tests, 470 FISPACT-II full simulations• 368 unit + integration tests• Over 100,000 assert statements• Collect code coverage from tests ~80% line coverage
Test type Line code coverage (%)
Function code coverage (%)
Implementation unit + integration tests only 8.6 16.1
API unit + integration tests only 16.4 31.1
Full unit + integration test suite 20.2 38.5
System tests only 68.3 66.3
Full test suite (unit + integration + system) 74.6 80.6
Full test suite (unit + integration + system) excluding unused code i.e. LSODES single precision solver
79.7 81.5
25594 lines2590 functions
|
• Inventories (nuclide quantities) can differ• Some nuclides at timesteps have different number of atoms < 0.01%• Very low impact on total heats and activities• Most differences around 1e-15
FISPACT-II API validation
Thomas Stainer - WPEC Subgroup 43 - June 201918
|
• Need rigorous validation system that can be fully automated and scheduled• People need to be able to trust the tools we are using in order to help adoption• Floating point issues can cause headaches, and difficult to trace• Write, Read, Write chains + compare to highlight serialization issues
Validation process
Thomas Stainer - WPEC Subgroup 43 - June 201919
ExistingENDF-6
fileIn-memory Xml file In-memory
New ENDF-6
file
ReadENDF
WriteXML
ReadXML
Write ENDF
Compare
|
Nuclear data interrogation using F-II
Thomas Stainer - WPEC Subgroup 43 - June 201920
Non trivial exercise in traditional file approach using F-II
|
• Command Line Tool only• Single executable program• Takes input file and produces output files• Use of keywords in input file to setup simulation• Human readable text output file
FISPACT-II traditional
Thomas Stainer - WPEC Subgroup 43 - June 201921
| Thomas Stainer - WPEC Subgroup 43 - June 201922
Problems with this approach…
Lots of files…Slow…
Reading nuclear data can take minutes
Reading and writing files is not only slow but introduces bugs*.i, *.out, *.json, *.tab1,
*.tab2, fluxes, files…..
Bugs…
| Thomas Stainer - WPEC Subgroup 43 - June 201923
More problems with this approach…
Cannot interface easily (at all?) with other codes
Must write input files from <insert transport code here>, float to string issues
Maybe we could make a GUI to read and write input and output files, but doesn’t scale, especially for web applications
|
What is an API?
In computer programming, an API can be considered as the suite of functions, commands, subroutines, protocols, objects and/or tools for building software and interacting with an application. Generally, it is the set of clearly defined methods, or interfaces, of communication among various components.
FISPACT-II API
Thomas Stainer - WPEC Subgroup 43 - June 201924
No more files! (well almost, some are still needed)
|
• Science must be reproducible• Easy way to define configuration and parameters of codes• Nuclear data is still text files
We still need files…
Thomas Stainer - WPEC Subgroup 43 - June 201925
but we can be smarter..• Less files• Support multiple formats• Use of databases instead of text files
|
Program 1
input files output files
Program 2
input files output files
Program 3
input files output files
Parser/Writer Parser/Writer
We still need files…
Thomas Stainer - WPEC Subgroup 43 - June 201926
Program 1
input files
Program 2 Program 3
output files
API API
Before
After
|
Numerical stability for nuclear instability
Thomas Stainer - WPEC Subgroup 43 - June 201927
212,640 FISPACT-II calculations
Took ~7 hours on local machine
Using exe instead would have taken ~9-10 days
(Actual – Expected)/Expected
100%
|
Numerical stability
Thomas Stainer - WPEC Subgroup 43 - June 201928
10 steps per half-life
|
Numerical stability
Thomas Stainer - WPEC Subgroup 43 - June 201929
100 steps per half-life
|
Error handling improvements
Thomas Stainer - WPEC Subgroup 43 - June 201930
Faster than writing to log file