PHY 604: Computatonal Methods in Physics and Astrophysics II
Summary
PHY 604: Computatonal Methods in Physics and Astrophysics II
Reminder: Evaluatons
● Please take the tme to fll out the online course evaluaton
● Feedback is helpful and appreciated
PHY 604: Computatonal Methods in Physics and Astrophysics II
Safe Code Practces
● You now have some experience with version control
– Set up version control for your own project
● This pays of the frst tme you need to go “back in tme” to fgure out what recent bug messed everything up
● Few research codes are done by a single person anymore—without version control, sharing is messy
● Any version control system is beter than none
PHY 604: Computatonal Methods in Physics and Astrophysics II
Reuse Your Code
● You've probably seen a number of instances when doing your homework where you built from existng code
– Not only is this much easier/quicker, it is safer
– You debug and maintain a single routne (e.g. in a library), and all of your projects can use it. This makes testng easier
● We've writen a number of Ex: matrix routnes
PHY 604: Computatonal Methods in Physics and Astrophysics II
Test Test Test
● Testng takes tme
– Test as you go along
– Write scripts to automate it as much as possible
● Unit testng
– For each major piece of your algorithm, think about how you might test it in isolaton from the rest of the code
● Regression Testng
– Test your output looking for changes in the results frequently
– Together with version control, this allows you to catch and recover from recently introduced bugs
● We saw some examples of these earlier in the class
PHY 604: Computatonal Methods in Physics and Astrophysics II
What Have We Learned
● One of the main goals of this class is that we should know what the packages we are using for our analysis, numerical simulaton, etc. are doing
– This does not mean that we should always reinvent the wheel and code everything on our own from scratch
● Actually, that's usually a bad idea
– Understanding the basics of these algorithms helps you understand their limitatons, assumptons, and failure modes
PHY 604: Computatonal Methods in Physics and Astrophysics II
What Have We Learned?
● Basics of programming
– We got a feel for how the computer is storing the numbers we feed it
– In most of our algorithms, we dealt with both roundof and truncaton error
– Roundof is mostly unavoidable (we discussed a few algorithmic tricks to minimize it)
– For truncaton error, a powerful test of your code is to check if your error converges at the expected rate
PHY 604: Computatonal Methods in Physics and Astrophysics II
What Have We Learned?
● Diferentaton/Integraton
– We saw diferent techniques for when you have f(x) defned contnuously vs. discretely
– This is where we frst really explored truncaton error
– Some neat methods: Gaussian quadrature
● Interpolaton
– Explored linear, polynomial interpolaton, splines
– Some nice examples of how higher-order is not always beter
– Trade-ofs between accuracy and smoothness
● Root-fnding
– Newton's method came up several tmes in later lectures
– Very powerful tool. We saw a few examples of what happens if your inital guess is not close enough though...
PHY 604: Computatonal Methods in Physics and Astrophysics II
What Have We Learned?
● ODEs
– 4th order Runge-Kuta is robust. It can serve most of your needs well.
– Stf systems require implicit methods for accuracy
– You should always use some form of error estmaton / adaptve stepping—otherwise you have no measure of your integraton error
● Linear algebra
– This is a huge topic
– Linear systems underlie many of the topics that follow
– We looked a direct and iteratve solvers
– Lots of great libraries exist that take advantage of sparsity or symmetries in your matrices
PHY 604: Computatonal Methods in Physics and Astrophysics II
What Have We Learned?
● FFTs
– Great example of how reorganizing the computatons can result in a much more efcient algorithm (stll does the same computaton)
– Also a great example of a method where the publicly available routnes are the way to go: they are fast, well tested, and fexible
● Fitng
– General linear least squares gives us a linear system that we can solve with the matrix techniques that are widely available
– General nonlinear least squares is much more difcult
● Likely that some linearizaton is done, with N-R to solve● Good inital guesses are critcal● Publicly available packages are the way to go—they are robust and tested
PHY 604: Computatonal Methods in Physics and Astrophysics II
What Have We Learned?
● PDEs:
– We saw diferent techniques for hyperbolic, elliptc, and parabolic PDEs
– Many many diferent types of discretzatons exist—we focused on fnite-volume and cell-centered fnite-diference
● Diferent strengths and weaknesses
– We discussed a bit about how to deal with scalar equatons and systems that mix the types of PDEs
– The Euler equatons was our introducton on how to deal with nonlinear hyperbolic systems
● Parallel programming:
– OpenMP for shared-memory—easy to get started, can do piece-by-piece
– MPI for distributed—need rethink the algorithm and layout of the data
– OpenACC provides a path to doing GPU programming
PHY 604: Computatonal Methods in Physics and Astrophysics II
What Have We Learned?
● Monte Carlo:
– Efcient for multdimensional integrals
– Basis for Markov chain Monte Carlo optmizaton
● Genetc algorithms:
– Like MCMC, a stochastc optmizaton method
PHY 604: Computatonal Methods in Physics and Astrophysics II
What Didn't We Cover
● SIAM Editors: Top 10 Algorithms of the 20th Century (SIAM News, 33, 4)
– Monte Carlo
● We did MC integrals and MCMC
– Simplex method for linear programming/optmizaton
● Looks to maximize: ● Real problems may be nonlinear
– Krylov subspace iteraton
● We briefy referenced one method: conjugate gradient● Works in the subspace of powers of A: A0b, Ab, A2b, ... ● Solve linear systems iteratvely
– Decompositonal approach to matrix computatons
● E.g. LU decompositon
– Fortran optmizing compiler
PHY 604: Computatonal Methods in Physics and Astrophysics II
What Didn't We Cover
● Contnued: SIAM Editors: Top 10 Algorithms of the 20th Century
– QR algorithm
● Method for computng eigenvalues
– Quicksort algorithm
● NlogN method for sortng numbers (very robust)
– FFT
● moves N2 discrete Fourier transform cost to N log N
– Integer relaton detecton algorithm
● Given x1, x2, ..., xn fnd numbers a1, a2, ..., an such that a1 x1 + a2 x2 + ...
+ an xn = 0 ● Apparently useful in Feynman diagram calculatons
– Fast multpole algorithm
● Reduces N2 calculatons in N-body calculaton to O(N) using multpole expansions and a hierarchical decompositon of the domain
PHY 604: Computatonal Methods in Physics and Astrophysics II
What Else Didn't We Cover?
● Chaos and nonlinear dynamics
– We saw a few examples, but there is a lot of explore here
● We didn’t cover diferent optmizaton methods as a single topic
– We briefy looked at Newton's method
– We looked a heuristc methods simulated annealing and GA
– We saw gradient/steepest descent in neural networks
– Didn’t discuss amoeba methods (Nelder-Mead heuristc)
● Statstcal measures of data sets (like the Students t-test)
● MCMC for parameter ftng (Bayesian statstcs)
PHY 604: Computatonal Methods in Physics and Astrophysics II
What Else Didn't We Cover?
● Computatonal fuid dynamics
– This builds on the ideas from advecton, but now we consider systems
● N-body/Tree codes
– Basic idea: we need to compute the N2 force interactons between all pairs of partcles. How can we do it in N logN or faster tme?
● Molecular dynamics
● SPH
– SPH is a popular hydrodynamics method used in astrophysics
– Instead of a grid, a collecton of partcles samples the mass distributon
– Contnuous quanttes are formed by integratng over nearby partcles with a smoothing kernel
PHY 604: Computatonal Methods in Physics and Astrophysics II
Where Do Physics Algorithms Get Published?
● Most journals publish algorithms
– Journal of Computatonal Physics is historically one of the main dedicated journals
● Lots of newer journals are popping up, especially those pursuing Open Access
– ApJ publishes many code papers and new algorithms, and has recently expanded its mission to allow pure sofware papers
● Also a new concept: for sofware package papers, allowing the author list to change in tme
– SIAM journals publish more applied math stuf
PHY 604: Computatonal Methods in Physics and Astrophysics II
Where Do Physics Algorithms Get Published?
● One of the difcultes you will encounter is that quite ofen, the litle details of algorithms are lef out of a paper
– This is both for space consideratons, or that it is assumed to be obvious
– If you spend months/years writng a code and then go to write the paper, you probably forget some of the minor design decisions, trade-ofs, nuances that were encountered over the development period
PHY 604: Computatonal Methods in Physics and Astrophysics II
Supercomputng Centers
● Supercomputng centers
– Natonal centers run by NSF (through XSEDE program) and DOE (NERSC, OLCF, ALCF)
– You can apply for tme—starter accounts available at most centers to get up to speed
– To get lots of tme, you need to demonstrate that your codes can scale to O(104) processors or more
● Queues
– You submit your job to a queue, specifying the number of processors (MPI + OpenMP threads) and length of tme
– Typical queue windows are 2 – 24 hours
– Job waits untl resources are available
PHY 604: Computatonal Methods in Physics and Astrophysics II
Supercomputng Centers
● Checkpoint/restart
– Long jobs won't be able to fnish in the limited queue window
– You need to write your code so that it saves all of the data necessary to restart where it lef of
● Archiving
– Mass storage at centers is provided (usually through HPSS)
– Typically you generate far more data than is reasonable to bring back locally—remote analysis and visualizaton necessary
PHY 604: Computatonal Methods in Physics and Astrophysics II
Should I Make My Code Public?
● Yes!
PHY 604: Computatonal Methods in Physics and Astrophysics II
Should I Make My Code Public?
● Yes!
● Experience shows that the benefts far out way the perceived risks
– Pros:
● You gain collaborators● People fnd and fx bugs● Your code paper gets lots of cites● Your results become reproducible—this is aferall one of the hallmarks of
science
– “Risks”:
● You'll get scooped!
– Reality: although this can happen, it has not been my experience. More likely is that someone will start to collaborate with you on related problems. Remember: you know your code best!
– Support
– Licensing
PHY 604: Computatonal Methods in Physics and Astrophysics II
Should I Make My Code Public?
● Things to think about:
– Support
– Licensing
PHY 604: Computatonal Methods in Physics and Astrophysics II
Documentaton
● No one likes to write documentaton
● Serves multple purposes:
– For developers, it preserves the train-of-thought that lead to design decisions, serves as a reference for complicated pieces of the algorithm, helps you get back into things afer some tme away
– For new users, it serves as the introducton
● Sharing your code? writng documentaton once saves you from having to answer the same questons over and over
– Start small: document pieces as questons arise or major design changes are done. This lets you gradually build up.
PHY 604: Computatonal Methods in Physics and Astrophysics II
I/O
● For many projects, output is not a major concern
– If you are just outputng a few columns of numbers, then ASCII works well, is portable
– Not very space-efcient, but typically doesn't mater with small outputs
● What if you need to write lots of data?
– Simulaton outputs of ~10 to 100 GB per fle are not unusual anymore
– You must store this in binary form
– You'll need to do parallel I/O
– Binary is generally not portable across machine architectures
PHY 604: Computatonal Methods in Physics and Astrophysics II
I/O and Endianness
● Diferent architectures store binary numbers with diferent byte orderings (endianness)
– Big-endian: most signifcant byte is stored frst (startng address)
● Examples: Motorola 68k, IBM POWER
– Litle-endian: least signifcant byte is stored frst
● Examples: x86, DEC Alpha, VAX
(Wikipedia)
PHY 604: Computatonal Methods in Physics and Astrophysics II
I/O and Endianness
● It is most efcient for the machine to write out in its natve format
● On the opposite endian machine, you will need to manually swap bytes as they are read in (can be slow)
● Some fle formats / libraries handle this automatcally
– HDF5: transparently handles endian diferences, supports parallel I/O, self-documentng
– NetCDF: similar to HDF5
– FITS: standard doesn't specify endianness, but NASA apparently says big-endian
● If you are going to make data publically available, using a standard I/O format/container may make things easier
PHY 604: Computatonal Methods in Physics and Astrophysics II
Final Thoughts...
● If you fnd yourself doing the same thing over and over, automate it
● Ask around
PHY 604: Computatonal Methods in Physics and Astrophysics II
Python for Scientfc Computng
Python has seen wide adopton in the scientfc community for data analysis, simulaton, prototyping, and visualizaton. It provides a simple, yet powerful means to build applicatons. This seminar introduces python and its use in scientfc computng.
● Flipped course format:
– We’ll work through interactve notebooks outside of class
– Class tme will be used for exercises that we discuss together
– Use slack for out-of-class communicaton (and learn how to integrate github + python + slack)
– Grading is based on partcipaton
● Sharing examples and discussion
● Advanced undergrads welcomed
● Topics include:
– Python
– Version control with git/github
– Jupyter notebooks /workfow management
– The NumPy array package
– The SciPy tools and basics of numerical methods
– Matplotlib and for visualizaton
– SymPy for symbolic mathematcs
– Pandas and the dataframe
– Building applicatons
– Interfacing with Fortran/C/C++
● Details:
– PHY 546, Spring 2018
– Mondays, 3:00-3:53pm
a weekly graduate seminar on techniques for scientfc programming
PHY 604: Computatonal Methods in Physics and Astrophysics II
● topics:
– the equatons of hydrodynamics
– viscous fows
– gas dynamics
– instabilites
– turbulence
– numerical methods for hydrodynamics
– reactve fows
– self-gravitatng and rotaton fows
– introducton to MHD
● logistcs
– ~ frst ½ of the course will be analytc discussions w/ applicatons to astro
– ~ ¼ of the class will focus on numerical methods for hydro
– traditonal problem sets + computatonal projects
– each student give a short presentaton on a topic related to the course
● course info
– instructor: Michael Zingale
– Spring 2018
– M/W 10:00 – 11:20am
PHY 688 (Special Topics in Astro)
Astrophysical Fluids and Plasmas
Can count for breadth requirement