D. L. Bruhwiler, 1 G. I. Bell, 1 A. Sobol, 1 P. Messmer, 1 J. Qiang, 2 R. Ryne, 2 W. Mori, 3 A....

D. L. Bruhwiler,1 G. I. Bell,1 A. Sobol,1

P. Messmer,1 J. Qiang,2 R. Ryne,2

W. Mori,3 A. Fedotov,4 I. Ben-Zvi,4

V. Litvinenko,4 S. Derbenev,5

R. Li,5 Y. Zhang,5 L. Merminga5

COMPASS – Community Petascale Project for Accelerator Science and

Simulation

Parallel Electron CoolingSimulations for SciDAC

1. Tech-X Corporation2. Lawrence Berkeley National Lab3. Univ. of California at Los Angeles4. Brookhaven National Lab5. Thomas Jefferson National Lab

COMPASS Kick-Off Meeting p. 2SciDAC – September 17, 2007 – Fermilab

Physics Motivation for Cooling Ion Beams

• A future polarized e-/ion collider is required to better probe the hadronic structure of matter– goal of the international nuclear physics community– ion luminosity of order 1033-1035 cm-2 s-1 is required

• orders of magnitude larger than present state-of-the-art– DOE Office of Nuclear Physics has two long-term concepts

• eRHIC (add e- linac+ring to the RHIC complex)• ELIC (add e- ring & ion linac+ring to Jefferson Lab)

• Higher luminosity requires cooling of ion beams– “electron cooling” is included in all present designs

• RHIC II luminosity upgrade requires e- cooling– near-term priority of the Office of Nuclear Physics– stochastic cooling is being implemented and may be good enough

• recent success of M. Blaskiewicz, M. Brennan & many collaborators• won’t work at higher intensities of eRHIC or ELIC

• Low-energy operation of RHIC is planned in 2010– recent idea to explore new physics– adequate statistics may require cooling for high luminosity

• Completely new idea: “coherent electron cooling”– combination of e- cooling and stochastic cooling– if it works, could provide 10x increase of friction or more


Courtesy of A. Fedotov et al., COOL ’07 Presentation (Sep. 10, 2007)


Electron Cooling for RHIC II


tp

Image courtesy of V. Litvinenko, BNL

Merminga, EIC2007, Apr 6-7 2007

ELIC LayoutELIC Layout

30-225 GeV protons15-100 GeV/n ions

3-9 GeV electrons3-9 GeV positronsGreen-field design of ion

complex directly aimed at full exploitation of science program.


Dynamical Friction/Diffusion is Long in the Tooth

• Case of isotropic plasma, with no external fields, was first explained 65 years ago– S. Chandrasekhar, Principles of Stellar Dynamics

(U. Chicago Press, 1942).– B.A. Trubnikov, Rev. Plasma Physics 1 (1965), p. 105.– NRL Plasma Formulary, ed. J.D. Huba (2000).

– Physics can be understood in two different ways• Binary collisions (integrate over ensemble of e-/ion collisions)• Dielectric plasma response (ion scatters off of plasma waves)

e-

e-

e-

e-


Idea for Electron Cooling is 40 Years Old

• Budker developed the concept in 1967– G.I. Budker, At. Energ. 22 (1967), p. 346.

• Many low-energy electron cooling systems:– continuous electron beam is generated– electrons are nonrelativistic & very cold– electrons are magnetized with a strong solenoid field

• suppresses transverse temperature & increases friction

• Fermilab has shown cooling of relativistic p-bar’s– S. Nagaitsev et al., PRL 96, 044801 (2006).

– 4.3 MeV e-’s (≈8) from a customized DC source– electrons are unmagnetized (solenoid for focusing)

• RHIC II, eRHIC, ELIC need “high-energy” cooler– 100 GeV/n -> ≈108 -> 54 MeV bunched electrons– Cooling is less efficient; new parameter regime


tp

RHIC II Electron Cooling System

Image taken from “RHIC II White Paper,” http://www.bnl.gov/cad/ecooling/docs/PDF/RHIC%20II_White_Paper.pdf

Merminga, EIC2007, Apr 6-7 2007

ELIC Circulator CoolerELIC Circulator Cooler

ion bunch

electron bunch

Electron circulator ring

Cooling section

solenoid

kickerkicker

SRF Linac

dump

electron injector

energy recovery


VORPAL supports use of BETACOOL

• BETACOOL code is used to model many turns• A.O. Sidorin et al., Nucl. Instrum. Methods A 558, 325 (2006).• A.V. Fedotov, I Ben-Zvi, D.L. Bruhwiler, V.N. Litvinenko, A.O. Sidorin,

New J. Physics 8, 283 (2006).

– a variety of electron cooling algorithms are available• in particular, models for the dynamical friction force

– many mechanisms for emittance growth are included

• VORPAL is used to study microphysics of friction– to increase understanding & make BETACOOL more effective


Numerical Approaches for Electron Cooling Simulations

• Langevin approach to solve Fokker-Planck equation– uses Rosenbluth potential (or Landau integral)– proof-of-principle demonstrated; to be extended to e-/ion interactions

• Fast multipole method (FMM) and tree-based algorithms– requires constant time step; inefficient for MD with a few close collisions– being reconsidered as part of the Langevin approach

• 4th-order predictor-corrector “Hermite” algorithm– taken from astrophysical dynamics community– generalized to include solenoid field– used successfully in molecular dynamics (MD) approach with a few ions– didn’t parallelize well, so we used a task farming approach

• astrophysicists use special “Grape” hardware to parallelize• Semi-analytic binary collision model

– also MD approach; very close connection to “Hermite” algorithm above– accurately models arbitrarily strong Coulomb collisions– arbitrary external fields included via 2nd-order operator splitting– scales well up to ~128 processors; must be generalized for petascale

• Electrostatic particle-in-cell (PIC)– cannot directly capture close Coulomb collisions– will rely on PETSc/Aztec00 for effective use of petascale hardware– could be combined effectively with “binary collision” model


Self-Consistent Langevin Solution of the Fokker-Planck/Landau Equation


A Test Example Showing Temperature Exchange in a 2-Species System


The Parallel VORPAL Framework is used to Simulate the Microphysics of Electron Cooling

• Electromagnetic PIC for laser-plasma– Nieter & Cary, J. Comp. Phys. (2004).

• Electrostatic PIC for beams & plasma– Messmer & Bruhwiler, Comp. Phys. Comm. (2004)

• Algorithms for simulating electron cooling physics– Fedotov, Bruhwiler, Sidorin, Abell, Ben-Zvi, Busby, Cary & Litvinenko, Phys. Rev.

ST/AB (2006).– Fedotov, Ben-Zvi, Bruhwiler, Litvinenko & Sidorin, New J. Phys. (2006).– Bell, Bruhwiler, Fedotov, Sobol, Busby, Stoltz, Abell, Messmer, Ben-Zvi &

Litvinenko, “Simulating the dynamical friction force on ions due to a briefly co-propagating electron beam,” J. Comp. Phys., in preparation.

• SRF Cavities, Electron guns, Dielectric structures (PBG)…– Nieter et al., J. Comp. Phys., in preparation.– Dimitrov, Bruhwiler, Smithe, Messmer, Cary, Kayran & Ben-Zvi, Proc. ICFA Beam

Dynamics Workshop on Energy Recovery Linacs (2007), in press.– Werner & Cary, J. Comp. Phys., in preparation.

• Large software development team (Tech-X & CU)– C++/MPI, object-oriented, template techniques, multi-physics– parallel or serial; cross-platform (Linux, AIX, OS X, Windows)

• Actively used throughout the beam & plasma communities– BNL, JLab, Fermilab, LBL, ANL, some universities, also outside the USA– commercial customers

• Development and use has been supported by several agencies since 2000– US DOE Office of Science (HEP, NP, FES & ASCR)– NSF (original grant); DOD (AFOSR, OSD)


Molecular dynamics approach – model each e-

Diffusive spreading of ion trajectories obscures any velocity drag due to dynamical friction.

For many millions of turns, friction forces will dominate diffusion.


Diffusive dynamics can obscure friction/drag

• Numerical trick of e-/e+ pairs can suppress diffusion– idea came from Alexey Burov– simulate with e-/e+ pairs that have identical initial conditions

• sign of external fields must be flipped for the positrons

– friction force, independent of sign of charge, is unchanged– diffusive kicks are approximately cancelled

• also use ~1,000 trajectories for each electron– RMS is reduced by Ntraj

1/2 from that of the original distribution

• from the Central Limit Theorem

• Use of many trajectories sometimes changes results !!– always true for field free case:

– in presence of external fields• some other limitation on min may hide this problem

• also,higher effective e- velocity can decrease res

trajreserel

trajcsimres N

nv

NN

,


CLT is used to pull <F> and error bars from binned data


Diffusive dynamics of ions can be correctly modeled


Perturbative calculations lead to Coulomb log

• Assume each e- trajectory is infinite and straight– integrate Coulomb force along trajectory to obtain v~-1

• integrating over all angles leads to zero

– by symmetry, v|| = 0 for each trajectory

• however, energy conservation requires v|| ≈ - v2 / 2v ~-2

• Approximation is very good for large – assumption of infinite trajectories not valid for finite

– choose physically reasonable cutoff max

• Approximation breaks down for small – choose cutoff min, for which v = vrel 90 deg scattering

e-

v=vi-ve vv||


Dynamical friction & finite-time effects on max

• For pe >> 2 and vi << e

– electron cloud screens ion charge; max = D = e/pe

• In opposite limit, pe < 2 (RHIC II param’s)– no screening of ion charge; choose max ~ max(vi,e)

– or calculate Coulomb log with finite-length trajectories• completely removes logarithmic singularity at large

for

for min ln d


Finite-time effects limit # of collisions for small

• Poisson statistics predict likelihood of Nc collisions– for all impact parameters less than

ne

= Nc


Finite-time effects on min lead to concept of res

• Friction force integrals assume, for all – there are plenty of trajectories to sample 4 sr

• so perpendicular kicks average out• so longitudinal kicks accumulate correctly

– and sufficient trajectories to sample e- velocities

• How many collisions are needed?– good agreement with simulations for Nc = 120

– for RHIC II parameters shown above• min ≈ 2.2 e-7 m res ≈ 1.8 e-5 m

• for max ~ vi ≈ 7.5 e-4 m >20% smaller Coulomb log


Physical Parameters from 2006 RHIC II Design


Failure to sample small reduces friction force

yields choice Nc ≈ 120


Numerical effects of domain size are understood


Unmagnetized high-energy cooling w/ undulator

• Purpose of the helical undulator magnet– provides focusing for electrons– suppresses e-/ion recombination

• Modest fields (~10 Gauss) effectively reduce recombination via ‘wiggle’ motion of electrons:

• What’s the effect of ‘wiggle’ motion on cooling?– increases minimum impact parameter of Coulomb log

][][104.1~ 232

GBmxvk ww

beamw

gyroosc

resosc ,2max min


VORPAL simulations verify effect of undulator

• Coherent e- wiggle motion – increases the effective minimum impact parameter– dynamical friction force is only reduced logarithmically


Simple model of single-wavelength error fields

zBzyxB erry 2sin,,

2,, max err

z

y BzyxBdzM

TmMm/sxMm

ev

eerr

65 10for 102

m 80 cm 10 MBerr 2 TmxMTm 56 10210

ctcBtyxE errx 2sin,,' Lorentz transform to beam frame:

e- oscillations:

mTmMmxvc

xerr 1 ,10for 109 2

67err


Long wavelengths increase effective e- temp


Short wavelengths increase effective min


Key Questions for Electron Cooling Simulations

• Electron cooling is key to future NP accelerators– never demonstrated for high-energy ions– cooling effectiveness may be “just enough”– simulations must help to improve design & reduce risk

• How best to suppress e-/ion recombination?– strong solenoid (used in all low-energy coolers)– helical undulator magnet (recent alternative concept)

• Quantify all phenomena that weaken cooling– e- wiggle motion in undulator– wide variety of magnetic field errors– high density of ions, complicated e- distributions, …

• New ideas must be supported– concept of “coherent electron cooling”– low-energy cooling for RHIC and/or RHIC II


Courtesy of V. Litvinenko & Y. Derbenev (FEL 2007)


Parallel Scaling for Electron Cooling Simulations

Execution times for Trillinos-based Poisson solve of a 3D Gaussian beam, for a 1026×65×65 grid (solid) and 4104×65×65 grid (dotted), using AMG precon-ditioned CGS (diamond) or Gauss-Seidel preconditioned CGS (stars).

To date, sim’s have been confined to ~256 processors or less.


Challenges for Petascale Cooling simulations

• Petascale path for MD involves move to PIC– O(Nion*Ne) scaling of pure MD limits problem size– hybrid electrostatic PIC/MD model will be considered

• PIC efficiently captures distant interactions– pure PIC will be useful, if min is bounded from below

• e.g. L, xerr, res

• Input needed from CETs– use of PETSc or Trilinos to rapidly solve Poisson eq.

• path to petascale for PIC and/or hybrid PIC/MD• VORPAL uses Trilinos for electrostatic PIC now

– plan to implement PETSc as an option– assistance in scaling and speed for other algorithms

• particle push, charge deposition, etc.– access to additional computing resources?

• will request 10x-50x larger allocation from NERSC this year– visualizing details of the electron wake

• one solution is IDL + AVS (used successfully in the past)• will try to benefit from ongoing work with VORPAL & VisIT


Future Plans for Langevin Approach

• Extend the Langevin approach to electrons and ions• Include effects of magnetic fields in the calculation of

Fokker-Planck collisional operator• Explore an efficient FMM-based technique to compute

the F-P collisional operator• Implement improved time integration schemes for

electrons and ion dynamics• Integrate the Fokker-Planck solver with our other

modules for multi-physics modeling


Acknowledgements

We thank O. Boine-Frankenheim, A. Burov, A. Jain, S. Nagaitsev, A. Sidorin, G. Zwicknagel & members of the Physics group of the RHIC Electron Cooling Project for many useful discussions.

We acknowledge assistance from the VORPAL team: J. Carlsson, J.R. Cary, B. Cowan, D.A. Dimitrov, A. Hakim, P. Messmer, P. Mullowney, C. Nieter, K. Paul, S.W. Sides, N.D. Sizemore, D.N. Smithe, P.H. Stoltz, S.A. Veitzer, D.J. Wade-Stein, G.R. Werner & N. Xiang.

Work at LBNL was supported by SciDAC-1.

Work at BNL and JLab was supported by the U.S. DOE Office of Science, Office of Nuclear Physics.

Work at Tech-X Corp. was supported by the U.S. DOE Office of Science, Office of Nuclear Physics under grants DE-FG03-01ER83313 and DE-FG02-04ER84094. We used computational resources of NERSC, BNL and Tech-X Corp.

D. L. Bruhwiler, 1 G. I. Bell, 1 A. Sobol, 1 P. Messmer, 1 J. Qiang, 2 R. Ryne, 2 W. Mori, 3 A....

Documents

Transcript of D. L. Bruhwiler, 1 G. I. Bell, 1 A. Sobol, 1 P. Messmer, 1 J. Qiang, 2 R. Ryne, 2 W. Mori, 3 A....