A Monte Carlo Model of Tevatron Operations

A Monte Carlo Model of A Monte Carlo Model of Tevatron OperationsTevatron Operations

Elliott McCroryFermilab/Accelerator Division

13 October 2005

Elliott McCrory, Fermilab/AD 3/7013 Oct 2005

Where is Fermilab?Where is Fermilab?


~50 km


Fox R

iver


Fermilab OverviewFermilab Overview

Main Injector& Recycler

TevatronLinac

Booster

Pbar Source


OutlineOutline

Overview of the Operations Model Monte Carlo ≡ Randomizations

SDA Sequenced Data Acquisition

Shot Data Analysis

Model Observations and Predictions

Effects of Future Improvements Note:

Several “extra” concepts relating to current Tevatron performance. May have to skip some.


Definitions of “Model”Definitions of “Model” Curiously subtle shades of

meaning!1. An example for imitation or

emulation• “My brother is a role model for my

son”2. Person who serves as a subject

for an artist or a fashion designer3. A Structural Design

• “We need a business model”4. A type or design of a product

• “I own a Volkswagen.” “Which model?” “Jetta.”

5. Something built to represent reality in a simplified way• “Model Airplane, 1:32 scale”

Here: 5


Fermilab TerminologyFermilab Terminology Stack

The antiprotons in the Accumulator Stash

The antiprotons in the Recycler Shot

The process of transferring antiprotons to the Tevatron

Done during a “Shot Setup” Store

Proton/antiproton collisions in the Tevatron Begins at the end of the Shot Setup Often used interchangeably with shot

Transfer AccumulatorRecycler antiproton transfer and its

associated setup time


Fermilab Operations: BasicsFermilab Operations: Basics Stacking

Antiproton production in DebuncherAccumulator• Every 2 to 3 seconds• 15E10 per hour

AccRecycler transfers• Three or four time per store, today

– Depends on stacking rate

Shot Setup • 100 to 200 minutes• Each step is 10 to 60 minutes

Tuning Transfer protons into Tevatron Transfer antiprotons into Tevatron Accelerate Squeeze/scrape

Collisions 20 to 40 hours

Between stores …


The RecyclerThe Recycler An antiproton Storage Ring

Main bends are permanent magnets Transfers into Recycler every few hours

• Offloading antiprotons from Accumulator

Advantages over Accumulator Electron cooling and Stochastic cooling Emittances are better

• 4π smaller transverse emittance• Longitudinal emittances are consistent and smaller

Transfers into Tevatron are better• Transmission efficiency is higher• Brighter antiprotons bunches at collisions

Can hold more antiprotons Stacking rate into Accumulator is better at smaller

stacks


Fermilab Operations: UpdateFermilab Operations: Update Averaging 18 pb-1 delivered per week to our two

experiments 107 ± 27 hours/week in collisions 10 E10 antiprotons/hour

Initial luminosity world record set on 4 October 1.42E32 [cm-2 sec-1]

Main Injector Slip stacking

Recycler Electron cooling achieved on July 9 Implemented for >50% of transfers to Tevatron in last 4

weeks• Cool by 70 eV-sec in 80 minutes, 250E10 particles

I am, by no means, an expert on these topics! An intelligent observer, perhaps


Recycler Electron CoolingRecycler Electron Cooling

Beam current: 250E10

Longitudinalemittance

53 minCool 70 eV-sec in 80 minutes

Transverseemittance


Tevatron Operations StatusTevatron Operations Status BPM Upgrade completed New lattice implemented last month

28 cm beta-star• Practical understanding of coupled machine• Partially equalized luminosity at 2 experiments• Reduced beta-beating in arcs between 2 experiments• Increase luminosity by ~15%

Previous lattice change December 2004 30% improvement in luminosity

Extra Orbit stabilization Crystal Collimator demonstration?


Orbit StabilizationOrbit StabilizationEXTRA


Crystal Collimator StudyCrystal Collimator StudyEXTRA

The Operations The Operations ModelModel


One Week of OperationOne Week of Operation

Recycler Stash

Luminosity

AccumulatorStack


One Simulated Week of OpsOne Simulated Week of Ops

Hours

Blu

e:

recy

cler

Sta

sh [

E1

0]

Red

: Lu

min

osi

ty [

1/(

cm2 s

ec)

]G

reen

: A

ccu

mu

lato

r S

tack

[E

10

]

Recycler Stash

Luminosity

AccumulatorStack


Basic IdeaBasic Idea Phenomenological representation of the

Tevatron Complex Mostly non-analytic

Monte Carlo (randomizations) Complexity is replaced by randomizations

Downtime For the Tevatron, stacking, PBar Source, etc.

Real data: Match model to reality This model’s genesis:

To develop intuition and provide guidance for optimizing luminosity

Now: Extrapolations/”What If”, based on today’s

performance• The effect of Recycler improvements


Complexity Complexity Randomness Randomness Variations in all realistic parameters

For example• Transmissions during a shot, • Luminosity lifetimes, • Extraction efficiency from antiproton sources, • Shot setup time, • Downtime for each sub-system,• Etc…

Model AssumptionsPerformance does not improve

• Random fluctuations around a specific set of parameters

• Performance determined largely by these parameters

• Better performance? Change parameters and run again.

No shutdown periods


Luminosity CharacterizationLuminosity Characterization One average proton & 36 antiprotons are

tracked Proton bunches are all the same Recycler & Accumulator antiproton bunches are different

L i(t=0) =

K H Np(0) NPBar, i(0)

[єp (0) + єPBar, i

(0)]

L (t) = L (0) e -t/τ(t)

τ(t) = τ(0) + C1 t C2

• τ(0) depends on L (0) and is adjusted to fit Real Data

• C1 = 3 ± 2

• C2 = f(C1) ≈ 0.5


Match Model to RealityMatch Model to Reality

GoalAppropriate range of values for

important parametersCorrelations among the parameters

Data SourcesSDA

• The “Supertable”• Other data tables

Data loggersWeekly summaries from operations

SDASDA


SDA: Overloaded AcronymnSDA: Overloaded Acronymn Sequenced Data Acquisition

Defines alternate “clock” for recording dataExtends definition of what can be stored

Shot Data AnalysisLook at Sequenced Data Acquisition database Look at conventional data loggersCreate summariesDo certain types of calculations

• More complicated (transmission efficiencies)• Time dependent (Emittances)

Observe/alert


Sequenced Data AcquisitionSequenced Data Acquisition More relevant “clock”

Shot/store number• Today: store # 4440

Case• Collider shot: 15 main

cases1. Proton Injection Porch2. Proton Injection tune up3. Eject Protons4. Inject Protons5. Pbar Injection Porch6. Inject Pbars7. (Defunct)8. Before Ramp9. Acceleration10. Flattop11. Squeeze 12. Initiate Collisions13. Remove Halo14. HEP15. Pause HEP

Set• Each case may have one or

more sets

For example: “What happened at 4401,

Inject Protons, second bunch injection [a.k.a. Set 2]?”

Other common processes use this clock abstraction AccumulatorRecycler

transfers Pbar Transfers to Tevatron


Sequenced Data AcquisitionSequenced Data Acquisition Data collection abstraction

All types of data can be acquired Implemented as a Java interface

SDA Database Detailed information

• 36 bunch data• Raw data from front ends

Indexed by Store Number• Accumulator to Recycler Transfer Number

30 GB today Data Loggers

Not strictly part of this, but very relevant Store <timestamp, value> pairs in relational DB

• Essentially Unix + milliseconds timestamp 70+ instances at Fermilab

• O(100 GB)


Shot Data AnalysisShot Data Analysis Data mining applications

Example

Sequenced Data Acquisition cross-checks Summary tables on the web The Supertable

A summary of key information, mostly from SDA database

Excel, HTML, AIDA/JAS One row = one store 224 columns for each store http://www-bd.fnal.gov/sda/supertable


SDA Database ExampleSDA Database Example


http://www-bd.fnal.gov/sda/supertablehttp://www-bd.fnal.gov/sda/supertable


Supertable ExampleSupertable Example


SDA Examples Relevant to SDA Examples Relevant to ModelModel

Using Excel Initial Luminosity versus Number of Antiprotons Initial Luminosity versus Initial Luminosity Lifetime Antiproton Emittances Uncertainty at the IP

• Beta-star changing??

Extras Lifetime fits

• Record luminosity vs. record integrated luminosity?

Antiproton Burn Rate Tevatron failure rate

• Not strictly SDA


Initial Luminosity vs. # PBarsInitial Luminosity vs. # PBars # 45 pbars at Remove Halo (1E09)

0

200

400

600

800

1000

1200

1400

1600

1800

2000

0 1000 2000 3000 4000 5000

Sequential Store Number

An

tip

roto

ns

at

Lo

w B

eta

[E

9]

# 10 MCR CDF initial lum - default (1E30)

0

20

40

60

80

100

120

140

160

0 500 1000 1500 2000

Antiproton Intensity [E9]

Init

ial

Lu

min

os

ity

at

CD

F [

E3

0]


Initl Lum Vs. Init Lum LifetimeInitl Lum Vs. Init Lum Lifetime # 23 CDF lum lifetime (hours)

0

2

4

6

8

10

12

14

16

18

20

0 20 40 60 80 100 120 140 160

Initial Luminosity at CDF [E10]

Av

era

ge

life

tim

e o

f C

DF

Lu

m o

ve

r 1

st

2

ho

urs

[h

rs]


PBar Emittance at ExtractionPBar Emittance at Extraction

0

2

4

6

8

10

12

14

16

0 50 100 150 200 250

Number of Antiprotons removed [E10]

Em

itta

nce

at

Ext

ract

ion

[95

% p

i mm

mr]

# 61 pbar H RR extracted emitt (pi-mm-mrad) # 62 pbar H Acc core emitt (pi-mm-mrad)

Accumulator

Recycler


PBar Emittance at ExtractionPBar Emittance at Extraction

Model generated Emittances

Real Emittances from Recycler

Number of Antiprotons Removed [E10]

Em

itta

nce


Luminosity Decay FitsLuminosity Decay Fitshttp://mccrory.fnal.gov/tevatronDecayFits

Three types of fits e(-t/tau) over first 2 hours e(-t/tau(t)) like in the model 1/t

EXTRA


Fit results for Stores 4332Fit results for Stores 4332

τ(t)L(t) = L(0) exp(-t/τ(t))τ(t) = τ(0) + c1 × t c2

Fourth best initial luminositySecond Place for Integrated Luminosity

EXTRA


Fit results for Stores 4332 & Fit results for Stores 4332 & 44314431

World record Initial Luminosity, 1.43E32Third Place for Integrated Luminosity

L(0) Hours IntegTau(0

)C1 C2

4332 1.28 30.8 5329. 4.39 1.27 0.541

4431 1.42 33.3 5265. 2.64 2.22 0.495

EXTRA


Antiproton Burn RateAntiproton Burn Rate

Calculated numerically using fitted results Removes data noise Cut: χ2/DOF < 30

• (error bars fixed: 0.05E9)

Luminosity Burn Rate [Rlum(t)] dN(A) / dt [E9 particles/hour] = −0.252 × (LCDF + LD0) [E30/cm2sec]

CDF & D0 Luminosities taken from SDA, Assumptions:

• Emittances, tunes, orbits, etc. are bunch independent (?!)

See Beamdocs # 1408 http://beamdocs.fnal.gov

EXTRA


Burn

Rate

[E9 p

bars

/hour]

Non-Luminosity Burn RateLuminosity Burn

RateTotal Burn Rate

Summary: 9/36 Bunches in Store Summary: 9/36 Bunches in Store 37443744

Hours into the Store

EXTRA


0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1800 2300 2800 3300 3800 4300

Store Number

Klu

dg

e F

acto

r to

get

CD

F L

um

ino

sity

Uncertainty at the IPUncertainty at the IP

Luminosity / (all known factors)

β* = 28 cm

Better emittance measurements Better lattice understanding Better instrumentation

EXTRA


= 0.975 / hour

Tevatron Failure RateTevatron Failure Rate

f(t) = e - t

σ = < t > = 1/

Time Between Tevatron Failures; Real Data

R ≈ 1 - ΔtΔt = 42 hours

e - t

Model data for Tevatron Failures


Failure Rate: InterpretationFailure Rate: Interpretation is Tevatron “Up Time” is measured directly from real data

< t > = σ = 1/ Probability of having stores of:

1 hour: 0.9752 hours: (0.975)2 = 0.95110 hours: (0.975)10 = 0.77620 hours: 0.60330 hours: 0.459

Failures are Independent of Time This is a random process!!


Reliability of Tevatron TodayReliability of Tevatron Today Tevatron is two machines

Low beta: Higher reliability ~ 0.988

– 20 hours: 0.785– 30 hours: 0.696

Injection and ramping: Lower reliability ~ 0.88 to 0.95

Recovery time Severe for superconducting machine

Classes of failures?Beyond the scope of this talk!

∴ Longer stores Tevatron is more reliable in collisions

Model Details Model Details and Predictionsand Predictions


State MachineryState Machinery All machines are implemented as Finite

State Machines Vary in complexity

Proton source: 5 states• Ready, Down, Sick, Studies, Access

Accumulator/Debuncher: 7 states• ReadyStacking, ReadyShot, ReadyRecTransfer, Down,

Recovery, Sick, Studies Tevatron: 17 states

• Ready, 7 shot-setup, 4 luminosity, Failure, Studies, Access, Recovery, Turn-Around

Recycler: 12 states• Ready, 4 transfers (2 in, 2 out), 2 down, recovery, 2

studies, access, cooling, turn-around.


Program StructureProgram Structure C++/Linux

800 weeks/minute• On 1.8 GHz Celeron

220+ parameters How does this work?

Step size = 0.1 hours “Listeners” provide connections among State Machines Main program guides time progression & venue for main

decisions• Stack

– Do transfer to Recycler?• “End-store” criterion satisfied? Start shot setup.

Repeat for N weeks, dumping lots of relevant data. Input parameters

Over200 input parameters to a model run Output handler

Lots of data files can be dumped


Random NumbersRandom NumbersR

an

dom

Like

ly(-

2,

12

, 8

)

Product of these two distributions

RandomLikely(0, 5, 2)

Linux drand48( )“RandomLikely”


DecisionsDecisions Same as reality Store

When to end the storeWhen to begin a store after a failure

• Answer: Wait for accumulation of antiprotons

AntiprotonsWhen and how much to transfer from

Accumulator to Recycler

CombinationHow many antiprotons to get from two

sources• Recycler only, Accumulator only, Combined Source


Some End-Store CriteriaSome End-Store Criteria Store Duration Integrated Luminosity to experiments Number of Antiprotons we have available How low L can the experiments use Best: Combination of last two:

Np expected luminosity R = Expected Luminosity / Actual Luminosity This criterion works very well algorithmically, but there are other

considerations in Real Life• Nowadays, the Run Coordinator ends a store based on this factor and many other

factors, e.g., time of day.

If Model is believable Can change the performance See how the End-Store criteria respond Find the Best criterion for ending stores for lots of

parameters


End-Store CriterionEnd-Store Criterion How to decide which is the “Best”

criterion? It integrates lots of luminosity It insensitive to natural fluctuations in

parameters• Some of these changes may be unnoticed• Random fluctuations or improvements?!

It is simple• Everyone can understand it!

– Some effective but complex schema have been rejected

Look at two end-store criteria Integrated LuminosityRatio

• Of Expected luminosity from available antiprotons to the luminosity now


Integrated Luminosity Integrated Luminosity CriterionCriterion

Num

ber

of

Anti

pro

tons

[E10

]O

r Lu

min

osi

ty [

E30/c

m2/s

ec]

Hours from start of simulated “Run”


Optimization of “Integ Lum”Optimization of “Integ Lum”

End store when Integrated Luminosity reaches this value [nb-1]

Avera

ge Inte

gra

ted L

um

inosi

ty f

or

the W

eek

[nb

-1]


Store Duration: Integ LumStore Duration: Integ Lum

Stop at Integ=3000 nb-1

4000 nb-1

5000 nb-1

6000 nb-1

7000 nb-1

Duration of stores ended intentionally [hours]


Target Ratio CriterionTarget Ratio CriterionN

um

ber

of

Anti

pro

tons

[E10

]O

r Lu

min

osi

ty [

E30/c

m2/s

ec]



Ratio: End at R>6; R(t)Ratio: End at R>6; R(t)



Ratio Criterion: OptimizationRatio Criterion: Optimization

End store when Ratio reaches this value

Avera

ge Inte

gra

ted L

um

inosi

ty f

or

the W

eek

[nb

-1]

22700 for integ


Ratio: Store DurationRatio: Store Duration

Store Duration [hours]

dN

/dt

[sto

res/

1 h

our

bin

]

End store when Ratio=2

4

6

810


Decisions Involving RecyclerDecisions Involving Recycler More decisions with Recycler

When to shoot from Accumulator to Recycler?• How much to take?

Whence do we get pbars for Tevatron?

Long story short … Shoot to Recycler when stack reaches 40 to 80 E10 Get pbars mostly (all?) from Recycler

• Presently, want to get all pbars from Recycler• But luminosity lifetime may be diminished because of

brighter pbars– Does integrated luminosity suffer???

• “use it or lose it”– Ignores antiprotons in Accumulator


Ongoing Studies on Optimum Ongoing Studies on Optimum RecyclerRecycler

Some crucial dependenciesTime required to transfer into Recycler

• Currently, 0.75 to 2 hours, most likely=1 hour.

• Plan: 15 minutes or lessTransmission efficiency

• Now ~90%Emittance from Recycler is ~4π less

than Accumulator• Improved L (0), but diminished initial

lifetime; – ∫L dt ?


# Pbars Avail vs. Transfer # Pbars Avail vs. Transfer TimeTime

Analytic calculation by D. McGinnis,BeamDocs # 1948

Predictions for Predictions for Future Future

PerformancePerformance


Predictions on Future Predictions on Future PerformancePerformance

Recycler improvements It may be able to hold 6E12 antiprotonsTransfers from Accumulator should

eventually take a minute or lessRecycling??????

• Collecting spent antiprotons from the Tevatron and re-cool them with Electron Cooling

Accumulator improvementsToday’s goal: 24E10 antiprotons per hour


Accum: 24E10/hr; Recy:6E12Accum: 24E10/hr; Recy:6E12N

um

ber

of

Anti

pro

tons

[E10

]O

r Lu

min

osi

ty [

E30/c

m2/s

ec]


Stack Size

Stash Size

Luminosity


Optimization of 24 mA/hr: 6E12Optimization of 24 mA/hr: 6E12A

vera

ge Inte

gra

ted L

um

/week

[1/n

b]

End-of-store Ratio

Best today

Future Performance

Summary & Summary & ConclusionsConclusions


ConclusionsConclusions This Operations Model has helped

us understand how to operate the Complex

SDA has been a crucial element to understanding the Tevatron and making this model workThe clock abstraction created by SDA has

been key


Lessons for LHC?Lessons for LHC?

Reliable, redundant, easily accessible performance data are crucial to understanding how you are operatingShot/Case/Set clock

Complexity of LHC loading may be better to model

Reliability matters (duh!)

Fin!Fin!A Monte Carlo Model of A Monte Carlo Model of

Tevatron OperationsTevatron Operations

Elliott McCroryFermilab/Accelerator Division

13 October 2005

A Monte Carlo Model of Tevatron Operations

Documents

Transcript of A Monte Carlo Model of Tevatron Operations