Report from the Statistics Forum
description
Transcript of Report from the Statistics Forum
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 1
Report from the Statistics Forum
ATLAS Analysis Strategies Workshop
CERN, 6 September, 2007
Glen CowanPhysics DepartmentRoyal Holloway, University of [email protected]/~cowan
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 2
The ATLAS Statistics Forum
Initiated at StatisticsWorkshop in January 07
Statistics Forum startedin March (Karl Jakobs)
Since then, meetings~monthly
(plus ~3 joint meetings with CMS)
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 3
Current MembershipConveners: Eilam Gross, Glen Cowan
“Statistics Experts”: GDC, Kyle Cranmer, Tom Junk,Bill Murray, Alex Read
Tool developers: Bill Quayle, Krzysztof Ciba
Representatives from ATLAS groups:Higgs: Eilam Gross & Guillaume UnalSUSY: Dan ToveySM: Maarten BoonekampExotics: Samir FerragB-Physics: Eduard BureloTop: Yoram RozenHeavy Ions, ...
Ex officio: Ian Hinchliffe & Karl Jakobs
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 4
Mandate
Define ATLAS Statistics Standards in contact with Physics Coordination and ATLAS groups.
Develop, validate and approve standard statistical toolsin close contact with the software group.
Represent ATLAS in discussions on statistical issues with CMS.
Discuss and make recommendations on statistical issues such as blind analyses, model independent searches, etc.
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 5
Web pagehttps://twiki.cern.ch/twiki/bin/view/Atlas/StatisticsTools
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 6
Hypernewshttps://hypernews.cern.ch/HyperNews/Atlas/get/physics-statistics.html
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 7
cf. BaBar Statistics Group Hypernews
Detaileddiscussions
Heavy traffic
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 8
Some ongoing activities
ATLAS Statistics Book
Higgs combination (different channels & ATLAS + CMS)
Software development (RooStat)
Sequential analysis
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 9
Twiki, i.e., a Living Document (thanks, Krzysztof) Core authors assigned, detailed outline discussed+ worked examples, particularly related to systematics+ FAQ, recommendationsGoal: first draft by October (need to work hard...)
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 10
Higgs CombinationHiggs suggested as a first major exercise for the group, but we would like to undertake similar efforts with other Physics Groups.
Many people involved; see next talk by Eilam.In particular, frequentist combination based on profilelikelihood very advanced (Kyle, Eilam, Ofer, ...)
But a brief comment on the Bayesian methods:Bayesian limits already widely used e.g. in CDF (→Tom J.).We are also investigating use of Bayes factors as a means of quantifying significance of a discovery:
Higgs
no Higgs Bayes factor B10prior odds
posterior odds
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 11
Higgs combination based on event countsN independent channels, countni events in search regions:
Expected number of signal events:( is global parameter, = 1 for SM).
Consider a fixed Higgs mass and assume SM branching ratios Bi.
Suggested method: constrain with limit up; consider mH excluded if upper limit up < 1.0.
For discovery, compute Bayes factor for H0 : = 0 vs. H1 : = 1
Constrain expected backgroundbi with sideband measurements:
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 12
Bayes factor ( = ratio of integrated likelihoods)
Here B10 and profile likelihood ratio close; comparisons ongoing.
B1
0
B10 at = 1 gives changein odds of SM Higgs vs. no Higgs in light of data,
i.e., measures of weight of evidence provided by datain support of one hypothesisover another.
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 13
Statistical softwareBill Quayle, Kyle Cranmer (from Statistics Forum) + developers of
TMVA (Andreas Hoecker et al.),
RooFit (Wouter Verkerke et al.),
MadTools/StatTools (Wisconsin group), ...
(see e.g. Bill Quayle’s 15 March 07 talk)
Some issues:
ATLAS specific (private) or HEP-wide (public)?
Use FrameworkXXX or, integrate ComponentXXX into anexisting framework?
Distribution (phystat.org, bundle with ROOT?)
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 14
RooFit/RooStats
Wouter Verkerke, Kyle Cranmer
Developments reported in ATLAS/CMS Forum, i.e., HEP-wide,not ATLAS-specific software.
E.g. Combination Toy Model (W. Verkerke -- 17 July 2007 ATLAS/CMS meeting)
‘Atlas’
Combined
Prof
ile li
kelih
ood
‘CMS’→ talk by Wouter
15
Looking at data as they are recorded and compute a new p-value or likelihood ratio test every 10 pb-1.
What is probability that p-value (or LR test) < 0.01 versus number of data looks ?
• looking 1 time at data : p(p-value < 0.01|H0) = 0.01• looking sequentially 100 times at data : p(at least 1 p-value < 0.01|H0) ~ 0.08 !
p-value
Likelihood ratio test
b = 200 fb
p-Value result versus L when H0 is true
P(at least 1 p-Value < 0.01) versus L when H0 is true
Sequential analysis (Renaud Bruneliere, 7 June and 3 July 07)
→ ‘Look-elsewhere effect in time’. Need to understand how to best quantify / deal with this!
G. CowanRHUL Physics Analysis Strategies Workshop -- Statistics Forum Report page 16
Conclusions, thoughtsStatistics Forum working since March, several projects ongoing:
Statistics BookHiggs combinationSoftware toolsSequential analysis
Want to increase interaction with Physics Groups,
If your group has a statistics issue/problem, is developing a statistical tool, etc. come tell us.(Also we will come to you...)