Evaluating Data Purpose of Experiments Evaluating Experiments Is to test theory? Let’s do it!!!
-
Upload
simon-shelton -
Category
Documents
-
view
217 -
download
2
Transcript of Evaluating Data Purpose of Experiments Evaluating Experiments Is to test theory? Let’s do it!!!
Evaluating DataPurpose of Experiments
Evaluating ExperimentsIs to test theory?
Let’s do it!!!
Evaluating DataPurpose of Experiments
Evaluating Experiments
U s e e x p e r i m e n t a l u n c e r t a i n t i e s a n d t h e p r o p e r t i e s o f p r o b a b i l i t y d i s t r i b u t i o n s t o e v a l u a t e t h e e x t e n t t o w h i c h d a t a s u p p o r t s o r r e f u t e s a p h y s ic a l m o d e l .
2006 LEP Constraints
Or is to win Nobel Prizes forTheorists?
July 4 2012 discovery announced
Evaluating Data
Purpose of Experiments
Evaluating Data
Purpose of ExperimentsPeter Higgs2013 Nobel Prize Winner
Austin Ball –Technical Director of CMS (He was in charge of building the CMS experiment)
Pu r p o s e o f Ex p e r im e n ts● Pur p o s e o f e x p e r im e n t :
t e s t e x is t in g m o d e l
“ M o d e l ” ( W e b s te r ) – a s y s t e m o f p o s tu la t e s , d a t a , a n d in f e r e n c e s p re se n t e d a s a m a t h e m a t ic a l d e sc r ip t io n o f a n e n t i t y o r s ta t e o f a f f a i r s
i.e. a f u n c t io n a l r e la t io n s h ip b e t w e e n v a r ia b le s
●
●
5
Standard Model of Particle Physics
https://en.wikipedia.org/wiki/Standard_Model_%28mathematical_formulation%29
CERN Experiments
http://home.web.cern.ch/about/experiments
U.S. HEP Experiments
http://www-sld.slac.stanford.edu/sldwww/sld.html
http://www-cdf.fnal.gov/about/index.html
Standard Model of Particle Physics
Electron-positron cross-section and Z-boson Lineshape
http://rsta.royalsocietypublishing.org/content/370/1961/805
W h a t c a n e x p e r im e n ts a c h i e v e?
● A n e x p e r im e n t c a n n o t p r o v e a m o d e l t o b e t r u e .
A n e x p e r im e n t c a n :– D e m o n s t r a te th a t a
m o d e l is f a ls e w i t h s o m e p r o b a b i l i t y
– Sh o w th a t a m o d e l is v a l id w ith in so m e p re c is ion
●
W h a t c a n e x p e r im e n ts a c h i e v e?
● A n e x p e r im e n t im p r o v e s th e s t r e n g th o f i t s s t a t e m e n ts b y d e c re a s in g u n c e r t a i n t ie s .
H o w d o I q u a n t i f y t h e e x t e n t to w h i c h th e d a t a to t h e r i g h t is in c o n s is t e n t w i t h O h m ' s l a w ?
●
13
E xp e r i m e n ta l E va lu a t i o n
● D e t e r m i n e s ig n i f i c a n c e o f y o u r m e a su r e m e n t .
W h a t d o e s t h e e x p e r im e n t sa y a b o u t t h e m o d e l y o u ' r e t e s t i n g ?
– D o e s i t su p p o r t o r r e f u te th e m o d e l?– Ca n w e q u a n t i f y o u r c o n f id e n c e in t h e
m o d e l?
●
G a u s s ia n D i s t r i b u t i o n
14
G a u s s ia n D i s t r i b u t i o n
A “ 1 s i g m a ” e r r o r in te r v a l is n o t a l l - in c lu s i v e !
15
9
U s e “ 2 /3 R u le ” a s a R o u g h G u id e
● Is d a t a c o n s is t e n t w i t h a Y a n d X h a v i n g a l in e a r r e l a t io n s h ip ?
10
U s e “ 2 /3 R u le ” a s a R o u g h G u id e
● Sp r e a d o f p o i n t s , e r r o r b a rs c o n s is t e n t w i t h n o r m a l e r r o r d is t r ib u t io n .
11
U s e “ 2 /3 R u le ” a s a R o u g h G u id e
● W h a t is t h e p r o b l e m h e re ?
12
U s e “ 2 /3 R u l e ” a s a R o u g h G u i d e
● W h a t is t h e p r o b l e m h e re ?
Ev a lu a t in g M o d e ls : 2
● D e f in i t i o n o f 2
● 2 h a s m e a n in g r e la t i v e t o t h e n u m b e r o f d e g r e e s o f f r e e d o m :
# D O F = ( # d a ta p o in t s ) -( # p a ra m e t e rs d e te r m in e d f ro m d a ta )
Ro u g h ly , 2 / # D O F ~ 1 f o r g o o d d a ta /m o d e l m a t c h
●
●
20
Ev a lu a t in g M o d e ls : 2
● D e f in i t i o n o f 2
● 2 h a s m e a n in g r e la t i v e t o t h e n u m b e r o f d e g r e e so f f r e e d o m :
# D O F = ( # d a ta p o in t s ) -( # p a ra m e t e rs d e te r m in e d f ro m d a ta )
Ro u g h ly , 2 / # D O F ~ 1 f o r g o o d d a ta /m o d e l m a t c h
●
●
“CONSTRAINTS”
21
15
2 is a m o r e q u a n t i t a t i v e m e t h o d :
16
2 is a m o r e q u a n t i t a t i v e m e t h o d :
What is #DOF?
17
2 is a m o r e q u a n t i t a t i v e m e t h o d :
● W h a t is t h e p r o b l e m h e re ?
18
2 is a m o r e q u a n t i t a t i v e m e t h o d :
● W h a t is t h e p r o b l e m h e re ?
Co u n t i n g D e g r e e s o f F r e e d o m● # D e g r e e s o f f r e e d o m i n
th is s t r a ig h t l in e f i t s h o u l d b e e a s y → w h a t is i t? ?
W h a t i f in te r c e p t w e r e a ls o to b e d e t e r m in e d f r o m th e d a t a ?
H o w m a n y c o n s t r a in ts i f I ' m f i t t in g a G a u ss ia n in a c o u n t in g e x p e r im e n t?
. . . i f I ' m f i t t i n g a G a u ss ia n in a n o n -c o u n t in g e x p e r im e n t?
●
●
●
27
p -v a l u e o r “ c h a n c e p r o b a b i l i t y ” : Th e p r o b a b i l i t y o f o b t a i n i n g a r e s u l t a t l e a s t a s e x t r e m e a s a g i v e n d a ta p o in t , a s su m i n g t h e d a t a p o i n t w a s t h e r e s u l t o f c h a n c e a lo n e
● A s t a t e m e n t a b o u t t h e p r o b a b i l i t y t h a t a n “ e f f e c t ” is r e a l ly a s t a t is t i c a l f lu c t u a t io n .
Re p l a c e s d is c re t e T r u e /Fa ls e (o r 1 /0 ) c h o ice w i t h a c o n t i n u u m r a n g i n g f r o m “ l ik e l y t r u e ” t o “ l ik e l y f a l s e ” .
T r u t h → “ Tr u t h in e s s ”
●
●
28
Ex a m p le : Su p p o s e t h e re a re 3 0 , 0 0 0 U n iv e r s it y o f U ta h s tu d e n t s , o f w h ic h 4 0 0 a re p e r m i t t e d t o c a r r y g u n s . If I 'm t e a c h in g a n a s t r o n o m y c la ss o f 1 2 0 s tu d e n t s , w h a t is t h e p ro b a b i l i t y t h a t o n e o r m o re is c a r r y in g a g u n ?
A n s w e r : T h e p r o b a b i l i t y d is t r i b u t i o n f o r t h e n u m b e r o f s tu d e n t s w it h w e a p o n s is
a p p r o x im a t e ly P o iss o n ia n , w it h a m e a n v a lu e o f ( 4 0 0 /3 0 , 0 0 0 ) x 1 2 0 = 1 .6
Th e p ro b a b i l i t y o f o b se r v in g o n e o r m o re in a P o iss o n d is t r ib u t i o n w it h m e a n 1 .6 is :
P ( > = 1 , 1 .6 ) = 1 – P ( 0 ,1 .6 ) = 1 – 0 .2 0 1 = 0 .7 9 9
P P
22
Fol lo w u p : Su p p o s e a m e t a l d e t e c t o r r e v e a ls t h a t th e r e a re 6 p e o p le in John B e l z ' s c la ss c a r r y in g g u n s. W h a t is th e p -v a lu e o f t h is o b se rv a t io n ? Sh o u ld JB b e w o r r ie d t h a t th e y ' re o u t to g e t h im ?
23
Fo l lo w u p : Su p p o s e a m e t a l d e t e c t o r r e v e a ls t h a t t h e r e a re 6 p e o p le in JB ' s c la ss c a r r y in g g u n s. W h a t is t h e p -v a lu e o f t h is o b se r v a t i o n ? Sh o u ld JB b e w o r r ie d t h a t t h e y ' re o u t to g e t h im ?
● A n s w e r : Th e p -v a l u e i s t h e s u m o f p r o b a b i l i t i e s f o r n = 6 , 7 , 8 , 9 . . .
● P-v a lu e P ( > = 6 , 1 . 6 )P
= 0 .0 0 6● Th is is a v e r y s m a l l
c h a n c e p ro b a b i l i t y . Per h a p s JB s h o u ld w e a r a b u l le t p r o o f v e s t !
31
Example: I f l ip a c o in 2 0 t im e s , 1 4 t im e s i t c o m e s u p h e a d s . U s e t h is d a ta a s a t e s t o f t h e h y p o th e s is t h a t t h e c o in is f a i r.
32
Example: I f l ip a c o in 2 0 t im e s , 1 4 t im e s i t c o m e s u p h e a d s . U s e t h is d a ta a s a t e s t o f t h e h y p o th e s is t h a t t h e c o in is f a i r.
p = ( B in o m ia l P r ob a b i l i t y o f > = 1 4 h e a d s ) + ( B i n o m ia l P r ob a b i l i t y o f < = 6 h e a d s )
= 0 .1 1 5
33
Example: I f l ip a c o in 2 0 t im e s , 1 4 t im e s i t c o m e s u p h e a d s . U s e t h is d a ta a s a t e s t o f t h e h y p o th e s is t h a t t h e c o in is f a i r.
p = ( B in o m ia l P r ob a b i l i t y o f > = 1 4 h e a d s ) + ( B i n o m ia l P r ob a b i l i t y o f < = 6 h e a d s )
= 0 .1 1 5
W h ile t h e “ f a ir c o in ” h y p o t h e s is d o e s n o t h a v e a h ig h le v e l o f t r u t h in e ss ( p -v a lu e ) , i t s n o t so lo w t h a t w e c a n sa y f o r su r e t h a t t h e c o in isn ' t f a ir.
Th e 2 a n d n u m b e r o f d e g r e e s o f f r e e d o m
34
c a n b e d i r e c t ly t r a n s la t e d in t o a p -v a l u e :
So u r c e : p d g . lb l . g o v
http://pdg.lbl.gov/2014/reviews/rpp2014-rev-statistics.pdf
http://pdg.lbl.gov/2014/reviews/contents_sports.html
Ta b le f r o m p a g e 2 9 3 o f Ta y lo r
35
Ex a m p le : Taylor 1 2 . 0 4I t h ro w t h r e e d ic e t o g e t h e r a to t a l o f 4 0 0 t im e s , re c o r d t h e n u m b e r o f s ix e s in e a c h t h r o w , a n d o b ta in t h e r e su l t s sh o w n b e lo w . U se t h e b in o m ia l d is t r i b u t io n to f in d t h e e x p e c t e dn u m b e r E f o r e a c h o f t h e t h r e e b in s a n d t h e n c a lc u la te 2 .
k
D o I h a v e re a so n t o s u s p e c t t h e d ic e a r e lo a d e d ?
36
Taylor 1 2 . 0 4
● N = 3 t r ia ls
= 0 , 1 , 2 , 3
su c c e sse s p = 1 /6
●
●
37
Taylor 1 2 . 0 4
38
Taylor 1 2 . 0 4
W h a t is t h e n u m b e r o f d e g r e e s o f f r e e d o m ?39
Taylor 1 2 . 0 4
40
●
2 / d o f =
4 . 1 /2 p -v a lu e
~ 0 .1 4
“Consistent with a statistical fluctuation at the 10% level”
●
●
A d d i t i o n a l R e a d i n g a n d Pr o b l e m s
● Re a d in Ta y l o r :
41
– Ch 5 : T h e N o r m a l D i s t r i b u t i o n
– Ch a p t e r 1 2 : Th e Ch i - Sq u a r e d Te s t f o r a D i s t r i b u t i o n
● Tr y t h e p r o b l e m s :– 5 .1 1 , 5 .1 2 , 5 .2 0 ,
5 .2 1– 1 2 . 4 , 1 2 . 6 , 1 2 .1 1 ,
1 2 .1 4 , 1 2 .1 6
CERN Root Analysis Software Tutorials
• Start RDP session on orion, draco or Cygnus• Open an x-terminal window and issue the following commands
• The following should spew forth….
• Then run the tutorials benchmark by typing the following command
• And now your odyssey begins
CERN Root Tutorials
CERN Root Tutorials
CERN Root Tutorials
CERN Root Tutorials
CERN Root Tutorials
CERN Root Tutorials
CERN Root Tutorials
CERN Root Tutorials• Now let’s play….• From the root command line issue the following command
• The following should spew forth….
• Open another xterm window• Copy and edit the “program” (Macro) tutorials/fit/fit1.C to
~/phys6719/root/sandbox/myfit1.C by typing the following commands.
• Modify the program …
Exercises
1. Use the examples from the tutorials to write a CERN root macro to generate a Histogram containing events sampled from the sum of an exponential probability distribution.
2. Fit this histogram with an exponential function. Verify that the fit returns parameters consistent with the probability distribution that you used to generate your histogram. Report the c2 /D.O.F. goodness of fit parameter.
3. Repeat exercises 1 and 2 for a Gaussian and a Poisson Distribution.4. Document your results including graphs and written descriptions in
a libreoffice document.