BayesianvsFrequentist

Xia,Ziqing (PurpleMountainObservatory)Duan,Kaikai (PurpleMontainObservatory)Centelles Chuliá,Salvador(Ific,valencia)

Srivastava,Rahul(Ific,Valencia)

Taken from xkcd

DifferentStatisticalQuestions

TheFrequentist likelihoodandtheBayesianposteriorasktwodifferentstatisticalquestionsofthedata:

Regionsofhighqualityoffit

Giventhepriorandthedata,extractthepdfoftheparameters.

Thenotionofprobability- Frequentist

• If thenumberoftrialsapproachesinfinity, therelativefrequencywillconverge exactly tothetrueprobability.

• Repeatabilityofanexperimentisthekeyconcept.

The number of trials where the event X occurred

The total number of trials

• TheMaximumLikelihoodEstimator:

Thenotionofprobability- Bayesian

Posterior

Marginal likelihood

PriorLikelihood function

It is a direct consequence of the Bayes theorem :

• Bayestheoremrelatestheposteriorprobability(whatweknowabouttheparameterafterseeingthedata)tothelikelihood (derivedfroma statisticalmodel fortheobserveddata) andtheprior (whatweknewabouttheparameterbeforewesawthedata).

• A generalruletoupdateourknowledgefromtheprior tothe posterior.

A SimpleProblem:PhotonCounts• Imaginethatweobservethelightcomingfromasinglestar.We assumethatthestar’struefluxisconstantwithtime.

• Giventhe measurementsanderrors,whatisthebestestimateofthetrueflux?

• we usePythontogeneratesometoydata ( Poissondistribution)

Frequentist ApproachtoSimplePhotonCounts• Theprobabilitydistributionofthemeasurement：

• constructthe likelihoodfunctionbycomputingtheproductoftheprobabilitiesforeachdatapoint:

• Thebestestimate：

BayesianApproachtoSimplePhotonCounts• Theposteriorprobability：

• Themodelprior: a standardchoiceistotakeauniformprior.

• TheBayesianprobabilityismaximizedatpreciselythesamevalueasthefrequentist result! InthecaseofaGaussian

likelihoodanduniformprior,theposteriorpdfandtheprofilelikelihoodareidenticalandthusthe

questionofwhichtochoosedoesnotarise.

BayesianApproachtoSimplePhotonCounts

• Theposteriorprobability：

TheBayesianBilliardGame

• AliceandBobcan’tseethebilliardtable.• Carolrollsaballdownthetable,andmarkswhereitlands.Oncethismarkisinplace,Carolbeginsrollingnewballsdownthetable.

• Iftheballlandstotheleftofthemark,Alicegetsapoint;ifitlandstotherightofthemark,Bobgetsapoint.

• Thefirstpersontoreach sixpointswinsthegame.• NowsaythatAliceisleadingwith5pointsandBobhas3points.WhatcanbesaidaboutthechancesofBobtowinthegame?

Frequentist (naive) ApproachtoTheBilliardGame• FiveballsoutofeightfellonAlice'ssideofthemarker• Maximumlikelihoodestimateof pthatanygivenrolllandsinAlice'sfavor.

• Assumingthismaximumlikelihoodprobability,wecancomputetheprobabilitythatBobwillwin,whichisgivenby:

Frequentist ProbabilityofBobWinning:0.05

BayesianApproachtoTheBilliardGame

• Theposteriorprobability：

PosteriorProbabilityofBobWinning:0.09?!

UniformPrior

MonteCarloApproachtoTheBilliardGame

• UseaMonteCarlosimulationtodeterminethecorrectanswer.

The correct ProbabilityofBobWinning:0.09!Bayesianwin!!!

Conclusions

• The Bayesian vs Frequentist debate has a long history and the battleis still going on

• People oftenmake statements claiming one to be better than other• In our opinion, it is not accurate to say that one is inherently superiorto other

• Both approaches can give poor results if not done carefully• However, frompractical point of view, Bayesian seems to bettersuited to handle “dirty” data

• Situation similar to classical numerical integration techniques vsMonte Carlo integration techniques

Open questions

• How to knowwhen to use one approach or the other?• What defines a good prior?• Is there another approach (the third one) can resolve this case?

References：[1] RobertoTrotta: BayesianMethodsinCosmology. arXiv: 1701.01467[2]JakeVanderPlas: Frequentism andBayesianism:APython-drivenPrimer.

arXiv: 1411.5018

Thankyou!