DR ADRIAN BEVAN MULTIVARIATE ANALYSIS AND …gae/Documents/Adrian1.pdfMULTIVARIATE ANALYSIS AND ITS...

MULTIVARIATE ANALYSIS AND ITS USE IN HIGH ENERGY PHYSICS 1) CUT BASED ANALYSES AND LINEAR DISCRIMINANTS

DR ADRIAN BEVAN

�1

Lectures given at the department of Physics at CINVESTAV, Instituto Politécnico Nacional, Mexico City 28th August - 3rd Sept 2018

A. Bevan

MULTIVARIATE ANALYSIS AND ITS USE IN HIGH ENERGY PHYSICS: CUT BASED ANALYSES AND LINEAR DISCRIMINANTS

LECTURE PLAN▸ These lectures can be found online at:

▸ https://pprc.qmul.ac.uk/~bevan/teaching/MLinHEP.html

▸ My teaching notes (linked from the above) include other courses on machine learning that may be of interest.

�2

https://pprc.qmul.ac.uk/~bevan/teaching/MLinHEP.html

A. Bevan


LECTURE PLAN▸ Introduction

▸ Cut based analyses

▸ Figures of merit for optimisation

▸ Linear discriminants

▸ Fisher’s linear discriminant

▸ Examples

▸ Summary

▸ Suggested reading

�3

A. Bevan


INTRODUCTION▸ In High Energy Physics we analyse data for many different

reasons: ▸ Event selection at the trigger level; ▸ Calibration of systems; ▸ Particle identification; ▸ Object reconstruction; ▸ Offline event selection for analysis (cut based analysis); ▸ Background suppression (candidate selection); ▸ As a part of a cut-based selection; ▸ As an input to a fit; either as one discriminating variable, or

as the single discriminating variable. ▸ Multivariate analysis (MVA) is a tool that helps us in all of these

areas.

�4

A. Bevan


INTRODUCTION▸ These lectures cover a number of different topics: ▸ Cut based analysis and linear discriminants ▸ Decision Trees ▸ Neural Networks ▸ Optimisation ▸ Deep Learning ▸ Support Vector Machines

▸ All of the topics are used in the field; the most common techniques are cut based analysis and the use of decision trees; but modern deep learning methods are gaining popularity.

▸ The concepts that underpin optimisation are also relevant for fit based approaches; although some of the methods used are tailored to machine learning (ML) methods.

�5

A. Bevan


INTRODUCTION▸ These lectures will not cover coding examples, although if

you are interested in coding examples there are several tools we can discuss outside of the lectures.

▸ The TMVA toolkit in ROOT has good documentation and would be a natural place to start working with ML in HEP as it has an interface to data via ROOT objects.

▸ The following set of lecture notes includes examples in TensorFlow, and includes a HEP analysis problem (the Kaggle Higgs challenge data). ▸ https://pprc.qmul.ac.uk/~bevan/teaching/PML.html

�6

https://pprc.qmul.ac.uk/~bevan/teaching/PML.html

A. Bevan


CUT BASED ANALYSES▸ A typical analysis workflow is schematically represented below:

▸ Cut-based selection (sometimes called a cut and count analysis) is the method of applying a set of cuts (usually rectangular) to the data in order to determine how many events of different species remain.

▸ The number of events left after applying cuts is then converted into a more interesting quantity such as a branching fraction, limit, or cross section.

▸ How do we determine the cuts?

�7

RAW DATA PRODUCED BY THE

DETECTOR

SEVERAL LEVELS OF TRIGGERING (ONLINE EVENT

SELECTION)

OFFLINE FILTERING TO DIVIDE DATA

INTO MORE MANAGEABLE

SETS

CUT BASED EVENT SELECTION TO

ENHANCE SIGNAL RELATIVE TO

BACKGROUND

DETAILED ANALYSIS LEADING TO FINAL RESULT

A. Bevan


CUT BASED ANALYSES: FIGURES OF MERIT FOR OPTIMISATION▸ Formally when we apply some cut we are imposing a threshold defined by some

condition. ▸ If the condition is satisfied then we retain the example for subsequent use. ▸ If the condition is not satisfied the example is discarded.

▸ The condition can be thought of as some decision boundary: ▸ this description will play an important role when we discuss ML algorithms.

▸ This is equivalent to applying a heavyside step function .

�8

H(x) =1

2(1 + sign(x))

<latexit sha1_base64="GlwR9ZhdZX7lU+r3dQDjwhPhih0=">AAACBnicbVDLSgMxFM3UV62vUZciBIvQIpRJEXQjFN10WcE+oB1KJs20oZnMkGTEMszKjb/ixoUibv0Gd/6NaTsLbT1w4XDOvdx7jxdxprTjfFu5ldW19Y38ZmFre2d3z94/aKkwloQ2SchD2fGwopwJ2tRMc9qJJMWBx2nbG99M/fY9lYqF4k5PIuoGeCiYzwjWRurbx/XSQxlewZ4vMUlQmlTTEjpTbCiMXu7bRafizACXCcpIEWRo9O2v3iAkcUCFJhwr1UVOpN0ES80Ip2mhFysaYTLGQ9o1VOCAKjeZvZHCU6MMoB9KU0LDmfp7IsGBUpPAM50B1iO16E3F/7xurP1LN2EiijUVZL7IjznUIZxmAgdMUqL5xBBMJDO3QjLCJhBtkiuYENDiy8ukVa0gp4Juz4u16yyOPDgCJ6AEELgANVAHDdAEBDyCZ/AK3qwn68V6tz7mrTkrmzkEf2B9/gDWeJbN</latexit><latexit sha1_base64="GlwR9ZhdZX7lU+r3dQDjwhPhih0=">AAACBnicbVDLSgMxFM3UV62vUZciBIvQIpRJEXQjFN10WcE+oB1KJs20oZnMkGTEMszKjb/ixoUibv0Gd/6NaTsLbT1w4XDOvdx7jxdxprTjfFu5ldW19Y38ZmFre2d3z94/aKkwloQ2SchD2fGwopwJ2tRMc9qJJMWBx2nbG99M/fY9lYqF4k5PIuoGeCiYzwjWRurbx/XSQxlewZ4vMUlQmlTTEjpTbCiMXu7bRafizACXCcpIEWRo9O2v3iAkcUCFJhwr1UVOpN0ES80Ip2mhFysaYTLGQ9o1VOCAKjeZvZHCU6MMoB9KU0LDmfp7IsGBUpPAM50B1iO16E3F/7xurP1LN2EiijUVZL7IjznUIZxmAgdMUqL5xBBMJDO3QjLCJhBtkiuYENDiy8ukVa0gp4Juz4u16yyOPDgCJ6AEELgANVAHDdAEBDyCZ/AK3qwn68V6tz7mrTkrmzkEf2B9/gDWeJbN</latexit><latexit sha1_base64="GlwR9ZhdZX7lU+r3dQDjwhPhih0=">AAACBnicbVDLSgMxFM3UV62vUZciBIvQIpRJEXQjFN10WcE+oB1KJs20oZnMkGTEMszKjb/ixoUibv0Gd/6NaTsLbT1w4XDOvdx7jxdxprTjfFu5ldW19Y38ZmFre2d3z94/aKkwloQ2SchD2fGwopwJ2tRMc9qJJMWBx2nbG99M/fY9lYqF4k5PIuoGeCiYzwjWRurbx/XSQxlewZ4vMUlQmlTTEjpTbCiMXu7bRafizACXCcpIEWRo9O2v3iAkcUCFJhwr1UVOpN0ES80Ip2mhFysaYTLGQ9o1VOCAKjeZvZHCU6MMoB9KU0LDmfp7IsGBUpPAM50B1iO16E3F/7xurP1LN2EiijUVZL7IjznUIZxmAgdMUqL5xBBMJDO3QjLCJhBtkiuYENDiy8ukVa0gp4Juz4u16yyOPDgCJ6AEELgANVAHDdAEBDyCZ/AK3qwn68V6tz7mrTkrmzkEf2B9/gDWeJbN</latexit>

x-1 -0.5 0 0.5 1

y(x)

0

0.2

0.4

0.6

0.8

1

Examples pass the cut

Examples fail the cut

We can map x to x—b if we want to place a cut at x=b in order to apply a non-trivial cut on the data.

We can invert the cut by mapping x to —x in the LHS of H(x).

A. Bevan


CUT BASED ANALYSES: FIGURES OF MERIT FOR OPTIMISATION▸ Cut values (cuts) are either determined in an ad-hoc way

driven by external constraints such as minimising resource:

▸ This can done when as a pre-selection step when it is “obvious” that we are throwing away more background than signal; and a final selection optimisation will then be made.

▸ If the preselection is imposed by some other resource driven constraint such as reducing the number of data to analyse in a fit.

▸ Or the cuts are determined by maximising some figure of merit (FOM) motivated by the problem being addressed.

�9

A. Bevan


CUT BASED ANALYSES: FIGURES OF MERIT FOR OPTIMISATION▸ Popular FOMs include:

�10

S/B<latexit sha1_base64="9k2XLA9WFm+VLlkzzxHgyl+l3zw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU01E0GOpF4+VWltoQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fDoUcepYthisYhVJ6AaBZfYMtwI7CQKaRQIbAfj25nffkKleSwfzCRBP6JDyUPOqLFSs3lR75crbtWdg6wSLycVyNHol796g5ilEUrDBNW667mJ8TOqDGcCp6VeqjGhbEyH2LVU0gi1n81PnZIzqwxIGCtb0pC5+nsio5HWkyiwnRE1I73szcT/vG5qwhs/4zJJDUq2WBSmgpiYzP4mA66QGTGxhDLF7a2EjaiizNh0SjYEb/nlVfJ4WfXcqnd/VanV8ziKcAKncA4eXEMN7qABLWAwhGd4hTdHOC/Ou/OxaC04+cwx/IHz+QOkF41c</latexit><latexit sha1_base64="9k2XLA9WFm+VLlkzzxHgyl+l3zw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU01E0GOpF4+VWltoQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fDoUcepYthisYhVJ6AaBZfYMtwI7CQKaRQIbAfj25nffkKleSwfzCRBP6JDyUPOqLFSs3lR75crbtWdg6wSLycVyNHol796g5ilEUrDBNW667mJ8TOqDGcCp6VeqjGhbEyH2LVU0gi1n81PnZIzqwxIGCtb0pC5+nsio5HWkyiwnRE1I73szcT/vG5qwhs/4zJJDUq2WBSmgpiYzP4mA66QGTGxhDLF7a2EjaiizNh0SjYEb/nlVfJ4WfXcqnd/VanV8ziKcAKncA4eXEMN7qABLWAwhGd4hTdHOC/Ou/OxaC04+cwx/IHz+QOkF41c</latexit><latexit sha1_base64="9k2XLA9WFm+VLlkzzxHgyl+l3zw=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU01E0GOpF4+VWltoQ9lsJ+3SzSbsboQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgpr6xubW8Xt0s7u3v5B+fDoUcepYthisYhVJ6AaBZfYMtwI7CQKaRQIbAfj25nffkKleSwfzCRBP6JDyUPOqLFSs3lR75crbtWdg6wSLycVyNHol796g5ilEUrDBNW667mJ8TOqDGcCp6VeqjGhbEyH2LVU0gi1n81PnZIzqwxIGCtb0pC5+nsio5HWkyiwnRE1I73szcT/vG5qwhs/4zJJDUq2WBSmgpiYzP4mA66QGTGxhDLF7a2EjaiizNh0SjYEb/nlVfJ4WfXcqnd/VanV8ziKcAKncA4eXEMN7qABLWAwhGd4hTdHOC/Ou/OxaC04+cwx/IHz+QOkF41c</latexit>

S/pS +B

<latexit sha1_base64="raNWG46W6I1IaGnDmAWCGj4C7lI=">AAAB83icbVDLSgNBEOz1GeMr6tHLYBAEIe6KoMcQLx4jMQ/ILmF2MpsMmX040yuEJb/hxYMiXv0Zb/6Nk2QPmljQUFR1093lJ1JotO1va2V1bX1js7BV3N7Z3dsvHRy2dJwqxpsslrHq+FRzKSLeRIGSdxLFaehL3vZHt1O//cSVFnH0gOOEeyEdRCIQjKKR3MaFqx8VZo3z2qRXKtsVewayTJyclCFHvVf6cvsxS0MeIZNU665jJ+hlVKFgkk+Kbqp5QtmIDnjX0IiGXHvZ7OYJOTVKnwSxMhUhmam/JzIaaj0OfdMZUhzqRW8q/ud1UwxuvExESYo8YvNFQSoJxmQaAOkLxRnKsSGUKWFuJWxIFWVoYiqaEJzFl5dJ67Li2BXn/qpcreVxFOAYTuAMHLiGKtxBHZrAIIFneIU3K7VerHfrY966YuUzR/AH1ucPgEqRUg==</latexit><latexit sha1_base64="raNWG46W6I1IaGnDmAWCGj4C7lI=">AAAB83icbVDLSgNBEOz1GeMr6tHLYBAEIe6KoMcQLx4jMQ/ILmF2MpsMmX040yuEJb/hxYMiXv0Zb/6Nk2QPmljQUFR1093lJ1JotO1va2V1bX1js7BV3N7Z3dsvHRy2dJwqxpsslrHq+FRzKSLeRIGSdxLFaehL3vZHt1O//cSVFnH0gOOEeyEdRCIQjKKR3MaFqx8VZo3z2qRXKtsVewayTJyclCFHvVf6cvsxS0MeIZNU665jJ+hlVKFgkk+Kbqp5QtmIDnjX0IiGXHvZ7OYJOTVKnwSxMhUhmam/JzIaaj0OfdMZUhzqRW8q/ud1UwxuvExESYo8YvNFQSoJxmQaAOkLxRnKsSGUKWFuJWxIFWVoYiqaEJzFl5dJ67Li2BXn/qpcreVxFOAYTuAMHLiGKtxBHZrAIIFneIU3K7VerHfrY966YuUzR/AH1ucPgEqRUg==</latexit><latexit sha1_base64="raNWG46W6I1IaGnDmAWCGj4C7lI=">AAAB83icbVDLSgNBEOz1GeMr6tHLYBAEIe6KoMcQLx4jMQ/ILmF2MpsMmX040yuEJb/hxYMiXv0Zb/6Nk2QPmljQUFR1093lJ1JotO1va2V1bX1js7BV3N7Z3dsvHRy2dJwqxpsslrHq+FRzKSLeRIGSdxLFaehL3vZHt1O//cSVFnH0gOOEeyEdRCIQjKKR3MaFqx8VZo3z2qRXKtsVewayTJyclCFHvVf6cvsxS0MeIZNU665jJ+hlVKFgkk+Kbqp5QtmIDnjX0IiGXHvZ7OYJOTVKnwSxMhUhmam/JzIaaj0OfdMZUhzqRW8q/ud1UwxuvExESYo8YvNFQSoJxmQaAOkLxRnKsSGUKWFuJWxIFWVoYiqaEJzFl5dJ67Li2BXn/qpcreVxFOAYTuAMHLiGKtxBHZrAIIFneIU3K7VerHfrY966YuUzR/AH1ucPgEqRUg==</latexit>

Signal to noise: For example this is a standard FOM used when testing detectors in order to control false positive rates. If a S/N of 10 is achieved for signal pulses coming out of a silicon detector in the laboratory then the chance of using spurious noise in track reconstruction is small.

Signal significance: This is one form of the signal significance metric. If the signal and background are both large then the denominator is equivalent to the uncertainty on the count rate for S+B. This FOM optimises the ratio of S to this uncertainty estimate.

A. Bevan


CUT BASED ANALYSES: FIGURES OF MERIT FOR OPTIMISATION▸ Consider the following toy example: two types of event described by

Gaussian distributions:

▸ Maximise to find an “optimal” cut.

�11

*While it is not often done, this same logic can be used when applying ML to our analysis. The problem with that approach is that it often requires a significant amount of computing resource to implement such an approach.

S/ S + B

SB

A. Bevan


CUT BASED ANALYSES: FIGURES OF MERIT FOR OPTIMISATION▸ If we have the ability to do so it is better to optimise the

MVA on a figure of merit related to a measurement we are making*. For example:

▸ Expected measurement precision

▸ Expected limit for a search

▸ Maximise purity (here we define purity as S/(S+B), and so we would maximise that as the FOM.

�12

*While it is not often done, this same logic can be used when applying ML to our analysis. The problem with that approach is that it often requires a significant amount of computing resource to implement such an approach.

A. Bevan


CUT BASED ANALYSES: FIGURES OF MERIT FOR OPTIMISATION▸ e.g. Consider a counting experiment that is a search for a

signal where there is a large known background, b.

▸ We can use the discovery significance Z0=√q0 as the figure of merit to optimise on:

▸ Where the likelihood is Poisson:

▸ From this we can compute

�13

*Here we invoke Wilks’ theorem to relate the likelihood ratio to a X2 distribution in order to be able to relate the significance to the likelihood ratio given by the null hypothesis of no signal as an outcome. e.g. see Cowan et al., Eur.Phys.J.C71:1554,2011.

ℒ =(s + b)ne(−s+b)

n!

q0 = − 2 ln(ℒ(s = 0)/ℒ)

q0 = − 2 ln ( bne−b

(s + b)ne−(s+b) )Z0 = q0 = 2(n ln(n/b) + b − n)

https://arxiv.org/abs/1007.1727

A. Bevan


CUT BASED ANALYSES: FIGURES OF MERIT FOR OPTIMISATION▸ Most cut based analyses consider cuts independently; so optimisation has

to be done either:

▸ Simultaneously for all cuts that are being made on an analysis. This approach results in MN iterations for the optimisation, where N is the number of cuts being made and M is the number of samples tested on each cut.

▸ This approach suffers from the curse of dimensionality.

▸ One at a time iteratively until such time as cut values do not change significantly [a pragmatic solution to overcome the MN scaling].

▸ Scales like some (usually small) number times MxN.

▸ How can we extract more information from our data?

�14

A. Bevan


LINEAR DISCRIMINANTS▸ We can construct combinations of input features (dimensions that we

want to cut on) to see if we are able to utilise the way that examples (events or candidates) cluster in the hyperspace that is formed by all of the features.

▸ Our ability to separate examples in this feature space will generally be better than considering lower dimensional projections of the problem and recursively iterating through until we have considered the whole space.

▸ The simplest approach is to construct a linear combination of features.

▸ In HEP the Fisher discriminant has been widely used on a number of experiments.

�15

A. Bevan


LINEAR DISCRIMINANTS: FISHER’S LINEAR DISCRIMINANT▸ This is an analytic algorithm that was inspired by the

classification problem for species of iris in the 1930’s[1].

▸ Starting point is the assumption that data are distributed according to a multi-Gaussian probability (e.g. random sampling of data), and that one wishes to maximise the separation between different classes (types) of iris.

▸ Maximise the separation of the mean distributions.

▸ Minimise the sum of the covariances.

�16

[1] R. A. Fisher, Ann. Eug., 7, 179188 (1936).

A. Bevan


LINEAR DISCRIMINANTS: FISHER’S LINEAR DISCRIMINANT▸ Consider the problem where we have two classes of event. Some

events of type A (signal) and some events of type B (background). ▸ These events are described by a 2D feature space, consisting of

the dimensions x and y. ▸ We want to compute F in order for us to be able to distinguish

between types A and B.

�17

[1] R. A. Fisher, Ann. Eug., 7, 179188 (1936).

A. Bevan


LINEAR DISCRIMINANTS: FISHER’S LINEAR DISCRIMINANT▸ The Fisher discriminant is given by

▸ α: a vector of weight parameters ▸ x: data with the dimension of the input feature space ▸ W: sum of covariance matrices for classes A and B ▸ μA,B: mean value of class A or B

�18

[1] R. A. Fisher, Ann. Eug., 7, 179188 (1936).

F = ↵Tx+ �

↵ = W�1(µA � µB)<latexit sha1_base64="77ByWewygWAI51ofP1uxfQ3pjeE=">AAACInicbVDLSgMxFM3UV62vqks3waJUpGVGBHUh1AriskJf0GnLnTRtQzMPkoxYhn6LG3/FjQtFXQl+jJm2C209cLmHc+4luccJOJPKNL+MxMLi0vJKcjW1tr6xuZXe3qlKPxSEVojPfVF3QFLOPFpRTHFaDwQF1+G05gyuY792T4VkvldWw4A2Xeh5rMsIKC210xc3+PAS28CDPrTK+AEf2w5VYNupiRa7tVaUs0Y4a7th+wrncNyLR+10xsybY+B5Yk1JBk1Raqc/7I5PQpd6inCQsmGZgWpGIBQjnI5SdihpAGQAPdrQ1AOXymY0PnGED7TSwV1f6PIUHqu/NyJwpRy6jp50QfXlrBeL/3mNUHXPmxHzglBRj0we6oYcKx/HeeEOE5QoPtQEiGD6r5j0QQBROtWUDsGaPXmeVE/ylpm37k4zheI0jiTaQ/soiyx0hgroFpVQBRH0iJ7RK3oznowX4934nIwmjOnOLvoD4/sH3sGgKg==</latexit><latexit sha1_base64="77ByWewygWAI51ofP1uxfQ3pjeE=">AAACInicbVDLSgMxFM3UV62vqks3waJUpGVGBHUh1AriskJf0GnLnTRtQzMPkoxYhn6LG3/FjQtFXQl+jJm2C209cLmHc+4luccJOJPKNL+MxMLi0vJKcjW1tr6xuZXe3qlKPxSEVojPfVF3QFLOPFpRTHFaDwQF1+G05gyuY792T4VkvldWw4A2Xeh5rMsIKC210xc3+PAS28CDPrTK+AEf2w5VYNupiRa7tVaUs0Y4a7th+wrncNyLR+10xsybY+B5Yk1JBk1Raqc/7I5PQpd6inCQsmGZgWpGIBQjnI5SdihpAGQAPdrQ1AOXymY0PnGED7TSwV1f6PIUHqu/NyJwpRy6jp50QfXlrBeL/3mNUHXPmxHzglBRj0we6oYcKx/HeeEOE5QoPtQEiGD6r5j0QQBROtWUDsGaPXmeVE/ylpm37k4zheI0jiTaQ/soiyx0hgroFpVQBRH0iJ7RK3oznowX4934nIwmjOnOLvoD4/sH3sGgKg==</latexit><latexit sha1_base64="77ByWewygWAI51ofP1uxfQ3pjeE=">AAACInicbVDLSgMxFM3UV62vqks3waJUpGVGBHUh1AriskJf0GnLnTRtQzMPkoxYhn6LG3/FjQtFXQl+jJm2C209cLmHc+4luccJOJPKNL+MxMLi0vJKcjW1tr6xuZXe3qlKPxSEVojPfVF3QFLOPFpRTHFaDwQF1+G05gyuY792T4VkvldWw4A2Xeh5rMsIKC210xc3+PAS28CDPrTK+AEf2w5VYNupiRa7tVaUs0Y4a7th+wrncNyLR+10xsybY+B5Yk1JBk1Raqc/7I5PQpd6inCQsmGZgWpGIBQjnI5SdihpAGQAPdrQ1AOXymY0PnGED7TSwV1f6PIUHqu/NyJwpRy6jp50QfXlrBeL/3mNUHXPmxHzglBRj0we6oYcKx/HeeEOE5QoPtQEiGD6r5j0QQBROtWUDsGaPXmeVE/ylpm37k4zheI0jiTaQ/soiyx0hgroFpVQBRH0iJ7RK3oznowX4934nIwmjOnOLvoD4/sH3sGgKg==</latexit>

NOTE: I like this algorithm as an example as it can help us understand the data and how to separate classes before looking in more detail at a neural network as the underlying equations appear later in the course.

It is also a simple algorithm that can be used as a benchmark to check that a more sophisticated algorithm performs at least as well as this one. Such a sanity check can be useful in low dimensional physics problems and it may or may not be useful for you machine learning work in the future.

A. Bevan


LINEAR DISCRIMINANTS: FISHER’S LINEAR DISCRIMINANT▸ Consider this 2D problem:

▸ The means (μ) and standard deviations (σ) describe the distribution of data; where σA,B are 2D covariance matrices.

▸ We can compute the mean (M) and variance (Σ) of the Fisher distribution using

�19

μA,x μA,yμB,x μB,yand (μA,y)T=(μA,x, μA,y).i.e. (μB,y)T=(μB,x, μB,y)

μA,B are the mean values of A and B, respectively, given by

µA,B;u =1

N

NX

i=1

ui, u = x, y<latexit sha1_base64="qGQudLsl6r6nqdbQ++1oel13WO0=">AAACK3icbVDLSsNAFJ34rPUVdelmsAguSklEUJBCrRtXUsE+oKlhMp20Q2eSMA+xhPyPG3/FhS584Nb/cPpYaOu5XDiccy8z9wQJo1I5zoe1sLi0vLKaW8uvb2xubds7uw0Za4FJHccsFq0AScJoROqKKkZaiSCIB4w0g8HlyG/eEyFpHN2qYUI6HPUiGlKMlJF8u+px7acXxeo51BksQy8UCKdull5nntTcY5RTJf2Ult3szohQ+7QIveKooC4/FOHQtwtOyRkDzhN3Sgpgippvv3jdGGtOIoUZkrLtOonqpEgoihnJ8p6WJEF4gHqkbWiEOJGddHxrBg+N0oVhLExHCo7V3xsp4lIOeWAmOVJ9OeuNxP+8tlbhWSelUaIVifDkoVAzqGI4Cg52qSBYsaEhCAtq/gpxH5m0lIk3b0JwZ0+eJ43jkuuU3JuTQqU6jSMH9sEBOAIuOAUVcAVqoA4weATP4A28W0/Wq/VpfU1GF6zpzh74A+v7B4fRpfg=</latexit><latexit sha1_base64="qGQudLsl6r6nqdbQ++1oel13WO0=">AAACK3icbVDLSsNAFJ34rPUVdelmsAguSklEUJBCrRtXUsE+oKlhMp20Q2eSMA+xhPyPG3/FhS584Nb/cPpYaOu5XDiccy8z9wQJo1I5zoe1sLi0vLKaW8uvb2xubds7uw0Za4FJHccsFq0AScJoROqKKkZaiSCIB4w0g8HlyG/eEyFpHN2qYUI6HPUiGlKMlJF8u+px7acXxeo51BksQy8UCKdull5nntTcY5RTJf2Ult3szohQ+7QIveKooC4/FOHQtwtOyRkDzhN3Sgpgippvv3jdGGtOIoUZkrLtOonqpEgoihnJ8p6WJEF4gHqkbWiEOJGddHxrBg+N0oVhLExHCo7V3xsp4lIOeWAmOVJ9OeuNxP+8tlbhWSelUaIVifDkoVAzqGI4Cg52qSBYsaEhCAtq/gpxH5m0lIk3b0JwZ0+eJ43jkuuU3JuTQqU6jSMH9sEBOAIuOAUVcAVqoA4weATP4A28W0/Wq/VpfU1GF6zpzh74A+v7B4fRpfg=</latexit><latexit sha1_base64="qGQudLsl6r6nqdbQ++1oel13WO0=">AAACK3icbVDLSsNAFJ34rPUVdelmsAguSklEUJBCrRtXUsE+oKlhMp20Q2eSMA+xhPyPG3/FhS584Nb/cPpYaOu5XDiccy8z9wQJo1I5zoe1sLi0vLKaW8uvb2xubds7uw0Za4FJHccsFq0AScJoROqKKkZaiSCIB4w0g8HlyG/eEyFpHN2qYUI6HPUiGlKMlJF8u+px7acXxeo51BksQy8UCKdull5nntTcY5RTJf2Ult3szohQ+7QIveKooC4/FOHQtwtOyRkDzhN3Sgpgippvv3jdGGtOIoUZkrLtOonqpEgoihnJ8p6WJEF4gHqkbWiEOJGddHxrBg+N0oVhLExHCo7V3xsp4lIOeWAmOVJ9OeuNxP+8tlbhWSelUaIVifDkoVAzqGI4Cg52qSBYsaEhCAtq/gpxH5m0lIk3b0JwZ0+eJ43jkuuU3JuTQqU6jSMH9sEBOAIuOAUVcAVqoA4weATP4A28W0/Wq/VpfU1GF6zpzh74A+v7B4fRpfg=</latexit>

[1] R. A. Fisher, Ann. Eug., 7, 179188 (1936).

A. Bevan


LINEAR DISCRIMINANTS: FISHER’S LINEAR DISCRIMINANT▸ Optimise J, where

▸ Thus:

which is optimised for

▸ resulting in: .

�20

‣ The α are given up to an arbitrary scale factor. ‣ The mean value of F can be offset by β arbitrarily.

[1] R. A. Fisher, Ann. Eug., 7, 179188 (1936).

A. Bevan


LINEAR DISCRIMINANTS: FISHER’S LINEAR DISCRIMINANT▸ Coming back to our

example problem we can computeF using

�21

[1] R. A. Fisher, Ann. Eug., 7, 179188 (1936).

F = ↵Tx+ �


‣ F is a linear combination of x and y (like a rotated axis in the (x, y) plane) along which we can project the data in order to obtain a smaller overlap than in either of the individual features (x or y).

‣ The offset β is arbitrary and is ignored here as it just changes the position of an example on the F axis without changing the separation between the classes.

‣ Separation in F is better than in either x or y. ‣ Separation in F is also better than that using

rectangular cuts.

A. Bevan


EXAMPLES: CUT AND COUNT▸ Measurement of jet fragmentation in Pb+Pb and pp

collisions at ATLAS (2.76 TeV).

▸ Use triggers to isolate potentially interesting events.

▸ Use event selection to identify jets; and from these compute the number of jets in a given pT and rapidity* range.

▸ This allows the computation of cross sections.

�22

*The rapidity y = 0.5 ln [ (E + pz ) / (E - pz) ], where E and pz are the energy and the component of the momentum along the beam direction, respectively.

ATLAS Collaboration, Eur. Phys. J. C 77 (2017) 379

A. Bevan


EXAMPLES: CUT AND COUNT▸ These plots are unfolded

charged particle distributions of the pT.

▸ Error bars indicate statistical uncertainties and the shaded bands indicate systematic uncertainties.

▸ The paper has a number of other results that use the same cut-and-count approach in order to extract information of interest.

�23

ATLAS Collaboration, Eur. Phys. J. C 77 (2017) 379

A. Bevan


EXAMPLES: FISHER DISCRIMINANT▸ Using a Fisher discriminant to suppress light quark

background in order to study B mesons. Produced via

▸ Light quarks are fast in the CM frame, whereas the B mesons are slow and decay isotropically.

�24

A. Bevan et al., Eur. Phys. J. C74 (2014) 3026

e+e� ! BB<latexit sha1_base64="EptcGH58LVStrz9ED4uX56keCek=">AAACAXicbVDLSgMxFM3UV62vUTeCm2ARBLHMiKDLUjcuK9gHdKYlk95pQzOTIckIZagbf8WNC0Xc+hfu/BszbRdaPRA4nHtuknOChDOlHefLKiwtr6yuFddLG5tb2zv27l5TiVRSaFDBhWwHRAFnMTQ00xzaiQQSBRxaweg6n7fuQSom4js9TsCPyCBmIaNEG6lnH0D3FLpn2NMC1zxhrPlNWW3Ss8tOxZkC/yXunJTRHPWe/en1BU0jiDXlRKmO6yTaz4jUjHKYlLxUQULoiAygY2hMIlB+Nk0wwcdG6eNQSHNijafqz42MREqNo8A4I6KHanGWi//NOqkOr/yMxUmqIaazh8KUYxM3rwP3mQSq+dgQQiUzf8V0SCSh2pRWMiW4i5H/kuZ5xXUq7u1FuVqb11FEh+gInSAXXaIqukF11EAUPaAn9IJerUfr2Xqz3mfWgjXf2Ue/YH18A2AUljQ=</latexit><latexit sha1_base64="EptcGH58LVStrz9ED4uX56keCek=">AAACAXicbVDLSgMxFM3UV62vUTeCm2ARBLHMiKDLUjcuK9gHdKYlk95pQzOTIckIZagbf8WNC0Xc+hfu/BszbRdaPRA4nHtuknOChDOlHefLKiwtr6yuFddLG5tb2zv27l5TiVRSaFDBhWwHRAFnMTQ00xzaiQQSBRxaweg6n7fuQSom4js9TsCPyCBmIaNEG6lnH0D3FLpn2NMC1zxhrPlNWW3Ss8tOxZkC/yXunJTRHPWe/en1BU0jiDXlRKmO6yTaz4jUjHKYlLxUQULoiAygY2hMIlB+Nk0wwcdG6eNQSHNijafqz42MREqNo8A4I6KHanGWi//NOqkOr/yMxUmqIaazh8KUYxM3rwP3mQSq+dgQQiUzf8V0SCSh2pRWMiW4i5H/kuZ5xXUq7u1FuVqb11FEh+gInSAXXaIqukF11EAUPaAn9IJerUfr2Xqz3mfWgjXf2Ue/YH18A2AUljQ=</latexit><latexit sha1_base64="EptcGH58LVStrz9ED4uX56keCek=">AAACAXicbVDLSgMxFM3UV62vUTeCm2ARBLHMiKDLUjcuK9gHdKYlk95pQzOTIckIZagbf8WNC0Xc+hfu/BszbRdaPRA4nHtuknOChDOlHefLKiwtr6yuFddLG5tb2zv27l5TiVRSaFDBhWwHRAFnMTQ00xzaiQQSBRxaweg6n7fuQSom4js9TsCPyCBmIaNEG6lnH0D3FLpn2NMC1zxhrPlNWW3Ss8tOxZkC/yXunJTRHPWe/en1BU0jiDXlRKmO6yTaz4jUjHKYlLxUQULoiAygY2hMIlB+Nk0wwcdG6eNQSHNijafqz42MREqNo8A4I6KHanGWi//NOqkOr/yMxUmqIaazh8KUYxM3rwP3mQSq+dgQQiUzf8V0SCSh2pRWMiW4i5H/kuZ5xXUq7u1FuVqb11FEh+gInSAXXaIqukF11EAUPaAn9IJerUfr2Xqz3mfWgjXf2Ue/YH18A2AUljQ=</latexit>

‣ BaBar used a fisher discriminant based on work by the CLEO collaboration. This divided the data up into 9 concentric cones to parameterise the energy flow for BB vs qq-bar (q=u, d, s, c) events.

‣ During the course of data taking new energy-flow variables were introduced that improved upon the “CLEO Fisher”.

F = ↵Tx+ �


A. Bevan


EXAMPLES: FISHER DISCRIMINANT▸ A typical Fisher used by BaBar is illustrated here:

▸ The discriminating variables are the sums L0, L1 and L2; typically a Fisher constructed from L0 and L2 is used.

▸ Usually F was used as a discriminating variable in a maximum likelihood fit. The aim of the fit would be to extract a signal yield or some other parameter (e.g. CP violation asymmetry).

�25

A. Bevan et al., Eur. Phys. J. C74 (2014) 3026

F = ↵Tx+ �


A. Bevan


SUMMARY▸ We’ve discussed the concept of cut-and count analyses

and their roles in our field.

▸ Linear discriminants are a simple extension of a cut based analysis:

▸ Dimensional reduction is achieved by combining information from many features into a single dimension.

▸ Examples from past publications have been discussed.

�26

A. Bevan


SUGGESTED TOOLS▸ TMVA has implementations of cut based optimisation and

Fisher discriminants implemented in it.

▸ scikit learn has linear (and quadratic) discriminants implemented.

▸ These are simple algorithms to implement if you want to try them out yourselves; you will obtain a deeper understanding of the process if you choose to follow that route.

�27

https://root.cern.ch/tmva

http://scikit-learn.org/stable/modules/lda_qda.html

A. Bevan


SUGGESTED READING (HEP RELATED)▸ There are numerous papers that have been written in the HEP

community that use the cut and count approach.

▸ Essentially all experimental results start off with this approach, and some will then apply more complicated methods in order to extract the measurements of interest from data.

▸ Fisher discriminants have been widely used in some experiments, particularly the B Factories.

▸ You will be able to find many examples of using these methods on the archive and in journals.

▸ Chapters 4 and 9 of A. Bevan et al., Eur. Phys. J. C74 (2014) 3026 may also be of use to you with understanding these methods.

�28

A. Bevan


SUGGESTED READING (NON-HEP)▸ There are many non-HEP related source of information on

this type of analysis. Below are a few suggested starting points:

▸ R. A. Fisher, Ann. Eug., 7, 179188 (1936).

▸ Vidal et al., Generalized Principal Component Analysis (2016), Springer.

▸ Cowan, Statistical Data Anslysis, (1998) Oxford University Press.

�29

DR ADRIAN BEVAN MULTIVARIATE ANALYSIS AND …gae/Documents/Adrian1.pdfMULTIVARIATE ANALYSIS AND ITS...

Documents

Transcript of DR ADRIAN BEVAN MULTIVARIATE ANALYSIS AND …gae/Documents/Adrian1.pdfMULTIVARIATE ANALYSIS AND ITS...