Data Visualisation & Interpretationmarmakoide.org › download › teaching › dm ›...

UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA SCHOOL OF SOFTWARE ENGINEERING OF USTC

Data Visualisation & InterpretationThe art of reading datasets

Devert AlexandreSchool of Software Engineering of USTC

14 February 2012 — Slide 1/1

Table of Contents

Devert Alexandre (School of Software Engineering of USTC) — Data Visualisation & Interpretation — Slide 2/1

Descriptive statistics

descriptive statistics helps to give a general summary ofdata

Example of descriptive statistics quantity

arithmetic mean

n∑i=1

Example of descriptive statistics quantity

arithmetic mean

n(a1 + a2 + · · ·+ an)

The mean is defined in Rn ⇒ geometric center

Mean computation

You think, it is easy to compute the mean ?

0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1

Mean computation

A naive summation algorithm will return this

>>> 0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.10.8999999999999999

Mean computation

An accurate summation algorithm will return this

>>> impor t math>>> math . fsum (0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1)0 .9

Mean computation

Algorithms like Kahan summation algorithm or Shewchuksummation algorithm reduces the numerical error

de f KahanSum( data ) :s = 0 .0c = 0 .0f o r i i n range ( l e n ( data ) ) :

y = data [ i ] − ct = s + yc = ( t − s ) − ys = t

r e t u r n s

Listing 1: Kahan summation

Central tendencyThe mean is a measure of central tendency ⇒ the mainbehaviour, the main value of some phenomenon

Mean robustnessThe mean is not a robust estimator of the centraltendency

Median

The median is the value such as 50% of the values arehigher, 50% of the values are lower

a = [6, 1, 7, 9, 6, 3, 4, 5, 2]

a = [1, 2, 3, 4, 5, 6, 6, 7, 9]

Median

The median is the value such as 50% of the values arehigher, 50% of the values are lower

a = [6, 1, 7, 9, 6, 3, 4, 8, 5, 2]

a = [1, 2, 3, 4, 5, 6, 6, 7, 8, 9]

2(5 + 8) = 6.5

Median computation

To compute the median, you can

1 sort the list of samples

2 • if size is odd → a = a n+12

• if size is even → a = 12(a n

2+ a n+1

Note that it is for indexes starting from 1

Median computation

Let’s code some python

de f median ( data ) :data . s o r t ( )i f l e n ( data ) % 2 == 0 :m = l e n ( data ) / 2r e t u r n 0 .5 ∗ ( data [m−1] + data [m] )

e l s e :r e t u r n data [ ( l e n ( data ) − 1) / 2 ]

Median computation

Let’s code some python

>>> a =[6 , 1 , 7 , 9 , 6 , 3 , 4 , 5 , 2 ]>>> median ( a )5

Median computationThe median have an equivalent in Rn ⇒ median center

Compute the median for each dimension to get themedian center

Median robustness

The median is a more robust estimator of the centraltendency

• green is the median

• pink is the arithmeticmean

Statistical dispersionThe following datasets have the same central tendency

Statistical dispersionBut they have different dispersions

Standard deviation

A traditional measure of dispersion is the standarddeviation sigma

σ2 =1

n − 1

N∑i=1

(ai − a)2

Standard deviation computation

Robust computation of the standard deviation ⇒Knuth-Welford algorithm

de f stdDev ( data ) :n = 0mean = 0M2 = 0meanEst imate = math . fsum ( data ) / l e n ( data )

f o r x i n data :y = x − meanEst imaten = n + 1d e l t a = y − meanmean = mean + d e l t a / nM2 = M2 + d e l t a ∗ ( y − mean )

r e t u r n math . s q r t (M2 / ( n − 1) )

Standard deviation

Standard deviation suffers from the same robustnessissues as mean. We will look why, later.

Quartiles

The lower quartile or first quartile is the value such as75% of the values are higher, 25% of the values are lower

a = [6, 1, 2, 7, 9, 6, 3, 4, 5, 2, 6]

a = [1, 2, 2, 3, 4, 5, 6, 6, 6, 7, 9]

q1 = 2

Quartiles

The higher quartile or third quartile is the value such as25% of the values are higher, 75% of the values are lower

a = [6, 1, 7, 9, 6, 3, 4, 5, 2, 6]

a = [1, 2, 2, 3, 4, 5, 6, 6, 6, 7, 9]

q3 = 6

Quartiles

Where is the second quartile ? ⇒ it’s the median

Interquartile range

The difference Q3− Q1 is the interquartile range or IQR⇒ it’s a more robust dispersion measure

normal distributionA model for random variables, with 2 parameters µ and σ

−6 −4 −2 0 2 4 60.00

normal distribution

The normal distributions have 2 parameters µ and σ.

Φ(x) =1√

2πσ2e

−(x−µ)2

This is the probability density of the normal distribution.

normal distribution

The normal distributions have 2 parameters µ and σ.

Φ(x) =1√

2πσ2e

−(x−µ)2

It tells the probability for x to appear, according to thisdistribution.

normal distributionµ is the mode, the central tendency of the normaldistribution

−6 −4 −2 0 2 4 60.00

normal distribution

If some data are following a normal distribution, then

µ = a

The more sample, the more ”true“ it will be

normal distributionσ controls the shape of the normal distribution

−6 −4 −2 0 2 4 60.0

normal distribution

If some data are following a normal distribution

σ2 =1

n − 1

N∑i=1

(ai − a)2

The standard deviation comes from here ⇒ dispersion ofa normal distribution

normal distributionµ and σ are completely independent parameters

−6 −4 −2 0 2 4 60.0

normal distribution

Practical interpretation of the normal distribution0.0

−2σ −1σ 1σ−3σ 3σµ 2σ

34.1% 34.1%

13.6%2.1%

13.6% 0.1%0.1%2.1%

68% of the values within [µ− σ, µ + σ]

normal distribution

−2σ −1σ 1σ−3σ 3σµ 2σ

34.1% 34.1%

13.6%2.1%

13.6% 0.1%0.1%2.1%

95% of the values within [µ− 2σ, µ + 2σ]

normal distribution

−2σ −1σ 1σ−3σ 3σµ 2σ

34.1% 34.1%

13.6%2.1%

13.6% 0.1%0.1%2.1%

99.7% of the values within [µ− 3σ, µ + 3σ]

skewed distributions

Your data might not have a symmetric distribution ⇒they might have a skewed distribution

0.0 0.5 1.0 1.5 2.0 2.5 3.00.0

• red is the true centraltendency

0.0 0.5 1.0 1.5 2.0 2.5 3.00.0

You can compute the skewness of your data

∑ni=1(ai − a)3(

∑ni=1(ai − a)2

multimodal distributionsYour data might have multiple modes

−3 −2 −1 0 1 2 3 4 50.0

multimodal distributionsIn such case, the mean, median and other descriptivequantities might have no reliable meaning

−3 −2 −1 0 1 2 3 4 50.0

Table of Contents

Observe your data

Descriptive statistics can completely miss importantinformations from your data !

Observe your dataThe Anscombe’s quartet

0 10 20

Observe your data

Those 4 datasets have exactly the same

• mean

• variance

• regression line

But they are not quite the same things !

BoxplotA nice way to summarize data distribution is the boxplot

Boxplot

The red mark shows the mean

Boxplot

The box goes from the lower quartile to the upperquartile

Boxplot

The box is thus centred on the median

Boxplot

The whiskers are the minimum and maximum values

Boxplot

Outliers values are shown as blue crosses

Outliers are values which are beyond 1.5× IQR from thequartiles

Scatter plotA scatter plot is simply a plot with the data as pointsalong 2 dimensions

−3 −2 −1 0 1 2 3−5

Data Visualisation & Interpretationmarmakoide.org › download › teaching › dm ›...

Documents

Transcript of Data Visualisation & Interpretationmarmakoide.org › download › teaching › dm ›...

Dra. Cardenas DM 2 en niños y adolecentes

I y h mam LEADERETTES ASSEMBLING OF PARADE ALERTNESS · \ h m « l aaa4ariata all *mmt th e w orld H k p h aen dM M ca h y ao « a a y larg e "M« t »nW>a t i n t ke«|> a rc u re

Flow meters DM, DE · R Z 25 40 50 80 100 150 -40 -160 DM 10 DM 16 DM 25 DM 40 DM 65 DM 100 DM 160 DM 250 DM 400 DM 650 DM 1000 = standard, = available Order example DM 16R25-40 5

Analogos de incretinas y glucosuricos en DM

CERN IT Department CH-1211 Genève 23 Switzerland t Storage Overview and IT-DM Lessons Learned Luca Canali, IT-DM DM Group Meeting 10-3-2009.

Ho ada y easd dm

Clase 12 -Reglam. Para El Registro , Control y Vigilancia San. de Pf, Dm y Ps , Copia

Desarrollo y Evolucion Embrionario Dm

T DM ST DM HISS - Start - Oscar Properties69 kvm 2 rok lgh 1:1201 88 kvm 4 rok lgh 1:1205 101 kvm 4 rok lgh 1:1204 63 kvm 2 rok lgh 1:1203 st t k/f dm t g k/f g dm st gstt k f dm trapphus

Math Version 2 Geo · a) 1dm b) 1, 2 dm g) 1,6 dm d) 1,8 dm e) 2,4 dm amocana 21 2 qula marTkuTxa koordinatTa sistemaSi gamosaxulia 1 1 3 x y ⎛⎞ =−⎜⎟ ⎝⎠ funqciis grafiki

Dm Etiopatogenia y Dx

DM-3551/DM-4551 DM DM DM DM DM DM D Network-Ready …files.sharpusa.com/.../MFPsPrinters/Brochures/cop_dow_DM3551_4551bro.pdf · DM DM DM DM DM DM DHigh Performance Features Do more

kamran/EE3301/class notes/ch7.pdf · y(t) = y transient + y steady state for t 0 y transient =[y(0) y( )]e t/ y steady state = y( ) for t 0 y transient = y(0)e t/ y( )e t/ for t 0

Diagnostico, Clasificacion y Etiopatogenia de Dm

DM CEMENT Dark DM CERAMIC Grey DM CERAMIC …docs.sibu.at/tl_files/uploads/katalog/DecoLine_2016...La gama DECO-LINE incluye superficies de espejo, “Vintage” y metálicas, así

Gr aph Theor y - Columbia Universityabishek/files/DM-Ch9.pdf · Gr aph Theor y 9.1 Intr oduction to Graphs 9.2 Graph T erminology 9.3 Represention and Isomorphism 9.4 Connectivity

Notified Products - Belgium · DM FSMP Abbott 1 9 Osmolite 1L T DM FSMP Abbott 1 10 Suplena 237ml blikjes T DM FSMP Abbott 1 11 Ensure plus 250ml T ... DM FSMP Fresenius Kabi 3 1

Gestacion y dm

Manual Operacion y Mtto Dm Series 3.6

DM 45 (40-60 TPH) DM 60 (90 -120 TPH) · Stationary four bin cold aggregate feeder unit is with each bin capacity of 8 T (in DM 45), 11 T (in DM 50) and 14 T (in DM 60).WARNING✕Site