Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an...
Transcript of Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an...
![Page 1: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/1.jpg)
Random Variate Generation and Testingin Interactive Environments
Josef Leydold and Wolfgang Hörmann
Department of Statistics and Mathematics, WU Wien, AustriaBogaziçi University, Istanbul, Turkey
8th MCQMC 2008Montréal, July 11, 2008
![Page 2: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/2.jpg)
Overview
Interactive environments help you to
detect coding errors in an implementation of a randomvariate generator;
detect serious bug in a design of an algorithm;
allow fast prototyping of simulation models.
The R programming language for statistical computing is wellsuited for this purpose.
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 2 / 46
![Page 3: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/3.jpg)
Development of Algorithms
1 Design an algorithms using mathematical principles.
2 Theoretical Proof of correctness.
3 Implement proof-of-concept using assembler / C / Fortran.
4 Run an empirical test to detect programming errors.
5 Estimate memory consumption and generation times.
6 Maybe the algorithms is coded and used in a productionenvironment.
7 Practitioners use the algorithm in their simulation using aprogramming library or some fancy GUI to build up amodel and run simulations.
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 3 / 46
![Page 4: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/4.jpg)
UNU.RAN
We have developed and implemented several (automatic)algorithm in a library called
UNU.RAN (Universal NonUniform Random variate generators),
available at http://statmath.wu-wien.ac.at/unuran.
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 4 / 46
![Page 5: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/5.jpg)
Example: Maxwell Inflow Distribution
double lpdf (double x) { log(1-x)-x*x; }double dlpdf (double x) { log(1-x)-x*x; }
int main(void) {UNUR_DISTR *distr; /* distribution object */UNUR_PAR *par; /* parameter object */UNUR_GEN *gen; /* generator object */
/* Create distribution object with required data */distr = unur_distr_cont_new();unur_distr_cont_set_logpdf(distr, lpdf);unur_distr_cont_set_dlogpdf(distr, dlpdf);unur_distr_cont_set_domain(distr, -UNUR_INFINITY, 1);
/* Choose a method: AROU */par = unur_arou_new(distr);
/* Create the generator object */gen = unur_init(par);
/* Sample */x = unur_sample(gen);
}
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 5 / 46
![Page 6: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/6.jpg)
UNU.RAN String Interface
int main(void) {UNUR_GEN *gen; /* generator object */
/* Create the generator object */gen = unur_str2gen(
"distr = cont; logpdf=’log(1-x)-x*x’; domain=(-inf,1) & \method=arou");
/* Sample */x = unur_sample(gen);
}
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 6 / 46
![Page 7: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/7.jpg)
Interactive Environments
In opposition to this library / GUI approach interactiveenvironments allow for
fast prototyping of algorithms;
playing around with the generator and look whether itbehaves like a source of (non-uniform) randomness;
In addition they provide a powerful language to analyse randomsequences.
Examples are R/S, Root , matlab, Mathematica, . . .
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 7 / 46
![Page 8: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/8.jpg)
The R Project for Statistical Computing
The R Project for Statistical Computing is well-suited for thispurpose:
many statistical tools
can be easily extended
large community provides add-on packages and support(more than 1000 packages on CRAN)
open source (i.e. you can learn from others)
comprehensive documentation(although a local Guru is helpful when you are new)
http://www.R-project.org
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 8 / 46
![Page 9: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/9.jpg)
An R Session
R version 2.7.0 (2008-04-22)Copyright (C) 2008 The R Foundation for Statistical ComputingISBN 3-900051-07-0
R is free software and comes with ABSOLUTELY NO WARRANTY.You are welcome to redistribute it under certain conditions.Type ’license()’ or ’licence()’ for distribution details.
R is a collaborative project with many contributors.Type ’contributors()’ for more information and’citation()’ on how to cite R or R packages in publications.
Type ’demo()’ for some demos, ’help()’ for on-line help, or’help.start()’ for an HTML browser interface to help.Type ’q()’ to quit R.
>
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 9 / 46
![Page 10: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/10.jpg)
Built-in Gaussian Generator
Create a Gaussian sample of size 1000> X <- rnorm(n=1000)
Show sample> X[1:4]
[1] -0.08014195 -1.47207477 -1.72954408 0.00723830
Transform into uniforms> U <- pnorm(X)> U[1:4]
[1] 0.46806218 0.07050034 0.04185588 0.50288764
Get some help> ?rnorm
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 10 / 46
![Page 11: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/11.jpg)
Testing Gaussian Generator
Run χ2 goodness-of-fit test> h <- hist(pnorm(X),plot=FALSE)> chisq.test(h$counts)
Chi-squared test for given probabilities
data: h$countsX-squared = 5.26, df = 9, p-value = 0.811
Run Kolmogorov-Smirnov test> ks.test(X,pnorm)
One-sample Kolmogorov-Smirnov test
data: XD = 0.0329, p-value = 0.2287alternative hypothesis: two-sided
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 11 / 46
![Page 12: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/12.jpg)
Testing Gaussian Generator
Run Shapiro-Wilk test of normality> shapiro.test(X)
Shapiro-Wilk normality test
data: XW = 0.9985, p-value = 0.5854
Run Anderson-Darling sample test> library("adk")> adk.test(pnorm(X), (1:1000-0.5)/1000)$adk
t.obs P-value extrapolationnot adj. for ties -0.34735 0.43139 1adj. for ties -0.34847 0.43173 1
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 12 / 46
![Page 13: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/13.jpg)
Histogram
Create histogram> hist(X,breaks=25)
Histogram of X
X
Fre
quen
cy
−2 0 2 4
050
100
150
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 13 / 46
![Page 14: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/14.jpg)
Fancy Histogram
Create fancy histogram> h <- hist(X, breaks=25, freq=FALSE,+ border="dark green", col="light green",+ xlab=NULL, ylab=NULL,+ main="Distribution of random sample")> xval <- seq(h$breaks[1],h$breaks[length(h$breaks)],+ length.out=100)> lines(xval, dnorm(xval), col="blue", lwd=2)
Define function> myhist <- function(x,pdf) {+ h <- hist(x,breaks=25,freq=FALSE,border="dark green",+ col="light green",xlab=NULL,ylab=NULL,+ main="Distribution of random sample")+ xval <- seq(h$breaks[1],h$breaks[length(h$breaks)],length.out=100)+ lines(xval,pdf(xval),col="blue",lwd=2)+ }
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 14 / 46
![Page 15: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/15.jpg)
Fancy Histogram
Create fancy histogram> h <- hist(X, breaks=25, freq=FALSE,+ border="dark green", col="light green",+ xlab=NULL, ylab=NULL,+ main="Distribution of random sample")> xval <- seq(h$breaks[1],h$breaks[length(h$breaks)],+ length.out=100)> lines(xval, dnorm(xval), col="blue", lwd=2)
Define function> myhist <- function(x,pdf) {+ h <- hist(x,breaks=25,freq=FALSE,border="dark green",+ col="light green",xlab=NULL,ylab=NULL,+ main="Distribution of random sample")+ xval <- seq(h$breaks[1],h$breaks[length(h$breaks)],length.out=100)+ lines(xval,pdf(xval),col="blue",lwd=2)+ }
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 14 / 46
![Page 16: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/16.jpg)
Fancy Histogram
Create histogram> myhist(X, dnorm)
Distribution of random sample
−2 0 2 4
0.0
0.1
0.2
0.3
0.4
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 15 / 46
![Page 17: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/17.jpg)
Transform to Uniforms
Create histogram> myhist(pnorm(X), dunif)
Distribution of random sample
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.4
0.8
1.2
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 16 / 46
![Page 18: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/18.jpg)
Transform to Uniforms
Distribution of transformed random sample
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.4
0.8
1.2
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 17 / 46
![Page 19: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/19.jpg)
Increase Sample Size
Sample size 1.e7
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 18 / 46
![Page 20: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/20.jpg)
Increase Sample Size
Sample size 1.e7
−4 −2 0 2 4
0.0
0.1
0.2
0.3
0.4
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 18 / 46
![Page 21: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/21.jpg)
Increase Sample Size
Sample size 1.e4
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
1.2
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 19 / 46
![Page 22: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/22.jpg)
Increase Sample Size
Sample size 1.e5
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
1.2
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 19 / 46
![Page 23: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/23.jpg)
Increase Sample Size
Sample size 1.e6
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
1.2
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 19 / 46
![Page 24: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/24.jpg)
Looking Closer
Sample size 1.e6
0.0 0.2 0.4 0.6 0.8 1.0
0.99
1.00
1.01
1.02
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 20 / 46
![Page 25: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/25.jpg)
Looking Closer
Sample size 1.e7
0.0 0.2 0.4 0.6 0.8 1.0
0.99
51.
000
1.00
51.
010
1.01
5
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 20 / 46
![Page 26: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/26.jpg)
Test with Increasing Sample Sizes
Run χ2 goodness-of-fit test1.e4 1.e5 1.e6 1.e7
p-value 0.03573423 0.4961224 1.046281e-05 1.500107e-89
Run Kolmogorov-Smirnov test1.e4 1.e5 1.e6 1.e7
p-value 0.5586853 0.5722032 0.009953565 3.152891e-09
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 21 / 46
![Page 27: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/27.jpg)
The Kinderman-Ramage Algorithm
The above error appeared in R prior to version 1.7.Then the default Gaussian generator was the algorithm byKinderman and Ramage [3] but one line of code was missing.
This generator was said to be the fastest Gaussian randomvariate generator and is implemented in several programs andlibraries.
Meanwhile the default Gaussian generator is the quantilefunction (inversion method). One can play with buggy generatorby setting the global generator using
> RNGkind(normal.kind="Buggy Kinderman-Ramage")
So what happens when we fix this bug and use the originallypublished algorithm?
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 22 / 46
![Page 28: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/28.jpg)
The Kinderman-Ramage Generator
Sample size 1.e8
−6 −4 −2 0 2 4 6
0.0
0.1
0.2
0.3
0.4
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 23 / 46
![Page 29: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/29.jpg)
The Kinderman-Ramage Generator
Sample size 1.e8
−0.2 −0.1 0.0 0.1 0.2
0.30
0.34
0.38
0.42
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 23 / 46
![Page 30: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/30.jpg)
The Kinderman-Ramage Generator
Sample size 1.e6
0.0 0.2 0.4 0.6 0.8 1.0
0.96
0.98
1.00
1.02
1.04
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 24 / 46
![Page 31: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/31.jpg)
The Kinderman-Ramage Generator
Sample size 1.e7
0.0 0.2 0.4 0.6 0.8 1.0
0.99
00.
995
1.00
01.
005
1.01
0
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 24 / 46
![Page 32: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/32.jpg)
The Kinderman-Ramage Generator
Sample size 1.e8
0.0 0.2 0.4 0.6 0.8 1.0
1.00
01.
005
1.01
01.
015
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 24 / 46
![Page 33: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/33.jpg)
The Kinderman-Ramage Algorithm
Run χ2 goodness-of-fit test1.e5 1.e6 1.e7 1.e8
p-value 0.822784 0.4299325 0.1891353 8.561413e-154
The error occured because a rejection statement was missing inthe proposed algorithm. However, in 1976 the authors of thepaper would have needed at least 8.5 hours of pure CPU time ona mainframe machine for just generating a sample of size 108.
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 25 / 46
![Page 34: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/34.jpg)
A “Perfect” Slice Sampler
Perfect sampler turn MCMC sampler into exact samplingalgorithms.
A popular method is Coupling from the past (CFTP) where twoextremal chains are generated by means of the same sequenceof random numbers. When the two chains coalesce, the state isreturned.
A variant based on the slice sampler stores all states of the twochains. Thus a modification of the algorithm has been proposedthat need only store the sequence of the uniform randomnumbers.
We ran the algorithm on a (truncated) exponential distribution.
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 26 / 46
![Page 35: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/35.jpg)
A “Perfect” Slice Sampler
Sample size = 105, t0 = 10
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
1.2
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 27 / 46
![Page 36: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/36.jpg)
Computational Limits
F distribution with ν1 = 10 and ν2 = 0.005 degrees of freedom
> X <- rf(1.e4, df1=10, df2=0.005)> h <- hist(pf(X,10,0.005),breaks=100)> chisq.test(h$counts)
Chi-squared test for given probabilities
data: h$countsX-squared = 26335.66, df = 99, p-value < 2.2e-16
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 28 / 46
![Page 37: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/37.jpg)
Computational Limits
F distribution with ν1 = 10 and ν2 = 0.005 degrees of freedom
0.0 0.2 0.4 0.6 0.8 1.0
0.0
1.0
2.0
3.0
> pf(1e300, df1=10, df2=0.005)
[1] 0.824609
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 29 / 46
![Page 38: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/38.jpg)
Computational Limits
Beta distribution with shape parameters 3 and 0.01> X <- rbeta(1e4, shape1=3, shape2=0.05)> h <- hist(pbeta(X,3,0.05),breaks=100)> chisq.test(h$counts)
Chi-squared test for given probabilities
data: h$countsX-squared = 26707.86, df = 99, p-value < 2.2e-16
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 30 / 46
![Page 39: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/39.jpg)
Computational Limits
Beta distribution with shape parameters 3 and 0.01
0.0 0.2 0.4 0.6 0.8 1.0
0.0
1.0
2.0
3.0
> pbeta(c(1-2^(-52), 1-2^(-53), 1-2^(-54)),+ shape1=3, shape2=0.05)
[1] 0.8224850 0.8285318 1.0000000
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 31 / 46
![Page 40: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/40.jpg)
Extending R by Your Own Code
R can load shared libraries. Thus
Time critical code can be implemented inC / C++ / Fortran / (Java) / . . .
The results of the code can be easily compared with theprototype code.
Production-ready code can be included, used and/orchecked within the R framework.
External libraries can be used by means of quite simplewrapper functions.
External code can be collected in packages for easier use.
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 32 / 46
![Page 41: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/41.jpg)
Uniform Random Numbers
Package rstream
implements a framework were random number generators aretreated as objects. This allows for different in/dependent streamsof random numbers from different sources.In particular it provides an interface to one of L’Ecuyer’spseudo-random number generators with multiple substreams.
> library(rstream)> rs <- new("rstream.lecuyer")> rstream.sample(rs,4)
[1] 0.8008996 0.2685858 0.4814772 0.3634042
> rstream.reset(rs); rstream.antithetic(rs) <- TRUE> rstream.sample(rs,4)
[1] 0.1991004 0.7314142 0.5185228 0.6365958
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 33 / 46
![Page 42: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/42.jpg)
Runuran
Package Runuran
provides an interface to our UNU.RAN library.
> library(Runuran)> lpdf <- function(x) { log(1-x)-x^2 }> dlpdf <- function(x) { -1/(1-x)-2*x }> distr <- unuran.cont(pdf=lpdf,dpdf=dlpdf,islog=TRUE,+ lb=-Inf,ub=1)> urg <- unuran.new(distr,"arou")> unuran.sample(urg,4)[1] -0.10216088 0.08524869 0.16917772 0.23213163
> X <- unuran.sample(urg,1e6)> nnpdf <- function(x) {exp(lpdf(x))}> c <- 1/integrate(nnpdf,-100,1)$value> pdf <- function(x) {c*exp(lpdf(x))}> myhist(X, pdf)
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 34 / 46
![Page 43: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/43.jpg)
Runuran
Package Runuran
provides an interface to our UNU.RAN library.
> library(Runuran)> lpdf <- function(x) { log(1-x)-x^2 }> dlpdf <- function(x) { -1/(1-x)-2*x }> distr <- unuran.cont(pdf=lpdf,dpdf=dlpdf,islog=TRUE,+ lb=-Inf,ub=1)> urg <- unuran.new(distr,"arou")> unuran.sample(urg,4)[1] -0.10216088 0.08524869 0.16917772 0.23213163
> X <- unuran.sample(urg,1e6)> nnpdf <- function(x) {exp(lpdf(x))}> c <- 1/integrate(nnpdf,-100,1)$value> pdf <- function(x) {c*exp(lpdf(x))}> myhist(X, pdf)
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 34 / 46
![Page 44: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/44.jpg)
Maxwell Inflow Distribution
Distribution of random sample
−3 −2 −1 0 1
0.0
0.1
0.2
0.3
0.4
0.5
0.6
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 35 / 46
![Page 45: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/45.jpg)
Hyperbolic Distribution
> lpdf <- function(x) { -sqrt(1+x^2) }> dlpdf <- function(x) { -x/sqrt(1+x^2) }> distr <- unuran.cont(pdf=lpdf,dpdf=dlpdf,islog=TRUE)> urg <- unuran.new(distr,"arou")> X <- unuran.sample(urg,1e6)
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 36 / 46
![Page 46: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/46.jpg)
Hyperbolic Distribution
Distribution of random sample
−10 −5 0 5 10
0.00
0.05
0.10
0.15
0.20
0.25
0.30
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 37 / 46
![Page 47: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/47.jpg)
Hyperpolic Distribution by Inversion
> lpdf <- function(x) { -sqrt(1+x^2) }> distr <- unuran.cont(pdf=lpdf,islog=TRUE)> urg <- unuran.new(distr,"pinv")> X <- unuran.sample(urg,1e6)
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 38 / 46
![Page 48: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/48.jpg)
Hyperpolic Distribution by Inversion
Distribution of random sample
−15 −10 −5 0 5 10
0.00
0.05
0.10
0.15
0.20
0.25
0.30
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 39 / 46
![Page 49: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/49.jpg)
Generalized Hyperpolic Distribution
> library(ghyp)
ghyp, 2007, Institute of Data Analysis and Process Design, GPL
> gh <- ghyp(lambda = -2, alpha.bar = 0.5,+ mu = 10, sigma = 5, gamma = 1)> pdf <- function(x) { dghyp(x,gh) }> distr <- unuran.cont(pdf=pdf,islog=FALSE)> urg <- unuran.new(distr,"pinv")> X <- unuran.sample(urg,1e6)
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 40 / 46
![Page 50: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/50.jpg)
Generalized Hyperpolic Distribution
Distribution of random sample
0 50 100 150 200
0.00
0.02
0.04
0.06
0.08
0.10
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 41 / 46
![Page 51: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/51.jpg)
Conclusion
Before using a new non-uniform random variate generator use aninteractive environment to
visualize the resulting samples;
run various tests;
play around with different parameters.
The R programming language for statistical computing is wellsuited for this purpose.
Beware:You can never verify the correctness of aparticular random variate generator!
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 42 / 46
![Page 52: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/52.jpg)
Conclusion
Before using a new non-uniform random variate generator use aninteractive environment to
visualize the resulting samples;
run various tests;
play around with different parameters.
The R programming language for statistical computing is wellsuited for this purpose.
Beware:You can never verify the correctness of aparticular random variate generator!
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 42 / 46
![Page 53: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/53.jpg)
Thank You
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 43 / 46
![Page 54: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/54.jpg)
References I
[1] Root. An object-oriented data analysis framework. CERN, Geneve,Switzerland. URL http://root.cern.ch/.
[2] R Development Core Team. R: A Language and Environment forStatistical Computing. R Foundation for Statistical Computing, Vienna,Austria, 2008. URL http://www.R-project.org. ISBN3-900051-07-0.
[3] A. J. Kinderman and J. G. Ramage. Computer generation of normalrandom variables. J. Am. Stat. Assoc., 71(356):893–898, 1976.
[4] Günter Tirler, Peter Dalgaard, Wolfgang Hörmann, and Josef Leydold.An error in the Kinderman-Ramage method and how to fix it.Computational Statistics and Data Analysis, 47(3):433–440, 2004. doi:doi:10.1016/j.csda.2003.11.019.
[5] James G. Propp and David B. Wilson. Exact sampling with coupledMarkov chains and applications to statistical mechanics. RandomStructures and Algorithms, 9:223–252, 1996.
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 44 / 46
![Page 55: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/55.jpg)
References II
[6] G. Casella, K. L. Mengersen, C. P. Robert, and D. M. Titterington. Perfectslice samplers for mixture distributions. Journal of the Royal StatisticalSociety B, 64:777–790, 2002.
[7] Anne Philippe and Christian P. Robert. Perfect simulation of positiveGaussian distributions. Statistics and Computing, 13:179–186, 2003.
[8] Christian P. Robert and George Casella. Monte Carlo StatisticalMethods. Springer Verlag, 2 edition, 2004.
[9] Pierre L’Ecuyer and Josef Leydold. xxxx. R News, x(x):x–x, 2005.
[10] Pierre L’Ecuyer, Richard Simard, E. Jack Chen, and W. David Kelton. Anobject-oriented random-number package with many long streams andsubstreams. Operations Research, 50(6):1073–1075, 2002.
[11] Josef Leydold and Wolfgang Hörmann. UNU.RAN – A Library forUniversal Non-Uniform Random Variate Generation. Department ofStatistics and Mathematics, WU Wien, A-1090 Wien, Austria,2000–2007. available at http://statmath.wu-wien.ac.at/unuran/.
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 45 / 46
![Page 56: Random Variate Generation and Testing in Interactive ... · Development of Algorithms 1 Design an algorithms using mathematical principles. 2 Theoretical Proof of correctness. 3 Implement](https://reader033.fdocuments.us/reader033/viewer/2022050300/5f69761fcf0b52712d00e3e2/html5/thumbnails/56.jpg)
References III
[12] Josef Leydold and Wolfgang Hörmann. Runuran – R interface to theUNU.RAN random variate generators. Department of Statistics andMathematics, WU Wien, A-1090 Wien, Austria, 2007. available athttp://cran.r-project.org/.
[13] Wolfgang Hörmann, Josef Leydold, and Gerhard Derflinger. AutomaticNonuniform Random Variate Generation. Springer-Verlag, BerlinHeidelberg, 2004.
MCQMC 2008 – Montréal Leydold & Hörmann – RV Generation and Testing in Interactive Environments – 46 / 46