Data generation, the hard parts
-
Upload
eric-torreborre -
Category
Technology
-
view
610 -
download
0
Transcript of Data generation, the hard parts
![Page 1: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/1.jpg)
Eric Torreborre / FP-Syd
Data generation
The hard parts
![Page 2: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/2.jpg)
![Page 3: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/3.jpg)
![Page 4: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/4.jpg)
NOT SO SIMPLE
![Page 5: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/5.jpg)
![Page 6: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/6.jpg)
Recursive data structuresPolymorphic functions
Constrained data
![Page 7: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/7.jpg)
![Page 8: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/8.jpg)
![Page 9: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/9.jpg)
![Page 10: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/10.jpg)
![Page 11: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/11.jpg)
Recursive data structures
![Page 12: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/12.jpg)
Agile Estimating and PlanningAgile ManagementAgile Product Management with ScrumAgile Product Planning and AnalysisAgile Project Management: Creating Innovative ProductsAgile Project Management For DummiesAgile Software Development, Principles, Patterns, and PracticesAgile Software Development with ScrumAgile Software Development with Distributed TeamsAgile Testing
![Page 13: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/13.jpg)
Agile + Estimating and Planning + Management + Product + Management with Scrum + Planning and Analysis
+ Project Management + Creating Innovative Products + For Dummies
+ Software Development + Principles, Patterns, and Practices + with Scrum + with Distributed Teams
+ Testing
![Page 14: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/14.jpg)
Generate trees
![Page 15: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/15.jpg)
![Page 16: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/16.jpg)
![Page 17: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/17.jpg)
Generate treesDepth?
Width?
Balanced?
Coverage?
Composition?
Uniformity?
Constraints?
Performance?
![Page 18: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/18.jpg)
programs
well-typed
![Page 19: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/19.jpg)
Size and dimension
BoltzmannModel
Combinatorialspecies
![Page 20: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/20.jpg)
![Page 21: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/21.jpg)
Same size bound
![Page 22: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/22.jpg)
505 valueson average for n=100
![Page 23: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/23.jpg)
18 constructors
P = 1 / 89
![Page 24: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/24.jpg)
P=1/4
P = 1 / 9
![Page 25: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/25.jpg)
![Page 26: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/26.jpg)
generating function
System of equations
solution + singularity
size in O(n)
![Page 27: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/27.jpg)
Enumerate structures
Sample uniformly
![Page 28: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/28.jpg)
![Page 29: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/29.jpg)
Set of labels
Family ofstructures
![Page 30: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/30.jpg)
![Page 31: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/31.jpg)
2
1
3
4
5
6
2
1
3
4
5
6
2
1
3
4
5
6
![Page 32: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/32.jpg)
b
a
c
d
e
f
b
a
c
d
e
f
b
a
c
d
e
e
![Page 33: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/33.jpg)
2
1
3
4
5
6
2
1
3
4
5
6
![Page 34: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/34.jpg)
Regular species
![Page 35: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/35.jpg)
0
![Page 36: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/36.jpg)
1
![Page 37: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/37.jpg)
X
11
![Page 38: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/38.jpg)
F
1
G
23
45
F1
23 4
5
G1
23 4
5
![Page 39: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/39.jpg)
X
1
0
X1
![Page 40: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/40.jpg)
X
1
1
X1
![Page 41: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/41.jpg)
1 1
1
1
![Page 42: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/42.jpg)
n 11 1 …
![Page 43: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/43.jpg)
n 11 1 …
![Page 44: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/44.jpg)
n 11 1 …
![Page 45: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/45.jpg)
F
1
G
23
45
F
1
2
3
4
5
G
![Page 46: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/46.jpg)
X 0
![Page 47: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/47.jpg)
X
1
1
X1
1
![Page 48: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/48.jpg)
X
1
X
X1
X
1
2
2
X2
X1
![Page 49: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/49.jpg)
X
1
XX
1X
2
2
3
XX
3
X1
X3
X2
X2
X1
X3
X3
X2
X1
![Page 50: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/50.jpg)
![Page 51: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/51.jpg)
L X1 L
L X1 L
L X
L X X
L X X X
![Page 52: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/52.jpg)
L
12
3
4
1 2 3 4
2 1 3 4
3 2 1 4
![Page 53: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/53.jpg)
L X1 L
LX
1
1
![Page 54: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/54.jpg)
2 3 4
1
5
9
6 7 8
10 11
No symmetries
![Page 55: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/55.jpg)
GF
12
3
45
1
2
3
4
5
G
F
G
G G
![Page 56: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/56.jpg)
R LX R
2
1
3
4
5
6 7
![Page 57: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/57.jpg)
F G
![Page 58: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/58.jpg)
F '
12
3
45
F
1
2
34
5
![Page 59: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/59.jpg)
F '
L L L'
C L'
![Page 60: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/60.jpg)
F |n|
1 23
n5
F
1
23
4
5 n
… …
… /= n
![Page 61: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/61.jpg)
Non regular species
![Page 62: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/62.jpg)
E
1 23
5
E
1
23
4
54
![Page 63: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/63.jpg)
C
1 23
5
C
4…
![Page 64: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/64.jpg)
E
CEP
![Page 65: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/65.jpg)
C
P CE
L 'C
L P
![Page 66: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/66.jpg)
GF
12
3
45
F G
4
1 3
25 G
4
1 3
25
G
4
1 3
25
![Page 67: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/67.jpg)
GF
EE |2|
EX |2|
![Page 68: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/68.jpg)
in code?
![Page 69: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/69.jpg)
![Page 70: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/70.jpg)
![Page 71: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/71.jpg)
![Page 72: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/72.jpg)
![Page 73: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/73.jpg)
![Page 74: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/74.jpg)
Maths…
![Page 75: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/75.jpg)
"seems it is doable
to find such a function, but needs
work."
"we have a noneasy
question"
![Page 76: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/76.jpg)
My strategy
![Page 77: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/77.jpg)
number of partitions having p sets
…
![Page 78: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/78.jpg)
int partitions of 6
change of representation
3-int partitions of n
![Page 79: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/79.jpg)
Given an index k
![Page 80: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/80.jpg)
![Page 81: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/81.jpg)
Proper notion of size
Uniformity
Species combinators for constraints
![Page 82: Data generation, the hard parts](https://reader036.fdocuments.us/reader036/viewer/2022062522/587ce8081a28ab564b8b4927/html5/thumbnails/82.jpg)
Eric Torreborre / FP-Syd
Data generation
The hard partsThanks!