Information theory for complex systems€¦ · Complex systems group, Department of Energy and...
Transcript of Information theory for complex systems€¦ · Complex systems group, Department of Energy and...
![Page 1: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/1.jpg)
complex systems group
Information theory for complex systems
Kristian LindgrenComplex systems group, Department of Energy and EnvironmentChalmers University of Technology, Gothenburg, Sweden
1: Cellular automata 2: Pattern formation 3: Spinn systems and Baker’s map
![Page 2: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/2.jpg)
complex systems group
Cellular automata and information
• 1-dimensional CA• Elementary CA rules
- Binary state (0 or 1, ”white” or ”black”)
- Nearest neighbour interaction
- CA state: bi-infinite sequence (... 0 1 1 1 0 0 1 1 ...)
• Dynamics given by deterministic local rule, updating all cells in parallel
![Page 3: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/3.jpg)
complex systems group
Rule table
• Example: Rule 110
111 110 101 100 011 010 001 000
0 1 1 0 1 1 1 0
(01101110)2=110
![Page 4: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/4.jpg)
complex systems group
Rule table
• Example: Rule 110
111 110 101 100 011 010 001 000
0 1 1 0 1 1 1 0
(01101110)8=110
![Page 5: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/5.jpg)
complex systems group
CA classes
Four classes of dynamics:(I) Towards homogenous
fixed point.(II) Towards inhomogenous
fixed point, shift, and/or periodic behavior.
(III) Irregular behavior – ”chaotic”
(IV) In between (II) and (III); long transients, ”complex”.
![Page 6: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/6.jpg)
complex systems group
Class III example: R22
”Chaotic”
![Page 7: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/7.jpg)
complex systems group
Difference pattern
A single cell state at the centered is changed, and the difference pattern illustrates how the disturbance is spread.
![Page 8: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/8.jpg)
complex systems group
Class IV example: R110
Computationally universal
![Page 9: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/9.jpg)
complex systems group
Information characteristics?
![Page 10: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/10.jpg)
complex systems group
Information in a symbol sequence
. . . 0 1 0 0 1 0 1 1 0 1 ?
Basic information
Statistics of the sequence give probabilities...
With probability p of the event, this is generalized to
![Page 11: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/11.jpg)
complex systems group
Information in a symbol sequence
. . . 0 1 0 0 1 0 1 1 0 1 ?
Probability of xm given x1, x2, ..., xm–1
Information gained when observing a symbol — local information
![Page 12: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/12.jpg)
complex systems group
Symmetric local information
Use probabilities that depend on either m–1 symbols to the left or to the right, pL or pR,
Local symmetric information combines ”left” and ”right”
? 1 0 1 1 1 0 0 0 0 1 0 . . .. . . 0 1 0 0 1 0 1 1 0 1 ?
![Page 13: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/13.jpg)
complex systems group
Regularity filter
• ”Local” information I applied to pattern of R110 (row-by-row)
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
![Page 14: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/14.jpg)
complex systems group
Regularity filter
• Applied to pattern of R110 (row-by-row)
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
m = 1
2 3 4 50 1 (bits)
![Page 15: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/15.jpg)
complex systems group
Regularity filter
• Applied to pattern of R110 (row-by-row)
m = 2
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 16: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/16.jpg)
complex systems group
Regularity filter
• Applied to pattern of R110 (row-by-row)
m = 3
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 17: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/17.jpg)
complex systems group
Regularity filter
• Applied to pattern of R110 (row-by-row)
m = 4
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 18: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/18.jpg)
complex systems group
Regularity filter
• Applied to pattern of R110 (row-by-row)
m = 5
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 19: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/19.jpg)
complex systems group
Regularity filter
• Applied to pattern of R110 (row-by-row)
m = 6
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 20: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/20.jpg)
complex systems group
Regularity filter
• Applied to pattern of R110 (row-by-row)
m = 7
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 21: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/21.jpg)
complex systems group
Regularity filter
• Applied to pattern of R110 (row-by-row)
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
m = 8
2 3 4 50 1 (bits)
![Page 22: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/22.jpg)
complex systems group
Regularity filter
• Applied to space-time pattern of R18 (row-by-row)
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
![Page 23: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/23.jpg)
complex systems group
Regularity filter
• Applied to pattern of R18 (row-by-row)
m = 1
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 24: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/24.jpg)
complex systems group
Regularity filter
• Applied to pattern of R18 (row-by-row)
m = 2
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 25: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/25.jpg)
complex systems group
Regularity filter
• Applied to pattern of R18 (row-by-row)
m = 3
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 26: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/26.jpg)
complex systems group
Regularity filter
• Applied to pattern of R18 (row-by-row)
m = 4
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 27: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/27.jpg)
complex systems group
Regularity filter
• Applied to pattern of R18 (row-by-row)
m = 5
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 28: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/28.jpg)
complex systems group
Regularity filter
• Applied to pattern of R18 (row-by-row)
m = 6
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 29: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/29.jpg)
complex systems group
Regularity filter
• Applied to pattern of R18 (row-by-row)
m = 7
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 30: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/30.jpg)
complex systems group
Regularity filter
• Applied to pattern of R18 (row-by-row)
m = 8
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 4 50 1 (bits)
![Page 31: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/31.jpg)
complex systems group
Average of the information quantity
• Average of the information quantity (here the ”Left” one):
• Two interpretations...
![Page 32: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/32.jpg)
complex systems group
Entropy
For a probability distribution P = {p(k)}k=1,...,n
quantifies
- the expected gain of information, or
- the lack of information — uncertainty about the state
![Page 33: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/33.jpg)
complex systems group
Entropy of a stochastic process
• The entropy per symbol, s, is the average uncertainty about the next symbol xm given the previously read ones x1...xm–1 in the limit of infinite m
• The entropy s quantifies the degree of ”randomness” of the sequence.
![Page 34: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/34.jpg)
complex systems group
Change of entropy in CA time evolution
• How does the entropy s change from one time step to the next in a CA?
In general, for deterministic rules, as entropy characterizes ”randomness”, entropy cannot increase,
![Page 35: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/35.jpg)
complex systems group
Relative information
• How does the information about the next symbol change when we extend the number of preceding symbols step-by-step?
• Correlation information
![Page 36: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/36.jpg)
complex systems group
Relative information
• Relative information or Kullback-Liebler information — quantifies how much information is gained when one distribution P0={p0(k)} is replaced by a new one P= {p(k)},
![Page 37: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/37.jpg)
complex systems group
Density information
• Without looking at preceding symbols, how does the information about the next symbol change when we learn the frequencies?
• Density information
![Page 38: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/38.jpg)
complex systems group
Decomposition of information
• The total information of 1 bit per cell can be decomposed into the entropy s and the redundant information kcorr,
• and the redundant information further into density information k1 and correlation information km (m=2, 3, ...)
![Page 39: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/39.jpg)
complex systems group
Information characteristics of CA time evolution
• Example: rule R110
10 20 30 40 50 60 70 80 90 100
0.05
0.1
0.15
0.2
0.25
0.3
Correlation information
km
k1 k2 k3 k4 k5 k6 k7 k8
![Page 40: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/40.jpg)
complex systems group
Regularity filter
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
• ”Local” information I applied to pattern of R60 (row-by-row)
![Page 41: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/41.jpg)
complex systems group
Regularity filter
• Applied to pattern of R60 (row-by-row)
m = 1
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 40 1 (bits)
![Page 42: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/42.jpg)
complex systems group
Regularity filter
• Applied to pattern of R60 (row-by-row)
m = 2
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 40 1 (bits)
![Page 43: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/43.jpg)
complex systems group
Regularity filter
• Applied to pattern of R60 (row-by-row)
m = 3
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 40 1 (bits)
![Page 44: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/44.jpg)
complex systems group
Regularity filter
• Applied to pattern of R60 (row-by-row)
m = 4
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 40 1 (bits)
![Page 45: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/45.jpg)
complex systems group
Regularity filter
• Applied to pattern of R60 (row-by-row)
m = 5
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 40 1 (bits)
![Page 46: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/46.jpg)
complex systems group
Regularity filter
• Applied to pattern of R60 (row-by-row)
m = 6
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 40 1 (bits)
![Page 47: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/47.jpg)
complex systems group
Regularity filter
• Applied to pattern of R60 (row-by-row)
m = 7
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 40 1 (bits)
![Page 48: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/48.jpg)
complex systems group
Regularity filter
• Applied to pattern of R60 (row-by-row)
m = 8
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 40 1 (bits)
![Page 49: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/49.jpg)
complex systems group
Regularity filter
• Applied to pattern of R60 (row-by-row)
m = 12
1 50 100 150 200
1
50
100
150
200
1 50 100 150 2001
50
100
150
200
2 3 40 1 (bits)
![Page 50: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/50.jpg)
complex systems group
Regularity filter
• R60 up to t = 21.
1 50 100 150 200
15101521
1 50 100 150 200
15101521
![Page 51: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/51.jpg)
complex systems group
Regularity filter
• R60 up to length 15 blocks
1 50 100 150 200
15101521
1 50 100 150 200
15101521
m = 1
![Page 52: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/52.jpg)
complex systems group
Regularity filter
• R60 up to length 15 blocks
1 50 100 150 200
15101521
1 50 100 150 200
15101521
m = 8
![Page 53: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/53.jpg)
complex systems group
Regularity filter
• R60 up to length 15 blocks
1 50 100 150 200
15101521
1 50 100 150 200
15101521
m = 12
![Page 54: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/54.jpg)
complex systems group
Regularity filter
• R60 up to length 15 blocks
1 50 100 150 200
15101521
1 50 100 150 200
15101521
m = 15
![Page 55: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/55.jpg)
complex systems group
Analytic solution
0 50 100 150 200
0
50
100
150
200
2500 50 100 150 200
0
50
100
150
200
250
Space-time diagram for rule R60 Local information, infinite m limit
![Page 56: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/56.jpg)
complex systems group
”Additive” CA rule R60The rule that adds two neighbouring states mod 2 (XOR operation) has a certain degree of reversibility.
An additive CA has a finite number of preimages to any state, and they define a class of ”almost reversible” CA. For these CA one can show that entropy is conserved in time,
This means that if one starts with a completely random state (with maximum entropy s = 1), the state at any time will also be completely random.
!
" x i = f (xi#1,xi) = xi#1 + xi (mod 2)
![Page 57: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/57.jpg)
complex systems group
”Additive” CA rule R60
Slightly ordered initial state: low density of 1’s, p(1) = 0.1, results in
2 4 8 16 32 64
0.1
0.2
0.3
0.4
0.5
Correlation information
![Page 58: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/58.jpg)
complex systems group
”Additive” CA rule R60 with noise
Noise added to the CA rule: with probability q a cell state is flipped (here q = 0.005). The noise destroys all long-range correlations.
The noise serves as an inflow of random information which leads to a steady increase in entropy until the state is completely random (s(t) = 1),
1 2 4 8 16 32 64
0.1
0.2
0.3
0.4
0.5
Correlation information
![Page 59: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/59.jpg)
complex systems group
Complexity quantities
• How to quantify ”complexity” in a symbol sequence?
• Entropy?
• How correlation information is distributed?
• How much information is there in the preceding symbols about the ones not yet read?
![Page 60: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/60.jpg)
complex systems group
Effective measure complexity or Excess entropy
• Information in the past ( ) about the future ( ), expressed by the relative information,
• The distribution of correlation information over block lengths,
![Page 61: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/61.jpg)
complex systems group
Excess entropy for rule R60
Assume an initial state (at t = 0) without correlations
• If s = 1 (maximum; equal densities of 0’s and 1’), then km = 0 all m ≥ 2 and
• If s < 1 (unequal densities of 0’s and 1’), then
![Page 62: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/62.jpg)
complex systems group
Excess entropy for R60
s = 1 s < 1
![Page 63: Information theory for complex systems€¦ · Complex systems group, Department of Energy and Environment Chalmers University of Technology, Gothenburg, Sweden 1: Cellular automata](https://reader033.fdocuments.us/reader033/viewer/2022052006/601acd5211f71c62300b5b8e/html5/thumbnails/63.jpg)
complex systems group
More information...
• Lecture notes (draft) available on course web site:
http://studycas.com/node/114
(Several papers can be provided on request.)