Post on 23-Feb-2016
description
Entropy and the 2nd Law of Thermodynamics
If you do a Google search on the term “entropy” or on the phrase “2nd Law of Thermodynamics”, you will get millions of hits… but they won’t all say the same thing! There are several different ways to state these ideas; they sound so different from each other, but they’re all related.
Flipping a Fair Coin
If we toss two fair coins, there are four possible outcomes:
It will be easier to list these outcomes with letters (H means “heads” and T means “tails”):
TT TH HT HHThese four outcomes are equally likely with fair coins. That’s true whether we flip one coin twice, or two coins at the same time and then arrange them in some sequential order.
When we list the possible outcomes, we see that there are eight (8):
TTT TTH THT HTT THH HTH HHT HHH
We’ll be exploring the outcomes with larger numbers of coins, so it will be convenient for us to categorize these outcomes. We’ll categorize them by the number of heads:
Now we flip 3 fair coins
Zero heads One head Two heads Three heads
TTT
TTHTHTHTT
THHHTHHHT HHH
(1 way) (3 ways) (3 ways) (1 way)
Now we flip 4 fair coins
Zero heads One head Two heads Three heads Four heads
TTTT
TTTHTTHTTHTTHTTT
TTHHTHTHTHHTHTTHHTHTHHTT
THHHHTHHHHTHHHHT HHHH
(1 way) (4 ways) (6 ways) (4 ways) (1 way)
The 16 possible outcomes are categorized above, and I’ve also included the number of specific outcomes in each category. We’ve seen these numbers before… they’re the rows of Pascal’s Triangle! It’s easy to calculate the numbers for each category – just use the “combination” function:
4C0 4C1 4C2 4C3 4C4
Entropy and the 2nd Law of Thermodynamics
“Entropy” is just the number of specific outcomes for a given category.
These numbers are shown in the table below, and are plotted on the vertical axis of the bar chart at right.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
50
100
150
200
250
300
Flipping 10 Fair Coins
Categories: Proportion of flips with heads facing up
Num
ber o
f Spe
cific
Out
com
es
The 2nd Law of Thermodynamics simply says that the system is most likely to be observed in the category having the largest entropy. Duh!
1 10 45 120 210 252 210 120 45 10 1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
50
100
150
200
250
300
Flipping 10 Fair Coins
Categories: Proportion of coins with heads facing up
Num
ber o
f Spe
cific
Out
com
es
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.00
0.05
0.10
0.15
0.20
0.25
0.30
Probability Distribution for 10 Fair Coins
Proportion of coins with heads facing up
Prob
abili
ty
Switching to Probabilities
So, we can use the combination function to calculate the number of specific outcomes in each category. The total number of all outcomes is given by 2n where n is the number of coins flipped.
Therefore, it is also easy to turn these into a probability of observing each category. Thus,
P(H=0.4) = 210/1024for example.
If we list the resulting probability for each category, we have something called a “probability distribution”. The following slides show probability distributions for increasing numbers of coins.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.00
0.05
0.10
0.15
0.20
0.25
0.30
10 Fair Coins
Proportion of coins with heads facing up
Prob
abili
ty
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.00
0.02
0.04
0.06
0.08
0.10
0.12
50 Fair Coins
Proportion of coins with heads facing up
Prob
abili
ty
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
100 Fair Coins
Proportion of coins with heads facing up
Prob
abili
ty
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.00
0.01
0.02
0.03
0.04
0.05
0.06
200 Fair Coins
Proportion of coins with heads facing up
Prob
abili
ty
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
500 Fair Coins
Proportion of coins with heads facing up
Prob
abili
ty
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.005
0.01
0.015
0.02
0.025
0.03
1,000 Fair Coins
Proportion of coins with heads facing up
Prob
abili
ty
0 10 20 30 40 50 600
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Sequential Coin Flips
Number of coin flips
Runn
ing
Prob
abili
ty T
otal
Alternatively, we could flip the coins one at a time and keep track of the evolving probability. Each coin flip has a random outcome, yet after many flips, the system will evolve toward its most likely outcome. Also, the variation away from the equilibrium value (i.e. the zig-zags in the graph) will disappear after many flips.
Evolution of the Coin System
0 200 400 600 800 1000 1200 14000
10
20
30
40
50
60
70
80
90
Entropy Simulation
Number of Collisions
Entr
opy
We began with 160 units of energy uniformly distributed among 40 particles, and this system quickly evolved to a more likely distribution of energy. With only 40 particles in our system, the variation in entropy persists , even at equilibrium… with a mole of particles, any variation would be undetectable.
Evolution of the Peg and Washer System
View of a Typical Example of an Evolved System
The image at left shows an entropy board after 1450 collisions. Describe the distribution of energy in this system.
Discussion Questions
1) If I toss 10 fair coins, which of the following specific outcomes is more likely?HHHHHHHHHH or HTHHTHTTTH ?
2) The first example above is from the category H=1.0 while the second is from the category H=0.5. Which category are we more likely to observe if we toss 10 coins?
3) Let’s say I toss 10 coins and then arrange them in some order. Then I tell you only which of the 11 categories I’ve got, but ask you to guess the specific order of heads and tails. Which category would give you the lowest chance of success? This is meant to show why the best definition may be, “Entropy is a measure of the amount of missing information”. With this new definition, restate the 2nd Law of Thermodynamics.
4) Using the trend observed in slides 7 – 12, what would the probability distribution look like if we tossed one billion coins?
5) Evaluate this statement from the new Next Generation Science Standards: “Uncontrolled systems always evolve toward more stable states—that is, toward more uniform energy distribution”.