Concise statistical thermodynamics

Elias Riedel Gårding

March 10, 2020

Contents1 Equilibrium 3

1.1 Physical systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Ensembles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 The microcanonical ensemble . . . . . . . . . . . . . . . . . . . . 41.4 The canonical ensemble . . . . . . . . . . . . . . . . . . . . . . . . 5

1.4.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4.2 The Boltzmann distribution . . . . . . . . . . . . . . . . . 51.4.3 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.5 The grand canonical ensemble . . . . . . . . . . . . . . . . . . . . 71.6 Examples and applications . . . . . . . . . . . . . . . . . . . . . . 9

1.6.1 The closed ideal gas . . . . . . . . . . . . . . . . . . . . . . 91.6.2 The open ideal gas . . . . . . . . . . . . . . . . . . . . . . . 91.6.3 Bosonic and fermionic modes . . . . . . . . . . . . . . . . 101.6.4 The equipartition theorem . . . . . . . . . . . . . . . . . . 12

2 Reversible processes 132.1 State representations . . . . . . . . . . . . . . . . . . . . . . . . . 132.2 Quasistatic, reversible and irreversible processes . . . . . . . . 132.3 Heat, work and the first law . . . . . . . . . . . . . . . . . . . . . 14

2.3.1 In the canonical ensemble . . . . . . . . . . . . . . . . . . 142.3.2 In the grand canonical ensemble . . . . . . . . . . . . . . 15

2.4 Gases and pressure . . . . . . . . . . . . . . . . . . . . . . . . . . 162.5 Thermodynamic potentials . . . . . . . . . . . . . . . . . . . . . . 162.6 The ideal gas, continued . . . . . . . . . . . . . . . . . . . . . . . 18

3 Irreversible processes 193.1 The second law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.2 Free energy change for special processes . . . . . . . . . . . . . . 20

3.2.1 Chemical reactions . . . . . . . . . . . . . . . . . . . . . . 20

1

A Mathematical preliminaries 21A.1 Partial derivatives and differential forms . . . . . . . . . . . . . 21

A.1.1 Coordinate systems on a manifold . . . . . . . . . . . . . 21A.1.2 Coordinate systems from constraints . . . . . . . . . . . . 21A.1.3 Partial derivatives . . . . . . . . . . . . . . . . . . . . . . . 22A.1.4 Differential forms . . . . . . . . . . . . . . . . . . . . . . . 22

A.2 The method of Lagrange multipliers . . . . . . . . . . . . . . . . 23

2

1 Equilibrium

1.1 Physical systemsA physical system consists of a set I of possible states i. The system isgoverned by a Hamiltonian H, which determines an energy H(i) = εi foreach state. Classically, I is the phase space, while quantum-mechanically,we take it to be the set of eigenstates of the Hamiltonian: H |i⟩ = εi |i⟩. Apartfrom energy, each state may have additional properties such as momentumpi, particle number ni and so on.

An example to keep in mind is the classical monoatomic ideal gas withN atoms of mass m confined to a region V whose volume we shall also callV . This system has states

i = (x1,p1, . . .xN ,pN) with xk ∈V and pk ∈R3

but a permutation of the atoms is considered to be the same state (that is,the space of states is I = (V×R3)N

SNwhere SN is the symmetric group of order

N). The system’s energies are

εi =N∑

k=1

p2k

2m.

1.2 EnsemblesAn ensemble consists of a large number M of copies S1, . . . , SM of a systemS. The states i of S are called microstates. The full state of an ensembleis the state of all its subsystems, (i1, . . . , iM). Removing the ordering of thesubsystems from the full state yields the macrostate (ai)i∈I where the oc-cupation number ai is the number of subsystems in microstate i, that is,ai =∑M

m=1(im = i). Clearly these numbers fulfil∑

i∈I ai = M.Ensembles are always analysed in the limit M →∞, where they are un-

derstood to model a probability distribution (pi)i∈I on the space of states of areal-world system. In the limit M →∞, the fraction pi ≡ ai/M is interpretedas the probability that the system is in microstate i. Equally well as theoccupation numbers (ai)i∈I , we can take these probabilities as defining themacrostate.

The entropy of the macrostate (pi)i∈I is defined as

S ≡ kB ⟨− ln pi⟩ =−kB∑i∈I

pi ln pi (1)

3

where kB is Boltzmann’s constant, equal to 1.380649×10−23 J/K in SI unitsor 1 in natural units. The entropy is a measure of the unpredictability of themicrostate given only information about the macrostate.

The internal energy of the macrostate is defined as the average energy

U ≡ ⟨εi⟩ =∑i∈I

piεi. (2)

We shall use ensembles to model a particular kind of macrostate, namelythermal equilibrium. Though tricky to define from first principles, it is in-tuitively easy to understand: The system is allowed to evolve for an infinitetime until it settles into a state where all macroscopically observable quan-tities no longer change with time.

1.3 The microcanonical ensembleThe microcanonical ensemble models a physical system completely isolatedfrom any environment. It consists of a large number of independent virtualcopies of a system, where each copy has the same energy E, that is, thestate is (i1, . . . , iM) with εim = E for all m. The law of conservation of energyensures that this property is preserved as the ensemble evolves in time.

The fundamental postulate of statistical mechanics says that, in thermalequilibrium, all microstates with energy E are equally likely. That is,

pi =

1Ω(E) if εi = E0 otherwise

(3)

where Ω(E)≡∑i∈I(εi = E) is the number of microstates with energy E.

Justifying the fundamental postulate is a complicated matter, and thereis no consensus on exactly how to best do it. The ergodic hypothesis positsthat an isolated system will, over time, explore all states compatible withits total energy, and spend an equal amount of time in each. This is anattractive idea, but it has a number of problems, not least that it may notbe true for many systems of interest. The principle of indifference is anotherroute. [1]

With the uniform probability distribution (3), the entropy (1) reduces to

S = kB lnΩ(E). (4)

Furthermore, it is an easy exercise to show that of all probability distribu-tions on a set of Ω(E) items, the uniform distribution (3) is the one withmaximal entropy. Thus, the fundamental postulate may also be stated as:In thermal equilibrium, the entropy of an isolated system is maximised.

4

1.4 The canonical ensemble1.4.1 Definition

The canonical ensemble models a physical system S in thermal equilibriumwith a large environment E . The composite system C ≡ S ⊗E is isolated,so its total energy Etot is conserved. Thus, in thermal equilibrium, it isdescribed by a microcanonical ensemble of a large number M of copies ofC, each with fixed energy Etot. That is, each microstate of the compositesystem with energy Etot is equally probable. Given this, we wish to find thedistribution of the microstates of S, ignoring those of E .

We need to make an assumption about the structure of the environment.The simplest is that it consists of a large number, M−1, of copies of S:

C =S︷︸︸︷

S1 ⊗E︷ ︸︸ ︷

S2 ⊗·· ·⊗SM .

This makes the composite system C itself an ensemble. It is, however, not amicrocanonical ensemble because the energy of individual subsystems is notfixed—the subsystems are not isolated from each other and may exchangeenergy while keeping the total energy fixed. It is called the canonical ensem-ble.

Formally, we are dealing with an “ensemble of ensembles”, because thereare M copies of the ensemble C. However, these copies are very muchvirtual—they are just another way of describing the uniform probability dis-tribution of the microstates of C, so it is easiest to forget about them andinstead imagine one copy of C, evolving in time and spending equal time inall its microstates according to the ergodic hypothesis. The copies of S, how-ever, interact with each other and represent an actual physical environment,so they are an essential part of the picture.

As before, the microstates of the composite system C are (i1, . . . , iM) whereim ∈ I are the microstates of S. We neglect any interaction energy betweenthe subsystems, so that εC

(i1,...,iM ) = εi1 +·· ·+εiM . The macrostate is (ai)i∈I or(pi)i∈I with pi = ai/M. The occupation numbers ai then fulfil∑

i∈Iai = M and

∑i∈I

aiεi = Etot. (5)

1.4.2 The Boltzmann distribution

In principle, the values of (ai)i∈I are different for different microstates. Thecanonical ensemble thus strictly describes a distribution of macrostates of C(this distribution formally constitutes the single macrostate of C). We will,

5

however, make an approximation and seek only the most likely macrostate ofC. It can be shown [2] that this approximation is completely justified in thelimit M →∞.

The most likely macrostate (ai)i∈I is the one that corresponds to thegreatest number of microstates (i1, . . . , iM). This number is

W ≡ M!∏i∈I ai!

. (6)

We thus seek the distribution (ai)i∈I that maximises W , or equivalently lnW ,subject to the two constraints (5). Introducing two Lagrange multipliers λand β, this is the same as the unconstrained maximum of

f (ai,λ,β)≡ lnW −λ(∑

i∈Iai −M

)−β

(∑i∈I

aiεi −Etot

).

By Stirling’s approximation (valid in the limit M →∞),

lnW ≈ M(ln M−1)−∑i∈I

ai(lnai −1),

so the condition for a maximum becomes

0= ∂ f∂ai

=− lnai −λ−βεi

by whichai = e−λ−βεi .

Rewriting in terms of pi = ai/M, we find the Boltzmann distribution

pi = 1Z

e−βεi (7)

where Z ≡ Meλ is called the partition function and is more usefully written

Z(εi,β)≡ ∑i∈I

e−βεi . (8)

We define the temperature T via

β= 1kBT

. (9)

6

1.4.3 Properties

From (8), we find that the internal energy (2) is simply derivable from thepartition function:

U =− ∂

∂βln Z. (10)

Another expression for the internal energy is U = Etot/M.We may also find an expression for the entropy (1). We compute

SkB

= ⟨− ln pi⟩ =⟨ln Z+βεi

⟩= ⟨ln Z⟩+β⟨εi⟩ = ln Z+βU ,

that isS = kB(ln Z+βU)= kB ln Z+ U

T. (11)

It is also instructive to compute

lnW ≈ M(ln M−1)−∑i∈I

ai(lnai −1)= M ln M−∑i∈I

ai lnai +0︷ ︸︸ ︷(∑

i∈Iai −M

)=−∑

i∈Iai ln

ai

M=−M

∑i∈I

pi ln pi = MkB

S,

that is,

S = 1M

kB lnW .

1.5 The grand canonical ensembleThe canonical ensemble describes the most probable macrostate given thatall microstates are equally likely, subject to the conservation of total energy.One may carry out an analogous derivation with additional conserved quan-tities. In the grand canonical ensemble, each state i has, in addition to theenergy εi, a particle number ni, and one postulates that the total particlenumber

∑i aini is conserved. Similarly to the derivation of the Boltzmann

distribution, we seek the maximum of W subject to the three constraints∑i∈I

ai = M and∑i∈I

aiεi = Etot and∑i∈I

aini = Ntot. (12)

Equivalently, we seek the unconstrained maximum of

f (ai,λ,α,β)≡ lnW −λ(∑

i∈Iai −M

)−β

(∑i∈I

aiεi −Etot

)−α

(∑i∈I

aini −Ntot

)where α is a Lagrange multiplier for the new constraint. The solution is

pi = 1Z

e−βεi−αni = 1Z

e−β(εi−µni) (13)

7

where µ≡−α/β is called the chemical potential and

Z (εi,β,µ)≡ ∑i∈I

e−β(εi−µni) (14)

is the grand canonical partition function.The average number of particles N becomes

N ≡ ⟨ni⟩ =∑i∈I

pini = 1β

∂

∂µlnZ . (15)

The internal energy is similar to in the canonical ensemble, but we have tokeep α=−βµ fixed in the partial derivative:

U =−(∂

∂β

)βµ

lnZ (16)

(the regular partial derivative − ∂∂β

lnZ = −(∂∂β

)µ

lnZ evaluates to U −µN).

The expression for the entropy is

S = kB ⟨− ln pi⟩ = kB⟨lnZ +βεi −βµni

⟩= kB(lnZ +βU −βµN

). (17)

Note the symmetry between (β,U) and (α, N) when this is written as S =kB

(lnZ +βU +αN

).

Setting µ= 0 in (14) reduces the grand canonical partition function to theregular partition function (8) of the canonical ensemble. Thus µ = 0 corre-sponds to the case where total particle number is not necessarily conserved.Note, however, that the canonical ensemble is often used for cases where theparticle number of each subsystem is fixed, such as in the standard treat-ment of the ideal gas (see below). Then one chooses the states i such thatthey all have the same number of particles. Such systems are called closed.The grand canonical ensemble models an open system, which may exchangeparticles with the environment, under the constraint that the total particlenumber is conserved.

The derivation above can be generalised to any number of conservedquantities X1 ≡∑

i aixi1, X2 ≡∑

i aixi2, . . . . The distribution becomes

pi = 1Z ′ e

−(α1xi1+α2xi

2+··· ) where Z ′ = ∑i∈I

e−(α1xi1+α2xi

2+··· ).

In this way, one could account for the conservation of momentum, angularmomentum and so on. In practice, however, it is very rarely useful to con-sider conserved quantities other than energy and particle number.

8

1.6 Examples and applications1.6.1 The closed ideal gas

The states and energy levels of the monoatomic ideal gas are detailed insection 1.1. Because the state space is continuous, the sum over states forone atom is replaced by an integral:∑

i∈I−−→ 1

h3

∫V

d3x∫R3

d3 p . (18)

Here, h is Planck’s constant, which in this context serves only to make theintegral dimensionless—as we shall see, its numerical value is not reflectedin any measurable quantities. The important aspect is that the number ofstates in a phase space volume

∫d3xd3 p is proportional to that volume. Li-

ouville’s theorem, which says that time evolution of an ensemble of statespreserves its phase space volume, then ensures that the number of mi-crostates, and therefore the probability of each one, is conserved.

The partition function for an ideal gas of N atoms in the canonical en-semble thus becomes

Z = 1N!

∫d3x1 d3 p1

h3 · · · d3xN d3 pN

h3 e−β

(p2

12m+···+p2

N2m

)(19)

where the factor 1/N! comes from removing the overcounting of permuta-tions of the atoms. The d3x integrals simply give a factor V N , while we

can use the Gaussian integral∫Rdpe−

p22a =p

2πa to evaluate the momentumintegrals; the final result is

Z = 1N!

(Vh3

)N(2πmβ

) 32 N

. (20)

We find ln Z = N(lnV − 32 lnβ+const), and thus

U =− ∂

∂βln Z = 3N

2β= 3

2NkBT (21)

andS = kB ln Z+ U

T= NkB

(lnV + 3

2lnT +const

). (22)

1.6.2 The open ideal gas

For the ideal monoatomic gas with a variable number of particles, the statesare

i = (n,x1,p1, . . .xn,pn) with i ∈N, xk ∈V and pk ∈R3

9

modulo permutations of the atoms. The grand canonical partition functionis

Z =∞∑

n=0

1n!

∫d3x1 d3 p1

h3 · · · d3xn d3 pn

h3 e−β

(p2

12m+···+ p2

n2m−µn

). (23)

Letting Zn be the canonical partition function (19) for a gas with n atoms,we may rewrite this as

Z =∞∑

n=0eβµnZn =

∞∑n=0

eβµn 1n!

(Z1)n =∞∑

n=0

1n!

(eβµZ1

)n = eeβµZ1 ,

That is, lnZ = eβµZ1. Inserting the known expression (20) for Z1, we find

lnZ = eβµVh3

(2πmβ

)3/2.

Then, the average particle number is

N = ⟨ni⟩ = 1β

∂

∂µlnZ = eβµ

Vh3

(2πmβ

)3/2= lnZ . (24)

By (16) the internal energy is

U =−(∂

∂β

)βµ

lnZ = 32β

lnZ = 32

NkBT = 32⟨ni⟩kBT. (25)

This agrees with the formula for the closed ideal gas (21), but N is now theaverage ⟨ni⟩ instead of a constant. The entropy is, according to (17),

S = kB(lnZ +βU −βµN

)=βkB(U + (1−βµ)N

)= (52−βµ

)NkB. (26)

1.6.3 Bosonic and fermionic modes

From the point of view of statistical physics, a mode is any system where thestates are described by a single number n, and the energy is a linear functionof n. In a bosonic mode, n ranges from zero to infinity, while in a fermionicmode, n can only be 0 or 1. The quantum harmonic oscillator is a bosonicmode, as are the modes of a bosonic quantum field like the electromagneticfield. The modes of a fermionic field, like the electron field (in free space orin a condensed matter system) are fermionic modes. In the latter two cases,we interpret n as the number of particles in the mode.

Quantum-mechanically, these modes are described by a pair of ladderoperators a and a† and a vacuum state |0⟩ such that a |0⟩ = 0. States are

10

constructed as |n⟩ ∝ (a†)n |0⟩ and are eigenstates of the number operatorn ≡ a†a with eigenvalues n. We take the Hamiltonian to be H = ~ω(n+ 1

2 )(but the ground state energy ~ω

2 is largely irrelevant). In a fermionic mode,the anticommutation relation a†, a†= 0 enforces (a†)2 = 0, whence there areonly two states, |0⟩ and |1⟩. This is the Pauli exclusion principle. For bosonicmodes, there is no such constraint.

The simple upshot of these quantum-mechanical considerations is thatwe are dealing with the systems

εn = ~ω(n+ 1

2

), n =

0,1,2,3, . . . bosonic mode0,1 fermionic mode.

(27)

We consider these systems in the grand canonical ensemble, that is, under aconstraint of conserved total energy and particle number.

In the bosonic case, the grand canonical partition function is

Z =∞∑

n=0e−β

(~ω

(n+ 1

2

)−µn

)= e−β~ω/2

1−e−β(~ω−µ) (28)

and the average occupation number is

N = ⟨n⟩ = 1β

∂

∂µlnZ = e−β(~ω−µ)

1−e−β(~ω−µ) =1

eβ(~ω−µ) −1(29)

This is the Bose–Einstein distribution.In the fermionic case, the grand canonical partition function is

Z =1∑

n=0e−β

(~ω

(n+ 1

2

)−µn

)= e−β~ω/2

(1+e−β(~ω−µ)

)(30)

and the average occupation number is

N = ⟨n⟩ = 1β

∂

∂µlnZ = e−β(~ω−µ)

1+e−β(~ω−µ) =1

eβ(~ω−µ) +1(31)

This is the Fermi–Dirac distribution.The internal energy for both cases is

U = ~ω(⟨n⟩+ 1

2

)= ~ω

(12+ 1

eβ(~ω−µ) ∓1

) (− for bosons+ for fermions

).

11

1.6.4 The equipartition theorem

The equipartition theorem states: For a system with N f quadratic degrees offreedom, the internal energy approaches N f

2 kBT+const at high temperatures.A “quadratic degree of freedom” means a variable x such that the energy

contains a term proportional to x2. “At high temperatures” means, in thiscontext, that the system has enough energy so that the possibly discrete setof values of x can be approximated as a continuum.

If our system has a state given by (x1, . . . , xN f ) and a Hamiltonian givenby

H = k1x21 +·· ·+kN f x2

N f+E0

with E0 being some arbitrarily chosen ground state energy, we may evaluatethe partition function as a Gaussian integral

Z ∝∫

dx1 . . .dxN f e−β

(k1x2

1+···kNf x2Nf

+E0

)=

√π

βk1· · · π

βkN f

e−βE0 ∝β−N f /2e−βE0 .

Then ln Z =−βE0 − N f2 lnβ+const and

U =− ∂

∂βln Z = E0 +

N f

2β= E0 +

N f

2kBT,

which is the content of the equipartition theorem.A monoatomic ideal gas has three quadratic degrees of freedom per atom,

namely the three components of the momentum. The equipartition theoremthus predicts that the internal energy should be U = 3

2 NkBT+const, which isexactly right. But a general ideal gas (not necessarily monoatomic) may haveother degrees of freedom, in particular molecular rotational and vibrationaldegrees of freedom. If there are f degrees of freedom in total per molecule(including the three momentum components) so that N f = N f , the partitionfunction becomes

Z ∝V NTN f /2 (32)

with internal energy

U = f2

NkBT. (33)

12

2 Reversible processes

2.1 State representationsWe have previously defined the macrostate of an ensemble as the probabilitydistribution (pi)i∈I on its microstates. As we are generally working in aknown ensemble—the canonical or sometimes the grand canonical—we mayrepresent the macrostate in different, equivalent ways. In full generality,the macrostate of a system is characterised by two kinds of parameters.

Parameters of the first kind are those that appear as Lagrange multi-pliers for the constraints of the ensemble. The temperature T (or β) andthe chemical potential µ belong to this class. We might call them “thermalparameters” or “environmental conditions”. They are not relevant for indi-vidual microstates, but appear only in the description of the macrostate.

Parameters of the second kind are those that appear in the Hamiltonianitself. What these parameters are depends on the system in question. Theseare V for an ideal gas (technically also m, although we will take it as aconstant because it doesn’t change in realistic situations), ω for a bosonic orfermionic mode, and so on. We may call them “Hamiltonian parameters” or“external constraints”. They directly influence the energies εi of individualmicrostates. In fact, the energy levels themselves, (εi)i∈I , form the mostgeneral set of Hamiltonian parameters.

Thus, a macrostate can always be represented as (T, εi) in the canoni-cal ensemble, and as (T,µ, εi) in the grand canonical ensemble. But specificsystems usually require much fewer parameters. For an ideal gas, for exam-ple, one can write the state as (T,V ) in the canonical ensemble and (T,µ,V )in the grand canonical ensemble. A bosonic or fermionic mode is describedby (T,ω) and (T,µ,ω) respectively.

Starting from these fundamental representations, one can represent sta-tes in terms of other variables. For example, for the (monoatomic) idealgas, one may find U and S via (21) and (22). The state is then equally wellspecified by (T,S), (S,V ), (U ,V ) or (S,U). (However, one cannot use (T,U)since knowledge of this pair is not sufficient to solve for V .)

2.2 Quasistatic, reversible and irreversible processesThermodynamics is largely concerned with the study of processes. By pro-cess we here mean any time-evolution of a system which starts and endsin an equilibrium state. That is, the start and end points can be describedas points in the space of macrostates parameterised as described above, forexample—for an ideal gas—in the two-dimensional (T,V ) space.

13

A process is quasistatic if the system changes slowly through a contin-uum of equilibrium states, i.e. it traces out a continuous curve in this space.

A process is said to be reversible if the system and its environment jointlyundergo a quasistatic process. In our framework, quasistatic and reversibleprocesses are the same, since we have only defined equilibrium with respectto the environment. However, in thermodynamics, one sometimes speaksof quasistatic irreversible processes, where the system is in “internal equi-librium” but not in equilibrium with its environment. Processes involvingfriction are one example. We shall not dwell on this distinction.

An irreversible process is one that is not reversible. In general, only theendpoints are equilibrium states, while the intermediate states are not. Inour framework, irreversible and non-quasistatic processes are the same.

A process is called cyclic if the initial and final states are the same.

2.3 Heat, work and the first law2.3.1 In the canonical ensemble

Consider a system in the canonical ensemble undergoing a quasistatic pro-cess. Take the differential of the internal energy and split it as follows:

dU = d(∑

i∈Ipiεi

)= ∑i∈Iεi dpi︸ ︷︷ ︸δQ

+∑i∈I

pi dεi︸ ︷︷ ︸−δW

. (34)

The first term on the right is called heat, δQ, while the second term is calledwork, −δW . Thus we have the first law of thermodynamics

dU = δQ−δW (35)

whereδQ ≡ ∑

i∈Iεi dpi and −δW ≡ ∑

i∈Ipi dεi . (36)

(The sign convention for work is somewhat unfortunate, and is there forhistorical reasons: +δW is the work done by the system and −δW is thework done on the system, which is practical when analysing systems suchas steam engines, which are meant to do positive net work on their surround-ings. The other sign convention, δW =∑

i pi dεi, is also in use. Beware!)The first law identifies two distinct ways that internal energy can be

added to or removed from a system: If the system starts in equilibrium, andone modifies the Hamiltonian parameters εi (section 2.1) by a small amountdεi, the internal energy increases by −δW = ⟨dεi⟩ = ∑

i pi dεi; this is calleddoing work on the system. However, the old distribution (pi)i∈I is no longer

14

the Boltzmann distribution with respect to the new energy levels (εi+dεi)i∈I .Thus, the system is no longer in equilibrium. The probabilities pi will there-fore change—by changing the thermal parameters (section 2.1)—by smallamounts dpi to re-establish equilibrium. The internal energy changes by anamount

∑i εi dpi at the expense of the environment’s internal energy; one

says that the system has absorbed or released heat.The entropy (1) is a direct function of the probabilities pi, and should

therefore change in a heat transfer but not directly when work is done. Letus compute the change in entropy:

dSkB

= d(−∑

ipi ln pi

)=−∑

idpi ln pi −

∑i

pi dln pi

=−∑i

dpi(−βεi − ln Z

)−∑i

dpi

=−∑i

dpi(−βεi

)− (∑i

dpi

)(1− ln Z)

=β∑i

dpi εi −d(∑

ipi

)︸ ︷︷ ︸

d(1)=0

(1− ln Z)

=βδQ,

in other words,δQ = T dS . (37)

2.3.2 In the grand canonical ensemble

Consider a system in the grand canonical ensemble undergoing a quasistaticprocess. We will not define heat and work according to (36), but instead wewill take eq. (37), δQ = T dS, as the definition of heat.

We compute, similarly to before,

dSkB

= d(−∑

ipi ln pi

)=−∑

idpi

(−βεi +βµni)

=β∑iεi dpi −βµ

∑i

ni dpi .

This gives the definition of heat in the grand canonical ensemble

δQ ≡ T dS =∑iεi dpi −µ

∑i

ni dpi =∑

i(εi −µni)dpi . (38)

Modifying the definition of work in a similar way,

−δW ≡∑i

pi dεi −µ∑

ipi dni =

∑i

pi(dεi −µdni

), (39)

15

we arrive at the first law of thermodynamics for the grand canonical ensem-ble:

dU = δQ−δW +µdN . (40)

In the grand canonical ensemble, the first law thus identifies three waysthat the internal energy may change: heat, work and particle exchange, thelatter considered neither heat nor work.

2.4 Gases and pressureIf the system in question is a gas, essentially the only parameter that can bevaried in the Hamiltonian is the volume V . The corresponding work is thusδW = pdV , where p is called the pressure. Naturally, δW contains a termpdV in any system where the volume is a parameters, including liquids andsolids. For a large class of substances, it is the only term in δW .

If δW = pdV , the first law takes the form

dU = T dS− pdV or dU = T dS− pdV +µdN (41)

in the canonical or grand canonical ensemble, respectively. The pressure pcan thus be expressed (defined) as

p =−(∂U∂V

)S

or p =−(∂U∂V

)S,N

. (42)

2.5 Thermodynamic potentialsLet us—for simplicity—concentrate on the case of a gas, or other substancefor which δW = pdV . In the grand canonical ensemble, we have the first law

dU = T dS− pdV +µdN , (43)

and all results derived from it below will be also valid for the canonical en-semble if we set dN = 0.

The first law expresses U as a function of S, V and N, its so-called natu-ral variables, and gives the name S, −p and µ to its partial derivatives withrespect to those variables. We may define other functions, called thermo-dynamical potentials, which are naturally functions of other variables. Inaddition to U , define the Helmholtz free energy (or just free energy) F, the

16

enthalpy H, the Gibbs free energy G and the grand potential Ω by

U dU = T dS− pdV +µdN (44)F ≡U −TS =⇒ dF =−S dT − pdV +µdN (45)H ≡U + pV =⇒ dH = T dS+V dp+µdN (46)G ≡U + pV −TS =⇒ dG =−S dT +V dp+µdN (47)Ω ≡U −TS−µN =⇒ dΩ =−S dT − pdV −N dµ . (48)

These relationships are useful in problem solving when manipulating dif-ferentials, but they also have a deeper meaning: They are each related to adifferent type of ensemble. We see that the potentials, including U , are nat-urally expressed as functions U(S,V , N), F(T,V , N), H(S, p, N), G(T, p, N)and Ω(T,V ,µ).

Take the Helmholtz free energy F, for example. Its natural variables, T,V and N, are precisely the variables that are kept constant in the canon-ical ensemble. Furthermore, we may rewrite it in terms of the canonicalpartition function Z, using the expression (11) for the entropy, as

F =−1β

ln Z, i.e. Z = e−βF . (49)

Or, take the grand potential Ω. Its natural variables, T, V and µ arethose kept constant in the grand canonical ensemble. The expression (17)for the entropy relates it to the grand canonical partition function Z as

Ω=−1β

lnZ , i.e. Z = e−βΩ. (50)

This leads us to a simple way to construct other ensembles. For example,for an ensemble with constant N, p and T, we see that these are the naturalvariables of the Gibbs free energy. So, we might conjecture that the partitionfunction of such an ensemble is given by

ZN pT = e−βG . (51)

Indeed, going through with the procedure from section 1.4, we let the volumeof each subsystem vary, but keep the total volume constant. We seek theglobal minimum of

f (ai,λ,β)≡ lnW −λ(∑

i∈Iai −M

)−β

(∑i∈I

aiεi −Etot

)−βp

(∑i∈I

aivi −Vtot

)and find pi = 1

ZN pTe−β(εi+pvi) with ZN pT = ∑

i e−β(εi+pvi). For the entropy, wehave S/kB = ⟨− ln pi⟩ =

⟨ln ZN pT +βεi +βpvi

⟩ = ln ZN pT +βU +βpV (whereV ≡ ⟨vi⟩). Thus

−1β

ln ZN pT =U + pV −TS =G and ZN pT = e−βG .

17

2.6 The ideal gas, continuedAs we found in sections 1.6.1 and 1.6.4, the ideal gas with f degrees of free-dom per molecule ( f = 3 for the monoatomic ideal gas) has the canonicalpartition function Z ∝ 1

N!VNTN f /2. Thus the free energy is F =−kBT ln Z =

−NkBT(lnV + f

2 lnT +const). From the definition (41) of pressure, we find

p =−(∂F∂V

)T= NkBT

V

which gives the famous ideal gas law

pV = NkBT. (52)

We may find the same law in the grand canonical ensemble; from (24) we

have lnZ = eβµ Vh3

(2πmβ

)3/2 = N. Using (48), we may calculate the pressure as

p =−(∂Ω

∂V

)T,µ

=−(∂

∂V

)T,µ

(−kBT lnZ )= kBTlnZ

V= NkBT

V,

again giving the ideal gas law pV = NkBT (but now with N being the aver-age ⟨ni⟩).

18

3 Irreversible processesIn an irreversible process, the system starts and ends in an equilibriumstate, with a distribution of microstates described by some ensemble. Whilethe process is taking place, however, we cannot say anything about the sys-tem’s state.

The key to analysing irreversible processes is assuming that the envi-ronment is in equilibrium throughout the whole process, although the sys-tem might not be. Essentially, we assume that the environment is too bigfor its state to be significantly affected by the comparatively small system.When we speak of the temperature T in an irreversible process, we thus al-ways mean the temperature of the environment—the system may not havea well-defined temperature during the process.

In a general process, the system S absorbs heat Q from the environment

E and does work W on it: S E .W

QSince S is not in equilibrium, we

cannot directly integrate the definitions (36) to split the change in internalenergy as ∆U = Q −W . However, as the environment is in equilibrium, wecan integrate those definitions with respect to the environment to define −Qand −W : −∆U =∆Uenv = (−Q)− (−W). This defines what we mean by Q andW in an irreversible process.

3.1 The second lawThe second law of thermodynamics says that processes in which the totalentropy decreases cannot occur:

∆Stot ≥ 0. (53)

For a system in contact with an environment

∆Stot =∆S+∆Senv. (54)

We may formulate the second law in terms of the system only. Since theenvironment is in equilibrium, ∆Senv =−Q

T . Inserting this into (53) and (54),we find the so-called generalised Clausius inequality

∆S ≥ QT

. (55)

(The original Clausius inequality says that QT ≤ 0 in a cyclic process, where

of course ∆S = 0.)

19

3.2 Free energy change for special processesConsider a general isothermal process. By the first law, Q = ∆U +W . Interms of the Helmholtz free energy F = U −TS we have ∆U = ∆F +T∆S,since the process is isothermal. Inserting these into the generalised Clausiusinequality (55),

∆S ≥ QT

= ∆U +WT

= ∆F +T∆S+WT

= ∆F +WT

+∆S.

Thus∆F +W ≤ 0 (56)

for a general isothermal process. We see that W ≤−∆F, that is, if the systemdoes work W , its free energy must decrease by at least the same amount. Thefree energy F can thus be interpreted as the maximum amount of work thesystem can do in an isothermal process.

Next, consider a system with volume V undergoing a general isothermaland isobaric process. We split the work done as W = p∆V+Wnon-pV . In termsof the Gibbs free energy G = F + pV , we get ∆F = ∆G − p∆V , so ∆F +W =∆G+Wnon-pV . Thus, (56) becomes

∆G+Wnon-pV ≤ 0 (57)

for a general isothermal and isobaric process. Evidently, G plays a similarrole as F, but for non-pV work (with the added constraint that p is constant).

3.2.1 Chemical reactions

20

A Mathematical preliminaries

A.1 Partial derivatives and differential formsA.1.1 Coordinate systems on a manifold

Our goal is to understand the partial derivatives of functions f : M → R,where M is some continuous set. Formally, we say that M is a manifold. Inthermodynamics, M corresponds to the space of macrostates for a thermo-dynamical system.

A point m ∈ M needn’t be a number or vector; in general, it is an ab-stract point. But one can describe m using numbers by choosing a coor-dinate system (formally called a chart). For concreteness, if M is a two-dimensional space, you can describe each point with two coordinates (x1, x2)as m = mx(x1, x2). But you could equally well choose another coordinate sys-tem (y1, y2) and write m = my(y1, y2). Here, mx and my are two functionsR2 → M.

Applying a function f : M →R after mx or my, we get functions f mx orf my : R2 →R:

( f mx)(x1, x2) ≡ f (mx(x1, x2))( f my)(y1, y2)≡ f (my(y1, y2)).

Usually, we are lazy and just write

f (mx(x1, x2)) = f (x1, x2)f (my(y1, y2))= f (y1, y2).

This notation hides the functions mx and my, but they are still there! IfI want to evaluate f (y1, y2), I cannot simply plug x1 = y1 and x2 = y2 intoa formula for f (x1, x2), because the symbol f actually represents differentfunctions R2 → R (different realisations of the same function f : M → R indifferent coordinate systems).

A.1.2 Coordinate systems from constraints

If we want to describe a two-dimensional space, we just need two coordi-nates. But it is often more practical to use three coordinates and one con-straint equation. For example, for the first octant of the two-dimensionalsphere with radius 1, it is very natural to use coordinates (x1, x2, x3) (all pos-itive numbers) for R3, subject to the constraint that x2

1+x22+x3

3 = 1. But eachpoint is determined by any two of the three coordinates: (x1, x2), (x2, x3) or(x1, x3) are all equally good coordinate systems. We could also use regularspherical coordinates (θ,ϕ).

21

An example from thermodynamics is the ideal gas. The state space canbe described by any two of the three variables volume V , pressure p and tem-perature T: (V , p), (T, p) or (V ,T). The third variable can be expressed as afunction of the other two: T(V , p), V (T, p), p(V ,T). But it is more natural tosay that the state space consists of the three variables (V , p,T), constrainedto obey the ideal gas law pV = NkBT where N and kB are constants (moreon this law in the main text).

The general framework is that we choose a system of three coordinates(x1, x2, x3) and then impose the constraint h(x1, x2, x3)= 0 where h : R3 →R issome function. For the spherical octant, we could pick h(x1, x2, x3)= x2

1+x22+

x23 −1 and for the ideal gas, h(V , p,T)= pV −NkBT.

A.1.3 Partial derivatives

Usually, when we have a function defined as

f (x1, x2)= (some expression involving x1 and x2)

we know what is meant by the partial derivative ∂ f∂x1

: It is the partial deriva-tive of f with respect to its first argument. We know that x1 is the firstargument because f was defined that way. Specifically, we know that weshould take the derivative while viewing x2 as a constant.

When we work with different coordinate systems, as we often do in ther-modynamics, this notation can be ill-defined. On the spherical octant, ifwe define a function f (x1, x2, x3) = x1x2x3, we can also view it as f (x1, x2),f (x2, x3) or f (x1, x3). Writing ∂ f

∂x1is then ambiguous: Do we mean the partial

derivative of f (x1, x2), f (x1, x3) or f (x1, x2, x3)?The solution is to introduce a notation to distinguish between these. We

write the above alternatives as(∂ f∂x1

)x2

,(∂ f∂x1

)x3

and(∂ f∂x1

)x2,x3

respectively. The subscript indicates which variables are to be held constant.

A.1.4 Differential forms

Differential forms, or just differentials, are an extremely convenient way tohandle partial derivatives. The basic idea is to view everything, includingthe coordinates themselves, as functions M → R. For any such function f ,one defines the differential d f .

22

Theorem. If a set of functions A, B and C are related by some constrainth(A,B,C)= 0, then (

∂A∂B

)C

(∂B∂C

)A

(∂C∂A

)B=−1. (58)

Proof. Since h is constant, dh = 0. Expand

0= dh = adA+bdB+ cdC .

Then (∂A∂B

)C=−b

a,

(∂B∂C

)A=− c

band

(∂C∂A

)B=−a

c,

so (∂A∂B

)C

(∂B∂C

)A

(∂C∂A

)B=

(−b

a

)(− c

b

)(−a

c

)= (−1)3 =−1.

A.2 The method of Lagrange multipliers

References[1] Jos Uffink. “Compendium of the Foundations of Classical Statistical

Physics”. In: Philosophy of Physics. Ed. by Jeremy Butterfield and JohnEarman. Handbook of the Philosophy of Science. Amsterdam: North-Holland, 2007, pp. 923–1074. DOI: https://doi.org/10.1016/B978-044451560 - 5 / 50012 - 9. URL: http : / / www . sciencedirect . com /science/article/pii/B9780444515605500129.

[2] Erwin Schrödinger. Statistical Thermodynamics. eng. Dover Books onPhysics. Newburyport: Dover Publications, 1989. ISBN: 1-5231-2517-9.

23