A Farewell to Entropy

Statistical Thermodynamics Based on Information

Arieh Ben-Naim

• Publisher: WSPC
• Year: 2008
• ISBN:
  9789814338288
• Topic: Physical Sciences
• Subtopic: Thermodynamics

The principal message of this book is that thermodynamics and statistical mechanics will benefit from replacing the unfortunate, misleading and mysterious term “entropy” with a more familiar, meaningful and appropriate term such as information, missing information or uncertainty. This replacement would facilitate the interpretation of the “driving force” of many processes in terms of informational changes and dispel the mystery that has always enshrouded entropy.

It has been 140 years since Clausius coined the term “entropy”; almost 50 years since Shannon developed the mathematical theory of “information” — subsequently renamed “entropy”. In this book, the author advocates replacing “entropy” by “information”, a term that has become widely used in many branches of science.

The author also takes a new and bold approach to thermodynamics and statistical mechanics. Information is used not only as a tool for predicting distributions but as the fundamental cornerstone concept of thermodynamics, held until now by the term “entropy”.

The topics covered include the fundamentals of probability and information theory; the general concept of information as well as the particular concept of information as applied in thermodynamics; the re-derivation of the Sackur-Tetrode equation for the entropy of an ideal gas from purely informational arguments; the fundamental formalism of statistical mechanics; and many examples of simple processes the “driving force” for which is analyzed in terms of information.

Contents:

• Elements of Probability Theory
• Elements of Information Theory
• Transition from the General MI to the Thermodynamic MI
• The Structure of the Foundations of Statistical Thermodynamics
• Some Simple Applications

Readership: Anyone interested in the sciences, students, researchers; as well as the layman.

Chapter 1

Introduction

As stated in the preface, the aim of this book is to show that statistical thermodynamics will benefit from the replacement of the concept of entropy by “information” (or any of its equivalent terms; see also Section 1.3 and Chapter 3). This will not only simplify the interpretation of what entropy is, but will also bring an end to the mystery that has befogged the concept of entropy for over a hundred years. In the first section (1.1) of this chapter, we shall briefly survey the historical milestones in the understanding of the key concept of thermodynamics: temperature and entropy. I shall try to convince the reader that had the molecular theory of gases (the so-called kinetic theory of heat) preceded the definition of the absolute temperature, the entropy would have been defined as a dimensionless quantity which, in turn, would have eased the acceptance of the informational interpretation of the entropy. In the following two sections (1.2 and 1.3), we shall discuss the pros and cons of the two main groups of interpretations of the entropy—one based on the idea of order-disorder (or any related concept), and the second on the concept of information or missing information (or any related concept).

1.1 A Brief History of Temperature and Entropy

Temperature and entropy are the two fundamental quantities that make the entire field of thermodynamics deviate from other branches of physics. Both of these are statistical quantities, crucially dependent on the existence of atoms and their properties. Temperature, like pressure, is an intensive quantity; both can be felt by our senses. Entropy, like volume, is an extensive quantity depending on the size of the system. Unlike pressure and volume, temperature and entropy (as well as the Second Law), would not have existed had matter not been made up of an immense number of atoms and molecules. It is true that both temperature and entropy were defined and measured without any reference to the atomic constituency of matter. However, the understanding of these quantities and in fact, their very existence, is dependent on the atomic constituency of matter.

Perhaps, the first and the simplest quantity to be explained by the dynamics of the particles is the pressure of a gas. The pressure is defined as the force exerted on a unit area. This definition is valid not only without reference to the atomic constituency of matter, but also if matter were not atomistic at all, i.e., if matter were continuous. Not so for the temperature. Although temperature can be sensed and measured without reference to atoms, its very existence and certainly its explanation is intimately dependent on the atomistic constituency of matter. During the 19^th century, it was widely believed that heat was a kind of substance that flows from a hot to a cold body. The association of the notion of temperature with motions of particles was one of the most important achievements of scientists of the late 19^th century.

Robert Boyle (1660) found that at a given temperature of the gas, the product of the volume and pressure is constant. This is now known as the Boyle–Marriote law:

Pressure could easily be explained by the particles incessantly bombarding the walls of the container. The pressure P is defined as force F per unit area, i.e.,

The force exerted by a moving particle of mass m and velocity v_x colliding with a wall perpendicular to the x-axis is equal to

Figure 1.1. Volume as a function of pressure for a gas at different temperatures. Temperature increases in the direction of the arrow.

the change in the momentum per unit of time. An elementary argument¹ leads to the expression for the pressure in terms of the average square velocity of the particles²

where ρ is the number density of the gas (i.e., the number of particles per unit volume), P is the (scalar) pressure, and

This explanation was first given by Daniel Bernoulli (1732), long before the mechanical interpretation of the temperature.

Although the sense of hot and cold has been experienced since time immemorial, its measurement started only in the 17^th and 18^th centuries. Various thermometers were designed by Ole Christensen

Rømer (1702), Daniel Gabriel Fahrenheit (1714), Anders Celsius (1742), and many others.

Following the invention of the thermometer, it became possible to make precise measurements of the temperature. In the early 19^th century, Jacques Charles and Joseph Louis Gay-Lussac discovered the experimental law that at constant pressure, the volume is linear in the temperature t (in °C), i.e.,

where C is a constant, proportional to the amount of gas.

From the graphs in Figure 1.2, it is clear that if we extrapolate to lower temperature all the curves converge to a single point. This led to the realization that there exists a minimal temperature, or an absolute zero temperature.

Combining the two empirical laws, one can write the equation of state of the ideal gas as:

Figure 1.2. Volume as a function of temperature at different pressures. The pressure decreases in the direction of the arrow. where T is the absolute temperature, n the number of moles, and the “constant,” R, for one mole of gas is

Having a statistical mechanical theory of pressure, i.e., an interpretation of the pressure in terms of the average kinetic energy of the particles, on the one hand, and the equation of state of the gas on the other, one could derive the expression for the temperature in terms of the average kinetic energy of the particles. Thus, from (1.1.3) and (1.1.6), one can derive the equation

where k_B = 1.380 × 10⁻23 J/K is now referred to as the Boltzmann constant, and N is the number of particles of the gas. In 1811, Amedeo Avogadro published his hypothesis that the average number of molecules in a given volume of any gas is the same (at a fixed temperature and pressure). With this finding, one can relate the Boltzmann constant k_B to the gas constant R by the relation

where N_AV = 6.023 × 10²³ (particles/mol) is the Avogadro number.

Sadi Carnot published a seminal work on the ideal heat engine in 1824.³ Carnot's prosaic style of writing was reformulated in mathematical terms by Clapeyron (1834). The latter laid the basis for the formulation of the Second Law by Clausius and by Kelvin.

William Thomson, later known as Lord Kelvin, established the existence of the absolute temperature (1854) and showed that by extrapolating from Charles’ and Gay-Lussac's law (see Figure 1.2), there exists an absolute zero temperature at about −273°C (today the Kelvin scale is defined in such a way that its value at the triple point of water is, by definition, 273.16 K).

During that period, the First Law of Thermodynamics was established by many workers, notably by Julius Robert Mayer, William Thomsom, James Prescott Joule, and John James Waterston.

The Second Law was first formulated by Clausius in 1849. A second formulation by William Thomson (Lord Kelvin) followed in 1850. In 1854, Clausius studied the quantity dQ/T in connection with heat engines, where dQ is a small amount of heat transferred and T , the absolute temperature. However, the concept of entropy was introduced only in 1863.

James Clerk Maxwell published the dynamical theory of gases in 1860. This was extended by Ludwig Boltzmann in 1867. These works established both the existence and the form of the equilibrium velocity distribution of the molecules in the gas, known today as the Maxwell–Boltzmann distribution. This distribution established the connection between the average kinetic energy of the molecules with the absolute temperature.

During the late 19^th and the early 20^th centuries, the molecular interpretation of the entropy was developed by Ludwig Boltzmann and by Josiah Willard Gibbs. At that time, the molecular theory of matter was far from being universally accepted. A turning point in the acceptance of the atomistic theory of matter was the publication of the theory of Brownian motion by Albert Einstein (1905) followed by the experimental corroboration by Jean Perrin (1910).

As we have seen in this brief history of temperature and entropy, the concept of temperature, the various thermometers and the various scales came before the establishment of the Maxwell–Boltzmann (MB) distribution of velocities. The acceptance of the Maxwell–Boltzmann distribution as well as Boltzmann's expression for the entropy came later. It is clear from this sequence of events that the units of temperature were determined at least two decades before the recognition and acceptance of the molecular interpretation of the temperature.

When Clausius formulated the Second Law in terms of dQ/T , it was only natural to define the entropy in units of energy divided by temperature. From that, it followed that Boltzmann's entropy had to be defined with the constant k_B (later known as Boltzmann's constant) having the same units as Clausius’ entropy, i.e., energy divided by temperature or Joule/Kelvin, or J/K.

Events could have unfolded differently had the identification of temperature with the average velocity of the atoms come earlier, in which case it would have been more natural to define temperature in units of energy,⁴ i.e., one would have arrived at the identity

(instead of 3k_B T/2 on the right-hand side). Having this identification of the temperature would not have any effect on the formal expression of either the efficiency of Carnot's engine,

or on the definition of Clausius’ entropy,

The only difference would have been that the entropy S would now be a dimensionless quantity. It would still be a state functio...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Dedication Page
• Contents
• List of Abbreviations
• Preface
• Chapter 1 - Introduction
• Chapter 2 - Elements of Probability Theory
• Chapter 3 - Elements of Information Theory
• Chapter 4 - Transition from the General MI to the Thermodynamic MI
• Chapter 5 - The Structure of the Foundations of Statistical Thermodynamics
• Chapter 6 - Some Simple Applications
• Appendices
• References
• Index