STRATA
Skip to study
XLVII

X.04 / SHARED INSTRUMENTS

Statistical mechanics, entropy, and coarse-graining

What information about a microscopic state survives an average over many particles?

A source-led introduction2 source references1 related experiments

Start with the question.

Statistical mechanics relates many microscopic configurations to macroscopic descriptions. Coarse-graining specifies which distinctions are retained and which are discarded. Wilson's scale-based analysis and a physical memory-erasure experiment offer complementary ways to examine that choice. Entropy should therefore be accompanied by the state space, probability distribution, and physical conditions under which it is being calculated or measured.

Count microscopic arrangements within a coarse description.

Consider four binary positions with exactly two ones. There are six arrangements sharing that coarse count. If the description retains only the count, it cannot distinguish those arrangements. Add a second summary, such as whether the ones are adjacent, and inspect how the grouping changes without changing the underlying configurations.

What would you inspect?

  • The microscopic state space, coarse variables, and assigned probabilities.
  • Physical constraints and the system boundary used in a thermodynamic account.

Take the next step.

Create two coarse descriptions of the same small system and explain which predictions each can support and which distinctions it loses.

Put part of the question to work.

These models isolate a mechanism you can vary. Their assumptions define the part of this topic they can illuminate.

Try entropy and compression.

Change one parameter, compare the numerical output, and export the result. The experiment loads when you open it.

Open its full page

Watch the quantities change.

12 seconds · Original model visualization · Silent

Entropy and compression

Compare symbol frequencies with the ordering that a code can exploit.

Read the explanation and method

The first sweep changes the probability of A while drawing each symbol independently. The other three symbols share the remaining probability.

As one symbol dominates, the observed symbol entropy falls. Huffman codes use short words for frequent symbols; the comparison excludes headers and the codebook.

The second sweep gives all four new draws equal probability and increases repetition. Long runs reward run-length coding, even when a symbol histogram cannot describe the dependence.

Method. The histogram gives an order-zero Shannon entropy. Huffman coding repeatedly joins the two least frequent branches to build a prefix code. The run-length format stores a two-bit symbol and an eight-bit count; counts run from 1 to 255, and a longer run uses another packet.

Read the supporting material.

  1. Renormalization Group and Critical Phenomena. I. Renormalization Group and the Kadanoff Scaling Picture

    Kenneth G. Wilson · Scale transformations and critical phenomena; supports coarse-graining and effective descriptions.

  2. Experimental verification of Landauer's principle linking information and thermodynamics

    Antoine Bérut and coauthors · A one-bit physical memory experiment approaching an erasure dissipation bound.

Continue through the versioned study records.

Follow the comparisons.

B01

Renormalization-group flow

When fine-scale degrees of freedom are averaged out, which parameters still predict the same observable? Compare the error at successive scales and identify the range over which the effective model holds.

B03

Entropy and information

What is being counted in each definition of entropy? Follow the change from messages to physical states and identify the assumptions behind each expression.

Read the neighboring questions.