Change the histogram.
Keep repetition at zero and move the probability of A from 25% toward 95%. Record entropy and Huffman bits per symbol at three settings.

XLVII / Experiments
Change a condition, inspect what happens, and keep the result. Six experiments connect an equation with a visible outcome, a numerical record, and questions you can pursue.
Entropy and compression
A message can repeat individual symbols or repeat whole stretches of the same symbol. Those patterns reward different codes. Generate a four-symbol message, then compare a fixed two-bit alphabet, a Huffman code, and a simple run-length format on exactly the same sequence.
Per symbol, from the observed histogram.
Per symbol; the codebook is excluded.
Per symbol; each packet costs 10 bits.
336 packets cover 512 symbols.
| Symbol | Count | Observed share | Huffman code |
|---|---|---|---|
| A | 283 | 55.3% | 1 |
| B | 85 | 16.6% | 00 |
| C | 73 | 14.3% | 011 |
| D | 71 | 13.9% | 010 |
The JSON includes parameters, model assumptions, sources, and full results. The CSV contains one row per symbol, with its zero-based position.
Method
The histogram gives an order-zero Shannon entropy. Huffman coding repeatedly joins the two least frequent branches to build a prefix code. The run-length format stores a two-bit symbol and an eight-bit count; counts run from 1 to 255, and a longer run uses another packet.
H = −Σ pᵢ log₂ pᵢ; Huffman bits = Σ countᵢ × code lengthᵢ; run-length bits = 10 × packets.
Keep repetition at zero and move the probability of A from 25% toward 95%. Record entropy and Huffman bits per symbol at three settings.
Keep new draws evenly weighted and raise repetition. Explain why a symbol-frequency code and a run-length code respond differently.
For a 64-symbol message, estimate how many extra codebook bits would erase a small payload saving. State which storage costs your estimate includes.
The original paper defines the information measure behind the symbol-entropy calculation.
Lecture notes derive prefix codes, the greedy tree construction, and expected code length.
Compare symbol frequencies with the ordering that a code can exploit.
The first sweep changes the probability of A while drawing each symbol independently. The other three symbols share the remaining probability.
As one symbol dominates, the observed symbol entropy falls. Huffman codes use short words for frequent symbols; the comparison excludes headers and the codebook.
The second sweep gives all four new draws equal probability and increases repetition. Long runs reward run-length coding, even when a symbol histogram cannot describe the dependence.
Method. The histogram gives an order-zero Shannon entropy. Huffman coding repeatedly joins the two least frequent branches to build a prefix code. The run-length format stores a two-bit symbol and an eight-bit count; counts run from 1 to 255, and a longer run uses another packet.
These dossiers explain how this experiment relates to a larger question, including the observation or comparison that the model leaves unresolved.