STRATA
Skip to study
XLVII

XLVII / Experiments

Workbench

Change a condition, inspect what happens, and keep the result. Six experiments connect an equation with a visible outcome, a numerical record, and questions you can pursue.

Evidence and base rates

How much should a flag change your estimate?

A factory inspects a batch of 10,000 components. Its screen detects some defective items and falsely flags some sound ones. Before interpreting a flag, account for both groups; a good detection rate alone does not tell you how many flagged items are defective.

Follow the population into the flagged group.

DefectiveSound
All 10,000 items before screening100 defective; 9900 soundOnly the 594 expected flagged items99 true flags; 495 false flags
The top bar is normalized to the whole batch. The lower bar is separately normalized to flagged items, so its orange share is the probability of a defect given a flag.
Defective among flagged
16.67%

The posterior probability after a flag.

Items flagged
594

Expected count in a batch of 10,000.

False flags
495

Sound items that the screen flags.

Missed defects
1.0

Defective items that pass the screen.

Every cell is an expected count in the same 10,000-item batch. Rows and columns conserve the whole population.
Actual conditionFlaggedPassedTotal
Actually defective991.0100
Actually sound49594059900
Total594940610000

Keep a reproducible record.

The JSON includes parameters, model assumptions, sources, and full results. The CSV contains a complete confusion table with unrounded expected counts.

Method

The calculation follows these rules.

Apply each conditional rate to its own population. True flags equal the defective population multiplied by the detection rate; false flags equal the sound population multiplied by the false-flag rate. Divide true flags by all flags to obtain the posterior probability.

P(defect | flag) = p × d / [p × d + (1 − p) × f].

Read the result within its assumptions.

  • These are expected counts for a hypothetical quality-control screen, not observations from a factory. Fractional counts are legitimate expectations.
  • Detection and false-flag rates are held constant within each group. The model does not include drift, repeated tests, or correlated screening errors.
  • If no item is expected to be flagged, conditioning on a flag has a zero denominator and the posterior is undefined.

Take the experiment further.

01

Read the denominator.

At the default settings, identify every item included in the flagged group. Explain why the posterior denominator is 594 rather than 100 or 10,000.

02

Choose the useful improvement.

For rare defects, compare improving detection from 99% to 100% with reducing false flags from 5% to 1%. Record which change alters the flagged group more.

03

Find the boundary.

Set both the defect rate and false-flag rate to zero. Explain why the screen produces no flagged group to interpret.

Read the sources behind the model.

  1. MIT 6.041, Conditioning and Bayes’ Rule

    John Tsitsiklis develops conditional probability and Bayes’ rule in a lecture with downloadable course material.

Watch one model run.

12 seconds · Original model visualization · Silent

Evidence and base rates

Follow a screening result back through the population that produced it.

Read the explanation and method

A hypothetical batch contains 10,000 components. Detection is fixed at 99 percent and false flags at 5 percent.

The defect rate rises from 1 percent toward 20 percent. Orange shows actual defects that are flagged; cream shows sound items that are falsely flagged.

The posterior is true flags divided by all flags. Changing the starting population changes this fraction even when both screening rates stay fixed.

Method. Apply each conditional rate to its own population. True flags equal the defective population multiplied by the detection rate; false flags equal the sound population multiplied by the false-flag rate. Divide true flags by all flags to obtain the posterior probability.

Apply the model to a research question.

These dossiers explain how this experiment relates to a larger question, including the observation or comparison that the model leaves unresolved.

Try another experiment.