The Prosecutor's Fallacy

Exercise 2.10 (attributed to Peter Lee) in volume 1 of Kevin P. Murphy's Probabilistic Machine Learning concerns legal reasoning. It says:

Suppose a crime has been committed. Blood is found at the scene for which there is no innocent explanation. It is of a type which is present in 1% of the population.

The prosecutor claims: "There is a 1% chance that the defendant would have the crime blood type if he were innocent. Thus there is a 99% chance he is guilty". This is known as the prosecutor's fallacy. What is wrong with this argument?

In its essence, the prosecutor's fallacy is when one improperly assumes that \(\text{Pr}[A|B] = \text{Pr}[B|A]\). This error occurs in many scenarios, including epidemiology [0], law [1], ecology [2], and likely anywhere probability theory is used.

Let's work out exercise 2.10 (above) to get a better understanding of this issue.

Let $$ \begin{align} E = \text{rare blood type collected from the crime scene} \\ B = \text{defendant has rare blood type from crime scene} \\ I = \text{defendant is innocent} \\ \neg{I} = \text{defendant is guilty} \end{align} $$

Now let's try to phrase the prosecutor's argument mathematically. It appears that the prosecutor has incorrectly asserted that $$ \text{Pr}\left[B|I,E\right] = 0.01 \implies \text{Pr}\left[\neg I|B,E\right] = 0.99 $$ How did the prosecutor arrive at this implication?

Recall that we are interested in the probability that the defendant is innocent given the evidence we've actually observed, \(\text{Pr}\left[I | B, E\right]\). Using Bayes' theorem, we may write $$ \text{Pr}\left[I|B, E\right] = \frac{\text{Pr}\left[B|I, E\right]\,\text{Pr}\left[I|E\right]}{\text{Pr}\left[B|I,E\right]\,\text{Pr}\left[I|E\right]+\text{Pr}\left[B|\neg I, E\right]\,\text{Pr}\left[\neg I | E\right]} $$

Now let's consider the conditions under which the prosecutor's implication is true. For their argument to work, we would need \(\text{Pr}\left[I|B,E\right] = \text{Pr}\left[B|I,E\right]\).

Observe that \(\text{Pr}\left[B|I,E]\right]=\text{Pr}\left[B\right] = 0.01\). Now, if the defendant were guilty, the probability that the rare blood at the crime scene was theirs is now 100%, so \(\text{Pr}\left[B|\neg I, E\right] = 1\)

We can then rewrite the conditional probability of innocence like so: $$ \text{Pr}\left[I|B,E\right] = \frac{\text{Pr}\left[B|I,E\right]\text{Pr}\left[I|E\right]}{\text{Pr}\left[B|I,E\right]\text{Pr}\left[I|E\right] + (1 - \text{Pr}\left[I|E\right])} $$

Recall that the prosecutor assumes that \(\text{Pr}\left[I|B,E\right] = \text{Pr}\left[B|I,E\right]\). We can use this to rewrite the equation above: $$ \text{Pr}\left[I|B,E\right]\left(1 - \frac{\text{Pr}\left[I|E\right]}{\text{Pr}\left[B|I,E\right]\text{Pr}\left[I|E\right] + (1 - \text{Pr}\left[I|E\right])}\right) = 0 $$

This forces the parenthetical expression to vanish: $$ 1 - \frac{\text{Pr}\left[I|E\right]}{\text{Pr}\left[B|I,E\right]\text{Pr}\left[I|E\right] + (1 - \text{Pr}\left[I|E\right])} = 0 $$

Multiplying by the denominator and performing some algebra, we find that $$ \text{Pr}\left[I|E\right](\text{Pr}\left[B|I,E\right] - 2) + 1 = 0 $$ or $$ \text{Pr}\left[I|E\right] = \frac{1}{2 - \text{Pr}\left[B|I,E\right]} $$

We were given the probability \(\text{Pr}\left[B|I,E\right] = 0.01\), so \(\text{Pr}\left[I|E\right] = \frac{100}{199} \approx 0.5\). This whole derivation only works if we're willing to accept that the base rate of guilt is essentially a coin flip.

Presumption of innocence is considered a fundamental right in many countries across the world. The burden of proof that an individual is guilty falls upon the prosecutor. In a criminal trial like the one outlined in the example, the prosecution has to establish beyond any reasonable doubt that the accused is in fact guilty.

If we're on a jury and the prosecutor tries the argument we analyzed above, we can only accept it if we think that the base rate of innocence is 50%. This is incompatible with a presumption of innocence, and thus violates the defendant's rights.

References