# Bayes Theorem

Bayes Theorem can be derived from the laws of conditional probability. The conditional probability $P(B|A)$ is the probability, that event $B$ happens given event $A$ happened before. To illustrate this idea consider the example, where two dice are rolled. What is the probability of counting $5$ (event B), given that we rolled a $2$ with the first dye (event A)?

Table
+ B=1 2 3 4 5 6
A=1 2 3 4 5 6 7
2 3 4 5 6 7 8
3 4 5 6 7 8 9
4 5 6 7 8 9 10
5 6 7 8 9 10 11
6 7 8 9 10 11 12

The table shows, that the condition of event A happening before event B resticts the sample space to $P(A)$ (table: marked in red). In our example we can only get B (rolling the second die and getting 5 alltogether), if we roll a $3$ with the second die. The subset $P(A \cap B)$ (both A and B happen) us thus given by one item $\{5 \}$ only. The conditional probability is like any probability given by the suitable subset $A \cap B$ devided by the sample space $A$ or equivalently their corresponding propabilities:

\begin{align} P(B|A)=\frac{P(A \cap B)}{P(A)} \tag{1} \end{align}

Similarly we could ask, what the conditional probability $P(A|B)$ is and we find with the same proceedure. \begin{align} P(A|B)=\frac{P(A \cap B)}{P(B)} \tag{2} \end{align} Equating the two equations for $P(A \cap B)$, we get

Bayes Theorem

The conditional probability $P(A|B)$ is related to the conditional probability $P(B|A)$ by the factor $\frac{P(A)}{P(B)}$ \begin{align} P(A|B)=\frac{P(B|A) \cdot P(A)}{P(B)} \end{align} Using that $P(B)$ can be written as the sum of the conditional probabilities $P(A)P(B|A)+P(B)P(B|\bar{A})$, that does not contain $P(B)$ anymore \begin{align} P(A|B)=\frac{P(B|A) \cdot P(A)}{P(A)P(B|A)+P(B)P(B|\bar{A})} \end{align}

The relation $P(B)=P(A)P(B|A)+P(B)P(B|\bar{A})$ is easily proofed. From eq (2) we know, that the probability of $A$ not happening $\bar{A}$ and $B$ happening is \begin{align} P(\bar{A} \cap B)=P(B|\bar{A})P(\bar{A}) \end{align} and \begin{align} P(A \cap B)=P(B|A)P(A). \end{align} Hence, \begin{align} P(B)=P(A \cap B)+P(\bar{A} \cap B)=P(B|A)P(A)+P(B|\bar{A})P(\bar{A}) \end{align}

## Example

A living cell has to make a decision of wheather or not to produce a certain protein or not. The cell does this by infering the state of the environment though measurement of a signal $S$. The probability, that the environment contains high amount of sugar, given $S$, can be inferred by Bayes's Rule \begin{align} P(high|S)=\frac{P(S|high) \cdot P(high)}{P(high)P(S|high)+P(low)P(S|low)} \end{align}