Lecture 1 - Basic Concepts - Matematica Computazione per L'Ingegneria

The best way to begin understanding probability is through a concrete example. Consider the simplest experiment of all: tossing an unbiased coin. This experiment has only two mutually exclusive outcomes:

Heads ( $H$ ).
Tails ( $T$ ).

Because the factors influencing the outcome are far too complex and numerous to model deterministically, we regard the result as random.

What is the probability of each outcome? Since the coin is unbiased, both outcomes are equally likely. We therefore assign probability $\frac{1}{2}$ to each outcome.

Generalizing this idea to any experiment with a finite number of mutually exclusive and equally likely outcomes, we say that the outcomes are equiprobable and obtain the following classical definition.

The Frequentist Approach¶

The frequentist approach interprets probability in terms of the empirical behavior of relative frequencies observed over repeated trials. Under this viewpoint, we imagine that an experiment can be repeated indefinitely under identical conditions.

Suppose that an experiment is repeated $n$ times in a sequence of independent trials. If $n(A)$ denotes the number of trials in which event $A$ occurs, then the ratio

f(A)=\frac{n(A)}{n}

is called the relative frequency of the event $A$ .

The frequentist interpretation identifies the probability of $A$ with the limiting relative frequency, whenever this limit exists:

\mathbb{P}(A) = \lim_{n\to\infty}\frac{n(A)}{n}.

(2)

This interpretation leads naturally to the need to count possible and favorable outcomes. Such questions belong to the field of combinatorics.

Counting Things¶

Combinatorics encompasses a large collection of techniques for counting finite sets. We will use a few fundamental principles that are particularly useful in probability.

Fundamental Principles of Counting¶

2 (Proof)

For $r=2$ , the result reduces to the previous theorem.

Suppose the statement holds for $r-1$ . Then there are precisely

n_2n_3\cdots n_r

distinct $(r-1)$ -tuples $(b_{i_2},\dots,x_{i_r})$ containing one element of each of the last $r-1$ kinds.

Regard these $(r-1)$ -tuples as elements of a new kind. Each $r$ -tuple

(a_{i_1},b_{i_2},\dots,x_{i_r})

can then be regarded as consisting of one element $a_{i_1}$ of the first kind and one $(r-1)$ -tuple of the new kind. By the previous theorem, the number of such $r$ -tuples is therefore

n_1(n_2\cdots n_r) n_1n_2\cdots n_r.

The result follows by mathematical induction.

This leads to the following fundamental rule.

Sampling Strategies¶

When drawing $r$ objects from a population of $n$ distinct objects, the number of possible arrangements depends on whether replacement is allowed and whether order matters.

Sampling with replacement (ordered).
Each draw has $n$ possibilities, so the number of distinct ordered samples is
$N=n^r.$
Sampling without replacement (ordered).
Each successive draw reduces the number of available choices by one. Therefore, the number of ordered samples, also called $r$ -permutations, is
$N = n(n-1)(n-2)\cdots(n-r+1) = \frac{n!}{(n-r)!}.$
Special case. If $r=n$ , this gives the total number of permutations of $n$ objects:
$N=n!.$

Combinations and Partitions

We introduce the following notation for this quantity.

Example: Quality Control

A batch of 100 manufactured items is inspected by examining 10 items selected at random. If none of the 10 inspected items is defective, the entire batch is accepted. Otherwise, the batch is subjected to further inspection.

What is the probability that a batch containing 10 defective items will be accepted?

Solution. The number of ways of selecting 10 items from a batch of 100 items is

N=\binom{100}{10} = \frac{100!}{10!,90!}.

By hypothesis, all such selections are equally likely. Let $A$ denote the event that the batch is accepted. Since the batch contains 10 defective items, it contains 90 non-defective items. The batch is accepted precisely when all 10 inspected items are selected from these 90 non-defective items. Hence,

N(A)=\binom{90}{10} = \frac{90!}{10!,80!}.

It follows from the classical definition of probability that

\mathbb{P}(A)= \frac{N(A)}{N}= \frac{90!,90!}{80!,100!} = \frac{81\cdot82\cdots90}{91\cdot92\cdots100}.

Numerically,

\mathbb{P}(A) \approx \left(1-\frac{1}{10}\right)^{10} \approx \frac{1}{e},

where $e=2.718\ldots$ is the base of the natural logarithm.

4 (Proof)

The subsets are distinguished by their prescribed sizes or roles, but the order of the elements within each subset is irrelevant.

We construct the partition in stages. First, choose a subset of $n_1$ elements from the original set. This can be done in

N_1=\binom{n}{n_1}

ways.

Next, choose a subset of $n_2$ elements from the remaining $n-n_1$ elements. This can be done in

N_2=\binom{n-n_1}{n_2}

ways.

Proceeding in the same way, after $k-2$ stages there remain $n_{k-1}+n_k$ elements. The $(k-1)$ -st subset can therefore be chosen in

N_{k-1} \binom{n-n_1-\cdots-n_{k-2}}{n_{k-1}}

ways, after which the final subset is determined.

By the multiplication principle, the total number of partitions is

N \binom{n}{n_1} \binom{n-n_1}{n_2} \cdots \binom{n-n_1-\cdots-n_{k-2}}{n_{k-1}}.

Substituting the definition of the binomial coefficients gives

\begin{aligned} N &= \frac{n!}{n_1!(n-n_1)!} \frac{(n-n_1)!}{n_2!(n-n_1-n_2)!} \cdots \ &\qquad \cdots \frac{(n-n_1-\cdots-n_{k-2})!} {n_{k-1}!(n-n_1-\cdots-n_{k-1})!} \\ &= \frac{n!}{n_1!n_2!\cdots n_k!}. \end{aligned}

The Sample Space¶

We call the mutually exclusive outcomes of a random experiment elementary events (or sample points), and denote them by the Greek letter $\omega$ .

The set of all possible elementary events associated with a given experiment is called the sample space, denoted by $\Omega$ .

An event $A$ associated with $\Omega$ is a set of elementary outcomes such that, for every $\omega\in\Omega$ , we can unambiguously determine whether $\omega$ leads to the occurrence of $A$ . We therefore identify an event with the corresponding subset of the sample space.

Special cases of events include:

The sure (or certain) event: this event always occurs, regardless of the outcome of the experiment. It is the entire sample space $\Omega$ .
The impossible event: this event never occurs and is represented by the empty set $\emptyset$ . Operations on Events

Given two events $A_1$ and $A_2$ :

Equivalence: $A_1$ and $A_2$ are identical, or equivalent, if they occur under exactly the same circumstances. We write $A_1=A_2$ .
Mutual exclusivity: $A_1$ and $A_2$ are mutually exclusive (or incompatible) if they cannot occur simultaneously, that is,

A_1\cap A_2=\emptyset.

We use the standard set operations on events:

Union: $A_1\cup A_2$ is the event that at least one of $A_1$ or $A_2$ occurs. For a collection $A_1,A_2,\ldots$ , we write

\bigcup_k A_k.

Intersection: $A_1\cap A_2$ is the event that both $A_1$ and $A_2$ occur. For a collection $A_1,A_2,\ldots$ , we write

\bigcap_k A_k.

Difference: $A_1\setminus A_2$ is the event that $A_1$ occurs but $A_2$ does not.
Complement: the complement of $A$ , denoted by $\overline{A}$ or $A^c$ , is the event that $A$ does not occur. It is given by

\overline{A}=\Omega\setminus A.

Constructing a Probability Model¶

A random experiment by itself does not determine a probability distribution. To describe uncertainty mathematically, we need to specify not only what outcomes are possible, but also how probability is assigned to those outcomes.

This leads to the distinction between three fundamental ingredients:

\boxed{ \text{random experiment} \quad\longrightarrow\quad \text{sample space} \quad\longrightarrow\quad \text{probability measure}. }

The random experiment is the physical, computational, or conceptual procedure whose outcome is uncertain. The sample space $\Omega$ specifies the set of all possible outcomes of the experiment. The probability measure $\mathbb P$ assigns probabilities to the events that can occur.

Together, these objects form a probability model,

(\Omega,\mathcal F,\mathbb P),

where $\mathcal F$ is the collection of events to which probabilities are assigned.

The distinction is important because specifying the sample space alone does not specify how likely its outcomes are.

The probability model of a die

Consider the experiment of throwing a six-sided die.

A natural sample space is

\Omega=\{1,2,3,4,5,6\}.

However, this does not yet tell us the probabilities of the six possible outcomes.

If the die is assumed to be fair, we use the probability model

\mathbb P(\{k\})=\frac16, \qquad k=1,\ldots,6.

For example, if

A=\{2,4,6\},

then

\mathbb P(A)=\frac36=\frac12.

The same sample space could describe a loaded die. For example, another probability model might assign

\begin{aligned} \mathbb P(\{1\})&=0.10,\\ \mathbb P(\{2\})&=0.15,\\ \mathbb P(\{3\})&=0.20,\\ \mathbb P(\{4\})&=0.20,\\ \mathbb P(\{5\})&=0.15,\\ \mathbb P(\{6\})&=0.20. \end{aligned}

The possible outcomes have not changed: the sample space is still $\Omega$ . What has changed is the probability measure.

Thus, the statement

\Omega=\{1,2,3,4,5,6\}

does not imply that the six outcomes are equally likely. Equal probabilities are an additional modeling assumption.

Probability models and physical experiments¶

In applications, the probability measure is usually not obtained from the experiment alone. It is a mathematical model based on available information, experimental data, physical considerations, or simplifying assumptions.

For example, consider the lifetime of a mechanical component. The experiment consists of selecting a component and observing how long it operates before failure. A natural sample space is

\Omega=(0,\infty),

since the lifetime is a positive quantity.

An event might be

A=\{\omega\in\Omega:\omega>1000\},

corresponding to the component surviving more than 1000 hours.

The sample space tells us which lifetimes are possible, but it does not determine the probability

\mathbb P(A).

To obtain this probability, we need a model for the distribution of component lifetimes. Such a model could be based on reliability experiments or on a physical theory of the failure mechanism.

This distinction is particularly important in engineering and computational applications. The mathematical model is an idealization of a physical system, and its usefulness depends on how well it represents the features of the experiment that are relevant to the question being studied.

Different models for the same experiment¶

The same physical experiment can sometimes be represented by different probability models, depending on the level of detail that is relevant.

For example, suppose that the experiment consists of measuring the diameter of a manufactured shaft.

At one level, we might record the actual diameter,

\Omega=(0,\infty).

At another level, if the manufacturing process only distinguishes whether the shaft satisfies a tolerance specification, we could use the simpler sample space

\Omega=\{\text{within tolerance},\text{out of tolerance}\}.

Both descriptions refer to the same physical process, but they answer different questions.

The first model retains quantitative information about the diameter, while the second records only whether the component satisfies the required specification.

Thus, constructing a probability model involves deciding which aspects of the random experiment need to be represented.

Discrete and continuous models¶

The nature of the sample space also depends on the experiment.

For a finite experiment, the sample space may contain only finitely many outcomes, such as

\Omega=\{1,2,\ldots,6\}

for a die.

A countably infinite sample space can arise when the possible outcomes are

\Omega=\{0,1,2,\ldots\}.

For example, an outcome might represent the number of trials required before a particular event occurs.

In other experiments, the possible outcomes form an interval or another uncountable set. For example, a measured time or length might take values in

\Omega=[0,1].

These different types of sample spaces require different ways of specifying the probability measure. In particular, when the sample space is continuous, individual outcomes can have probability zero even though the corresponding outcomes are possible.

The distinction between discrete and continuous probability models will become central when we introduce random variables and their probability distributions.

From an experiment to a mathematical model¶

The construction of a probability model can therefore be viewed as a sequence of questions:

\boxed{ \begin{aligned} &\text{What is the random experiment?}\\ &\text{What are the possible outcomes?}\\ &\text{Which collections of outcomes are events?}\\ &\text{How should probability be assigned to those events?} \end{aligned} }

The first three questions determine the structure of the model, while the last specifies the probability measure.

A well-chosen probability model should capture the aspects of the experiment that are relevant to the problem under consideration. Once the model has been specified, the axioms of probability provide the rules that all probability calculations must satisfy.

In the following lectures, we will develop additional tools for working within such models. In particular, we will see how probabilities change when additional information is available, leading naturally to the concepts of conditional probability, Bayes’ theorem, and independence.

From the Algebra of Events to the Computation of Probabilities¶

We now derive the basic algebraic rules satisfied by probabilities.

We assume that all events under consideration have well-defined probabilities. Moreover, all events obtained from a given collection of events by taking unions, intersections, differences, and complements are also assumed to have well-defined probabilities.

Consider two mutually exclusive events $A_1$ and $A_2$ , and let

A=A_1\cup A_2.

Suppose that we repeat the experiment a large number of times under identical conditions. Let $n$ be the total number of trials, and let $n(A_1)$ , $n(A_2)$ , and $n(A)$ denote the numbers of trials in which $A_1$ , $A_2$ , and $A$ occur, respectively.

Because $A_1$ and $A_2$ are mutually exclusive, whenever $A$ occurs, exactly one of $A_1$ or $A_2$ occurs. Therefore,

n(A)=n(A_1)+n(A_2),

and hence

\frac{n(A)}{n} = \frac{n(A_1)}{n} + \frac{n(A_2)}{n}.

Passing to the corresponding probabilities gives

\mathbb{P}(A) = \mathbb{P}(A_1)+\mathbb{P}(A_2).

Similarly, if $A_1$ , $A_2$ , and $A_3$ are mutually exclusive, then

\mathbb{P}(A_1\cup A_2\cup A_3) = \mathbb{P}(A_1) + \mathbb{P}(A_2) + \mathbb{P}(A_3).

More generally, for $n$ mutually exclusive events $A_1,\ldots,A_n$ , we obtain the addition law for probabilities:

\mathbb{P}\left(\bigcup_{k=1}^n A_k\right) = \sum_{k=1}^n\mathbb{P}(A_k).

5 (Proof)

The inequality

0\leqslant\mathbb{P}(A)\leqslant1

follows from the fact that every relative frequency satisfies

0\leqslant\frac{n(A)}{n}\leqslant1.

For two arbitrary events $A_1$ and $A_2$ , we can decompose them as

A_1 = (A_1\setminus A_2) \cup (A_1\cap A_2),

A_2 = (A_2\setminus A_1) \cup (A_1\cap A_2),

and

A_1\cup A_2 = (A_1\setminus A_2) \cup (A_2\setminus A_1) \cup (A_1\cap A_2).

The three events on the right-hand side are mutually exclusive. Applying the addition law therefore gives

\mathbb{P}(A_1) = \mathbb{P}(A_1\setminus A_2) + \mathbb{P}(A_1\cap A_2),

\mathbb{P}(A_2) = \mathbb{P}(A_2\setminus A_1) + \mathbb{P}(A_1\cap A_2),

and

\mathbb{P}(A_1\cup A_2) = \mathbb{P}(A_1\setminus A_2) + \mathbb{P}(A_2\setminus A_1) + \mathbb{P}(A_1\cap A_2).

Rearranging the first two equations gives

\mathbb{P}(A_1\setminus A_2) = \mathbb{P}(A_1)-\mathbb{P}(A_1\cap A_2),

and

\mathbb{P}(A_2\setminus A_1) = \mathbb{P}(A_2)-\mathbb{P}(A_1\cap A_2).

Substituting these expressions into the formula for $\mathbb{P}(A_1\cup A_2)$ yields

\begin{aligned} \mathbb{P}(A_1\cup A_2) &= \mathbb{P}(A_1)-\mathbb{P}(A_1\cap A_2) +\mathbb{P}(A_2)-\mathbb{P}(A_1\cap A_2) \\ &\qquad +\mathbb{P}(A_1\cap A_2) \\ &= \mathbb{P}(A_1) + \mathbb{P}(A_2) - \mathbb{P}(A_1\cap A_2). \end{aligned}

Finally, suppose that $A_1\subseteq A_2$ . Then

A_1\cap A_2=A_1.

Hence,

\mathbb{P}(A_2) = \mathbb{P}(A_1) + \mathbb{P}(A_2\setminus A_1),

and therefore

\mathbb{P}(A_1) = \mathbb{P}(A_2) - \mathbb{P}(A_2\setminus A_1) \leqslant \mathbb{P}(A_2).

More generally, Theorem 5 can be used to consider the case in which $A_1,\ldots,A_n$ are pairwise disjoint events, then

\mathbb P\left(\bigcup_{i=1}^n A_i\right) = \sum_{i=1}^n\mathbb P(A_i).

(3)

Finite additivity is particularly useful when an event can be decomposed into simpler, mutually exclusive events.

If two events are not disjoint, we cannot simply add their probabilities. Indeed, if $A$ and $B$ overlap, then the outcomes in $A\cap B$ would be counted twice:

\mathbb P(A)+\mathbb P(B).

To correct this double counting, we subtract the probability of the intersection using the complete formula from Theorem 5.

Inclusion-Exclusion Principle¶

The addition law can be generalized to events that are not mutually exclusive.

6 (Proof)

For $n=2$ , the statement reduces to the formula for two events proved above.

Suppose the formula holds for $n-1$ events. Then

\begin{aligned} \mathbb{P}\left(\bigcup_{k=2}^n A_k\right) &= \sum_{i=2}^n\mathbb{P}(A_i) - \sum_{2\leqslant i<j\leqslant n} \mathbb{P}(A_i\cap A_j) \\ &\quad+ \sum_{2\leqslant i<j<k\leqslant n} \mathbb{P}(A_i\cap A_j\cap A_k) -\cdots. \end{aligned}

Moreover,

\begin{aligned} \mathbb{P}\left( A_1\cap\bigcup_{k=2}^n A_k \right) &= \sum_{i=2}^n\mathbb{P}(A_1\cap A_i) - \sum_{2\leqslant i<j\leqslant n} \mathbb{P}(A_1\cap A_i\cap A_j) \\ &\quad+ \sum_{2\leqslant i<j<k\leqslant n} \mathbb{P}(A_1\cap A_i\cap A_j\cap A_k) -\cdots. \end{aligned}

Applying the two-event addition formula to $A_1$ and $\bigcup_{k=2}^n A_k$ gives

\mathbb{P}\left(\bigcup_{k=1}^n A_k\right) = \mathbb{P}(A_1) + \mathbb{P}\left(\bigcup_{k=2}^n A_k\right) - \mathbb{P}\left( A_1\cap\bigcup_{k=2}^n A_k \right).

Substituting the two expansions above and collecting terms yields

\begin{aligned} \mathbb{P}\left(\bigcup_{k=1}^n A_k\right) &= \mathbb{P}(A_1) +\sum_{i=2}^n\mathbb{P}(A_i) -\sum_{2\leqslant i<j\leqslant n}\mathbb{P}(A_i\cap A_j) \\ &\quad -\sum_{i=2}^n\mathbb{P}(A_1\cap A_i) +\sum_{2\leqslant i<j\leqslant n} \mathbb{P}(A_1\cap A_i\cap A_j) -\cdots \\ &= P_1-P_2+P_3-\cdots+(-1)^{n+1}P_n. \end{aligned}

This completes the induction.

Multiple failure modes in an engineering component

Consider a manufacturing process for mechanical components. A component may exhibit three different types of defects:

$A$ : a dimensional defect;
$B$ : a surface defect;
$C$ : a material defect.

Suppose that quality-control data give the following probabilities:

\begin{aligned} \mathbb P(A)&=0.05,\\ \mathbb P(B)&=0.04,\\ \mathbb P(C)&=0.03, \end{aligned}

together with the pairwise and triple intersections

\begin{aligned} \mathbb P(A\cap B)&=0.012,\\ \mathbb P(A\cap C)&=0.008,\\ \mathbb P(B\cap C)&=0.006, \end{aligned}

and

\mathbb P(A\cap B\cap C)=0.003.

We want to determine the probability that a randomly selected component has at least one type of defect.

The event of interest is

A\cup B\cup C.

The three events are not assumed to be disjoint, so we use the inclusion–exclusion principle:

\begin{aligned} \mathbb P(A\cup B\cup C) ={}& \mathbb P(A)+\mathbb P(B)+\mathbb P(C)\\ &-\mathbb P(A\cap B) -\mathbb P(A\cap C) -\mathbb P(B\cap C)\\ &+\mathbb P(A\cap B\cap C). \end{aligned}

Substituting the data gives

\begin{aligned} \mathbb P(A\cup B\cup C) ={}& 0.05+0.04+0.03\\ &-0.012-0.008-0.006\\ &+0.003\\ ={}&0.097. \end{aligned}

Therefore,

\mathbb P(A\cup B\cup C)=0.097.

Thus, approximately $9.7\%$ of the manufactured components have at least one of the three types of defects.

The different terms in the formula correct successive levels of multiple counting. When we first add

\mathbb P(A)+\mathbb P(B)+\mathbb P(C),

a component having two defects is counted twice, while a component having all three defects is counted three times. We therefore subtract the three pairwise intersections. However, a component having all three defects has then been subtracted three times, so it has been removed completely from the count. We therefore add the triple intersection once more.

The inclusion–exclusion principle can thus be viewed as a systematic correction for overlapping events:

\text{single events} - \text{pairwise overlaps} + \text{triple overlap}.

Notice that the probabilities of the intersections are supplied directly by the quality-control data, we will discuss in the following how the probability of intersection works in general.

Infinite Sequences and Continuity Properties¶

The addition law for disjoint events is not restricted to a finite number of events. In many probability models, however, it is natural to consider a sequence of events

A_1,A_2,A_3,\ldots

rather than only a finite collection.

This occurs, for example, when an experiment can produce infinitely many possible outcomes, or when we are interested in whether an event eventually occurs during a sequence of trials. Consider an experiment that is repeated indefinitely and let $A_k$ denote the event that a specified outcome occurs for the first time on the $k$ -th trial. The event that the outcome occurs at some point during the experiment is then

\bigcup_{k=1}^{\infty}A_k.

If the events $A_1,A_2,\ldots$ are mutually exclusive, exactly one of them can occur. We would therefore expect the probability of their union to be obtained by adding their individual probabilities, just as in the finite case.

This leads naturally to the question of whether the addition law should continue to hold when the number of events is infinite. The answer is one of the fundamental principles of probability: probability is countably additive.

More precisely, if $A_1,A_2,\ldots$ are pairwise disjoint events, then

\mathbb{P}\left(\bigcup_{k=1}^{\infty}A_k\right) = \sum_{k=1}^{\infty}\mathbb{P}(A_k).

This is the countable additivity property of probability.

It is important to distinguish this statement from the finite addition law. For any finite collection of pairwise disjoint events,

\mathbb{P}\left(\bigcup_{k=1}^{n}A_k\right) = \sum_{k=1}^{n}\mathbb{P}(A_k),

whereas countable additivity concerns the limit of such finite unions:

\bigcup_{k=1}^{\infty}A_k = \lim_{n\to\infty} \bigcup_{k=1}^{n}A_k.

Thus, countable additivity allows the probability model to remain consistent when we pass from finite collections of events to infinite sequences of mutually exclusive events.

This property is essential because infinite sequences arise naturally in probability. Repeated experiments, waiting times, random walks, convergence questions, and many other probabilistic constructions require us to consider what happens after arbitrarily many trials. Countable additivity provides the mathematical rule that connects these infinite constructions with the probabilities of the individual events.

7 (Proof)

Define

B_1=A_1,

and, for $n\geqslant2$ ,

B_n=A_n\setminus A_{n-1}.

Since the sequence is increasing, the events $B_1,B_2,\dots$ are mutually exclusive. Moreover,

\bigcup_{k=1}^n B_k=A_n

and

\bigcup_{k=1}^{\infty}B_k = \bigcup_{k=1}^{\infty}A_k.

By countable additivity,

\begin{aligned} \mathbb{P}\left(\bigcup_{k=1}^{\infty}A_k\right) &= \mathbb{P}\left(\bigcup_{k=1}^{\infty}B_k\right) \\ &= \sum_{k=1}^{\infty}\mathbb{P}(B_k) \\ &= \lim_{n\to\infty} \sum_{k=1}^n\mathbb{P}(B_k) \\ &= \lim_{n\to\infty} \mathbb{P}\left(\bigcup_{k=1}^nB_k\right) \\ &= \lim_{n\to\infty}\mathbb{P}(A_n). \end{aligned}

Countable additivity gives us a way to compute the probability of a countable union of disjoint events. In applications, however, we often encounter sequences of events that are not disjoint but instead become progressively smaller or more restrictive.

For example, suppose that $A_n$ represents the event that an experiment satisfies a certain condition after $n$ successive tests. As more tests are performed, the condition may become increasingly restrictive, leading to a decreasing sequence

A_1\supseteq A_2\supseteq A_3\supseteq\cdots.

The event that satisfies the condition at every stage, no matter how far the sequence is continued, is

\bigcap_{k=1}^{\infty}A_k.

It is therefore natural to ask whether the probability of this limiting event can be obtained by taking the limit of the probabilities of the successive events:

\mathbb{P}\left(\bigcap_{k=1}^{\infty}A_k\right) \stackrel{?}{=} \lim_{n\to\infty}\mathbb{P}(A_n).

This question is the event-theoretic analogue of a familiar idea from analysis: when a sequence of sets becomes smaller and smaller and converges to a limiting set, we would like the corresponding probabilities to converge to the probability of that limiting set.

For decreasing sequences of events, this is indeed the case.

8 (Proof)

Taking complements gives the increasing sequence

A_1^c\subseteq A_2^c\subseteq\cdots.

By continuity from below,

\mathbb{P}\left(\bigcup_{k=1}^{\infty}A_k^c\right) = \lim_{n\to\infty}\mathbb{P}(A_n^c).

Using De Morgan’s law and $\mathbb{P}(A^c)=1-\mathbb{P}(A)$ , we obtain

\begin{aligned} \mathbb{P}\left(\bigcap_{k=1}^{\infty}A_k\right) &= 1-\mathbb{P}\left(\bigcup_{k=1}^{\infty}A_k^c\right) \\ &= 1-\lim_{n\to\infty}\mathbb{P}(A_n^c) \\ &= \lim_{n\to\infty} \left[1-\mathbb{P}(A_n^c)\right] \\ &= \lim_{n\to\infty}\mathbb{P}(A_n). \end{aligned}

The result is called continuity from above because the events decrease as the index increases, while their probabilities converge to the probability of their limiting intersection.

This property will be useful whenever an event is described as satisfying an infinite sequence of increasingly restrictive conditions. It also illustrates an important feature of a probability measure: countable additivity is not only a rule for adding probabilities, but also provides a consistent notion of taking limits of events.

The inclusion–exclusion principle gives an exact expression for the probability of a finite union of events by correcting for overlaps. For two events,

\mathbb{P}(A\cup B) = \mathbb{P}(A)+\mathbb{P}(B)-\mathbb{P}(A\cap B).

The correction term is necessary because adding $\mathbb{P}(A)$ and $\mathbb{P}(B)$ counts the outcomes in $A\cap B$ twice.

For a larger collection of events, inclusion–exclusion requires increasingly complicated intersections. In particular, for a countably infinite sequence

A_1,A_2,\ldots,

an exact inclusion–exclusion formula would involve intersections of two events, three events, four events, and so on.

In many applications, however, we do not need an exact value. It is enough to have a simple upper bound for the probability that at least one of the events occurs.

The basic idea is straightforward: if we add the probabilities

\mathbb{P}(A_1)+\mathbb{P}(A_2)+\cdots,

any outcome belonging to several of the events may be counted more than once. Therefore, the sum can overestimate the probability of the union, but it cannot underestimate it.

This leads to the following fundamental bound.

9 (Proof)

Define the mutually exclusive events

B_1=A_1, \qquad B_n=A_n\setminus\bigcup_{k=1}^{n-1}A_k, \quad n\geqslant2.

Then $B_n\subseteq A_n$ for every $n$ , and

\bigcup_{k=1}^{\infty}B_k = \bigcup_{k=1}^{\infty}A_k.

By monotonicity,

\mathbb{P}(B_k)\leqslant\mathbb{P}(A_k).

Therefore,

\begin{aligned} \mathbb{P}\left(\bigcup_{k=1}^{\infty}A_k\right) &= \mathbb{P}\left(\bigcup_{k=1}^{\infty}B_k\right) \\ &= \sum_{k=1}^{\infty}\mathbb{P}(B_k) \\ &\leqslant \sum_{k=1}^{\infty}\mathbb{P}(A_k). \end{aligned}

The result is also called the union bound. It is particularly useful because it does not require the events to be disjoint or independent, and it avoids the need to compute any intersection probabilities.

For a finite collection, the same idea gives

\mathbb{P}\left(\bigcup_{k=1}^{n}A_k\right) \leqslant \sum_{k=1}^{n}\mathbb{P}(A_k).

Thus, the probability of at least one event occurring is never larger than the sum of the individual probabilities.

The inequality becomes an equality when the events are pairwise disjoint, since in that case there is no multiple counting. When the events overlap, the sum generally overestimates the probability of the union.

The importance of Boole’s inequality goes beyond its simple proof. It provides a way of controlling complicated events by decomposing them into simpler events. This idea will recur throughout probability theory: rather than computing a probability exactly, we can often obtain a useful bound by covering the event of interest with a collection of simpler events.

Reliability of a system with multiple failure modes

Consider an engineering system monitored during operation. Suppose that, during a given operating period, the following five undesirable events may occur:

\begin{aligned} A_1 &: \text{failure of the power supply},\\ A_2 &: \text{failure of the cooling system},\\ A_3 &: \text{failure of a temperature sensor},\\ A_4 &: \text{failure of a pressure sensor},\\ A_5 &: \text{failure of the control unit}. \end{aligned}

Assume that reliability testing provides the following probabilities:

\begin{aligned} \mathbb P(A_1)&=0.002,\\ \mathbb P(A_2)&=0.001,\\ \mathbb P(A_3)&=0.003,\\ \mathbb P(A_4)&=0.002,\\ \mathbb P(A_5)&=0.001. \end{aligned}

We are interested in the probability that at least one failure occurs during the operating period. This is the event

A_1\cup A_2\cup A_3\cup A_4\cup A_5.

To compute this probability exactly, we would need information about intersections such as

A_1\cap A_2, \qquad A_1\cap A_3, \qquad A_2\cap A_5,

as well as higher-order intersections. In practice, such information may be difficult to obtain, particularly when the different failure mechanisms are dependent.

Boole’s inequality provides an immediate upper bound:

\begin{aligned} \mathbb P\left(\bigcup_{k=1}^{5}A_k\right) &\leqslant \sum_{k=1}^{5}\mathbb P(A_k)\\ &= 0.002+0.001+0.003+0.002+0.001\\ &= 0.009. \end{aligned}

Therefore,

\mathbb P\left(\text{at least one failure}\right)\leq 0.009.

Thus, even without knowing how the different failure mechanisms interact, we can conclude that the probability of at least one failure during the operating period is no greater than $0.9\%$ .

If two or more failure mechanisms can occur simultaneously, simply adding their probabilities counts those situations more than once. This is precisely why the sum provides an upper bound rather than, in general, an exact probability.

The example illustrates an important practical use of Boole’s inequality: when the exact probability of a complicated failure event is difficult to determine, we can obtain a conservative bound by decomposing it into simpler failure modes.