Probability Fundamentals

Probability Fundamentals

Learning Objectives

Define fundamental probability terminology, including sample spaces, events, and set operations.
Apply the multiplication rule, permutations, and combinations to determine the size of a sample space.
Calculate the theoretical probability of an event using Kolmogorov's Axioms.
Apply the general and special addition rules to calculate the probability of the union of events.
Use the complement rule to determine the probability that an event does not occur.

Basic probability theory, sample spaces, events, counting rules, and probability rules.

In engineering, we rarely have perfect information. Probability provides a mathematical framework for quantifying uncertainty and making rational decisions when outcomes are unpredictable (e.g., predicting the exact lifespan of a bridge component).

Core Concepts and Set Theory

The foundational terminology and set theory operations used in probability.

Experiment

Any process or action that generates an observation or outcome (e.g., testing the compressive strength of a concrete cylinder).

Sample Space ( $S$ )

The set of all possible, mutually exclusive outcomes of an experiment. For example, when inspecting a welded joint, $S = \{\text{Acceptable}, \text{Defective}\}$ .

Event ( $E$ )

A subset of the sample space; a collection of specific outcomes. An event occurs if the outcome of the experiment is an element of that subset.

Venn Diagrams

A graphical representation of sets and their relationships. The sample space $S$ is typically represented by a rectangle, and events (subsets of $S$ ) are represented by circles drawn inside the rectangle. They are extremely useful for visualizing intersections, unions, and mutually exclusive events.

Set Operations (Unions, Intersections, Complements)

Operations used to combine and relate different events.

Union ( $A \cup B$ ): The event that either $A$ occurs, or $B$ occurs, or both occur. Contains all outcomes in $A$ or $B$ .
Intersection ( $A \cap B$ ): The event that both $A$ and $B$ occur simultaneously. Contains all outcomes common to both $A$ and $B$ .
Complement ( $A'$ or $A^c$ ): The event that $A$ does not occur. Contains all outcomes in $S$ that are not in $A$ .
Mutually Exclusive (Disjoint) Events: Two events that cannot occur simultaneously ( $A \cap B = \emptyset$ ).

Interactive Simulation

Interact with the Venn Diagram simulation below to visualize set operations such as unions, intersections, and complements.

Engineering Data Analysis • Topic 3

Venn Diagram & Set Operations

Set Operation Presets

Set Mathematical Notation

\varnothing \quad \text{(Empty Set)}

Set Operations Guide

• Click on any region of the Venn diagram (including Set A, Set B, the central intersection lens, or the surrounding Universal space) to toggle highlights and construct custom set formulas dynamically.

Counting Rules

Overview of Counting Rules

Techniques for determining the size of the sample space without listing every outcome.

When the sample space is large, counting outcomes manually is impractical. We use counting rules to systematically determine the total number of possible outcomes.

The Multiplication Rule (Fundamental Counting Principle)

If an operation can be performed in $n_1$ ways, and a second operation can be performed in $n_2$ ways, then the two operations can be performed together in $n_1 \times n_2$ ways.

Example: If a building design offers 3 foundation types and 4 framing materials, there are $3 \times 4 = 12$ distinct structural combinations.

Permutations (Order Matters)

An arrangement of objects in a specific order.

Permutations

The number of permutations of n distinct objects taken r at a time.

P(n, r) = \frac{n!}{(n - r)!}

Variables

Symbol	Description	Unit
$P(n, r)$	Number of permutations	-
$n$	Total number of distinct objects	-
$r$	Number of objects selected	-

Combinations (Order Does Not Matter)

A selection of objects without regard to order.

Combinations

The number of combinations of n distinct objects taken r at a time.

C(n, r) = \binom{n}{r} = \frac{n!}{r!(n - r)!}

Variables

Symbol	Description	Unit
$C(n, r)$	Number of combinations	-
$n$	Total number of distinct objects	-
$r$	Number of objects selected	-

Axioms and Rules of Probability

The fundamental mathematical laws that govern probability calculations.

Probability of an Event, $P(A)$

A numerical measure of the likelihood that an event will occur, ranging from 0 (impossible) to 1 (certain). If all outcomes in the sample space $S$ are equally likely, it is calculated as the ratio of outcomes in $A$ to the total outcomes in $S$ .

Probability of an Event

Calculates the theoretical probability of an event when all outcomes are equally likely.

P(A) = \frac{\text{Number of outcomes in } A}{\text{Total number of outcomes in } S}

Variables

Symbol	Description	Unit
$P(A)$	Probability of event A occurring	-
$A$	The specific event of interest	-
$S$	The sample space containing all possible outcomes	-

Kolmogorov's Axioms of Probability

The rigorous foundation of probability theory:

Axiom 1: For any event $A$ , $P(A) \ge 0$ .
Axiom 2: The probability of the entire sample space is 1 ( $P(S) = 1$ ).
Axiom 3: If $A_1, A_2, A_3, \dots$ are mutually exclusive events, then $P(A_1 \cup A_2 \cup A_3 \dots) = P(A_1) + P(A_2) + P(A_3) + \dots$

The Addition Rule

Used to calculate the probability of the union of two events.

General Addition Rule

A rule to calculate the probability that either event A or event B occurs. We subtract the intersection $P(A \cap B)$ so that outcomes common to both $A$ and $B$ are not counted twice.

General Addition Rule

Calculates the probability of the union of two events.

P(A \cup B) = P(A) + P(B) - P(A \cap B)

Variables

Symbol	Description	Unit
$P(A \cup B)$	Probability that either event A or event B occurs	-
$P(A)$	Probability of event A occurring	-
$P(B)$	Probability of event B occurring	-
$P(A \cap B)$	Probability that both events A and B occur simultaneously	-

Special Addition Rule (Mutually Exclusive Events)

If $A$ and $B$ are mutually exclusive (they cannot both happen, so $P(A \cap B) = 0$ ).

Special Addition Rule

Calculates the probability of the union of two mutually exclusive events.

P(A \cup B) = P(A) + P(B)

Variables

Symbol	Description	Unit
$P(A \cup B)$	Probability that either event A or event B occurs	-
$P(A)$	Probability of event A occurring	-
$P(B)$	Probability of event B occurring	-

The Complement Rule

Complement Rule

The probability that an event does not happen is 1 minus the probability that it does happen.

Complement Rule

Calculates the probability of the complement of an event.

P(A') = 1 - P(A)

Variables

Symbol	Description	Unit
$P(A')$	Probability that event A does not occur	-
$P(A)$	Probability that event A occurs	-

Interactive Simulation

Interact with the simulation below to explore the Law of Large Numbers and probability fundamentals.

Engineering Data Analysis

Law of Large Numbers Simulation

Auto-Simulation Speed

SlowFast

Total Flips (n)

Heads (0.0%)

Tails (0.0%)

Empirical Probability vs. Theoretical $P(H)$

P=0.5

Mathematical Concept

The Law of Large Numbers (LLN) guarantees that the long-term relative frequency of an event approaches the theoretical probability as the number of trials increases.

\lim_{n \to \infty} \frac{n_H}{n} = P(H) = 0.5

Key Takeaways

Sample Space ( $S$ ): All possible outcomes of an experiment.
Events: Subsets of the sample space, combined using unions, intersections, and complements.
Counting Rules: Use Permutations when order matters, and Combinations when order is irrelevant.
Probability Bounds: Probability must always fall between $0$ and $1$ .
Addition Rule: Used for "A OR B" scenarios ( $A \cup B$ ); subtract the intersection to avoid double-counting unless events are mutually exclusive.

Previous TopicDescriptive Statistics - Examples & Applications

Quiz Me

Next TopicProbability Fundamentals - Examples & Applications

Prev Next

Quiz Me

Learning Objectives

Core Concepts and Set Theory

Experiment

Sample Space (SSS)

Event (EEE)

Venn Diagrams

Set Operations (Unions, Intersections, Complements)

Interactive Simulation

Engineering Data Analysis • Topic 3

Set Operation Presets

Set Operations Guide

Counting Rules

Overview of Counting Rules

The Multiplication Rule (Fundamental Counting Principle)

Permutations (Order Matters)

Permutations

Combinations (Order Does Not Matter)

Combinations

Axioms and Rules of Probability

Probability of an Event, P(A)P(A)P(A)

Probability of an Event

Kolmogorov's Axioms of Probability

The Addition Rule

General Addition Rule

General Addition Rule

Special Addition Rule (Mutually Exclusive Events)

Special Addition Rule

The Complement Rule

Complement Rule

Complement Rule

Interactive Simulation

Engineering Data Analysis

Empirical Probability vs. Theoretical P(H)P(H)P(H)

Mathematical Concept

Sample Space ( $S$ )

Event ( $E$ )

Probability of an Event, $P(A)$

Empirical Probability vs. Theoretical $P(H)$