The Addition Rule
Probability Theory
When Does A or B Happen? Count Without Double-Counting
The addition rule calculates the probability of A or B (or both) occurring. The key insight: subtract the overlap so you never count it twice.
- General rule — P(A or B) equals P(A) plus P(B) minus P(A and B)
- Mutually exclusive — When A and B cannot both occur, the formula simplifies
- Inclusion-exclusion — The principle extends to three or more events
- Venn diagrams — Visual proof of why double-counting happens and how to fix it
The addition rule is your tool for computing "at least one" probabilities.
What is the Addition Rule?
Definition
The addition rule calculates the probability of A or B (or both) occurring. It accounts for the overlap between events to avoid double-counting.
General Addition Rule
Here,
- =Probability of A or B (or both)
- =Probability of event A
- =Probability of event B
- =Probability of both A and B occurring
import numpy as np
import matplotlib.pyplot as plt
# Example: Drawing a card
S = [f"{rank} of {suit}"
for suit in ['Hearts', 'Diamonds', 'Clubs', 'Spades']
for rank in ['A', '2', '3', '4', '5', '6', '7', '8', '9', '10', 'J', 'Q', 'K']]
A = [c for c in S if 'Hearts' in c] # Event A: Hearts
B = [c for c in S if 'A' in c.split()[0]] # Event B: Aces
p_a = len(A) / len(S)
p_b = len(B) / len(S)
p_a_and_b = len(set(A) & set(B)) / len(S)
p_a_or_b = p_a + p_b - p_a_and_b
print(f"P(Hearts) = {p_a:.4f} = {len(A)}/52")
print(f"P(Ace) = {p_b:.4f} = {len(B)}/52")
print(f"P(Heart ∩ Ace) = {p_a_and_b:.4f} = {len(set(A) & set(B))}/52")
print(f"P(Heart ∪ Ace) = {p_a_or_b:.4f} = {p_a + p_b - p_a_and_b:.4f}")
Mutually Exclusive Events
When A and B cannot occur together (A ∩ B = ∅), the addition rule simplifies:
Addition Rule for Mutually Exclusive Events
Here,
- =Mutually exclusive: no overlap
# Mutually exclusive: rolling a die
# Event A: even number {2, 4, 6}
# Event B: odd number {1, 3, 5}
S = {1, 2, 3, 4, 5, 6}
A = {2, 4, 6}
B = {1, 3, 5}
p_a = len(A) / len(S)
p_b = len(B) / len(S)
p_a_or_b = p_a + p_b # No need to subtract intersection (it's 0)
print(f"P(Even) = {p_a:.4f}")
print(f"P(Odd) = {p_b:.4f}")
print(f"P(Even or Odd) = {p_a_or_b:.4f}")
Inclusion-Exclusion for Three Events
Three-Event Addition Rule
Here,
- =Probability of at least one event occurring
# Three events: die roll
S = {1, 2, 3, 4, 5, 6}
A = {1, 2, 3} # ≤ 3
B = {2, 4, 6} # Even
C = {4, 5, 6} # ≥ 4
# Count using inclusion-exclusion
n_a = len(A)
n_b = len(B)
n_c = len(C)
n_ab = len(A & B)
n_ac = len(A & C)
n_bc = len(B & C)
n_abc = len(A & B & C)
n_union = n_a + n_b + n_c - n_ab - n_ac - n_bc + n_abc
p_union = n_union / len(S)
print(f"n(A ∪ B ∪ C) = {n_union}")
print(f"P(A ∪ B ∪ C) = {p_union:.4f}")
Addition Rule in Machine Learning
| ML Application | Addition Rule Usage | Why |
|---|---|---|
| Multi-class metrics | P(A∪B) for combined outcomes | Macro/micro averaging |
| Feature independence | P(A∪B) = P(A) + P(B) - P(A∩B) | Naive Bayes assumption |
| Ensemble voting | Probability of any model correct | Combine predictions |
import numpy as np
# Addition rule in multi-label classification
# P(cat OR dog) = P(cat) + P(dog) - P(cat AND dog)
p_cat = 0.3
p_dog = 0.4
p_both = 0.1
p_either = p_cat + p_dog - p_both
print(f"P(cat): {p_cat}")
print(f"P(dog): {p_dog}")
print(f"P(both): {p_both}")
print(f"P(cat OR dog) = {p_cat} + {p_dog} - {p_both} = {p_either}")
print("This is critical for multi-label classification metrics!")
Key Takeaways
Summary: Addition Rule
- General rule: P(A ∪ B) = P(A) + P(B) − P(A ∩ B) — subtract overlap to avoid double-counting
- Mutually exclusive: P(A ∪ B) = P(A) + P(B) — no overlap to subtract
- Inclusion-exclusion extends to three or more events with alternating signs
- P(A ∪ B) ≤ P(A) + P(B) — union probability is always ≤ sum of individual probabilities
- Common mistake: forgetting to subtract P(A ∩ B) when events overlap
- The complement rule connects: P(Aᶜ) = 1 − P(A)