Expected Value

Probability Theory

The Long-Run Average — What Happens on Average

Expected value is the weighted average of all possible values a random variable can take, weighted by their probabilities. It is the single number that summarizes a random variable.

Law of Large Numbers — The average of many trials converges to the expected value
Linearity of expectation — E[X + Y] equals E[X] plus E[Y], always
Decision theory — Choose the action with the highest expected payoff
Fair game — A game is fair if its expected value is zero

Expected value is the most important concept in probability. It tells you what to expect on average.

What is Expected Value?

Definition

Expected value is the weighted average of all possible values a random variable can take, weighted by their probabilities. It is the long-run average value over infinitely many repetitions of the experiment.

Expected Value (Discrete)

E[X] = \mu = \sum_{i=1}^{n} x_i \cdot P(X = x_i)

Here,

$E[X]$ =Expected value of random variable X
$x_i$ =The i-th possible value of X
$P(X = x_i)$ =Probability of X taking value x_i

import numpy as np
import matplotlib.pyplot as plt

# Fair die
values = np.array([1, 2, 3, 4, 5, 6])
probs = np.array([1/6, 1/6, 1/6, 1/6, 1/6, 1/6])
expected = np.sum(values * probs)

print(f"E[Die] = {' + '.join(f'{v}×(1/6)' for v in values)}")
print(f"       = {expected:.4f}")
print(f"       = 21/6 = 3.5")

Properties of Expected Value

ThLinearity of Expectation

Constant: $E[c] = c$
Scalar multiplication: $E[cX] = c \cdot E[X]$
Additivity: $E[X + Y] = E[X] + E[Y]$ (always true, even if X and Y are dependent)

# Verify linearity
np.random.seed(42)
n_sim = 100000

X = np.random.choice([1, 2, 3, 4, 5, 6], n_sim)
Y = np.random.choice([1, 2, 3, 4, 5, 6], n_sim)

print(f"E[X] ≈ {X.mean():.4f} (theoretical: 3.5)")
print(f"E[Y] ≈ {Y.mean():.4f} (theoretical: 3.5)")
print(f"E[X+Y] ≈ {(X+Y).mean():.4f} (theoretical: 7.0)")
print(f"E[X]+E[Y] ≈ {X.mean()+Y.mean():.4f}")

Expected Value of Functions

Expected Value of g(X)

E[g(X)] = \sum_{i} g(x_i) \cdot P(X = x_i)

Here,

$g(X)$ =A function of the random variable X

# E[X²] for a fair die
squared_values = values**2
e_x_squared = np.sum(squared_values * probs)
variance = e_x_squared - expected**2

print(f"E[X²] = {' + '.join(f'{v}²×(1/6)' for v in values)}")
print(f"      = {e_x_squared:.4f}")
print(f"\nVar(X) = E[X²] - (E[X])² = {e_x_squared:.4f} - {expected**2:.4f} = {variance:.4f}")

Expected Value in Decision Making

# Expected Value of a Lottery Ticket
cost = 10
prizes = np.array([1000, 100, 10, 0])
probabilities = np.array([0.001, 0.01, 0.1, 0.889])

expected_prize = np.sum(prizes * probabilities)
net_ev = expected_prize - cost

print("Lottery Expected Value:")
for p, prob in zip(prizes, probabilities):
    print(f"  Prize ${p:>5}: P = {prob:.3f}")
print(f"\n  E[Prize] = ${expected_prize:.2f}")
print(f"  Cost     = ${cost:.2f}")
print(f"  Net EV   = ${net_ev:.2f}")
print(f"  -> {'Unfavorable' if net_ev < 0 else 'Favorable'} bet!")

Expected Value ≠ Typical Outcome

Expected value is a long-run average. In any single trial, you will almost never observe the expected value. A lottery with EV = - $2 doesn't mean you lose$ 2 every time — you either win big or lose your stake.

Conditional Expected Value

# E[X | condition]
np.random.seed(42)
scores = np.random.normal(75, 12, 1000)

overall_mean = scores.mean()
high_scores = scores[scores > 80]
low_scores = scores[scores <= 80]

print(f"E[Score] = {overall_mean:.2f}")
print(f"E[Score | Score > 80] = {high_scores.mean():.2f}")
print(f"E[Score | Score ≤ 80] = {low_scores.mean():.2f}")

Expected Value in Machine Learning

ML Application	Expected Value Usage	Why
Loss functions	E[loss] = expected risk	Minimize expected loss
Reinforcement learning	E[reward] = value function	Maximize expected return
Thompson sampling	E[outcome \| arm]	Multi-armed bandits
Value at Risk	E[loss \| threshold]	Financial ML

import numpy as np

# Expected value = what you expect on average
# E[X] = Σ x × P(x)

# Example: portfolio expected return
returns = np.array([0.10, 0.05, -0.03, 0.08])
probabilities = np.array([0.3, 0.4, 0.1, 0.2])

expected_return = np.sum(returns * probabilities)
print(f"Returns: {returns}")
print(f"Probabilities: {probabilities}")
print(f"Expected return: {expected_return:.4f} ({expected_return*100:.2f}%)")

# Reinforcement learning: Q-value = expected cumulative reward
# Q(s,a) = E[G_t | s_t=s, a_t=a]
print(f"\nIn RL: Q(s,a) = E[total reward | state s, action a]")
print(f"This is just the expected value of future rewards!")

Key Takeaways

Summary: Expected Value

E[X] = Σ xᵢ · P(X = xᵢ) — probability-weighted average of all possible values
Linearity: E[aX + bY] = aE[X] + bE[Y] — always holds, even for dependent variables
E[constant] = constant — the expected value of a constant is itself
Var(X) = E[X²] − (E[X])² — expected value is used to compute variance
Expected value guides decisions: choose the option with the highest expected payoff
Expected value is a theoretical long-run average — not the outcome of any single trial

Expected Value — The Mean of a Random Variable