Welcome to Chapter 2! In the previous chapter, we introduced the ‘why’ of probability and set up our Python environment. Now, we dive into the fundamental vocabulary and concepts that form the bedrock of probability theory. Understanding these core ideas – sample spaces, events, and the rules governing them – is crucial before we can tackle more complex problems and distributions. We’ll use set theory as our language and Python to make these abstract concepts tangible.
Learning Objectives¶
Understand and define experiments, outcomes, and sample spaces.
Distinguish between discrete and continuous sample spaces.
Define events as subsets of the sample space.
Review basic set operations (union, intersection, complement) and their relevance to probability.
Grasp the fundamental Axioms of Probability.
Apply basic probability rules like the Complement and Addition rules.
Use Python sets and lists to represent sample spaces and events.
Calculate empirical probabilities through simulation.
Experiments, Outcomes, Sample Spaces¶
In probability, an experiment is any procedure or process that can be repeated infinitely (in theory) and has a well-defined set of possible results. Think of flipping a coin, rolling a die, measuring a patient’s temperature, or recording the time it takes for a website to load.
Each possible result of an experiment is called an outcome.
For flipping a coin, the outcomes are Heads (H) or Tails (T).
For rolling a standard six-sided die, the outcomes are the integers 1, 2, 3, 4, 5, 6.
For measuring temperature, an outcome could be 37.2°C.
For website load time, an outcome could be 1.34 seconds.
The sample space, often denoted by or (Omega), is the set of all possible outcomes of an experiment.
Discrete vs. Continuous Sample Spaces¶
Sample spaces can be categorized based on the nature of their outcomes:
Discrete Sample Space: Contains a finite or countably infinite number of outcomes. The outcomes can be listed.
Example (Finite): Rolling a standard six-sided die. The sample space is .
Example (Countably Infinite): Flipping a coin until the first Head appears. The sample space is . Although infinite, we can map each outcome to a positive integer (1st flip, 2nd flip, 3rd flip, etc.).
Continuous Sample Space: Contains an infinite number of outcomes that form a continuum. The outcomes cannot be listed individually because there are infinitely many possibilities between any two given outcomes.
Example: Measuring the exact height of a randomly selected adult. The sample space might be all real numbers between 0.5 meters and 3.0 meters, .
Example: Recording the time until a component fails. .
Python Representation (Discrete): We can easily represent finite discrete sample spaces using Python sets or lists. Sets are often conceptually closer as they inherently handle uniqueness and order doesn’t matter.
Python Implementation
# Sample space for rolling a single six-sided die
sample_space_die = {1, 2, 3, 4, 5, 6}
# Sample space for flipping a coin
sample_space_coin = {'Heads', 'Tails'}
print(f"Sample space (Die): {sample_space_die}")
print(f"Sample space (Coin): {sample_space_coin}")Sample space (Die): {1, 2, 3, 4, 5, 6}
Sample space (Coin): {'Heads', 'Tails'}
Events as Subsets¶
An event is any subset of the sample space. It represents a specific outcome or a collection of outcomes of interest. Events are usually denoted by capital letters (A, B, E, etc.).
Experiment: Rolling a die ()
Event A: Rolling an even number. . Note that .
Event B: Rolling a number greater than 4. . Note that .
Event C: Rolling a 3. . Simple event (contains only one outcome).
Event D: Rolling a number less than 10. . This is the certain event.
Event E: Rolling a 7. or . This is the impossible event (the empty set).
Python Representation: Events, being subsets, can also be represented using Python sets.
Python Implementation
# Continuing the die roll example
S = {1, 2, 3, 4, 5, 6}
# Event A: Rolling an even number
A = {2, 4, 6}
# Event B: Rolling a number greater than 4
B = {5, 6}
# Check if they are subsets of S
print(f"Is A a subset of S? {A.issubset(S)}")
print(f"Is B a subset of S? {B.issubset(S)}")
print(f"Event A: {A}")
print(f"Event B: {B}")Is A a subset of S? True
Is B a subset of S? True
Event A: {2, 4, 6}
Event B: {5, 6}
Set Theory Refresher¶
Since events are sets, the language and operations of set theory are fundamental to probability. Let A and B be two events in a sample space S.
Union (): The set of outcomes that are in A, or in B, or in both. Corresponds to the logical ‘OR’.
Example: For the die roll, = “Rolling an even number OR a number greater than 4” = .
Intersection (): The set of outcomes that are in both A and B. Corresponds to the logical ‘AND’.
Example: For the die roll, = “Rolling an even number AND a number greater than 4” = .
Complement ( or ): The set of outcomes in the sample space S that are not in A. Corresponds to the logical ‘NOT’.
Example: For the die roll, = “NOT rolling an even number” = “Rolling an odd number” = .
Disjoint Events: Two events A and B are disjoint or mutually exclusive if they have no outcomes in common, i.e., their intersection is the empty set ().
Example: The event “Rolling an even number” (A={2,4,6}) and the event “Rolling an odd number” (A’={1,3,5}) are disjoint. The event “Rolling a 1” ({1}) and “Rolling a 6” ({6}) are disjoint.
Venn Diagrams: These are useful visual aids. The sample space S is represented by a rectangle, and events are represented by circles or shapes within it. Overlapping areas show intersections, and the area outside a circle represents its complement.
(We won’t draw Venn diagrams directly in code here, but libraries like matplotlib_venn can be used for this. Conceptually, imagine S as a box containing numbers 1-6. Circle A encloses 2, 4, 6. Circle B encloses 5, 6. The overlap contains only 6. The area outside A contains 1, 3, 5.)
Python Set Operations:
Python Implementation
S = {1, 2, 3, 4, 5, 6}
A = {2, 4, 6} # Even numbers
B = {5, 6} # Numbers > 4
C = {1, 3, 5} # Odd numbers (A's complement)
# Union (A or B)
union_AB = A.union(B)
print(f"A union B: {union_AB}") # Corresponds to {2, 4, 5, 6}
# Intersection (A and B)
intersection_AB = A.intersection(B)
print(f"A intersection B: {intersection_AB}") # Corresponds to {6}
# Complement of A (relative to S)
complement_A = S.difference(A)
print(f"Complement of A: {complement_A}") # Corresponds to {1, 3, 5}
print(f"Is complement of A equal to C? {complement_A == C}")
# Check for disjoint events (A and C)
intersection_AC = A.intersection(C)
print(f"A intersection C: {intersection_AC}") # Corresponds to {}
print(f"Are A and C disjoint? {intersection_AC == set()}") # Empty set means disjointA union B: {2, 4, 5, 6}
A intersection B: {6}
Complement of A: {1, 3, 5}
Is complement of A equal to C? True
A intersection C: set()
Are A and C disjoint? True
Axioms of Probability¶
The entire structure of probability theory is built upon three fundamental axioms, proposed by Andrey Kolmogorov. Let S be a sample space, and P(A) denote the probability of an event A.
Non-negativity: For any event A, the probability of A is greater than or equal to zero.
Probabilities cannot be negative.
Normalization: The probability of the entire sample space S is equal to 1.
This means that some outcome within the realm of possibility must occur. The maximum possible probability is 1.
Additivity for Disjoint Events: If is a sequence of mutually exclusive (disjoint) events (i.e., for all ), then the probability of their union is the sum of their individual probabilities:
For a finite number of disjoint events, say A and B, this simplifies to: If , then .
Examples based on the axioms:
Consider rolling a fair six-sided die with . Assuming fairness, each outcome has probability 1/6.
Example 1 - Non-negativity (Axiom 1):
Each individual outcome has non-negative probability:
Example 2 - Normalization (Axiom 2):
The event “Roll < 7” equals the entire sample space: .
Since the individual outcomes are disjoint, we can apply Axiom 3:
Example 3 - Impossible events:
The event “Roll > 6” is impossible: .
To find , note that and these sets are disjoint. By Axiom 3:
Since (Axiom 2):
This shows that the probability of any impossible event is 0.
Basic Probability Rules¶
Several useful rules can be derived directly from the axioms:
Probability Range: For any event A:
(Follows from Axioms 1 & 2 and ).
Complement Rule: The probability that event A does not occur is 1 minus the probability that it does occur.
Derivation: A and A’ are disjoint () and their union is the entire sample space (). By Axiom 3, . By Axiom 2, . Therefore, , which rearranges to the rule.
Example: What is the probability of not rolling a 6? Let , so . = “not rolling a 6” = . .
Addition Rule (General): For any two events A and B (not necessarily disjoint), the probability that A or B (or both) occurs is:
Intuition: If we simply add P(A) and P(B), we have double-counted the probability of the outcomes that are in both A and B (the intersection). So, we subtract to correct for this. If A and B are disjoint, and , which reduces this rule to Axiom 3 for two events.
Example: What is the probability of rolling an even number (A={2,4,6}) or a number greater than 4 (B={5,6})? The intersection is , so . Using the Addition Rule: . Let’s check the outcomes in . There are 4 outcomes, each with probability 1/6. So the total probability is indeed . It works!
Hands-on Python Practice¶
Let’s use Python to solidify these concepts through simulation. We often don’t know the theoretical probabilities beforehand, or the situation is too complex to calculate. Simulation allows us to estimate probabilities by running the experiment many times and observing the outcomes. This estimated probability is called the empirical probability.
Empirical Probability:
The Law of Large Numbers (which we’ll study later) tells us that as the number of trials increases, the empirical probability converges to the true theoretical probability.
Setup: We’ll need NumPy for efficient random number generation.
Python Implementation
import numpy as np
import matplotlib.pyplot as plt
# Configure plots for better readability
plt.style.use('seaborn-v0_8-whitegrid')With our libraries imported, we can now work through several examples demonstrating how Python helps us understand probability concepts.
1. Representing Sample Spaces and Events¶
We’ve already seen how to use Python sets. Let’s reiterate for a coin flip.
Python Implementation
# Sample Space
S_coin = {'H', 'T'} # Using H for Heads, T for Tails
# Events
E_heads = {'H'}
E_tails = {'T'}
print(f"Sample Space (Coin): {S_coin}")
print(f"Event (Heads): {E_heads}")
print(f"Is Heads an event in S_coin? {E_heads.issubset(S_coin)}")Sample Space (Coin): {'T', 'H'}
Event (Heads): {'H'}
Is Heads an event in S_coin? True
2. Simulating Simple Experiments¶
Let’s simulate rolling a fair six-sided die many times.
Python Implementation
import numpy as np
# Simulate 1000 dice rolls
num_rolls = 1000
rolls = np.random.randint(1, 7, size=num_rolls) # Generate random integers between 1 (inclusive) and 7 (exclusive)
# Display the first 20 rolls
print(f"First 20 rolls: {rolls[:20]}")
print(f"Total rolls simulated: {len(rolls)}")First 20 rolls: [2 2 2 6 4 4 4 6 5 5 1 6 3 3 1 3 2 6 4 6]
Total rolls simulated: 1000
3. Calculating Empirical Probabilities¶
Example: What is the empirical probability of rolling a number greater than 4?
Theoretical answer: Event . .
Python Implementation
# Define the event B: rolling > 4
# We can count how many rolls satisfy this condition
outcomes_greater_than_4 = rolls[rolls > 4]
num_success = len(outcomes_greater_than_4)
# Calculate empirical probability
empirical_prob_B = num_success / num_rolls
print(f"Number of rolls > 4: {num_success}")
print(f"Total rolls: {num_rolls}")
print(f"Empirical P(Roll > 4): {empirical_prob_B:.4f}")
print(f"Theoretical P(Roll > 4): {1/3:.4f}")Number of rolls > 4: 324
Total rolls: 1000
Empirical P(Roll > 4): 0.3240
Theoretical P(Roll > 4): 0.3333
Note: The rolls variable used here comes from the dice roll simulation above. Try running the simulation and this calculation multiple times—you’ll notice the empirical probability fluctuates slightly but remains close to the theoretical value of 1/3, especially with a large num_rolls.
Example: What is the empirical probability of rolling an even number?
Theoretical answer: Event . .
Python Implementation
# Event A: rolling an even number
# An outcome is even if outcome % 2 == 0
outcomes_even = rolls[rolls % 2 == 0]
num_even = len(outcomes_even)
# Calculate empirical probability
empirical_prob_A = num_even / num_rolls
print(f"Number of even rolls: {num_even}")
print(f"Total rolls: {num_rolls}")
print(f"Empirical P(Roll is Even): {empirical_prob_A:.4f}")
print(f"Theoretical P(Roll is Even): {0.5:.4f}")Number of even rolls: 473
Total rolls: 1000
Empirical P(Roll is Even): 0.4730
Theoretical P(Roll is Even): 0.5000
4. Visualizing Events and Outcomes¶
We can use histograms to visualize the distribution of outcomes from our simulation.
Python Implementation
import matplotlib.pyplot as plt
# Plotting the frequency of each outcome
plt.figure(figsize=(8, 5))
# Count occurrences of each die face
faces = [1, 2, 3, 4, 5, 6]
counts = [np.sum(rolls == face) for face in faces]
# Create bar plot
plt.bar(faces, counts, color='skyblue', edgecolor='black', alpha=0.7)
plt.title(f'Frequency of Outcomes for {num_rolls} Die Rolls')
plt.xlabel('Die Face')
plt.ylabel('Frequency Count')
plt.xticks(faces)
# Add a line for the expected frequency for a fair die
expected_frequency = num_rolls / 6
plt.axhline(expected_frequency, color='red', linestyle='--', label=f'Expected Frequency ({expected_frequency:.1f})')
plt.legend()
plt.show()
The histogram shows the counts for each outcome (1 through 6). For a fair die and a large number of rolls, we expect the bars to be roughly the same height, close to the theoretical expected frequency (total rolls / 6).
We can also visualize the outcomes that constitute a specific event. For instance, let’s highlight the rolls that were greater than 4.
Python Implementation
# Create a boolean mask for the event
event_mask_B = rolls > 4 # True if roll > 4, False otherwise
# Simple textual visualization: show rolls and whether they met the condition
print("Roll | > 4?")
print("----|----")
for i in range(15): # Show first 15
print(f"{rolls[i]:<4}| {event_mask_B[i]}")
# Highlight on a plot (can be more complex, here just re-emphasize the count)
print(f"\nFrom the simulation:")
print(f"- The event 'Roll > 4' (outcomes {5, 6}) occurred {num_success} times.")
print(f"- The empirical probability is {empirical_prob_B:.4f}")Roll | > 4?
----|----
2 | False
2 | False
2 | False
6 | True
4 | False
4 | False
4 | False
6 | True
5 | True
5 | True
1 | False
6 | True
3 | False
3 | False
1 | False
From the simulation:
- The event 'Roll > 4' (outcomes (5, 6)) occurred 324 times.
- The empirical probability is 0.3240
This visualization shows which specific outcomes from our simulation satisfy the event condition (rolling > 4). It demonstrates how we can programmatically filter and analyze events from our simulated data.
Chapter Summary¶
This chapter introduced the basic language of probability theory, grounded in set theory.
We defined experiments, outcomes, and sample spaces (discrete and continuous).
We conceptualized events as subsets of the sample space.
We reviewed set operations (union, intersection, complement) and how they relate to combining or modifying events.
We established the foundational Axioms of Probability (Non-negativity, Normalization, Additivity).
We derived essential rules like the Complement Rule and the Addition Rule.
Crucially, we used Python to represent these concepts (using
setsandnumpy) and performed simulations to calculate empirical probabilities, comparing them to theoretical values.
Understanding this vocabulary and these basic rules is essential. In the next chapter, we will build upon this foundation by learning systematic ways to count the number of outcomes in sample spaces and events, which is often necessary for calculating theoretical probabilities, especially when outcomes are equally likely.
Exercises¶
Set Operations and Events: In a survey of 100 students:
60 students study Mathematics (M)
50 students study Physics (P)
30 students study both Mathematics and Physics
a) How many students study Mathematics or Physics (or both)? b) How many students study neither Mathematics nor Physics? c) How many students study exactly one of the two subjects?
AnswerLet be the set of students studying Mathematics, be the set of students studying Physics.
Given:
Total students = 100
a) Students studying M or P (or both): Using the inclusion-exclusion principle:
Answer: 80 students
b) Students studying neither:
Answer: 20 students
c) Students studying exactly one subject: Only M: Only P: Total:
Answer: 50 students
Probability Axioms: A box contains 5 red balls, 3 blue balls, and 2 green balls. You randomly select one ball.
a) What is the probability of selecting a red ball? b) What is the probability of not selecting a red ball? c) What is the probability of selecting a red or blue ball? d) Verify that the probabilities of all possible outcomes sum to 1.
Addition Rule: A fair six-sided die is rolled. Let be the event “roll an even number” and be the event “roll a number greater than 3”.
a) List the outcomes in events , , and b) Calculate , , and c) Use the Addition Rule to find d) Verify your answer by directly counting favorable outcomes
AnswerSample space: , each outcome has probability
a) Events:
(even numbers)
(greater than 3)
(even AND greater than 3)
b) Probabilities:
c) Using Addition Rule:
d) Direct verification: , so ✓
De Morgan’s Laws: In a deck of 52 playing cards, let:
= event that the card is a Heart
= event that the card is a face card (Jack, Queen, King)
Find the probability that a randomly drawn card is: a) Neither a Heart nor a face card (use De Morgan’s Law: ) b) Verify your answer by calculating directly
AnswerIn a standard deck:
Hearts: 13 cards, so
Face cards: cards (J, Q, K in each suit), so
Heart face cards: 3 cards (J♥, Q♥, K♥), so
a) Using De Morgan’s Law:
First, find using the Addition Rule:
Then apply Complement Rule:
b) Direct calculation: Total cards: 52 Hearts or face cards: 22 (calculated above) Neither:
✓
Mutually Exclusive vs. Independent: Consider rolling a fair six-sided die.
Let = “roll a 2”
Let = “roll an odd number”
a) Are events and mutually exclusive? Explain. b) Calculate , , and c) If events were independent, what would equal? Compare with your answer in (b).
Answera) Mutually exclusive?
Yes, and are mutually exclusive because:
(cannot roll both 2 and an odd number simultaneously)
b) Probabilities:
(since )
c) If independent:
If and were independent, then:
But we found
Conclusion: Events and are NOT independent. Mutually exclusive events (with positive probability) can never be independent—knowing one occurred tells us the other didn’t occur.