Appendix E: Summary of Formulas

This appendix provides a summary of the key formulas introduced in Chapters 1–8.

Chapter 2: The Language of Probability: Sets, Sample Spaces, and Events¶

Axioms of Probability¶

Let $S$ be a sample space, and $P(A)$ denote the probability of an event $A$.

1. Non-negativity: For any event $A$, the probability of $A$ is greater than or equal to zero. $P(A)\ge 0$

2. Normalization: The probability of the entire sample space $S$ is equal to 1. $P(S)=1$

3. Additivity for Disjoint Events: If $A_1,A_2,A_3,\dots$ is a sequence of mutually exclusive (disjoint) events (i.e., $A_i\cap A_j=\emptyset$ for all $i\ne j$), then the probability of their union is the sum of their individual probabilities.

   $$P(A_1\cup A_2\cup A_3\cup \cdots)=P(A_1)+P(A_2)+P(A_3)+\cdots$$

   (1)

   • For a finite number of disjoint events, say $A$ and $B$: If $A\cap B=\emptyset$, then $P(A\cup B)=P(A)+P(B)$

• Probability of Impossible Event: The probability of an impossible event (the empty set, $\emptyset$) is 0. $P(\emptyset)=0$

Basic Probability Rules¶

1. Probability Range: For any event $A$: $0\le P(A)\le 1$

2. Complement Rule: The probability that event $A$ does not occur is 1 minus the probability that it does occur. $P(A^c)=1-P(A)$

3. Addition Rule (General): For any two events $A$ and $B$ (not necessarily disjoint), the probability that $A$ or $B$ (or both) occurs is:

   $$P(A\cup B)=P(A)+P(B)-P(A\cap B)$$

   (2)

Empirical Probability¶

The empirical probability of an event $A$ is estimated from simulations:

Chapter 3: Counting Techniques: Permutations and Combinations¶

The Multiplication Principle¶

If a procedure can be broken down into a sequence of $k$ steps, with $n_1$ ways for the first step, $n_2$ for the second, $\dots$, $n_k$ for the $k$-th step, then the total number of ways to perform the entire procedure is:

Permutations (Order Matters)¶

1. Permutations without Repetition: The number of permutations of $n$ distinct objects taken $k$ at a time:

   $$P(n,k)=\frac{n!}{(n-k)!}$$

   (5)

   • Special Case: Arranging all $n$ distinct objects: $P(n,n)=n!$

2. Permutations with Repetition (Multinomial Coefficients): The number of distinct permutations of $n$ objects where there are $n_1$ identical objects of type 1, $n_2$ of type 2, $\dots$, $n_k$ of type $k$ (such that $n_1+n_2+\cdots+n_k=n$):

   $$\frac{n!}{n_1!,n_2!\cdots n_k!}$$

   (6)

Combinations (Order Doesn’t Matter)¶

1. Combinations without Repetition: The number of combinations of $n$ distinct objects taken $k$ at a time (also “$n$ choose $k$”):

   $$C(n,k)=\binom{n}{k}=\frac{n!}{k!(n-k)!}$$

   (7)

   • Relationship to permutations:

     $$C(n,k)=\frac{P(n,k)}{k!}$$

     (8)

2. Combinations with Repetition: The number of combinations with repetition of $n$ types of objects taken $k$ at a time:

   $$\binom{n+k-1}{k}=\frac{(n+k-1)!}{k!(n-1)!}$$

   (9)

Probability with Equally Likely Outcomes¶

The probability of an event $E$ when all outcomes in the sample space $S$ are equally likely:

Chapter 4: Conditional Probability¶

Definition of Conditional Probability¶

For any two events $A$ and $B$ from a sample space $S$, where $P(B)>0$, the conditional probability of $A$ given $B$ is defined as:

The Multiplication Rule for Conditional Probability¶

Rearranging the definition of conditional probability gives:

Similarly, if $P(A)>0$:

For three events $A,B,C$:

The Law of Total Probability¶

Let $B_1,B_2,\dots,B_n$ be a partition of the sample space $S$. Then, for any event $A$ in $S$:

Expanded form:

Chapter 5: Bayes’ Theorem and Independence¶

Bayes’ Theorem¶

Provides a way to “reverse” conditional probabilities. If $P(B)>0$:

Where $P(B)$ can often be calculated using the Law of Total Probability (e.g., with a partition ${A,A^c}$):

Independence of Events¶

1. Formal Definition: Events $A$ and $B$ are independent if and only if:

   $$P(A\cap B)=P(A)P(B)$$

   (19)

2. Alternative Definition (using conditional probability):

   • If $P(B)>0$, $A$ and $B$ are independent if and only if: $P(A\mid B)=P(A)$

   • Similarly, if $P(A)>0$, independence means: $P(B\mid A)=P(B)$

Conditional Independence¶

Notation:

Definition (with $P(C)>0$):

Equivalent “no extra information” form: if $P(B\cap C)>0$, then

Likewise, if $P(A\cap C)>0$, then

Chapter 6: Discrete Random Variables¶

Probability Mass Function (PMF)¶

For a discrete random variable $X$, the PMF $p_X(x)$ is:

Properties of a PMF:

1. $p_X(x)\ge 0$ for all possible values $x$.

2. $$\sum_x p_X(x)=1$$

   (25)

   (sum over all possible values $x$).

Cumulative Distribution Function (CDF)¶

For a random variable $X$, the CDF $F_X(x)$ is:

For a discrete random variable $X$:

Properties of a CDF:

1. $0\le F_X(x)\le 1$ for all $x$.

2. If $a<b$, then $F_X(a)\le F_X(b)$ (non-decreasing).

3. $$\lim_{x\to -\infty} F_X(x)=0$$

   (28)
4. $$\lim_{x\to +\infty} F_X(x)=1$$

   (29)

5. $P(X>x)=1-F_X(x)$

6. $P(a<X\le b)=F_X(b)-F_X(a)$ for $a<b$.

7. $$P(X=x)=F_X(x)-\lim_{y\to x^-}F_X(y)$$

   (30)

   (for a discrete RV, this is the jump at $x$).

Expected Value (Mean)¶

For a discrete random variable $X$:

Variance¶

For a random variable $X$ with mean $\mu_X$:

For a discrete random variable $X$:

Computational formula for variance:

Where $E[X^2]$ for a discrete random variable is:

Standard Deviation¶

The positive square root of the variance:

Functions of a Random Variable¶

If $Y=g(X)$:

1. PMF of $Y$ (for discrete $X$):

   $$p_Y(y)=P(Y=y)=P(g(X)=y)=\sum_{x:,g(x)=y} p_X(x)$$

   (37)

2. Expected Value of $Y=g(X)$ (LOTUS - Law of the Unconscious Statistician): For a discrete random variable $X$:

   $$E[Y]=E[g(X)]=\sum_x g(x)\cdot p_X(x)$$

   (38)

Chapter 7: Common Discrete Distributions¶

Bernoulli Distribution¶

Models a single trial with two outcomes (success=1, failure=0). Parameter: $p$ (probability of success).

• PMF:

  $$P(X=k)=p^k(1-p)^{1-k}\quad \text{for }k\in{0,1}$$

  (39)

  • Alternatively:

    $$P(X=k)= \begin{cases} p, & k=1 \ 1-p, & k=0 \ 0, & \text{otherwise} \end{cases}$$

    (40)

• Mean: $E[X]=p$

• Variance: $\operatorname{Var}(X)=p(1-p)$

Binomial Distribution¶

Models the number of successes in $n$ independent Bernoulli trials. Parameters: $n$ (number of trials), $p$ (probability of success on each trial).

• PMF:

  $$P(X=k)=\binom{n}{k}p^k(1-p)^{n-k}\quad \text{for }k=0,1,\dots,n$$

  (41)

• Mean: $E[X]=np$

• Variance: $\operatorname{Var}(X)=np(1-p)$

Geometric Distribution¶

Models the number of trials ($k$) needed to get the first success. Parameter: $p$ (probability of success on each trial).

• PMF (for $X=$ trial number of first success):

  $$P(X=k)=(1-p)^{k-1}p\quad \text{for }k=1,2,3,\dots$$

  (42)

• Mean (trial number of first success):

  $$E[X]=\frac{1}{p}$$

  (43)

• Variance (trial number of first success):

  $$\operatorname{Var}(X)=\frac{1-p}{p^2}$$

  (44)

Negative Binomial Distribution¶

Models the number of trials ($k$) needed to achieve $r$ successes. Parameters: $r$ (target number of successes), $p$ (probability of success on each trial).

• PMF (for $X=$ trial number of $r$-th success):

  $$P(X=k)=\binom{k-1}{r-1}p^r(1-p)^{k-r}\quad \text{for }k=r,r+1,r+2,\dots$$

  (45)

• Mean (trial number of $r$-th success):

  $$E[X]=\frac{r}{p}$$

  (46)

• Variance (trial number of $r$-th success):

  $$\operatorname{Var}(X)=\frac{r(1-p)}{p^2}$$

  (47)

Poisson Distribution¶

Models the number of events occurring in a fixed interval of time or space. Parameter: $\lambda$ (average number of events in the interval).

• PMF:

  $$P(X=k)=\frac{e^{-\lambda}\lambda^k}{k!}\quad \text{for }k=0,1,2,\dots$$

  (48)

• Mean: $E[X]=\lambda$

• Variance: $\operatorname{Var}(X)=\lambda$

Hypergeometric Distribution¶

Models the number of successes in a sample of size $n$ drawn without replacement from a finite population of size $N$ containing $K$ successes. Parameters: $N$ (population size), $K$ (total successes in population), $n$ (sample size).

• PMF:

  $$P(X=k)=\frac{\binom{K}{k}\binom{N-K}{n-k}}{\binom{N}{n}}$$

  (49)

  for $k$ such that $\max(0,n-(N-K))\le k\le \min(n,K)$.

• Mean:

  $$E[X]=n\frac{K}{N}$$

  (50)

• Variance:

  $$\operatorname{Var}(X)=n\frac{K}{N}\left(1-\frac{K}{N}\right)\left(\frac{N-n}{N-1}\right)$$

  (51)

  • Finite Population Correction Factor:

    $$\frac{N-n}{N-1}$$

    (52)

Chapter 8: Continuous Random Variables¶

Probability Density Function (PDF)¶

For a continuous random variable $X$, the PDF $f_X(x)$ describes the relative likelihood of $X$. Properties of a PDF:

1. $f_X(x)\ge 0$ for all $x$.

2. $\int_{-\infty}^{\infty} f_X(x)\,dx = 1$ (total area under curve is 1).

3. $P(a\le X\le b)=\int_a^b f_X(x)\,dx$.

4. For any specific value $c$: $P(X=c)=\int_c^c f_X(x)\,dx=0$.

Cumulative Distribution Function (CDF)¶

For a continuous random variable $X$, the CDF $F_X(x)$ is:

Properties of a CDF:

1. $F_X(x)$ is non-decreasing.

2. $$\lim_{x\to -\infty} F_X(x)=0$$

   (54)
3. $$\lim_{x\to \infty} F_X(x)=1$$

   (55)

4. $P(a<X\le b)=F_X(b)-F_X(a)$.

5. $$f_X(x)=\frac{d}{dx}F_X(x)$$

   (56)

   (where the derivative exists).

Expected Value (Mean)¶

For a continuous random variable $X$:

Variance¶

For a continuous random variable $X$ with mean $\mu$:

Computational formula:

Where

Standard Deviation¶

The positive square root of the variance:

Percentiles and Quantiles¶

The $p$-th percentile $x_p$ is the value such that $F_X(x_p)=P(X\le x_p)=p$. The quantile function $Q(p)$ is the inverse of the CDF:

Functions of a Continuous Random Variable¶

If $Y=g(X)$:

1. CDF of $Y$:

   $$F_Y(y)=P(Y\le y)=P(g(X)\le y)$$

   (63)

2. PDF of $Y$ (Change of Variables Formula): If $g(x)$ is monotonic with inverse $x=g^{-1}(y)$, then:

   $$f_Y(y)=f_X(g^{-1}(y))\left|\frac{dx}{dy}\right|$$

   (64)

3. Expected Value of $Y=g(X)$ (LOTUS):

   $$E[Y]=E[g(X)]=\int_{-\infty}^{\infty} g(x)f_X(x),dx$$

   (65)

Chapter 9: Common Continuous Distributions¶

1. Uniform Distribution¶

X∼U(a,b)

• PDF (Probability Density Function):

  $$f(x; a, b) = \begin{cases} \frac{1}{b-a} & \text{for } a \le x \le b \\ 0 & \text{otherwise} \end{cases}$$

  (66)

• CDF (Cumulative Distribution Function):

  $$F(x; a, b) = P(X \le x) = \begin{cases} 0 & \text{for } x < a \\ \frac{x-a}{b-a} & \text{for } a \le x \le b \\ 1 & \text{for } x > b \end{cases}$$

  (67)

• Expected Value: E[X]=2a+b

• Variance: Var(X)=12(b−a)2

2. Exponential Distribution¶

T∼Exp(λ)

• PDF (Probability Density Function):

  $$f(t; \lambda) = \begin{cases} \lambda e^{-\lambda t} & \text{for } t \ge 0 \\ 0 & \text{for } t < 0 \end{cases}$$

  (68)

• CDF (Cumulative Distribution Function):

  $$F(t; \lambda) = P(T \le t) = \begin{cases} 1 - e^{-\lambda t} & \text{for } t \ge 0 \\ 0 & \text{for } t < 0 \end{cases}$$

  (69)

• Survival Function: P(T>t)=1−F(t)=e−λt

• Expected Value: E[T]=λ1

• Variance: Var(T)=λ21

• Memoryless Property: P(T>s+t∣T>s)=P(T>t) for any s,t≥0.

3. Normal (Gaussian) Distribution¶

X∼N(μ,σ2)

• PDF (Probability Density Function):

  $$f(x; \mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{ - \frac{(x-\mu)^2}{2\sigma^2} }$$

  (70)

  for −∞<x<∞.

• Expected Value: E[X]=μ

• Variance: Var(X)=σ2

• Standardization (Z-score): Z=σX−μ where Z∼N(0,1).

4. Gamma Distribution¶

X∼Gamma(k,λ) (using shape k and rate λ) or X∼Gamma(k,θ) (using shape k and scale θ=1/λ)
The Gamma function is Γ(k)=∫0∞xk−1e−xdx. For positive integers k, Γ(k)=(k−1)!.

• PDF (Probability Density Function): Using shape k and rate λ:

  $$f(x; k, \lambda) = \frac{\lambda^k x^{k-1} e^{-\lambda x}}{\Gamma(k)} \quad \text{for } x \ge 0$$

  (71)

  Using shape k and scale θ=1/λ:

  $$f(x; k, \theta) = \frac{1}{\Gamma(k)\theta^k} x^{k-1} e^{-x/\theta} \quad \text{for } x \ge 0$$

  (72)

• Expected Value: E[X]=λk=kθ

• Variance: Var(X)=λ2k=kθ2

5. Beta Distribution¶

X∼Beta(α,β)
The Beta function is B(α,β)=∫01tα−1(1−t)β−1dt=Γ(α+β)Γ(α)Γ(β).

• PDF (Probability Density Function):

  $$f(x; \alpha, \beta) = \frac{1}{B(\alpha, \beta)} x^{\alpha-1} (1-x)^{\beta-1} = \frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)} x^{\alpha-1} (1-x)^{\beta-1}$$

  (73)

  for 0≤x≤1.

• Expected Value: E[X]=α+βα

• Variance: Var(X)=(α+β)2(α+β+1)αβ

Chapter 10: Joint Distributions¶

Joint Probability Mass Functions (PMFs)¶

For two discrete random variables $X$ and $Y$:

• Joint PMF Definition:

  $$p_{X,Y}(x, y) = P(X=x, Y=y)$$

  (74)

• Conditions:

  1. $p_{X,Y}(x, y) \ge 0$ for all $(x, y)$

  2. $\sum_{x} \sum_{y} p_{X,Y}(x, y) = 1$

Joint Probability Density Functions (PDFs)¶

For two continuous random variables $X$ and $Y$:

• Probability over a Region A:

  $$P((X, Y) \in A) = \iint_A f_{X,Y}(x, y) \,dx \,dy$$

  (75)

• Conditions:

  1. $f_{X,Y}(x, y) \ge 0$ for all $(x, y)$

  2. $\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f_{X,Y}(x, y) \,dx \,dy = 1$

Marginal Distributions¶

• Marginal PMF of X (Discrete):

  $$p_X(x) = P(X=x) = \sum_{y} P(X=x, Y=y) = \sum_{y} p_{X,Y}(x, y)$$

  (76)

• Marginal PMF of Y (Discrete):

  $$p_Y(y) = P(Y=y) = \sum_{x} P(X=x, Y=y) = \sum_{x} p_{X,Y}(x, y)$$

  (77)

• Marginal PDF of X (Continuous):

  $$f_X(x) = \int_{-\infty}^{\infty} f_{X,Y}(x, y) \,dy$$

  (78)

• Marginal PDF of Y (Continuous):

  $$f_Y(y) = \int_{-\infty}^{\infty} f_{X,Y}(x, y) \,dx$$

  (79)

Conditional Distributions¶

• Conditional PMF of Y given X=x (Discrete):

  $$p_{Y|X}(y|x) = P(Y=y | X=x) = \frac{P(X=x, Y=y)}{P(X=x)} = \frac{p_{X,Y}(x, y)}{p_X(x)}$$

  (80)

  (provided $p_X(x) > 0$)

• Conditional PDF of Y given X=x (Continuous):

  $$f_{Y|X}(y|x) = \frac{f_{X,Y}(x, y)}{f_X(x)}$$

  (81)

  (provided $f_X(x) > 0$)

Joint Cumulative Distribution Functions (CDFs)¶

• Joint CDF Definition:

  $$F_{X,Y}(x, y) = P(X \le x, Y \le y)$$

  (82)

• Discrete Case:

  $$F_{X,Y}(x, y) = \sum_{x_i \le x} \sum_{y_j \le y} p_{X,Y}(x_i, y_j)$$

  (83)

• Continuous Case:

  $$F_{X,Y}(x, y) = \int_{-\infty}^{x} \int_{-\infty}^{y} f_{X,Y}(u, v) \,dv \,du$$

  (84)

• Properties:

  1. $0 \le F_{X,Y}(x, y) \le 1$

  2. $F_{X,Y}(x, y)$ is non-decreasing in both $x$ and $y$.

  3. $\lim_{x \to \infty, y \to \infty} F_{X,Y}(x, y) = 1$

  4. $\lim_{x \to -\infty} F_{X,Y}(x, y) = 0$ and $\lim_{y \to -\infty} F_{X,Y}(x, y) = 0$

Chapter 11: Independence, Covariance, and Correlation¶

Independence of Random Variables¶

Two random variables $X$ and $Y$ are independent if for any sets $A$ and $B$:

This is equivalent to:

• Discrete:

  $$P(X=x, Y=y) = P(X=x) P(Y=y)$$

  (86)

  (Joint PMF = Product of Marginal PMFs)

• Continuous:

  $$f_{X,Y}(x,y) = f_X(x) f_Y(y)$$

  (87)

  (Joint PDF = Product of Marginal PDFs)

Covariance¶

The covariance between two random variables $X$ and $Y$:

• Definition:

  $$\mathrm{Cov}(X, Y) = E[(X - E[X])(Y - E[Y])]$$

  (88)

• Computational Formula:

  $$\mathrm{Cov}(X, Y) = E[XY] - E[X]E[Y]$$

  (89)

• Properties:

  1. $\mathrm{Cov}(X, X) = \mathrm{Var}(X)$

  2. $\mathrm{Cov}(X, Y) = \mathrm{Cov}(Y, X)$

  3. $\mathrm{Cov}(aX + b, cY + d) = ac \mathrm{Cov}(X, Y)$

  4. $\mathrm{Cov}(X+Y, Z) = \mathrm{Cov}(X, Z) + \mathrm{Cov}(Y, Z)$

  5. If $X$ and $Y$ are independent, then $\mathrm{Cov}(X, Y) = 0$.

Correlation Coefficient¶

The Pearson correlation coefficient between two random variables $X$ and $Y$:

• Definition:

  $$\rho(X, Y) = \frac{\mathrm{Cov}(X, Y)}{\sigma_X \sigma_Y} = \frac{\mathrm{Cov}(X, Y)}{\sqrt{\mathrm{Var}(X) \mathrm{Var}(Y)}}$$

  (90)

• Properties:

  1. $-1 \le \rho(X, Y) \le 1$

  2. $\rho(aX + b, cY + d) = \mathrm{sign}(ac) \rho(X, Y)$, (assuming $a \ne 0, c \ne 0$)

Variance of Sums of Random Variables¶

For any two random variables $X$ and $Y$, and constants $a$ and $b$:

• General Formula:

  $$\mathrm{Var}(aX + bY) = a^2 \mathrm{Var}(X) + b^2 \mathrm{Var}(Y) + 2ab \mathrm{Cov}(X, Y)$$

  (91)

• Sum of Variables ($a=1, b=1$):

  $$\mathrm{Var}(X + Y) = \mathrm{Var}(X) + \mathrm{Var}(Y) + 2 \mathrm{Cov}(X, Y)$$

  (92)

• Difference of Variables ($a=1, b=-1$):

  $$\mathrm{Var}(X - Y) = \mathrm{Var}(X) + \mathrm{Var}(Y) - 2 \mathrm{Cov}(X, Y)$$

  (93)

• If $X$ and $Y$ are independent ($\mathrm{Cov}(X, Y) = 0$):

  $$\mathrm{Var}(aX + bY) = a^2 \mathrm{Var}(X) + b^2 \mathrm{Var}(Y)$$

  (94)
  $$\mathrm{Var}(X + Y) = \mathrm{Var}(X) + \mathrm{Var}(Y)$$

  (95)
  $$\mathrm{Var}(X - Y) = \mathrm{Var}(X) + \mathrm{Var}(Y)$$

  (96)

• Extension to Multiple Variables ($X_1, X_2, ..., X_n$):

  $$\mathrm{Var}\left(\sum_{i=1}^n a_i X_i\right) = \sum_{i=1}^n a_i^2 \mathrm{Var}(X_i) + \sum_{i \ne j} a_i a_j \mathrm{Cov}(X_i, X_j)$$

  (97)

  or

  $$\mathrm{Var}\left(\sum_{i=1}^n a_i X_i\right) = \sum_{i=1}^n a_i^2 \mathrm{Var}(X_i) + 2 \sum_{i < j} a_i a_j \mathrm{Cov}(X_i, X_j)$$

  (98)

• If all $X_i$ are independent:

  $$\mathrm{Var}\left(\sum_{i=1}^n a_i X_i\right) = \sum_{i=1}^n a_i^2 \mathrm{Var}(X_i)$$

  (99)

Chapter 12: Functions of Multiple Random Variables¶

Sums of Independent Random Variables (Convolution)¶

Let $X$ and $Y$ be two random variables, and $Z = X+Y$.

• Discrete Case (PMF of Z):

  $$P(Z=z) = \sum_{k} P(X=k, Y=z-k)$$

  (100)

  If $X$ and $Y$ are independent:

  $$P(Z=z) = \sum_{k} P(X=k)P(Y=z-k)$$

  (101)

  This is the discrete convolution of the PMFs.

• Continuous Case (PDF of Z):

  $$f_Z(z) = \int_{-\infty}^{\infty} f_{X,Y}(x, z-x)dx$$

  (102)

  If $X$ and $Y$ are independent:

  $$f_Z(z) = \int_{-\infty}^{\infty} f_X(x)f_Y(z-x)dx = (f_X * f_Y)(z)$$

  (103)

  This is the convolution of the PDFs.

General Transformations (Jacobian Method for PDFs)¶

If $Y_1 = g_1(X_1, X_2)$ and $Y_2 = g_2(X_1, X_2)$ are transformations of random variables $X_1, X_2$, and these transformations are invertible such that $X_1 = h_1(Y_1, Y_2)$ and $X_2 = h_2(Y_1, Y_2)$.

• Joint PDF of $Y_1, Y_2$:

  $$f_{Y_1,Y_2}(y_1, y_2) = f_{X_1,X_2}(h_1(y_1,y_2), h_2(y_1,y_2)) |J|$$

  (104)

  Where $|J|$ is the absolute value of the determinant of the Jacobian matrix.

• Jacobian Determinant (J):

  $$J = \det \begin{pmatrix} \frac{\partial x_1}{\partial y_1} & \frac{\partial x_1}{\partial y_2} \\ \frac{\partial x_2}{\partial y_1} & \frac{\partial x_2}{\partial y_2} \end{pmatrix}$$

  (105)

Order Statistics¶

Let $X_1, X_2, \dots, X_n$ be $n$ independent and identically distributed (i.i.d.) random variables with CDF $F_X(x)$ and PDF $f_X(x)$. Let $X_{(1)}, X_{(2)}, \dots, X_{(n)}$ be the order statistics (sorted values).

• CDF of the Maximum ($Y_n = X_{(n)}$):

  $$F_{Y_n}(y) = P(X_{(n)} \le y) = [F_X(y)]^n$$

  (106)

• PDF of the Maximum ($Y_n = X_{(n)}$):

  $$f_{Y_n}(y) = n[F_X(y)]^{n-1}f_X(y)$$

  (107)

• CDF of the Minimum ($Y_1 = X_{(1)}$):

  $$F_{Y_1}(y) = P(X_{(1)} \le y) = 1 - [1-F_X(y)]^n$$

  (108)

• PDF of the Minimum ($Y_1 = X_{(1)}$):

  $$f_{Y_1}(y) = n[1-F_X(y)]^{n-1}f_X(y)$$

  (109)

• PDF of the $k$-th Order Statistic ($Y_k = X_{(k)}$):

  $$f_{Y_k}(y) = \frac{n!}{(k-1)!(n-k)!} [F_X(y)]^{k-1} [1-F_X(y)]^{n-k} f_X(y)$$

  (110)