A random variable is a variable whose possible values are numerical outcomes of a random phenomenon. There are two main types of random variables: discrete and continuous.

1. Discrete Random Variables: These are random variables that can take on a countable number of values. For example, the number of heads in 10 coin flips is a discrete random variable.

2. Continuous Random Variables: These are random variables that can take on an infinite number of values within a given range. For example, the time it takes for a computer to solve a problem is a continuous random variable.

Formally, a random variable is a measurable function that maps outcomes of a random process to real numbers. This mapping allows us to assign probabilities to different outcomes and analyze them statistically using existing mathematical tools.

Let $\Omega$ be the sample space of a random process, and let $X: \Omega \to \mathbb{R}$ be a random variable. The function $X$ assigns a real number to each outcome in $\Omega$. The probability distribution of a random variable describes how the probabilities are distributed over the possible values of the random variable.

Probability Functions

For discrete random variables, we use the Probability Mass Function (PMF): $p_X(x) = P(X = x)$

Properties:

• $p_X(x) \geq 0$ for all $x$
• $\sum_{x} p_X(x) = 1$

For continuous random variables, we use the Probability Density Function (PDF): $f_X(x) \text{ where } P(a \leq X \leq b) = \int_a^b f_X(x)dx$

Properties:

• $f_X(x) \geq 0$ for all $x$
• $\int_{-\infty}^{\infty} f_X(x)dx = 1$

Cumulative Distribution Function (CDF)

The CDF is defined for both discrete and continuous random variables: $F_X(x) = P(X \leq x)$

For discrete: $F_X(x) = \sum_{t \leq x} p_X(t)$

For continuous: $F_X(x) = \int_{-\infty}^{x} f_X(t)dt$

In summary, random variables are functions that map outcomes of random processes(sample space) to real numbers, allowing us to analyze and quantify the behavior of random phenomena.

A more rigorous definition is possible by introducing measurement and probability space, you may access here optionally: Random Variable - stackexchange.

For more details on expectation and variance calculations, see Expectation and Variance.

Joint random variables describe the behavior of two or more random variables defined on the same probability space. For two random variables $X$ and $Y$, their joint distribution specifies the probability of $X$ taking value $x$ and $Y$ taking value $y$ simultaneously.

Joint Probability Functions

For discrete random variables, we use the Joint Probability Mass Function: $p_{X,Y}(x,y) = P(X = x, Y = y)$

Properties:

• $p_{X,Y}(x,y) \geq 0$ for all $x,y$
• $\sum_{x}\sum_{y} p_{X,Y}(x,y) = 1$

For continuous random variables, we use the Joint Probability Density Function: $f_{X,Y}(x,y) \text{ where } P(a \leq X \leq b, c \leq Y \leq d) = \int_a^b \int_c^d f_{X,Y}(x,y) dy dx$

Properties:

• $f_{X,Y}(x,y) \geq 0$ for all $x,y$
• $\iint_{\mathbb{R}^2} f_{X,Y}(x,y) dx dy = 1$

Marginal Distributions

The marginal distribution of one variable can be obtained from the joint distribution:

For discrete:

• $p_X(x) = \sum_{y} p_{X,Y}(x,y)$
• $p_Y(y) = \sum_{x} p_{X,Y}(x,y)$

For continuous:

• $f_X(x) = \int_{-\infty}^{\infty} f_{X,Y}(x,y) dy$
• $f_Y(y) = \int_{-\infty}^{\infty} f_{X,Y}(x,y) dx$

Independence

Random variables $X$ and $Y$ are independent if: $p_{X,Y}(x,y) = p_X(x) \cdot p_Y(y) \text{ (discrete)}$ $f_{X,Y}(x,y) = f_X(x) \cdot f_Y(y) \text{ (continuous)}$

This means the joint distribution factors into the product of marginal distributions.