11 Characteristic Function, PGF

#PGF #CharacteristicFunction #FourierTransform #PoissonDistribution #BinomialDistribution

1 Characteristic Function

We have used continuity theorem on multiple occasions to obtain several convergence results.
However, the MGF $M_{X} (t)$ may not exist for some RVs. E.g., for $X \sim Cauchy$ , $M_{X} (t) = \infty, \forall t \neq 0$ .
Fortunately, the Fourier transformation of a RV $X$ , defined as $φ_{X} (t) = E [e^{i t X}]$ is finite $\forall t \in R$ . This function is called the characteristic function.

Theorem (Levy's Continuity Theorem)

$X_{n} \overset{d}{\to} X \Leftrightarrow φ_{X_{n}} (t) \to φ_{X} (t) .$

2 PGF

PGF

Let $X$ be a non-negative integer valued RV. For $| t | \leq 1$ , $G_{X} (t) = E [t^{X}] = \sum_{n = 0}^{\infty} t^{n} P (X = n) .$

Theorem (Uniqueness)

Suppose $X, Y$ are non-negative RVs. If $G_{X} = G_{Y}$ , then $X \overset{d}{=} Y$ .

Theorem

Let $X_{1}, \dots, X_{n}$ be independent, non-negative integer valued RVs. Then the PGF for $S_{n} = X_{1} + \dots + X_{n}$ satisfies $G_{S_{n}} (t) = \prod_{k = 1}^{n} G_{X_{i}} (t) .$

Proof

By independence $\begin{aligned} G_{S_{n}} (t) & = E (t^{X_{1} + \dots + X_{n}}) = E [\prod_{k = 1}^{n} e^{t X_{k}}] = \prod_{k = 1}^{n} E [e^{t X_{k}}] = \prod_{k = 1}^{n} G_{X_{k}} (t) . \end{aligned}$

Poisson (λ) + Poisson (μ) \overset{d}{=} Poisson (λ + μ)

$X \sim Poisson (λ), λ > 0$ , then $G_{X} (t) = \sum_{n = 0}^{\infty} t^{n} \frac{λ^{n} e^{- λ}}{n!} = e^{λ (t - 1)}$ .

Moreover, if $Y \sim Poisson (μ), X ⊥ ⊥ Y$ , then $G_{X + Y} (t) = e^{(λ + μ) (t - 1)},$ so by uniqueness theorem $X + Y \sim Poisson (μ + λ)$ .

Here is a very useful theorem:

Theorem (Compounding)

Let $X_{1}, X_{2}, \dots$ be a sequence of iid non-negative integer-valued RVs with common PGF $G_{X}$ , while $N$ is a non-negative inter-valued RV independent of $X_{1}, X_{2}, \dots$ , with PGF $G_{N}$ . Let $S_{N} = X_{1} + \dots + X_{N}$ . Then $G_{S_{N}} (t) = G_{N} (G_{X} (t)) = \sum_{n = 0}^{\infty} (G_{X} (t))^{n} P (N = n) .$

Proof

By direct computation $\begin{aligned} G_{S_{N}} (t) & = \sum_{k = 0}^{\infty} t^{k} P (S_{n} = k) \\ = \sum_{k = 0}^{\infty} t^{k} \sum_{n = 0}^{\infty} P (S_{N} = k | N = n) P (N = n) \\ = \sum_{k = 0}^{\infty} t^{k} \sum_{n = 0}^{\infty} P (S_{N} = k) P (N = n) \\ = \sum_{n = 0}^{\infty} [\sum_{k = 0}^{\infty} t^{k} P (S_{n} = k)] P (N = n) \\ = \sum_{n = 0}^{\infty} G_{S_{n}} (t) P (N = n) \\ = \sum_{n = 0}^{\infty} [G_{X} (t)]^{n} P (N = n) . \end{aligned}$