1 Beta Distribution

In the last note we introduced Beta distribution in this example. Now we give a formal definiton.

Here Gamma function is$$\mathrm{\Gamma }(\alpha )={\int }_0^{\mathrm{\infty }}{t}^{\alpha -1}{\mathrm{e}}^{-t}\mathrm{d}t.$$For ${\displaystyle n\in \mathbb{N}}$, ${\displaystyle \mathrm{\Gamma }(n)=(n-1)!}$. ${\displaystyle \mathbb{E}(Z)=\frac{\alpha }{\alpha +\beta }}$.

${\displaystyle f_Z(z)}$ can take quite different shapes for different ${\displaystyle \alpha ,\beta }$ value.

Generalization: ${\displaystyle X_i\sim \mathrm{Gamma}({\alpha }_i,\lambda ),i=1,\cdots ,n}$. ${\displaystyle X_1,\cdots ,X_n}$ are independent. Then $$X_1+\cdots +X_n\perp \perp \frac{(X_1,\cdots ,X_n)}{X_1+\cdots +X_n}\sim \mathrm{Dirichlet}({\alpha }_1,\cdots ,{\alpha }_n),$$ with p.d.f $$f(x_1,\cdots ,x_n)=\frac{\mathrm{\Gamma }({\alpha }_1+\cdots +{\alpha }_n)}{\mathrm{\Gamma }({\alpha }_1)\cdots \mathrm{\Gamma }({\alpha }_n)}\mathbf{1}\{\sum _{i=1}^n{x}_i=1\}\prod _{i=1}^n\mathbf{1}\{x_i>0\}{x}_i^{{\alpha }_i-1}.$$

2 Order Statistics

2.1 Distribution of Order Statistics

Suppose ${\displaystyle X_1,\cdots ,X_n\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}}{\sim }F}$.

Take derivative:$$f_{X_{(1)}}(x)=\frac{\mathrm{d}}{\mathrm{d}x}{F}_{X_{(1)}}(x)=nf(x)[1-F(x){]}^{n-1}.$$

Similarly,

2.2 General Order Statistics

Recall quantile function:

Then, for general ${\displaystyle X_1,\cdots ,X_n\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}}{\sim }{F}_X}$, ${\displaystyle X_{(j)}\stackrel{\mathrm{d}}{=}{q}_X(U_{(j)})}$, where ${\displaystyle U_{(j)}\sim \mathrm{Beta}(j,n-j+1)}$ for ${\displaystyle j=1,\cdots ,n}$.

2.3 Joint Distribution of Order Statistics

2.4 Discussion on Gaps

Denote ${\displaystyle U_1=U_{(1)},L_j=U_{(j)}-U_{(j-1)},j=2,\cdots ,n}$, and ${\displaystyle L_{n+1}=1-U_{(n)}}$. ${\displaystyle L_j}$ are the gaps between order statistics. Then $$f_{L_1,\cdots ,L_n}(l_1,\cdots ,l_n)=n!\mathbf{1}\{l_1+\cdots +l_n<1\}\prod _{i=1}^n\mathbf{1}\{l_i>0\}.$$
RHS is the joint density of ${\displaystyle \frac{(W_1,\cdots ,W_{n+1})}{W_1+\cdots +W_{n+1}}}$, where ${\displaystyle W_1,\cdots ,W_{n+1}\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}}{\sim }\mathrm{Exp}(\lambda )=\mathrm{Gamma}(1,\lambda )}$. So $$\begin{array}{rl}& (L_1,\cdots ,L_{n+1})\stackrel{\mathrm{d}}{=}\frac{(W_1,\cdots ,W_{n+1})}{W_1+\cdots +W_{n+1}},\\ & (L_1,\cdots ,L_{n+1})\sim \mathrm{Dirichlet}(1,\cdots ,1),\\ & f(l_1,\cdots ,l_{n+1})=n!\mathbf{1}\{\sum _{i=1}^{n+1}{l}_i=1\}\sum _{i=1}^n\mathbf{1}\{l_i>0\}.\end{array}$$
The gaps are not independent, but ${\displaystyle (L_1,\cdots ,L_{n+1})}$ is exchangeable. So $$L_1\stackrel{\mathrm{d}}{=}\cdots \stackrel{\mathrm{d}}{=}{L}_{n+1},L_1=X_{(1)}\sim \mathrm{Beta}(1,n).$$
${\displaystyle \mathrm{\forall }i=1,\cdots ,n+1}$, ${\displaystyle \mathbb{E}[L_i]=\frac{1}{1+n}}$.