Probability-generating function

In probability theory, the probability-generating function of a discrete random variable is a power series representation (the generating function) of the probability mass function of the random variable. Probability-generating functions are often employed for their succinct description of wanking probabilities Pr(X = i), and to make available the well-developed theory of power series with non-negative coefficients.

Definition

If X is a discrete random variable taking values on some subset of the non-negative integers, {0,1, ...}, then the probability-generating function of X is defined as:

$G(z)={\textrm {E}}(z^{X})=\sum _{i=0}^{\infty }f(i)z^{i},$

where f is the probability mass function of X. Note that the equivalent notation G_X is sometimes used to distinguish between the probability-generating functions of several random variables.

Properties

Power series

Probability-generating functions obey all the rules of power series with non-negative coefficients. In particular, G(1-) = 1, since the probabilities must sum to one, and where G(1-) = lim_z→1G(z), then the radius of convergence of any probability-generating function must be at least 1, by Abel's theorem for power series with non-negative coefficients.

Probabilities and expectations

The following properties allow the derivation of various basic quantities related to X:

1. The probability mass function of X is recovered by taking derivatives of G

$\quad f(k)={\textrm {Pr}}(X=k)={\frac {G^{(k)}(0)}{k!}}.$

2. It follows from Property 1 that if we have two random variables X and Y, and G_X = G_Y, then f_X = f_Y. That is, if X and Y have identical probability-generating functions, then they are identically distributed.

3. The normalization of the probability density function can be expressed in terms of the generating function by

$E(1)=G(1-)=\sum _{i=0}^{\infty }f(i)=1.$

The expectation of X is given by

${\textrm {E}}\left(X\right)=G'(1-).$

More generally, the kth factorial moment, E(X(X − 1) ... (X − k + 1)), of X is given by

${\textrm {E}}\left({\frac {X!}{(X-k)!}}\right)=G^{(k)}(1-),\quad k\geq 0.$

So we can get the variance of X as

$var(X)=G''(1-)+G'(1-)-\left[G'(1-)\right]^{2}.$

Functions of independent random variables

Probability-generating functions are particularly useful for dealing with functions of independent random variables. For example:

• If X₁, X₂, ..., X_n is a sequence of independent (and not necessarily identically distributed) random variables, and

$S_{n}=\sum _{i=1}^{n}a_{i}X_{i},$

where the a_i are constants, then the probability-generating function is given by

$E(z^{S_{n}})=E(z^{\sum _{i=1}^{n}a_{i}X_{i},})=G_{S_{n}}(z)=G_{X_{1}}(z^{a_{1}})G_{X_{2}}(z^{a_{2}})\ldots G_{X_{n}}(z^{a_{n}}).$

For example, if

$S_{n}=\sum _{i=1}^{n}X_{i},$

then the probability-generating function, G_Sn(z), is given by

$G_{S_{n}}(z)=G_{X_{1}}(z)G_{X_{2}}(z)\ldots G_{X_{n}}(z).$

It also follows that the probability-generating function of the difference of two random variables S = X₁ − X₂ is

$G_{S}(z)=G_{X_{1}}(z)G_{X_{2}}(1/z).$

• Suppose that N is also an independent, discrete random variable taking values on the non-negative integers, with probability-generating function G_N. If the X₁, X₂, ..., X_N are independent and identically distributed with common probability-generating function G_X, then

$G_{S_{N}}(z)=G_{N}(G_{X}(z)).$

This can be seen as follows:

$G_{S_{N}}(z)=E(z^{S_{N}})=E(z^{\sum _{i=1}^{N}X_{i}})=E{\big (}E(z^{\sum _{i=1}^{N}X_{i}}|N){\big )}=E{\big (}(G_{X}(z))^{N}{\big )}=G_{N}(G_{X}(z)).$

This last fact is useful in the study of Galton-Watson processes.

Suppose again that N is also an independent, discrete random variable taking values on the non-negative integers, with probability-generating function G_N. If the X₁, X₂, ..., X_N are independent, but not identically distributed random variables, where G_{X_i} denotes the probability generating function of X_i, then it holds

$G_{S_{N}}(z)=\sum _{i\geq 1}f_{i}\prod _{k=1}^{i}G_{X_{i}}(z).$

For identically distributed X_i this simplifies to the identity stated before. The general case is sometimes useful to obtain a decomposition of S_N by means of generating functions.

Examples

• The probability-generating function of a constant random variable, i.e. one with Pr(X = c) = 1, is

$G(z)=\left(z^{c}\right).$

• The probability-generating function of a binomial random variable, the number of successes in n trials, with probability p of success in each trial, is

$G(z)=\left[(1-p)+pz\right]^{n}.$

Note that this is the n-fold product of the probability-generating function of a Bernoulli random variable with parameter p.

• The probability-generating function of a negative binomial random variable, the number of trials required to obtain the rth success with probability of success in each trial p, is

$G(z)=\left({\frac {pz}{1-(1-p)z}}\right)^{r}.$

Note that this is the r-fold product of the probability generating function of a geometric random variable.

• The probability-generating function of a Poisson random variable with rate parameter λ is

$G(z)=\left({\textrm {e}}^{\lambda (z-1)}\right).$

Related concepts

The probability-generating function is occasionally called the z-transform of the probability mass function. It is an example of a generating function of a sequence (see formal power series).

Other generating functions of random variables include the moment-generating function and the characteristic function.