Large Sample Techniques for Statistics

Abstract

In a way, the world is made up of approximations, and surely there is no exception in the world of statistics. In fact, approximations, especially large sample approximations, are very important parts of both theoretical and applied statistics. The Gaussian distribution, also known as the normal distribution, is merely one such example, due to the well-known central limit theorem. Large-sample techniques provide solutions to many practical problems; they simplify our solutions to difficult, sometimes intractable problems; they justify our solutions; and they guide us to directions of improvements. On the other hand, just because large-sample approximations are used everywhere, and every day, it does not guarantee that they are used properly, and, when the techniques are misused, there may be serious consequences. Example 1 (Asymptotic χ2 distribution). Likelihood ratio test (LRT) is one of the fundamental techniques in statistics. It is well known that, in the “standard” situation, the asymptotic null distribution of the LRT is χ2, with the degrees of freedom equal to the difference between the dimensions, defined as the numbers of free parameters, of the two nested models being compared (e.g., Rice 1995, pp. 310). This might lead to a wrong impression that the asymptotic (null) distribution of the LRT is always χ2. A similar mistake might take place when dealing with Pearson’s χ2-test—the asymptotic distribution of Pearson’s χ2-test is not always χ2 (e.g., Moore 1978). Example 2 (Approximation to a mean). It might be thought that, in a large sample, one could always approximate the mean of a random quantity by the quantity itself. In some cases this technique works. For example, suppose X1,...,Xn are observations that are independent and identically distributed (i.i.d.) such that μ = E(X1) = 0. Then one can approximate E(n i=1 Xi) = nμ by simply removing the expectation sign, that is, by n i=1 Xi. This is because the difference n i=1 Xi − nμ = n i=1(Xi − μ) is of the order O( √n), which is lower than the order of the mean of n i=1 Xi. However, this technique completely fails if one considers (n i=1 Xi)2 instead. To see this, let us assume for simplicity that Xi ∼ N(0, 1). Then E(n i=1 Xi)2 = n. On the other hand, since n i=1 Xi ∼ N(0, n), (n i=1 Xi)2 = n{(1/ √n) n i=1 Xi}2 ∼ nχ2 1, where