RANDOM SAMPLING

Assuming that you’re not going to choose a biased sample, the next question is: how do I select a fair sample?

The gold standard is (simple) random sampling.

The word “random” has a very specific meaning in Statistics.

A sample is random if:

1. Every member of the population has an equal chance of being selected, and

2. Every possible sample (of the size that is to be collected) has an equal chance of being the sample.

The first condition is pretty self-explanatory. The second one is less obvious. It’s easier to understand if we look at a sampling method that passes the first test, but fails the second.

Suppose I wish to take a sample of size 10 from a population of size 60. I first construct a list of the 60 members of the population. I then roll a die and I select whichever of the first six people in the list corresponds to the number on the die: so if I roll a five, I take the fifth person.

After that, I then take every sixth person in the list. So if I rolled a five, I take the fifth person, and then the 11th, 17th, 23rd, and so on, up to the 59th person.

Each of the first six people has an equal chance of being selected, assuming the die is fair. Since the remaining nine people in the sample are determined by who is chosen first, they also all have an equal chance of being selected — though only because the size of the sample divides the size of the population. 60 is divisible by 10. Specifically, 60 ÷ 10 = 6, and this was the number of sides on the die, and the size of the jump from one sample member to the next.

Had there been 61 people in the population, one of two things could go wrong. Either we stick with having a sample of size 10 no matter what, in which case the 61st person can never be chosen. Or we take every sixth person after the first person has been determined by the roll of the die, in which case sometimes we will have a sample of size 11.

This method is called systematic sampling — literally sampling with a system. And, provided the sample size divides the population size, Test 1 of randomness — that every member of the population has an equal chance of being selected — is satisfied.

However, systematic sampling fails Test 2: not every possible sample has an equal chance of being the sample. In fact, in the example we’ve been discussing, only ten samples are possible at all: {1, 7, 13, …, 55}, {2, 8, 14, …, 56}, and so on, up to {6, 12, 18, …, 60}.

There is a huge number of different samples of size 10 you could take from a population of size 60 — over 75 billion, in fact. This method only results in ten of these! For example, the sample {1, 2, 3, …, 10} has zero chance of being the sample.

This doesn’t make this sampling method bad. It’s just not random. Indeed, systematic sampling has a lot going for it. It’s simple to implement and it can often result in a satisfactorily representative sample.

On the other hand, random sampling can result in a sample that is not representative: it could generate the sample {1, 2, 3, …, 10}. If it did, how would you feel about that sample: the first ten people on the list? Politically it might look suspicious; despite being random, people might not believe that it was. And, depending on the order in which the population was listed, it may be highly unrepresentative. Staff lists of teachers in schools are sometimes arranged by seniority: the first ten people on the list would be the ten most senior. Or sometimes they are grouped by teaching subject: so the first ten people might only represent two academic departments in the school.

One way to address this issue is to construct a stratified sample. Suppose we know that 40% of a particular population is male and 60% is female. We take a random sample from each sub-population, called a stratum. Crucially the size of each sample is proportional to the size of the stratum. So, in this example, a sample of total size 20 would comprise 8 men and 12 women. This way the sample is guaranteed to be representative of the structure of the population.

All statistical theory is based on the assumption that your sample is random. Yet, as we shall see later, samples are rarely collected at random.