What is the right sample size in quantitative research?

Image courtesy of: Glenn Carstens-Peters @glenncarstenspeters

There is no single “right” sample size for quantitative research. A useful starting point in many market research surveys is around 100 respondents in each subgroup you want to analyse separately — but the appropriate sample depends on the precision you need, the comparisons you want to make and your budget.

Here is a practical way to think about it.

Sample sizes and depth of analysis

For example, you might do a simple survey with a sample size of 100 “completes” (i.e. 100 respondents who have completed the whole questionnaire).

But if you know that you will want to analyse the survey results for women separately to those for men, then you might decide you need 100 completes for each, giving a total sample size of 200 completes.

You may also decide you will want to analyse the results for younger and older respondents, split by males and females. So, you will want 100 completes in each group, giving a total required sample size of 400 completes.

In practice, market researchers often work backwards from the analysis they expect to conduct: decide which groups need to be examined separately, then make sure there are enough respondents in each.

The precision of results

Some researchers, who use the above approach, might decide that 100 completes per group-to-analyse will not give sufficiently precise results. They might choose to have 200 completes per group, for example, or more.

It is easiest to explain this with an example. Suppose you run a survey and ask respondents what is their favourite day of the week? And imagine 40% of them say “Saturday”. That 40% is an estimate, not a perfectly precise measure of the wider population. This is because the survey would give a slightly different result each time it is run with a new set of respondents, but the % saying “Saturday” will generally be somewhere in a range around 40%.

Using the conventional 95% confidence level, and assuming a simple random sample, a result of 40% from 100 respondents has a margin of error of roughly ±10 percentage points — so approximately 30% to 50%.

With 200 respondents, the margin is roughly ±7 points — approximately 33% to 47%.

The larger the sample, therefore, the more precise the estimate tends to be. But the improvement gets progressively smaller as the sample grows.

But survey respondents cost money. So researchers planning a survey need to think about whether the gain they will get in terms of narrower percentage ranges from larger sample sizes is worth the additional cost.

What determines the right sample size?

The right sample size depends on what you need the research to tell you. In practice, the main considerations are:

• How many groups you need to analyse separately. If you need reliable results for men and women, different age groups or users and non-users, each subgroup needs enough respondents of its own.

• How much precision you need. Larger samples produce narrower margins of error, although the improvement becomes progressively smaller as sample size increases.

• How small a difference you need to detect. If an important decision depends on distinguishing between two fairly similar results, you may need a larger sample.

• How common the target audience is. Specialist or low-incidence groups can be much more expensive to recruit, which may affect what is practical.

• Your budget. Increasing sample size improves precision, but it also increases fieldwork cost.

For many market research projects, the most useful starting point is therefore not simply “How large should the sample be?” but “What comparisons do we need to make, and how precise do those results need to be?”

Is 100 a good sample size for quantitative research?

Sometimes. A sample of 100 can be perfectly useful for directional market research, particularly if you mainly need results for the sample as a whole.

But if you want reliable comparisons between several subgroups — for example men versus women, different age groups or users versus non-users — 100 respondents in total may quickly become too small. What matters is not simply the total sample but the number of respondents left in each group you need to analyse.

Contact me if you want to discuss the right sample for a particular study.