The secret to getting properly qualified participants

Why most screeners fail — and how to fix them.

Most online research — surveys and focus groups — relies on panels for recruitment.
People sign up to a panel, answer the questions, and get a reward.

It’s a practical system. But it has a flaw that many marketers never see.

Panel members slowly become experts at qualifying.

How respondents learn to “beat” screeners (recruitment questions)

After enough invitations, patterns appear.

People start to learn:

  • which topics tend to pay

  • which answers get them into more projects

  • which words look like the “right” ones

So if a screener asks:

“Do you buy pesto regularly?”

it doesn’t take long for people to work out that:

  • saying “yes” gets them into the study

  • saying “no” gets them screened out

So they say yes — whether it’s true or not.

This isn’t really dishonesty. It’s human nature. People follow incentives.

The result?

  • “buyers” who don’t actually buy

  • “heavy users” who barely engage

  • “loyalists” who tried the brand once

And then the research goes off-track before it has even begun.

The real problem: predictable questions

Most screeners are far too obvious.

They:

  • name the category

  • hint at the “correct” answer

  • use direct, literal wording

  • make it clear what the recruiter is looking for

So respondents can game them with very little effort.

The issue isn’t “bad respondents”.
It’s predictable questions.

The fix: write screeners that can’t be guessed

The solution is simple but important:

Write screening questions that don’t reveal what you’re really looking for.

In practice, that means:

  • Ask about adjacent behaviours.
    Instead of “Do you buy pesto regularly?”, ask about how they cook, where they shop, or which meal occasions they care about.

  • Avoid signalling the category.
    Don’t flag “pesto lovers” in the question. Build up a picture through small, neutral items.

  • Use wording with no obvious “right” answer.
    Keep options balanced and believable, so there’s no clear “qualify me” choice.

  • Include natural filters people can’t anticipate.
    For example, a combination of behaviours, store choices and product types that only true category users will match.

When your screener is disguised, people can’t work out how to “cheat” it.
They simply answer honestly, because there’s no benefit in doing anything else.

What you get when recruitment is done properly

Designing un-gameable screeners takes a little more thought. But the pay-off is big.

You get:

  • genuinely qualified participants

  • richer, more grounded answers

  • less noise in the data

  • findings you can actually trust

And you avoid the quiet disaster of basing decisions on the wrong people.

The takeway

Most screening problems don’t come from unreliable respondents.
They come from screeners that are too easy to read.

Disguise the screener, and you dramatically improve the quality of your sample — before you ask a single question about ideas, packs or messaging.

If you’re interested in how this links to question design, you might also like our article on

Why surveys can outperform focus groups for qualitative insight