Are you flipping your online survey rating scales?

This might seem a strange question, but if you ever run online surveys you’ll probably recognize what it is referring to.  Essentially, if you have a survey question which asks respondents to rate something, should your scale go from negative to positive, or the other way round?

The reason this is important is the “first order effect”. This basically says that, all other things being equal, respondents are more likely to click on the first response to a closed question than they are to click any other answer.

To overcome this effect it is common in online surveys to randomise the order in which closed question responses are displayed.  But this is not so easy for rating scale questions, apart, that is, from 3 point scales, in which the first and last elements can be randomised, so that they, in effect, swop places with each other.

But there is something called “flipping”. This means that within a survey rating scales can be shown from negative-positive to some respondents and positive- negative to other respondents.  And this can be done on vertical or horizontal scales, including those used within grid questions.

And just to mention, if flipping is used within a questionnaire then it will consistently show any rating scale questions the same way round. In other words, if a respondent is shown the first scale negative-positive, then all other rating scale questions in the questionnaire will also be shown negative-positive.

Of course, there is also the matter of how the question itself is phrased (whether negative-positive or vice versa), because this should accord with the order of the scale.  Luckily this is not difficult, subject to what software is being used, of course.

Finally, just to mention that if you are using a third party to script your online questionnaire then they will not necessarily think about or mention flipping.  So it will be up to you to brief them.