While collaborating on a project, Jeffrey and I struck up a conversation on scales, and I mentioned that our company has had tremendous success replacing traditional 5-point scales with 3-point scales.
You can read the original article here, but the main arguments I make in advocating the 3-point scale are:
1. They are visually more appealing to respondents, and thus create more buy-in
2. They generate more variance than 1-5 scales (where ~90% of all respondents say 4 or 5)
3. They decrease response set
All three are important, but from a client’s viewpoint, the main advantage is that a 3-point scale allows us to distinguish between people who are perfectly satisfied with your product/service/brand from people who are REALLY enthusiastic about your product/service/brand.
And that’s crucial.
Interview respondents after they use a 5-point scale and you’ll hear some disconcerting comments that make you really question validity.
People who are extremely enthusiastic about your product/service/brand might only score it as a 4 and argue, “It’s terrific, but things can always be better than they are.”
Other people might assign a rating of 5; and when pressed for their reasoning, they’ll argue, “Well, things are fine; I have no complaints.” Not exactly a ringing endorsement.
In these two examples, you’d like the first person to have assigned a 5, and the second person to have assigned a 3.
I argue that the 3-point scale helps us improve our validity (though certainly not perfect it) by more accurately capturing respondents’ sentiments.
We’ll offer respondents options such as:
- Truly outstanding
- Perfectly acceptable
- Can use some improvement
A few things to note.
First, when using a 5-point scale, respondents with mild quibbles or complaints often feel like a score of 1, 2, or 3 is too harsh. So they will still assign a 4, or a 3 at worst. Thus, with the 3-point scale above, we have made the “negative” category sound less severe to encourage more people to choose it.
Clearly, the people with strong negative opinions (the few who DO use the 1 or 2 on a 5-point scale) will still use our “Can use some improvement” category. But that category now includes many more respondents who aren’t irate or dissatisfied at all, but who do have constructive criticism or minor concerns to share.
Further, we’ve made it more acceptable to answer with the middle category. Whereas respondents using a 1-5 scale feel like a score of 3 is harsh, our middle item is still framed positively.
Thus, we are encouraging more people to use the middle option, which will reduce the number of respondents who use our “best” category even when they really aren’t that enthusiastic.
This should leave us with only our most enthusiastic respondents in the most positive category, all of our “things are fine” respondents in the middle category, and anyone with a complaint in our bottom category, which is exactly what we want.
Is it perfect? Hah. But I think it gets us a little closer to perfection, and that’s a lot in research.
* * * * *
I recently observed Carbonite using a 3-point scale, which is a perfect choice for them.
They have a pretty simple business model. They back up your data. Either you think they’re doing it or you don’t. No need for 5-point scales here.
So their survey does a few smart things. First, it’s short and clearly formatted, and very unintimidating.
Next, the categories for the closed-ended questions are Satisfied/Neutral/Dissatisfied.
It’s a great choice. Some customers might have actively recovered data using their services and are happy. Others probably have never needed to retrieve files, and they’re neutral, while everyone else probably had an issue and is dissatisfied. Three types of customers, three categories of response.
If you’re a research geek, it’s gratifying to see conceptualization so perfectly match operationalization.