Sensory scientists are constantly asking our panelists to answer these questions, which as you can see can become very challenging because we're trying to measure a response that is already ambiguous.
Enter scaling. We've all seen them, those little surveys that are left in hotel rooms or handed to us by our server at a restaurant. Something like ----
How much do you agree with these statements:
The flavor of the overall meal exceeded my expectations.
My food was served hot and fresh.
The quality of the food was excellent.
And then you check off one of the following for each statement:
Neither agree nor disagree
Strongly agreeThese surveys are mostly useless for many reasons. First and foremost, surveys should provide objective information that is actionable, but instead surveys like this are designed by managers to stroke egos.
Lets break my simple survey down.
"The flavor of the overall meal exceeded my expectations" is leading. It encourages us to already think about how great the meal was, rather than rate the flavor objectively. Second, if the soup was terrible but the entree sublime, how do you answer this question? If you rate it based on the entree, then we mistakenly think the soup was perfect.
"My food was served hot and fresh."
This question is asking two things. Food can be hot and not fresh, fresh and not hot, both hot and fresh, or neither. Again, the answer is unactionable. Second, what does fresh mean? Fresh from the farm? Fresh/raw? Fresh-not-sitting-under-a-heatlamp-for-20-minutes? Or are they asking if the food tasted stale and old and moldy? Not sure here, but hopefully its not the latter.
"The quality of the foods is excellent."
Besides that this is another example of a leading question, the food is excellent compared to what? The corn chowder I hypothetically just ate was excellent compared to a low fat frozen microwavable corn chowder, but was mediocre at best compared to the chowder I had last Christmas at Ashley's in Little Rock. And what is quality? Microbiological quality? Color? Taste? Odor? Cooking? It is extremely ambiguous.
The rating doesn't have enough options, either. I hate to break it to all of you, but no matter how much time you spend trying to decide whether or not you "Slightly Agree" or "Strongly Agree" with a statement, most restaurants lump these two categories together and determine that you were simply happy.
Finally, you can't really average this scale together, at least not legitimately (sadly I'm sure some places do). This is because if you assign the numbers 1 to 5 to the categories you're assuming an equal interval of "agreement" or liking between the numbers. For example, is the distance between "strongly disagree - slightly disagree" the same as "neither agree nor disagree - slightly disagree"? Probably not for most of you. So it doesn't make sense to say that slightly agree is twice as much agreement as slightly disagree.
So what's the answer? Well, I will spare you the lecture in psychophysics (for now), but I assure you it involves a lunatic/genious named Gustav Fechner.a nightmare, astronomy, and the fate of your soul (or at least Fechner's). So simply trust me that this works:
Howard Schutz, a legend in the sensory community who has been publishing non-stop since around 1950. It happens to be for liking, so you'd have to rephrase your questions in the restaurant survey, but you could do it. What is important here is to notice the top and bottom anchor - "greatest imaginable liking." This helps to "center" everyone on the same scale, no matter their frame of reference. If my greatest imaginable food liking is my grandmother's Thanksgiving dinners, then this scale works, and if it is a meal cooked in the greatest restaurants in the world, it also works. Notice that the phrases aren't evenly distributed. That psychophysics stuff taught us how to space these things so that we CAN apply a ruler to them and average them together from different people. There's other scales (i.e. the army quartermaster 9 point hedonic scale) that also work in similar fashion. Your tax money via the army research labs developed both of these, btw.
My backup plan is to re-design restaurant surveys, because most are truly comical. I hope that restaurants don't give them too much weight, and now you don't either, unless of course I designed it!