Suppose you have a 3 x 3 experimental design in a stated preference survey with cost $100, $250, and $500 and benefit is 25%, 50% and 75% improvement (in quality or quantity) variables. You ask more than one choice question. SurveyMonkey spits out the data with a number code which you assume is ordinal. For example:
| Cost | Code |
| $100 | 1 |
| $250 | 2 |
| $500 | 3 |
But actually you find that the code is not in order. For example:
| Cost | Code |
| $100 | 3 |
| $250 | 2 |
| $500 | 1 |
Similar things happen to the benefit variable. There is no pattern to the number code assignment across choice question. So, when coding the variable in your statistical software you must compare the the qualitative data (don't forget your labels John Whitehead!) to the quantitative data.
SurveyMonkey recently "upgraded" their product and this quirk seems to be an unintended result.
*For example, no scope effect! BP-funded economists are going to hop on that train big-time.
