What is the appropriate action when using poorly worded scales from previous literature? I work with questionnaire data and am fairly sensitive to good practice in questionnaire construction. Usually I generate my own, but when I draw on existing literature to use reliably reproduced measurement scales, sometimes the items are poorly worded or vague enough that I would not consider them appropriate or valid. Consider the item:

"Weeds should be eradicated because they inhibit the full development of useful and ornamental plants."

A participant can say that they disagree with this item. So do they think weeds shouldn't be eradicated? Or that they should be eradicated, but not for the reason given. Maybe they think that weeds do inhibit the full development of useful plants, but that is not sufficient cause for eradication of weeds. The question is vague enough that it doesn't confer any useful information about the participant's broader view on weeds except in the one fringe case where they agree with it as stated. Clearly it should be split into two items and already the scale is much improved by it.

There are other examples that are much more shockingly bad, but have been reproduced enough times that they form reliable scales. Is it better to use them as-is, with all their flaws, or to use them as fully as possible while fixing any glaring errors, or not to use them at all?
If you change the items in a questionnaire, you have a different questionnaire. All the psychometric properties such as reliability or validity and the normative data no longer apply. You need to be able to compare your measurement to a normative sample to receive a meaningful measurement. If you change a questionnaire, you have to test it against your construct, calculate its reliability etc., and normalize it over a large representative sample. Otherwise any conclusions you draw from your experiments will be at best tentative and potentially false.

The meaning of a questionnaire item is not the literal meaning you make of that "sentence", but the relation that the behavior of answering this item has with other behavior that you know represents your construct. For example, the item "Weeds should be eradicated because they inhibit the full development of useful and ornamental plants." is from a questionnaire on environmental attitudes (Milfont &amp; Duckitt, 2004) and measures "necessity of development" (NoD). If a significant number of your sample answer "yes" to the question about weeds, and they also rate high in NoD in a valid NoD task, then this item can be used to measure NoD. If you change the words, this correlation might no longer hold.




Milfont, T. L., &amp; Duckitt, J. (2004). The structure of environmental attitudes: A first- and second-order confirmatory factor analysis. Journal of Environmental Psychology, 24, 289–303. Available online at http://gip.uniovi.es/gdiyad/docume/arti_jep01.pdf.