Thursday, March 7, 2013

SLOPpy Survey

A SLOP, in survey parlance, is a Self Selected Opinion Poll.  In other words, people choose to participate, so the "sample" is not random.  Confused?  I found today an example of one here.  The lede says:
MANILA, Philippines—Poverty is the number one concern that candidates should focus on, INQUIRER.net’s survey reveals on Thursday. 
Okay, it's hard to argue with doing something about poverty and, at my last glance, a sample size of 5,980.  Except, of course, there's nothing random about this sample, so we can't say with any certainty that it represents the population as a whole.

We are seeing a teeny tiny shift away from random surveys, mainly because they're expensive to do and the response rates to telephone surveys -- the gold standard of the industry -- are dipping into the single digits.  A lot of folks are turning to huge panels, with a large N (that's number of folks surveyed).  I'm not buying it, at least not yet.  If you read the story linked to above, you'll notice at no time, unless I missed it, does the writer tell us the survey has no scientific value whatsoever.  That's the main problem here.  It's okay to run these surveys, for serious and for fun topics, but it's not okay to paint them as being scientifically sound.  Instead, high up in the story, admit that it's mostly bullshit.

No comments: