Demonic Ninja Llama

He can't keep quiet anymore

OMG Those Scary Wingnut Republicans!!!!!

The Lefty side of the Blogosphere is buzzing today about a new Harris poll that reputedly portrays the attitudes people have towards Barack Obama, and it is considered especially newsworthy for the percentages of Republicans who allegedly view him as a “socialist”, a “muslim”, or “the Anti-Christ.”

A summary of some of the “results” of this poll:

Majorities of Republicans believe that President Obama:

  • Is a socialist (67%)
  • Wants to take away Americans’ right to own guns (61%)
  • Is a Muslim (57%)
  • Wants to turn over the sovereignty of the United States to a one world government (51%); and
  • Has done many things that are unconstitutional (55%).

Also large numbers of Republicans also believe that President Obama:

  • Resents America’s heritage (47%)
  • Does what Wall Street and the bankers tell him to do (40%)
  • Was not born in the United States and so is not eligible to be president (45%)
  • Is the “domestic enemy that the U.S. Constitution speaks of” (45%)
  • Is a racist (42%)
  • Want to use an economic collapse or terrorist attack as an excuse to take dictatorial powers (41%)
  • Is doing many of the things that Hitler did (38%).

Even more remarkable perhaps, fully 24% of Republicans believe that “he may be the Anti-Christ” and 22% believe “he wants the terrorists to win.”

Now I think it is obvious by my attitude that I have suspicions about these results.  Something here does not pass the smell test.  The Harris press release states that “a new book, Wingnuts: How the Lunatic Fringe Is Hijacking America by John Avlon describes the large numbers of Americans who hold extreme views of President Obama. This Harris Poll seeks to measure how many people are involved.”  So, either this poll was commissioned by Mr. Avlon, who obviously has his bias, or it was influenced by his argument.  That there is a certain expectation of a result is the first thing that concerns me.  It’s too easy to let your hypothesis guide your methodology in such a way as to give you the result you desire.

Second, as far as I can tell, the question was framed as a strictly True/False dichotomy.  No room for uncertainty.  No “I don’t know” option.  So, a respondent, when faced with this choice, if he really doesn’t know or have a strong feeling about a question, but suspects that there might be some chance of it being true, will pick the “True” option.  This seems to me a likely explanation for the question on whether Obama was not born in the US.  Most people cannot prove that he was, and so they don’t feel certain about the “False” option.  There is enough uncertainty in their minds that they pick the “True” option, when they really don’t know.  I also suspect that simply by framing the question that way, there was a bias for “True” to be picked more often.  If the question had been phrased, “Barack Obama was born in the US, and is therefore eligible to be President,”  I suspect that the percentage who would answer “True” would have been greater than the number who answered “False” with the survey’s wording.  Answering “False” to a negative statement requires a closer reading of the question, and a greater certainty of one’s own opinion.

By way of illustration, if I myself had taken this quiz, when confronted with the choice of “true or false” as to whether Obama is a socialist, I believe I would have answered “true.”  I don’t know if he really is.  But I certainly don’t feel confident that he is not.  So “false” seems like a less appropriate answer to me.

Let me say here that I am certain there is a percentage of Americans who do believe these things about Mr. Obama.  I happen to agree that he had done things that are unconstitutional (but that’s another blog post).  But even that question is problematic.  It states:  “he has done many things that are unconstitutional.” (emphasis mine)  How am I to approach such a question?  I think he has done unconstitutional things, but “many”?  How many is “many”?  I would lean towards answering yes on this, simply because I feel he has done some things.  But if I were to state to you, “I feel Obama has done many unconstitutional things,” I do sound kinda nutty.  The respondent is trapped.

So, yes, there is some basis for this poll’s results.  But I suspect the numbers are inflated.

An Online Poll

I have no experience or expertise to judge the reputation of the Harris organization.  I’m not a statistician, nor do I play one on TV.  However, examining the fine print on the poll news release yields this:

This Harris Poll was conducted online within the United States March 1 and 8, 2010 among 2,320 adults (aged 18 and over). Figures for age, sex, race/ethnicity, education, region and household income were weighted where necessary to bring them into line with their actual proportions in the population. Propensity score weighting was also used to adjust for respondents’ propensity to be online.

All sample surveys and polls, whether or not they use probability sampling, are subject to multiple sources of error which are most often not possible to quantify or estimate, including sampling error, coverage error, error associated with nonresponse, error associated with question wording and response options, and post-survey weighting and adjustments. Therefore, Harris Interactive avoids the words “margin of error” as they are misleading. All that can be calculated are different possible sampling errors with different probabilities for pure, unweighted, random samples with 100% response rates. These are only theoretical because no published polls come close to this ideal.

Respondents for this survey were selected from among those who have agreed to participate in Harris Interactive surveys. The data have been weighted to reflect the composition of the adult population. Because the sample is based on those who agreed to participate in the Harris Interactive panel, no estimates of theoretical sampling error can be calculated.

All sorts of things jumped out at me.  Now, I understand online polling is what Harris does, and I assume they are good at it.  But online polling is fraught with perils.  How can you be sure that your sample is an accurate reflection of the broader society, when it is already biased towards those who are A) online and  B) willing or interested in participating in your survey.  Harris states that they weight the respondents for age, sex, education, etc. to bring them into line with the general population, and that they weight as well for “online propensity.”  Fine, but I’d like to know more about this.  The data are being manipulated behind the scenes, and I couldn’t find much info on their website about their methodology (probably because it is proprietary).

Then there is this marvelous sentence:

All sample surveys and polls, whether or not they use probability sampling, are subject to multiple sources of error which are most often not possible to quantify or estimate, including sampling error, coverage error, error associated with nonresponse, error associated with question wording and response options, and post-survey weighting and adjustments.

In my opinion, that invalidates the whole exercise.  Harris does not even offer a margin of error.  For all we know, the potential errors and adjustments involved in this particular survey could mean that the numbers are off by a significant scale of magnitude.  Instead of 24% thinking Obama is the anti-Christ, which sounds like a lot, it could just as easily be 14%, which seems a lot smaller.

So, my liberal friends, feel free to allow these purported poll results to reinforce your view of Republicans as scary nutjobs.  But I hope open-minded people will recognize that this poll, while accurately identifying a deep unease with Barack Obama as President, does not reveal specific opinions or the breadth of their sway with any kind of precision.

Update: If my comments aren’t convincing to you, take note of what the director of polling at ABC News has to say about this poll.

A wealth of academic literature, neatly summarized here, demonstrates that questions constructed in this fashion – true/false, agree/disagree – carry a heavy dose of what’s known as acquiescence bias. They overstate agreement with whatever’s been posited, often by a very substantial margin….

Using all negative statements, rather than a mix of negative and positive ones, reflects another non-standard approach, one that can further bias responses.

Read the whole thing.  Emphasis mine.

Advertisements

March 24, 2010 - Posted by | Uncategorized

No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: