Comments on: How to Read Polls

By: Spencer

Spencer — Wed, 26 Nov 2008 20:46:31 +0000

Wow, I found this fascinating! Great explanation. Who knew that statistics could be manipulated in so many ways… (hint of sarcasm)

By: danvk

danvk — Sun, 28 Sep 2008 19:32:06 +0000

That would be true if the pollsters reported a standard deviation, the formula for which is:

sigma = sqrt(p * (1-p) / N)

But what they’re really reporting is 1/sqrt(N). Whether that quantity should be referred to as a “margin of error” is mostly semantics. It’s the quantity that’s reported, so it’s the quantity I based my calculations on.

By: starwed

starwed — Sun, 28 Sep 2008 07:11:48 +0000

I believe you made an error here... The margin of error depends not only on the sample size, but also the recorded result: Margin of error

By: danvk

danvk — Fri, 26 Sep 2008 18:05:31 +0000

That’s a really interesting podcast. I wonder what the real margin of error would look like on one of these polls.

I figured out where the 1.644 was coming from. It’s equal to

sqrt(2) * erfinv(2*0.95 – 1) = 1.64485

Plug in values other than 0.95 to get other confidence levels.

By: Matt Rosencrantz

Matt Rosencrantz — Fri, 26 Sep 2008 04:54:28 +0000

I found this podcast:
http://www.econtalk.org/archives/2008/07/rivers_on_polli.html

fascinating on why these margins of error systematically underestimate the true uncertainty. Of course I don’t know anything about the particular poll you are pointing too, but if what this speaker says is true, sloppy statistics are pretty pervasive. In particular the part about not adjusting error calculations for sample re-weighting.