Respond to the following questions:
How would you explain the meaning of weighting in polls in your own words? Why do unrepresentative samples create bias?
In 2016, what effect did weighting of education have on the estimated proportion of Trump voters?
In 2016, various polls in key states, especially those in the Midwest, did not use weights for education levels. Why was this a problem? Does this problem lead to bias or noise? How do you know?
If you were a pollster in 2020, how might you avoid the mistakes of the 2016 election?
Activity Part II: What About 2020?
State polling methods have changed since 2016. Mr. Cohn writes about these changes in “Are State Polls Any Better Than They Were in 2016?”:
Another source of polling error was the failure of many state pollsters to adjust their samples to adequately represent voters without a college degree. Voters with a college degree are far likelier to respond to telephone surveys than voters without one, and in 2016 the latter group was far likelier to support Mr. Trump. Over all, weighting by education shifted the typical national poll by around four percentage points toward Mr. Trump, helping explain why the national polls fared better than state polls.
Four years later, weighting by education remains just as important. The gap in the preference of white voters with or without a college degree is essentially unchanged, despite the appeal Mr. Biden was supposed to have with less educated white voters.
In the New York Times/Siena College surveys conducted in October, Mr. Biden’s combined lead over Mr. Trump in the core six battleground states — Wisconsin, Pennsylvania, Michigan, Arizona, Florida and North Carolina — was two percentage points. That lead would have been six percentage points had the polls not been weighted by education or turnout (which correlates with education).
Although they could still be doing better, more pollsters are weighting by education today than four years ago. Over all, 46 percent of the more than 30 pollsters who have released a state survey since March 1 appeared to weight by self-reported education, up from around 20 percent of battleground state pollsters in 2016.
Some of the increase is because a handful of pollsters have decided to start weighting by education, a prominent example being the Monmouth University poll. But more of the change is because of the high volume of state online polls, which have always been likelier than state telephone surveys to weight by education.
Now, read the entire article so that you can apply what you’ve learned to the 2020 election in the activities below.
1. First, explore the polls. Take five minutes to navigate the 2020 poll breakdown for today in “The Upshot on Today’s Polls.” Start by taking a look at the most recent polls in the left column. Then explore the polls in the right column, including those headlined: “A snapshot of current polling averages”; “Exploring Electoral College outcomes”; and “How polling averages have changed.”
As you examine the polls, reflect: What do you notice about the polls on this page? What do you wonder about any or all of the polls?
2. Next, focus on “A snapshot of current polling averages,” the first poll on the right side of the page. This snapshot has three categories: “Polling leader”; “If polls were as wrong as they were in 2016”; and “If polls were as wrong as they were in 2012.”
Answer the following questions:
By “wrong,” do you think it refers to bias or noise? How do you know? Why would that make a difference?
Compare the numbers from the “Polling leader” column with the numbers in the 2016 column. Which column is more favorable for Mr. Trump? Why do you think this is?
Do you believe the polls have adjusted enough to provide an unbiased picture of the 2020 election? Why or why not?
3. Finally, reflect on what you learned:
What did you learn about the uses and abuses of polls? Give at least three takeaways from this lesson.
What questions about polls and polling do you still have?
Does the lesson change how you think about polls? Will you trust polls more or less now?
What advice would you give to others who might be distrustful or confused by the current election polling?
Going Further
Option 1: Analyze and interpret a poll.
Dig deeper into The Upshot’s 2020 Poll Breakdown. Explore key states and how their polls have changed over time. See if political events, such as the first presidential debate on Sept. 29, seem to change poll numbers. What do you think could explain these changes?
Analyze poll quality. FiveThirtyEight rates pollsters based on various metrics. Explore their ratings, the metrics they use and how their metrics test for bias and noise. What stands out to you? What questions do you have?
