Does Mode Matter For Modeling Political Choice? Evidence From the - - PowerPoint PPT Presentation

Does Mode Matter For Modeling Political Choice? Evidence From the 2005 British Election Study

by Harold Clarke David Sanders Marianne Stewart Paul Whiteley

Survey Modes For National Election Studies - Pros & Cons

• Traditional In-Person - VERY

Expensive (Big Chunks of NSF & ESRC PSCI Budgets!), Very Slow

• RDD Telephone - Increasingly

Expensive, Fast

• Self-Completion Mail Questionnaires -

Inexpensive, Slow

• Internet - Inexpensive, VERY Fast

The Rap on Internet Surveys: Limited Coverage and NonProbability Samples

• In-Person Surveys - The ANES Gold Standard
• RDD Surveys - The CES Gold Standard
• In-Person & RDD Surveys - Probability Samples,

but Potential Respondents Select Out

• Unit Non-Response - Now Large in both In-Person

and RDD, Sometimes Huge in RDD

• Internet Surveys - Non-Probability Samples (but

KN), Potential Respondents Select In

• All Modes Have Selection Biases

The 2005 BES

Figure 1: Probability and Internet Panel Survey Design in the 2005 British Election Study

Wave 1 Pre-election Probability Sample, Face-to-Face N=3589 128 PSUs Wave 2 Post-election Probability Sample, Face-to-Face N=4161 Including top-up, mail-back; 128 PSUs Wave 3 One Year Out Internet users from Wave 2 Probability Sample, Internet Survey Method N=983

BES 2005 CORE FACE-TO-FACE PANEL BES 2005 INTERNET CAMPAIGN PANEL SURVEY:

Wave 1 Pre-campaign Baseline Survey N=7793 Wave 2 Campaign survey 200 interviews per Day for 30 days N=6068 Wave 3 Post-election Interview N=5910 Wave 4 One Year Out Interview N=6186 Probability Internet sample versus traditional Internet sample Sampling Experiment: Face-to-face vs Internet sampling experiment (2) Face-to-face vs Internet sampling experiment (1)

Survey Houses

• In-Person -> National Center for

Social Research ‘Natcen’ - conducted 1983 - 1997 BES

• Internet -> YouGov - also

conducting NSF-sponsored 'Valence Politics and the Dynamics

• f Party Support' Project

Figure 2. Reported Vote In-Person and Internet Post-Election Surveys and Actual Vote in Britain, 2005 General Election

39.6 31.7 22.4 6.3 36.1 30.8 24.4 8.7 36.2 33.2 22.6 7.9

5 10 15 20 25 30 35 40 45 50 Labour Conservative Liberal Democrat Other Parties Percentage In-Person Internet Actual Vote

Figure 3. Party Identification in Pre- and Post- Election In-Person and Internet Surveys

34 25 12 6 23 33 24 11 9 23 37 26 13 6 18 36 25 12 7 21

5 10 15 20 25 30 35 40 45 50 Labour Conservative Liberal Democrat Other Parties None, DK Percentage In-Person Pre Internet Pre In-Person Post Internet Post

Figure 4. Reported Turnout in In-Person and Internet Surveys and Actual Turnout in 2005 British General Election

71.7 82.9 61.1

10 20 30 40 50 60 70 80 90 100 In-Person Internet Actual Percentage

Data Quality? Comparative Overeports

• f Turnout in National Election Studies

34.7 24.8 24.7 17.3 37.9 17.8 20.2 27.2 25.2 21.5 18.7 9.9 10.6 21.8 5 10 15 20 25 30 35 40 2004 NAES, RDD 2000 NAES, RDD 2004 ANES, IP, trad 2004 ANES, IP, rev 2002 ANES, RDD, trad 2002 ANES, RDD, rev 2000 ANES, IP, rev 2000 ANES, RDD, rev 2004 CES, RDD 2000 CES, RDD 2001 BES, RDD 1964-2001 BES, IP 2005 BES, IP 2005 BES, internet

Composite Labour Vote Model

• Party Leader Images
• Party Best Most Important Issue
• Party Identification
• Party-Issue Proximities
• Economic Evaluations
• Opinions about Iraq War
• Tactical Voting
• Demographics

Table 5. Comparative Performance of Rival Party Choice Models

McFadden R2 McKelvey R2 AIC BIC

• A. Models Estimated Using In-Person Survey Data

Social Class .01 .02 2794.20 2805.51 All Demographics .03. .06 2753.54 2810.08 Economic Evaluations .07 .13 2633.38 2644.69 Issue Proximities .12 .22 2507.63 2530.25 Most Important Issue .27 .40 2079.75 2108.02 Party Identification .37 .48 1794.87 1823.14 Leader Images .40 .65 1692.95 1715.56 Composite Model .58 .76 1256.45 1414.76

• B. Models Estimated Using Internet Survey Data

Social Class .01 .01 6409.16 6422.16 All Demographics .02 .04 6328.65 6400.17 Economic Evaluations .14 .24 5564.96 5577.97 Issue Proximities .19 .34 5229.46 5255.52 Most Important Issue .33 .48 4299.71 4332.29 Party Identification .36 .50 4163.88 4196.45 Leader Images .44 .64 3617.93 3643.94 Composite Model .59 .76 2715.98 2898.40

Table 6. Rival Models of Labour Voting Comparative Predictive Power

In-Person Survey Internet Survey % Correctly % Correctly Predicted Lambda Predicted Lambda Models Social Class 60.5 .00 63.9 .00 All Demographics 63.1 .07 64.3 .01 Economic Evaluations 65.3 .12 70.4 .18 Issue Proximities 67.9 .19 72.3 .23 Most Important Issue 78.3 .45 80.8 .47 Party Identification 83.5 .59 82.6 .52 Leader Images 82.0 .55 83.7 .55 Composite Model 87.3 .68 88.6 .68

Figure 5. Cross-Predicting Labour Voting in the In-Person and Internet Surveys

87.3 88.1 86.8 88.6 10 20 30 40 50 60 70 80 90 100 In-Person Data Internet Data Percent Correctly Classified In-Person Model Internet Model

Conclusions

• Mode Doesn’t Matter for Modeling Electoral

Choice in Britain

• Internet Surveys – The Future?
• Very Cost Effective
• Huge N’s - Study Election Outcomes
• Super Fast
• Cool Experiments – e.g., Feedback to

Respondents

• Do British Findings Travel Well? How far is it

from Wivenhoe Park to Ann Arbor? To Montreal? Encouraging Findings from our 2006 Congressional & 2006 Canadian election studies

The 2009/10 BES

• More Mode Comparisons
• Survey Experiments
• Huge Internet Campaign Survey
• Monthly Continuous Monitoring Survey, with

Research Opportunities Like TESS – you can send us your proposal!

• Links to CCAP, and hopefully ANES and

PSNZ