Spanish exit polls. Sampling error or nonresponse bias?
DOI:
https://doi.org/10.3989/ris.2016.74.3.043Keywords:
Election night forecasts, measurement error, multi-hypergeometric distribution, nonresponse, Spanish regional electionsAbstract
Countless examples of misleading forecasts on behalf of both pre-election and exit polls can be found all over the world. Non-representative samples due to differential nonresponse have been claimed as being the main reason for inaccurate exit-poll projections. In real inference problems, it is seldom possible to compare estimates and true values. Electoral forecasts are an exception. Comparisons between estimates and final outcomes can be carried out once votes have been tallied. In this paper, we examine the raw data collected in seven exit polls conducted in Spain and test the likelihood that the data collected in each sampled voting location can be considered as a random sample of actual results. Knowing the answer to this is relevant for both electoral analysts and forecasters as, if the hypothesis is rejected, the shortcomings of the collected data would need amending. Analysts could improve the quality of their computations by implementing local correction strategies. We find strong evidence of nonsampling error in Spanish exit polls and evidence that the political context matters. Nonresponse bias is larger in polarized elections and in a climate of fear
Downloads
References
Bautista, R., Callegaro, M., Vera, J.A. and Abundis, F. 2007. Studying nonresponse in Mexican exit polls. International Journal of Public Opinion Research, 19, 492– 503. http://dx.doi.org/10.1093/ijpor/edm013
Beltrán, U. and Valdivia, M. 1999. Accuracy and error in electoral forecasts: The case of Mexico. International Journal of Public Opinion Research, 11, 115–134. http://dx.doi.org/10.1093/ijpor/11.2.115
Bernardo, J.M. 1984. Monitoring the 1982 Spanish socialist victory: A Bayesian analysis, Journal of the American Statistical Association, 79, 510–515. http://dx.doi.org/10.1080/01621459.1984.10478077
Bernardo, J.M. 1997. Probing public opinion: The state of Valencia experience. In C. Gatsonis, J.S. Hodges, R.E. Kass, R. McCulloch, P. Rossi, and N.D. Singpurwalla, (eds.), Bayesian Case Studies 3 (pp. 3–21). New York: Springer-Verlag. http://dx.doi.org/10.1007/978-1-4612-2290-3_1
Bernardo, J.M. and Girón, F.J. 1992. Robust sequential prediction from non-random samples: The election night forecasting case. In J.M. Bernardo, J.O. Berger, A.P. Dawid and A.F.M. Smith (eds.), Bayesian Statistics 4 (pp. 61–77). Oxford: Oxford University Press.
Best, S.M. and Krueger, B.S. 2012. Exit Polling: Surveying the American Electorate, 1972–2010. Los Angeles: CQ Press, Sage. http://dx.doi.org/10.4135/9781452234410
Biemer, P.P. 2010. Total Survey Error. Design, implementation and evaluation. Public Opinion Quarterly, 74, 817–848. http://dx.doi.org/10.1093/poq/nfq058
Blom, A.G., de Leeuw, E.D. and Hox, J.J. 2011. Interviewer effects on nonresponse in the European Social Survey. Journal of Official Statistics, 27, 359–377.
Cresie, N. and Read, T.R.C. 1984. Multinomial goodness-of-fit tests. Journal of the Royal Statistical Society B, 46, 440–464.
Childs, A. and Balakrishnan, N. 2000. Some approximations to the multivariate hypergeometric distribution with applications to hypothesis testing. Computational Statistics and Data Analysis, 35, 137–154. http://dx.doi.org/10.1016/S0167-9473(00)00007-4
Cleophas, T.J. and Zwinderman, A.H. 2011. Log likelihood ratio tests. In Statistical Analysis of Clinical Data on a Pocket Calculator (pp. 37–38). Springer. http://dx.doi.org/10.1007/978-94-007-1211-9_13
Díaz de Rada, V. 2013. La no respuesta en encuestas presenciales realizadas en Espa-a. Revista Internacional de Sociología, 71, 357–381. http://dx.doi.org/10.3989/ris.2012.02.07
Díaz de Rada, V. 2014. Analysis of incidents in face-to-face surveys: Improvements in fieldwork. Revista Espa-ola de Investigaciones Sociológicas, 145, 43–72.
Dopp, K. 2006. Vote Miscounts or Exit Poll Error? New Mathematical Function for Analyzing Exit Poll Discrepancy. http:// http://www.electionmathematics.org/em-exitpolls/Exit-Poll-Analysis.pdf. Last access 07/02/2014.
Edison Media Research and Mitofsky International 2005. Evaluation of Edison/Mitofsky election system 2004. http:// abcnews.go.com/images/Politics/EvaluationofEdisonMitofskyElectionSystem.pdf. Last access 07/02/2014.
El País. 2012. ¿Por qué fallaron las encuestas?, El País, 26 March. http://politica.elpais.com/politica/2012/03/26/ actualidad/1332790007_767368.html. Last access 06/03/2014.
Frankovic, K., Panagopoulos, C. and Shapiro, R. 2009. Opinion and election polls. In D. Pfeffermann and C.R. Rao, (eds.), Handbook of Statistics—Sample Surveys: Design Methods and Applications, Volume 29A (pp. 567– 596) Netherlands: Elsevier. http://dx.doi.org/10.1016/S0169-7161(08)00022-9
Groves, R.M., Dillman, D.A., Eltinge, J.L. and Little, R.J.A. 2002. Survey Nonresponse. New York: Wiley.
Haan, M., Ongena, P.Y. and Aarts, K. 2014. Reaching Hard-to- Survey Populations: Mode Choice and Mode Preference. Journal of Official Statistics, 30, 355–379. http://dx.doi.org/10.2478/jos-2014-0021
Haunberger, S. 2010. The effects of interviewer, respondent and area characteristics on cooperation in panel surveys: a multilevel approach. Quality & Quantity, 44, 957–969. http://dx.doi.org/10.1007/s11135-009-9248-5
Keeter, S. 2011. Public opinion polling and its problems. In K. Goidel (ed.), Political Polling in the Digital Age: The Challenge of Measuring and Understanding Public Opinion (pp. 28–53). Baton Rouge: Louisiana State University Press.
Klofstad, C.A. and Bishin, B.G. 2012. Exit and entrance polling: A comparison of election survey methods. Field Methods, 24, 429–437. http://dx.doi.org/10.1177/1525822X12449711
Krumpal, I. 2013. Determinants of social desirability bias in sensitive surveys: a literature review. Quality & Quantity, 47, 2025–2047. http://dx.doi.org/10.1007/s11135-011-9640-9
Martín Arroyuelos, A.M., Férnandez Aguirre, K. and Modro-o Herrán, J.A. 2014. Early forecasting of Parliamentary seats distribution: the representative polling stations method. In J. Bozeman, V. Girardin and C.H. Skiadas (eds.), New Perspectives on Stochastic Modeling and Data Analysis (pp. 123–135).
Menold, N. and Kemper, C.J. 2014. How do real and falsified data differ? Psychology of survey response as a source of falsification indicators in face-to-face surveys. International Journal of Public Opinion Research, 26, 41– 65. http://dx.doi.org/10.1093/ijpor/edt017
Merkle, D.M. and Edelman, M. 2002. Nonresponse in exit polls: a comprehensive analysis. In R.M. Groves, D.A. Dillman, J.L. Eltinge and R.J.A. Little (eds.), Survey Nonresponse (pp. 243–257). New York: Wiley. PMid:12186630 PMCid:PMC1222982
Mitofsky, W. 1991. A short history of exit polls. In P.J Lavrakas and J.K. Holley (eds.), Polling and Presidential Election Coverage (pp. 83–99). Newbury Park, CA: Sage.
Mokrzycki, M., Keeter, S. and Kennedy, C. 2009. Cell-phone-only voters in the 2008 exit poll and implications for future noncoverage bias. Public Opinion Quarterly, 73, 845–865. http://dx.doi.org/10.1093/poq/nfp081
Noelle-Neumann, E. 1984. The Spiral of Silence: Public Opinion – Our Social Skin. Chicago: The University of Chicago Press.
Orriols, L. and Martínez, A. 2014. The role of the political context in voting indecision, Electoral Studies, 35, 12–23. http://dx.doi.org/10.1016/j.electstud.2014.03.004
Panagopoulus, C. 2013. Who participates in exit polls? Journal of Elections, Public Opinion and Parties, 23, 444–455. http://dx.doi.org/10.1080/17457289.2013.811585
Pavía, J.M. 2010. Improving predictive accuracy of exit polls. International Journal of Forecasting, 26, 68–81. http://dx.doi.org/10.1016/j.ijforecast.2009.05.001
Pavía, J.M. and García-Cárceles, B. 2012. Una aproximación empírica al error de dise-o muestral en las encuestas electorales del CIS. Metodología de Encuestas, 14, 45–62.
Pavía, J.M. and Larraz, B. 2012. Nonresponse bias and superpopulation models in electoral polls. Revista Espa-ola de Investigaciones Sociológicas, 137, 237–264. http://dx.doi.org/10.5477/cis/reis.137.237
Pavía, J.M., Larraz, B. and Montero, J.M. 2008. Election forecasts using spatiotemporal models. Journal of the American Statistical Association, 103, 1050–1059. http://dx.doi.org/10.1198/016214507000001427
Pavía, J.M., Mu-iz, P. and Álvarez, J.A. 2001. Proyecciones en la noche electoral. Estadística Espa-ola, 43, 225–239.
Pavía, J.M. and López-Quilez, A. 2013. Spatial vote redistribution in redrawn polling units. Journal of the Royal Statistical Society A, 176, 655–678. http://dx.doi.org/10.1111/j.1467-985X.2012.01055.x
Pavía-Miralles, J.M. 2005. Forecasts from non-random samples: The election night case, Journal of the American Statistical Association, 100, 1113–1122. http://dx.doi.org/10.1198/016214504000001835
Pavía-Miralles, J.M. and Larraz-Iribas, B. 2008. Quick-counts from non-selected polling stations. Journal of Applied Statistics, 35, 383–405. http://dx.doi.org/10.1080/02664760701834881
Radcliff, B. 2005. Exit polls. In S.J. Best and B. Radcliff (eds.), Polling America: An Encyclopedia of Public Opinion, Volume I (A-O). Westport, CT: Greenwood Press.
Rechner, A.C. 2002. Methods of Multivariate Analysis. New York: John Wiley & Sons. PMCid:PMC1721411
Selb, P., Herrmann, M., Munzert, S. Schübel, T. and Shikano, S. 2013. Forecasting runoff elections using candidate evaluations from first round exit polls. International Journal of Forecasting, 29, 541-547. http://dx.doi.org/10.1016/j.ijforecast.2013.02.001
Slater, M. and Christensen, H. 2008. Applying AAPOR's final disposition codes and outcome rates to the 2000 Utah colleges' exit poll. In F.J. Scheuren and W. Alvey (eds.), Elections and Exit Polling (pp. 37–50). Hoboken, NJ: Wiley.
Sobrino, V.M. 2012. Los avances de resultados en una jornada electoral. Más Poder Local, 13, 42–44.
Tourangeau, R., Rips, L.J. and Rasinki, K. 2000. The Psychology of Survey Response. Cambridge, UK: Cambridge University Press. http://dx.doi.org/10.1017/CBO9780511819322
Turner, C.F., Ku, L., Rogers, S.M., Lindberg, P.D., Pleck, J.H. and Sonenstein, F.L. 1998. Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science, 280, 867–873. http://dx.doi.org/10.1126/science.280.5365.867 PMid:9572724
Published
How to Cite
Issue
Section
License
Copyright (c) 2016 Consejo Superior de Investigaciones Científicas (CSIC)
This work is licensed under a Creative Commons Attribution 4.0 International License.
© CSIC. Manuscripts published in both the print and online versions of this journal are the property of the Consejo Superior de Investigaciones Científicas, and quoting this source is a requirement for any partial or full reproduction.
All contents of this electronic edition, except where otherwise noted, are distributed under a Creative Commons Attribution 4.0 International (CC BY 4.0) licence. You may read the basic information and the legal text of the licence. The indication of the CC BY 4.0 licence must be expressly stated in this way when necessary.
Self-archiving in repositories, personal webpages or similar, of any version other than the final version of the work produced by the publisher, is not allowed.
