Grid questions in telephone surveys. Do the answers change when the questions are asked independently?
DOI:
https://doi.org/10.3989/ris.2023.81.4.M23-2Keywords:
Telephone survey, Multiple question, Liker scale, Acquiescence, Straightlining, Response quality in surveysAbstract
This work deals with the so-called “grid questions”: a series of items is presented (usually in rows), sharing a common set of response options (usually in columns), asking one or more questions about each item. These questions, very common in questionnaires, have advantages and disadvantages compared to the decomposition of the different elements into independent questions. This paper uses a telephone survey of the general population to analyze the extent to which the questions that are part of a battery are answered worse than when they are independent questions. Three questions have been considered, one attitudinal and two behavioral, all of them with five response categories. The distribution of the questions, together with the use of seven quality indicators in the answer, shows that the battery questions are not answered worse than when the same questions are considered independently.
Downloads
References
Alwin, Duane F. y Jon A. Krosnick. 1985. "The measurement of Values in Surveys: A comparison of ratings and rankings". Public Opinion Quarterly 49: 535-552. https://doi.org/10.1086/268949
Alwin, Duane F. y Jon A. Krosnick. 1991. "The Reliability of Survey Attitude Measurement: The Influence of Question and Respondent Attributes". Sociological Methods and Research 20: 139-81. https://doi.org/10.1177/0049124191020001005
Azofra, María José. 1999. Cuestionarios. Madrid: Centro de Investigaciones Sociológicas.
Bethlehem, Jeike y Silvia Biffignandi. 2011. Handbook of web surveys. Neva York: Wiley. https://doi.org/10.1002/9781118121757 PMid:21374080
Bell, Douglas. Carol Mangione y Charles Kahn. 2001. "Randomized testing of alternative survey formats using anonymous volunteers on the world wide web". Journal of the American Medical Informatics Association 8: 616-620. https://doi.org/10.1136/jamia.2001.0080616 PMid:11687568 PMCid:PMC130072
Bishop, George y Andrew Smith. 2001. "Response-Order Effects and the Early Gallup Split-Ballots". Public Opinion Quarterly 65: 479-505. https://doi.org/10.1086/323575
Centro de Investigaciones Sociológicas (CIS) (2022a), Barómetro de enero 2022, estudio nº 3347, https://www.cis.es/cis/opencm/ES/2_bancodatos/estudios/ver.jsp?estudio=14604&cuestionario=17590&muestra=25078.
Centro de Investigaciones Sociológicas (CIS) (2022b), Barómetro de febrero 2022, estudio nº 3351, https://www.cis.es/cis/opencm/ES/2_bancodatos/estudios/ver.jsp?estudio=14611&cuestionario=17597&muestra=25091.
Centro de Investigaciones Sociológicas (CIS) (2022c), Barómetro de marzo 2022, estudio nº 3355, https://www.cis.es/cis/opencm/ES/2_bancodatos/estudios/ver.jsp?estudio=14616&cuestionario=17601&muestra=25113.
Centro de Investigaciones Sociológicas (CIS) (2022d), Barómetro de abril 2022, estudio nº 3359, https://www.cis.es/cis/opencm/ES/2_bancodatos/estudios/ver.jsp?estudio=14623&cuestionario=17608&muestra=25132.
Centro de Investigaciones Sociológicas (CIS) (2022e), Barómetro de mayo 2022, estudio nº 3363, https://www.cis.es/cis/opencm/ES/2_bancodatos/estudios/ver.jsp?estudio=14630&cuestionario=17621&muestra=25153.
Centro de Investigaciones Sociológicas (CIS) (2022f), Barómetro de junio 2022, estudio nº 3366, https://www.cis.es/cis/opencm/ES/2_bancodatos/estudios/ver.jsp?estudio=14634&cuestionario=17635&muestra=25202.
Chang, Linchat y Jon Krosnick. 2010. "Comparing Oral Interviewing with Self-Administered Computerized Questionnaires, An Experiment". Public Opinion Quarterly 74: 154-167. https://doi.org/10.1093/poq/nfp090
Couper, Mick, Rogert Tourangeau., Frederick Conrad y Chan Zang. 2013. "The design of grids in web surveys". Social Science Computer Review 31 (3): 322-341. https://doi.org/10.1177/0894439312469865 PMid:25258472 PMCid:PMC4172361
Couper, Mick, Michael Traugott y Mark Lamias. 2001. "Web survey design and administration". Public Opinion Quarterly 65(2): 230-253. https://doi.org/10.1086/322199 PMid:11420757
de Leeuw, Edith. 1992. Data Quality in Mail, Telephone and Face to Face Surveys. Amsterdam: TT-Publikaties.
De Leeuw, Edith. y Joop Hox. 2015. "Survey mode and mode effects". Pp. 22-35 en Improving survey methods. Lessons from recent research, compilado por Uriel Engel, Benn Jann, Peer Lynn et al. Nueva York and London: Routledge and European Association of Methodology.
Díaz de Rada, Vidal. 2001. Diseño y elaboración de cuestionarios para la investigación comercial. Madrid: Esic.
Díaz de Rada, Vidal. 2018. "Encuestas presenciales con cuestionario de papel y ordenador: una comparativa en preguntas de actitudes". Papers 103/2: 199-227. https://doi.org/10.5565/rev/papers.2301
Díaz de Rada, Vidal. 2022. "Strategies to improve response rates to online surveys". Papers 107 (4). https://doi.org/10.5565/rev/papers.3073
Dillman, Don, Jolene D. Smyth y Leath Christian. 2014. Internet, Phone, Mail and Mixed-Mode Surveys: The Tailored Design Method (4th edición). Nueva York: Wiley. https://doi.org/10.1002/9781394260645
Fowler, Floyd J. y Tom Mangione. 1990. Standarized survey interviewing. Newbury Park, CA: Sage, Applied Social Research Methdods Series, vol 18. https://doi.org/10.4135/9781412985925
García Ferrando, Manuel y Ramón Llopis. 2015. "La encuesta". Pp. 331-336 en El Análisis de la Realidad Social, compilado por Manuel García Ferrando, Francisco Alvira, Louis Enrique Alonso y Modesto Escobar. Madrid: Alianza.
Graäf, Lorenz. 2002. "Assessing internet questionnaires: The online pretest lab". Pp. 69-79 en Online social sciences, compilado por B. Batinic, U.-D. Reips y M. Bosnjak. WA. Seattle: Hogrefe & Huber.
Greene, Jessica, Howard Speizer y Wyndy Wiitala. 2008. "Telephone and web: Mixed-mode challenge". Health Services Research 43: 230-248. https://doi.org/10.1111/j.1475-6773.2007.00747.x PMid:18211527 PMCid:PMC2323139
Groves, Robert y Robert Kahn. 1979. Surveys by telephone, a national comparison with personal interviews. San Francisco: Academic Press.
Guinaliu Blasco, Miguel y Vidal Díaz de Rada. 2021. "Combining sources of information to increase survey response rates". Spanish Journal of Marketing-ESIC 25 (1): 29-45. https://doi.org/10.1108/SJME-04-2020-0060
Haberman, Shelby. 1973. "The Analysis of Residuals in Cross-Classified Tables". Biometrics 29: 205-220. https://doi.org/10.2307/2529686
Heerwegh, Dirk. 2009. "Mode differences between face-to-face and web surveys: An experimental investigation of data quality and social desirability effects". International Journal of Public Opinion Research 21: 111-120. https://doi.org/10.1093/ijpor/edn054
Herzog, A. Regula y Jerald G. Bachman. 1981. "Effects of questionnaire length on response quality". Public Opinion Quarterly 45: 549-559. https://doi.org/10.1086/268687
Holbrook, Allyson, Melanie C. Green y Jon Krosnick. 2003. "Telephone versus face-to-face interviewing of national probability samples with long questionnaires". Public Opinion Quarterly 67: 79-125. https://doi.org/10.1086/346010
Holbrook, Alyson, Jon Krosnick, David Moore y Rogert Tourangeau. 2007. "Response order effects in dichotomous categorical questions presented orally: the impact of question and respondent attributes". Public Opinion Quarterly 71: 325-348. https://doi.org/10.1093/poq/nfm024
Jordan, Lawrence, Alfred Marcus y Lee Reeder. 1980. "Response styles in telephone and household interviewing, a field experiment". Public Opinion Quarterly 44: 102-111. https://doi.org/10.1086/268585
Kennedy, Courtney, Nicholas Hatley, Arnold Lau, Andrew Mercer, Scott Keeter, Joshua Ferno y Dorene Asare-Marlo. 2021. "Strategies for detecting insincere respondents in online surveys". Public Opinion Quarterly 85 (4): 1050-1075. https://doi.org/10.1093/poq/nfab057
Kim, Yujin, Jennifer Dykema, Paul Moberg, John Stevenson, Penny Black y Paul Moberg. 2019. "Straightlining: Overview of Measurement, Comparison of Indicators, and Effects in Mail-Web Mixed-Mode Surveys". Social Science Computer Review 37(2): 214-233. https://doi.org/10.1177/0894439317752406
Krosnick, Jon A. 1991. "Response strategies for coping with the cognitive demands of attitude measures in surveys". Applied Cognitive Psychology 5: 213-236. https://doi.org/10.1002/acp.2350050305
Krosnick, Jon A. 1999. "Survey research". Annual Review of Psychology 50: 537-567. https://doi.org/10.1146/annurev.psych.50.1.537 PMid:15012463
Krosnick, Jon A. y Duane F. Alwin. 1988. "A test of the form-resistant correlation hypothesis. Ratings, rankings, and the measurement of values". Public Opinion Quarterly 52(3): 526-538. https://doi.org/10.1086/269128
Krosnick, Jon A. y Katthew Berent. 1993. "Comparison of party identification and policy preferences: the impact of survey question format". American Journal of Political Science 37(3): 941-964. https://doi.org/10.2307/2111580
Krosnick, Jon A., Allyson Holbrook, Matthew Berent, Richard T. Carson, W. Michael Hanemann, Raymond J. Koop, Robert Cameron Mitchell, Stanley Presser, Paul A. Ruud, V.Kerry Smith, Wendy R. Moody, Melanie C. Green y Michael Conaway 2002. "The Impact of No Opinion Response Options on Data Quality, Non-Attitude Reduction or an Invitation to Satisfice?". Public Opinion Quarterly 66: 371-403. https://doi.org/10.1086/341394
Liu, Mingnan. 2017. "Labelling and Direction of Slider Questions Results from web survey experiments". International Journal of Market Research 59(5): 601-624.
Liu, Mingnan y Alexandru Cernat. 2018. "Item-by-item Versus Matrix Questions: a web survey experiment". Social Sciences Computer Review 36(3): 690-706. https://doi.org/10.1177/0894439316674459
Lozar Manfreda, Katja, Vasja Vehovar y Gasper Koren. 2008. "Internet surveys". Pp. 271-283 en Handbook of public opinion research, editado por Wolfgang Donsbach y Michael Traugott. Thousand Oaks, CA: Sage. https://doi.org/10.4135/9781848607910.n26
McCarty, John A. y Lion Shrum. 2000. "The measurement of personal values in survey research, a test of alternative rating procedures". Public Opinion Quarterly 64: 271-298. https://doi.org/10.1086/317989 PMid:11114269
Narayan, Sowmya y Jon Krosnick. 1996. "Education Moderates some Response Effects in a Attitude Measurement". Public Opinion Quarterly 60: 58-88. https://doi.org/10.1086/297739
National Research Council. 2013. Nonresponse in social science surveys: a research agenda. R. Tourangeau y T. Plewer, editores. Panel on a Research Agenda for the Future of Social Science Data Collection, Committee on National Statistics. Division of Behavioral and Social Sciences and Education. Washington DC: The National Academic Press.
Nicolaas, Gerry, Katarina Thomson y Peter Lynn. 2000. The feasibility of conducing Electoral Surveys in the UK by telephone. Londres: NCSR.
Olson, Kristen, Jolene Smyth y Beth Cochran. 2018. "Item Location, the Interviewer-Respondent Interaction, and Responses to Battery Questions in Telephone Surveys". Sociological Methodology 48 (1): 225-268. https://doi.org/10.1177/0081175018778299
Pardo, Antonio y Rafael San Martín. 2010. Análisis de datos. Madrid: Síntesis.
Peterson, Thomas. 2008. "Split ballots and an experimental approach to public opinion research". Pp. 322-329 en The SAGE Handbook of Public Opinion Research, compilado por Wolfgang Donsbach y Michael W. Traugott. Thousand-Oaks, LA: Sage. https://doi.org/10.4135/9781848607910.n30
Peytchev, Andy. 2011. "Breakoff and unit nonresponse across web surveys". Journal of Official Statistics 27: pp. 33-47.
Peytchev, Andy, Mick Couper, Sean McCabe y Scott Crawford. 2006. "Web survey design, paging versus scrolling". Public Opinion Quarterly 70: 596-607. https://doi.org/10.1093/poq/nfl028
Revilla, Melanie, Willem Saris y Jon Krosnick. 2014. "Choosing the Number of Categories in Agree-Disagree Scales". Sociological Methods y Research 43(1): 73-97. https://doi.org/10.1177/0049124113509605
Revilla, Melanie, Daniele Toninelli y Carlos Ochoa. 2017. "An experiment comparing grids and item-by-item formats in web surveys completed through PCs and smartphones". Telematics and Informatics 34: 30-42. https://doi.org/10.1016/j.tele.2016.04.002
Revilla, Melanie y Mick Couper. 2018. "Comparing Grids With Vertical and Horizontal Item-by-Item Formats for PCs and Smartphones". Social Sciences Computer Review, 36(3): 349-368. https://doi.org/10.1177/0894439317715626
Schuman, Howard. 2008. Method and Meaning in Polls & Surveys. Harvard: Harvard University Press.
Silver, Henning, Joss Robmann y Tobias Gummer. 2018. "When near means related: evidence from three web survey experiments on inter-item correlations in grid questions". International Journal of Social Research Methodology 21 (3): 275-288. https://doi.org/10.1080/13645579.2017.1381478
Sudman, Seymour y Norman Bradburn 2002: Asking questions. New York: Wiley.
Sudman, Seymour, Norbert Bradburn y Norman Schwarz. 1996. Thinking about answers. San Francisco: Jossey-Bass.
Sykes, Wendy y Martin Collins. 1988. "Effects of Mode of Interview, Experiments in the UK". Pp. 30-306 en Telephone Survey Methodology, compilado por Robert Groves et al. Nueva York: Wiley.
Terentev, Evgeniy y Natalia Maloshonok. 2019. "The impact of response options ordering on respondents' answers to rating questions: results of two experiments". International Journal of Social Research Methodology 22(2): 179-198. https://doi.org/10.1080/13645579.2018.1510660
Thorndike, Frances P.; Per Carlbring, Frederick Smyth; Joshua Magee, Linda Gorder-Frederick, Lars, Göran Ost y Lee M. Ritterband 2009. "Web-based measurement: effect of completing single or multiple items per webpage". Computers in Human Behavior 25: 393-401. https://doi.org/10.1016/j.chb.2008.05.006
Toepoel, Vera y Don Dillman. 2011. "How visual design affects the interpretability of survey question". Pp. 165-190 en Social and behavioral research and the internet, compilado por Marcel Das, Peter Ester y Lars Kaczmirek. Nueva York.: Routledge. https://doi.org/10.4324/9780203844922-7
Toepoel, Vera, Brenda Vermeeren y Baran Metin. 2019. "Smileys, Stars, Hearts, Buttons, Tiles or Grids: Influence of Response Format on Substantive Response, Questionnaire Experience and Response Time". Bulletin de Méthodologie Sociologique 142: 57-74. https://doi.org/10.1177/0759106319834665
Toepoel, Vera, Marcel Das, y Arthur van Soest 2009: "Design of web questionnaires: The effects of he number of items per screen". Field Methods 21: 200-213. https://doi.org/10.1177/1525822X08330261
Tourangeau, Rogert, Mick Couper y Frederick Conrad. 2004. "Spacing, position, and order: interpretive heuristics for visual features of survey questions". Public Opinion Quarterly 68(3): 368-393. https://doi.org/10.1093/poq/nfh035
Tourangeau, Rogert, Lance Rips y Kennet Rasinski. 2000. The psychology of survey response. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511819322
Visauta, Restituto. 1989. Técnicas de investigación social. Barcelona: PPU.
Wojtowicz, Tamara. 2001. "Designing Lengthy Internet Questionnaires: Suggestions and Solutions". Pp. 25-32 en The Challenge of the Internet; Proceedings of the ASC International Conference on Survey Research Methods, compilado por Andrew Westlake, Wendy Sykes, Tony Manners y Malcom Rigg. Londres: Association for Survey Computing.
Yan, Tim 2008. "Nondifferentiation". Pp. 520-521 en Encyclopedia of survey research methodology, compilado por Paul Lavrakas. Newbury Park, CA: Sage.
Ye, Cong, Jenna Fulton y Rogert Tourangeau. 2011. "More positive or More Extreme? A Meta-Analysis of Mode Differences in Response Choice". Public Opinion Quarterly 75: 349-365. https://doi.org/10.1093/poq/nfr009
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Consejo Superior de Investigaciones Científicas (CSIC)
This work is licensed under a Creative Commons Attribution 4.0 International License.
© CSIC. Manuscripts published in both the printed and online versions of this Journal are the property of Consejo Superior de Investigaciones Científicas, and quoting this source is a requirement for any partial or full reproduction.All contents of this electronic edition, except where otherwise noted, are distributed under a “Creative Commons Attribution 4.0 International” (CC BY 4.0) License. You may read here the basic information and the legal text of the license. The indication of the CC BY 4.0 License must be expressly stated in this way when necessary.
Self-archiving in repositories, personal webpages or similar, of any version other than the published by the Editor, is not allowed.
