The recoding of scale variables is a common step in the analysis of survey data. It is not immune, however, to certain pitfalls, such as the introduction of biases, or potential data distortion. This paper presents a methodological proposal for the validation of any recoding process, whether it involves metric- or categorical-scale variables. The aim of the proposed methodology is to verify the adequacy of the re-codification by indicating how close in structure the re-coded data are to the original data. The basis of the methodology is a factorial analysis technique, Multiple Factor Analysis (MFA), which is performed on a global data table juxtaposing the original-scale and recoded-scale data. The procedure is tested on real-world data drawn from a public opinion poll on perceptions of leading politicians in the Spanish Parliament.

La recodificación de variables es una etapa habitual en el análisis de datos de encuesta. No está exenta de riesgos, como pueden ser la introducción de sesgos o las posibles deformaciones de los datos originales. En este trabajo se propone una metodología de validación de cualquier recodificación, tanto de variables de escala métrica, como de escala categórica para reducir el número de categorías. El objetivo de la metodología propuesta es comprobar la adecuación de la recodificación, indicando en qué medida los datos recodificados mantienen la misma estructura de los datos originales. La metodología se basa en el uso de un método factorial, Multiple Factor Analysis (MFA), aplicado sobre una tabla global obtenida como yuxtaposición de los datos medidos con la escala original y los mismos datos medidos con la escala recodificada. El procedimiento ha sido testado a partir de un conjunto de datos reales extraídos de una encuesta de opinión pública sobre los principales políticos del Parlamento Español.

The questionnaire is one of the most widely-used sociological data gathering tools. After establishing the number of questions to be included, it is very important to select an appropriate question format. For surveys involving complex issues, one of the most frequent options is to use the so-called battery question format, where the same set of response options is assigned to the whole set of questions (Revilla et al.,

The battery question requires less response time than other formats and, according to Bell et al. (

After deciding on the type of question format, the next crucial step is to select the response scale. This decision influences the quality of the responses and determines the statistical method to be used for the survey data analysis (Conrad and Kreuter,

The

As a rule of thumb, the scale recoding should not generate any loss of key information that could seriously compromise the analysis of the original data. Consequently, the researcher must verify or validate the recoded scale to ensure that it does not lead to a biased interpretation of the original data. In consequence, we propose a methodology for finding an objective criterion to help the researcher in this choice, that is, a methodology for scale recoding validation.

The factorial method Multiple Factor Analysis (MFA) is the core of the methodology for scale recoding validation. MFA provides numerical and graphic indicators to show how closely the recoded data resemble the original data. Furthermore, it is a factorial method designed to handle mixed data, that is, to analyse metric and categorical tables simultaneously. This is the case studied later, where the original variables, which are ratings of 0 or 1 (lowest) to 10 (highest) usually considered as metric variables, are recoded as categorical ones. In general, the MFA scale validation methodology is very flexible since it enables the comparison of different types of original scales (categorical or metric) with different types of recoded scales (usually categorical), according to the research objectives.

Abascal and Díaz de Rada (

It follows from the above that it is common practice to recode the original scale in order to obtain a categorical scale with a small number of categories. It goes without saying that, when the recoding process involves merging consecutive categories, it needs to be carried out very carefully, only after a prior analysis and never as a matter of course. The problem is even greater when categorizing a metric variable. In this case, Abascal et al. (

This paper is structured as follows. The next section presents data obtained from a survey of public perceptions regarding leading members of the Spanish Parliament. After this, there comes a section explaining each stage of the scale recoding validation MFA methodology, followed by another showing the main results of the empirical check of the factor stability between the original and recoded scales in order to illustrate the appropriateness of the MFA methodology. The main conclusions of the MFA methodology, its limitations and proposals for ways to address them, are presented in the final section.

The data used in this paper is drawn from a Spanish household survey conducted by the Sociological Research Centre, Centro de Investigaciones Sociológicas (CIS,

The survey question selected to illustrate the MFA recoding scale validation methodology reads as follows:

Please indicate whether you know each of the following political leaders [SHOW CARD]. How do you rate the political performance of NAME OF POLITICAL LEADER KNOWN on a scale of 0 to 10, where 0 means “very poor” and 10 means “very good”?

With the aim of illustrating the MFA recoding validation methodology, we use the ratings for the leaders of the four main parties as given by those respondents who recognized them. The same criterion is used in Abascal and Díaz de Rada (

The final sample comprises 1,704 respondents who gave their opinions of the four leaders at the time of the survey. Mariano Rajoy, the Spanish president and president of the Popular Party, PP, right wing. Alfredo Pérez Rubalcaba, president of the main opposition party, the Spanish socialist party, PSOE, a moderate left-winger. Cayo Lara, president of the second most important opposition party, United Left, IU/ICU, a left- winger. Finally, Rosa Díez, ex-socialist, co-founder of the UPyD –i.e., the Progressive and Democratic Union, Member of the European Parliament, and a regular media figure.

The four bar charts in

On the whole, the scale recoding validation methodology must enable the comparison of several tables of data to determine whether they have a similar structure. There are several factorial methods for this purpose, as can be seen in Dazy and Le Barzic (

The MFA, proposed by Escofier and Pagés (

There are three main reasons for using MFA as the scale validation method:

MFA can be used simultaneously with groups of metric and categorical variables, the only proviso being that all the variables within each group must be of the same type. The validation of a scale recoding requires an MFA of two groups of variables drawn from the same data source: one group having been measured on the original scale (all variables within the group will be coded either as metric or as categorical) and another group having been measured on the recoded scale.

MFA balances out the influence of the two groups (original and recoded variables). The main factors of the MFA reveal the variability between cases, such that the influence of each group is comparable. The result of applying MFA is to balance the influence of the original data against that of the recoded data, such that neither group biases the analysis.

MFA provides a rich source of numerical and graphical indicators. As well as the usual indicators used in factorial analysis methods, MFA provides measures of relationships between groups and different factor planes, thereby facilitating interpretation of the data, as will be illustrated in the section that follows.

Essentially, MFA is a weighted Principal Component Analysis (PCA) (

_{1} and X_{2 }(the cases are the rows and the variables are the columns). In this stage, a factorial analysis is performed on each of the tables and the first eigenvalue, λ_{1}^{(1)} and λ_{1}^{(2)}, in each analysis is retained. The factors obtained in each separate analysis are called

_{1}|X_{2}]. Each table is weighted by the inverse of the first eigenvalue retained in the partial analysis, 1/λ_{1}^{(1)} and 1/λ_{1}^{(2)}. The factors obtained in the global analysis are called

The structure of each table, X_{1} and X_{2}, is maintained by giving the same weighting to all the variables which form one group

The influence of the groups is balanced because the maximum inertia of any of the ^{’}) in the global cloud,

Where _{(1)}_{(1)} and _{(1)}_{(1)} are two individuals characterized only by the variables of group _{(2)}_{(2)} and _{(2)}_{(2)} are the same individuals characterised only by the variables of group

One noteworthy characteristic of MFA is that it can be used to analyze tables of different types of variables (quantitative and categorical), known as mixed tables. The results from quantitative tables are equivalent to those obtained via PCA, while the results from categorical tables are equivalent to those obtained by means of Multiple Correspondence Analysis (MCA). This greatly facilitates their comparison.

As well as providing the classical results of other factor analysis techniques (PCA or MCA), MFA has the distinctive feature of treating the group of variables as one more element of analysis. In the case at hand, there are two groups: that of the variables measured on the original scale and that of the variables measured on the recoded scale. This yields a large number of specific results. Thus, in MFA, the terms “partial” and “global” take on special importance. In terms of cases (observations), we speak of “partial cases” when the variables of each group are characterized separately. In the context that concerns us, each case gives two partial cases per respondent. One of these partial cases represents the respondent’s responses on the original scale (the score of 0 to 10 assigned to the politician, plus the DK/NA option). The other partial case represents the same respondent’s replies on the recoded scale (an evaluation based on one of four categories). We speak of “global cases” when the characterization is based on all the information from all the groups of variables. In our context, there is one global case per respondent. This global case captures all of a respondent’s answers, taking into account both scales (original and recoded). Global cases can be interpreted as the midpoint of their respective partial cases. In our context, the global cases are the synthesis obtained by combining the two measuring scales (original and recoded).

For a deeper explanation of MFA, the interested reader is referred to Lebart, Morineau and Piron (

The methodology for the validation of a recoding process consists of an MFA of the global data table obtained from the juxtaposition of the original with the recoded tables (

The results yielded by the MFA are analyzed in the pre-established order, which can be observed in the validation methodology. The focus in each stage is on the issues that are relevant for the objective of the validation process:

At this point, the researcher has sufficient objective criteria to decide whether to proceed with recoding. If the results indicate weak factor stability between the two scales, an alternative form of recoding must be found and subjected to the validation procedure. Otherwise, the researcher can proceed to stage 2.

Once the above two stages are complete, the researcher is equipped to make the final decision regarding the validity of the proposed recoding, and, if a replacement is deemed necessary, the information obtained in the previous stages will indicate the direction the adjustment must take.

In the survey chosen for the purposes of this study, the CIS uses a scale of 0 to 10 for the politician ratings plus one more category for the “don’t know /no answer” response option. This is usually treated as a metric scale, that is, by analyzing the average rating for each politician and adding the percentage of non-response, (don’t know/no answer). Such data structures are frequently analyzed by PCA, the drawbacks of which are noted and duly explained in Abascal and Díaz de Rada,

Having justified this first decision, we need to note that the resulting number of categories is 12 (11 from the scale of 0-10, plus the non-response category, “don’t know/no answer”) for each rating of each of the four politicians. The total dimension of the data table, therefore, is 1,704 rows (cases) by 48 columns (categories). Under these conditions, the excessive number of categories in the MCA can give rise to negative consequences which can be summed up as follows (Lebart et al.,

The first factors may be capturing the influence of “rare categories”, that is, those selected in only a few cases, and may therefore be eclipsing more common response behaviour or relegating it to a higher order.

The factors, overall, may be formed by very few categories, which would suggest that they are hardly worth taking into consideration since they do not indicate a general trend and can be difficult to interpret.

There could be too many factors to examine simultaneously.

The MCA of the data considered in this analysis reveals that the first factors, in fact, present the characteristics mentioned above. Thus, the principal axis, or first factor, is determined mainly by the “non-response” category”, in other words, it highlights those respondents who select “don’t know/no answer” (for almost all their politician ratings). Close examination of the plane formed by factors 2 and 3 (

There are too many response categories: contiguous scores are very close, and almost overlapping (for example, 2 and 3 in quadrant 4; or 5, 6 and 7 in quadrant1).

Likewise, the vertical axis (factor 3) shows that ratings lower than 5 (low on the negative side) are the opposite of the ratings higher than 5 (high on the positive side).

The zero rating (category 11) virtually alone determines the horizontal axis (factor 2).

The response categories ordered from 0 to 10 depict a sort of parabola or horseshoe shape, indicating the so-called Guttman effect (Lebart et al.,

The shortcomings of these results reveal the need to recode the original scale. By reducing the number of categories, it will be possible not only to mitigate their negative effects when MCA is used, but also to retain the aforementioned advantages of using a categorical scale.

In view of the above results, the scale will be recoded as follows:

Keep the category “don’t know/no answer” (label: -N).

Keep the no rating category, that is, “zero ratings” (label: -Z), since it behaves very differently from a score of 1, and it appears with very high frequency (

Define a new category to group ratings of less than 5 (excluding zero) and label it “fail mark” (label:-F)

Define a new category to group ratings of 5 or more, to be labelled “pass mark”

This considerably reduces the size of the data table, from 12 categories per initial question to 4. It does, however, result in some loss of information and also carries the risk of introducing biases or distortions of the original data. It is for this reason that a recoding validation stage is considered necessary before proceeding with the data analysis. The kernel of this paper, therefore, is a proposal of methodology for achieving this objective. Its underlying philosophy and adequacy are described in the following sections.

This section illustrates how the proposed methodology permits examination of the stability of the selected recoding described in the preceding subsection. The procedure consists of a MFA of the duplicated data table, as initially coded into 12 categories (group 1) and as recoded into 4 categories (group 2). Both groups are described on a categorical scale. The results obtained in both stages confirm the appropriateness of the recoding, which considerably reduces the size of the problem while maintaining the initial structure of the data. Some of the results are presented below.

The first factors of each group show structures common to both. This is clear from the high correlation between factors of the same order, for the different groups (

The correlations between the global factors and the factors for the separate groups analyzed (

The numerical data interpreted so far are illustrated and complemented with graphs. Thus, the very similar overall pattern (internal structure) of the two tables analyzed is confirmed in

In this stage, all the ratings of the politicians using the four recoded and twelve original response categories are projected onto the principal factor planes and the distances between them are analyzed.

For the purposes of this example, we take the factor plane formed by the second and third factors (plane 2, 3) because the first factor still captures the “don’t know/no answer” category, as commented on in the preceding subsection. The next step is the interpretation of the distances between the ratings for a single politician, Rajoy (President of Spain at the time of the survey), which is enough to show how the procedure works. Note that the categories corresponding to fail marks are denoted on the original scale by the values 1, 2, 3 and 4 and the recoded fail mark category is denoted by Rajoy _F.

The procedure could then continue with the ratings of the other three politicians, but, when both scales show high factor stability, as in this case, it is not strictly necessary. In other words, the measures of global similarity obtained in stage 1 and the particular analysis of Rajoy’s ratings, performed in this second stage, enable the validation of the recoding of the original categorical scale. Therefore, despite substantially reducing the dimensions of the studied phenomenon, the recoding causes no relevant loss of original information.

The usefulness of the second stage becomes apparent when the global indicators are found to reveal low factor stability between the two scales, indicating that the recoding has distorted the structure of the initial data. A situation of this nature raises the need to analyze the distances between the variables or categories on the original scale and those on the recoded scale. The purpose of such an analysis is to identify the precise differences between the two scales, show which categories are the least stable, and thus indicate the changes required in the recoding.

The proposed methodology is able to detect when a recoding is inappropriate and liable to lead to major information loss or biases. In this subsection, the aim is to illustrate the capability of the methodology by analyzing an inappropriate recoding. In the new scale, the null (zero) rating is added to the fail category, leaving only 3 categories: “don’t know/no answer”, “pass” and “fail”. It is known,

The aim here is to show briefly how the proposed methodology is able to detect this serious recoding error. Thus, the original scale variables form group 1 and the recoded scale variables form group 2. We have included as many indicators from the first stage as are required to illustrate the distortion of the original information due to inappropriate recoding.

The correlations between the partial factors from the analysis of the original scale and those from the new one are very weak, except for the first factor. This is to be expected, given that the recoding maintains the “don’t know/no answer” category, which, as seen in the preceding subsection, defines the first factor. The correlation between the second partial factors, however, is very weak (0.36), and actually drops to 0.0 between the third partial factors and subsequent ones. In consequence, the correlation between the partial factors of the same order is weak; so the structure of the original data and the recoded data has changed. In addition, there is strikingly high correlation (0.85) between partial factor 3 in the original data and partial factor 2 in the recoded data. This indicates that the new scale has “deleted” a piece of relevant information, captured by the second principal dimension of the MCA of the original scale. By merging the zero rating with its adjacent categories, the recoding conceals a key piece of information and the proposed methodology detects this easily.

A quick comparison of

It is common practice for researchers to recode the original scale chosen to measure a phenomenon of interest. There are various reasons for recoding, two of the main ones being to reduce the dimension of the data (reduce the number of categories in a categorical scale) or to change the nature of the data to fit more appropriate methodologies (transform metric scales into categorical scales). It is important, however, to ensure that the new scale does not result in any transformation or loss of relevant parts of the original information, since this could substantially alter the results. Validation of the recoding is therefore one more stage in the research process.

The results in the two preceding sections provide clear proof of the adequacy of MFA for finding the answer when faced with the decision of whether to recode the original scale. If the researcher recodes the original scale correctly, the methodology provides various graphic and numerical signals which capture and measure the main similarities between the recoded scale and the original one. Thanks to this methodology, an incorrect recoding can be quickly discarded based on a simple analysis of the correlation structure of the partial factors.

Thus, it is worth emphasizing to potential users the major advantage of this validation methodology - its simplicity. The user does not need to be an expert either in factorial analysis or in multi-table analysis. All the results required can be obtained automatically using existing statistical software, some of which is freely available. The decision to recode can be adopted or abandoned after simply following the guidelines indicated by the authors in the example (and counter-example). The proposal for this methodology involves a multi-stage protocol in which the results are interrelated and must therefore be analyzed jointly and in an ordered manner before validating or rejecting the proposed recoding.

One of the most relevant contributions of this paper is that it does not propose a specific recoding criterion, but provides researchers, instead, with a tool that will, ideally, lead them towards a validated recoding. The underlying idea is to determine how far the initial relationship structure among the variables remains unaltered after recoding. The methodology is able to handle both metric and categorical variables, variables with high heterogeneity, variables with different ranges and even variables expressed in different units. The core concept behind the proposed method is the study of factor stability to ensure that the new scale has not produced distortions in the original data structure. We therefore consider it coherent with the philosophy underlying the factorial analysis of multiple tables. From the family of methods developed in recent years for the simultaneous study of multiple data matrices, the choice was MFA. Use of this method is not constrained by the nature of the original data, because it can be used for the analysis of so-called mixed data tables. Thus, one of the tables might contain what was originally either metric or categorical data and the other might be the result of recoding into fewer categories in order to reduce the dimension of the problem.

A practical case based on the opinion poll ratings assigned to four leading Spanish politicians provides a framework in which to demonstrate the suitability of the MFA recoding validation methodology. The usefulness of the procedure is then demonstrated by comparing the structures of two groups of variables, one for the ratings of the politicians on a twelve-category nominal scale (the original scale), and another for their ratings on the recoded four-category nominal scale. Recoding greatly reduces the dimension of the problem, which is an advantage in itself, without provoking any significant change in the initial structure of the data under analysis. A second, intentionally “erroneous”, recoding is carried out to demonstrate that the method has the further capabilities of detecting when a recoding is inappropriate and likely to lead to significant loss of information or biases, and of indicating the direction in which it needs to be modified.

Looking ahead, we have proposals for four future lines of research. The first and most obvious is to apply the validation methodology to quantitative variables, which was not possible in this paper due to lack of space. The context for this is the recoding of different types of household consumptions (in Euros), following an idea proposed in Abascal et al. (

Recovered from

Note that this type of marking system is deeply rooted in all stages of education in Spain.