Skip to main content
placeholder image

Individualised rating-scale procedure: a means of reducing response style contamination in survey data?

Conference Paper


Download full-text (Open Access)

Abstract


  • Response style bias has been shown to seriously contaminate the substantive results drawn from

    survey data; particularly those conducted using cross-cultural samples. As a consequence. identification of

    response formats that suffer least from responst style bias has been called for. Previous studies show that

    respondents' personal characteristics, such as age, education level and culture, are connected with response

    style manifestation.

    Differences in the way respondents interpret and utilise researcher-defined fixed rating-scales (e.g. Likert

    formats), poses a problem for survey researchers. Techniques that are currently used to remove response

    bias from survey data are inadequate as they cannot accurately determine the level of contamination present

    and frequently blur true score variance. Inappropriate rating-scales can impact on the level of response style

    bias manifested, insofar as they may not represent respondents' cognitions. Rating-scale lengths that are too

    long present respondents with some response categories that are not 'meaningful', whereas rating-scales that

    are too short force respondents into compressing their cognitive rating-scales into the number of response

    categories provided (this can cause ERS contamination - extreme responding). We are therefore not able to

    guard against two respondents, who share the same cognitive position on a continuum, reporting their stance

    using different numbers on the rating-scale provided. This is especially problematic where a standard fixed

    rating-scale is used in cross-cultural surveys.

    This paper details the development of the Individualised Rating-Scale Procedure (IRSP), a means of

    extracting a respondent's 'ideal' rating-scale length, and as such 'designing out' response bias, for use as the

    measurement instrument in a survey. Whilst the fundamental ideas for self-anchoring rating-scales have been

    posited in the literature, the IRSP was developed using a series of qualitative interviews with participants.

    Finally, we discuss how the IRSP's reliability and validity can be quantitatively assessed and compared to

    typical fixed researcher-defined rating-scales, such as the Likert format.

Authors


  •   Chami-Castaldi, Elisa (external author)
  •   Reynolds, Nina L.
  •   Wallace, James (external author)

Publication Date


  • 2008

Citation


  • Chami-Castaldi, E., Reynolds, N. & Wallace, J. (2008). Individualised rating-scale procedure: a means of reducing response style contamination in survey data?. 7th European Conference on Research Methodology for Business and Management Studies (pp. 99-109). Reading, United Kingdom: Academic Publishing Limited.

Ro Full-text Url


  • http://ro.uow.edu.au/cgi/viewcontent.cgi?article=1537&context=buspapers

Ro Metadata Url


  • http://ro.uow.edu.au/buspapers/535

Start Page


  • 99

End Page


  • 109

Place Of Publication


  • Reading, United Kingdom

Abstract


  • Response style bias has been shown to seriously contaminate the substantive results drawn from

    survey data; particularly those conducted using cross-cultural samples. As a consequence. identification of

    response formats that suffer least from responst style bias has been called for. Previous studies show that

    respondents' personal characteristics, such as age, education level and culture, are connected with response

    style manifestation.

    Differences in the way respondents interpret and utilise researcher-defined fixed rating-scales (e.g. Likert

    formats), poses a problem for survey researchers. Techniques that are currently used to remove response

    bias from survey data are inadequate as they cannot accurately determine the level of contamination present

    and frequently blur true score variance. Inappropriate rating-scales can impact on the level of response style

    bias manifested, insofar as they may not represent respondents' cognitions. Rating-scale lengths that are too

    long present respondents with some response categories that are not 'meaningful', whereas rating-scales that

    are too short force respondents into compressing their cognitive rating-scales into the number of response

    categories provided (this can cause ERS contamination - extreme responding). We are therefore not able to

    guard against two respondents, who share the same cognitive position on a continuum, reporting their stance

    using different numbers on the rating-scale provided. This is especially problematic where a standard fixed

    rating-scale is used in cross-cultural surveys.

    This paper details the development of the Individualised Rating-Scale Procedure (IRSP), a means of

    extracting a respondent's 'ideal' rating-scale length, and as such 'designing out' response bias, for use as the

    measurement instrument in a survey. Whilst the fundamental ideas for self-anchoring rating-scales have been

    posited in the literature, the IRSP was developed using a series of qualitative interviews with participants.

    Finally, we discuss how the IRSP's reliability and validity can be quantitatively assessed and compared to

    typical fixed researcher-defined rating-scales, such as the Likert format.

Authors


  •   Chami-Castaldi, Elisa (external author)
  •   Reynolds, Nina L.
  •   Wallace, James (external author)

Publication Date


  • 2008

Citation


  • Chami-Castaldi, E., Reynolds, N. & Wallace, J. (2008). Individualised rating-scale procedure: a means of reducing response style contamination in survey data?. 7th European Conference on Research Methodology for Business and Management Studies (pp. 99-109). Reading, United Kingdom: Academic Publishing Limited.

Ro Full-text Url


  • http://ro.uow.edu.au/cgi/viewcontent.cgi?article=1537&context=buspapers

Ro Metadata Url


  • http://ro.uow.edu.au/buspapers/535

Start Page


  • 99

End Page


  • 109

Place Of Publication


  • Reading, United Kingdom