Browse > Article
http://dx.doi.org/10.3743/KOSIM.2011.28.3.335

A Study on Measuring the Change of the Response Results in Likert 5-Point Scale Measurement  

Noh, Young-Hee (Department of Library & Information Science, Konkuk University)
Publication Information
Journal of the Korean Society for information Management / v.28, no.3, 2011 , pp. 335-353 More about this Journal
Abstract
This study examines how and which direction respondents who participated in 5-point Likert scale surveys change their initial responses when they are given an identical second survey after certain treatments. The research employs three identical questionnaires (first, second and third surveys) to analyze survey results based on group differences, kinds of treatment, survey purposes, and response change direction and the degree. This paper concludes that, first, it is significant that specialist groups do not change their initial responses compared to a general librarian group. Second, there are no differences by survey purpose; however, participants tend to change their initial responses by others' opinions rather than by previous use experiences. Third, participants who initially answered positively tend not to change their responses, and most participants who answered negatively change their initial responses in a positive direction. Fourth, when there are changes, participants change their initial responses by less than two points, and most of them change by one point. Finally, the hypothesis that middle responses change most and that participants who respond at both ends do not change their opinion was rejected by the finding that participants who answered on the negative end tend to change their initial responses in a positive direction.
Keywords
likert-type scales; response rate; response changing; negative response; positive response; neutral point;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Shcuman, H. and S. Presser. 1981. Questions and Answers in Attitude Surveys. New York: Academic Press.
2 Sheehan, B. K. and J. S. McMillan. 1999. "Response variation in e-mail surveys: An exploration." Journal of Advertising Journal, 39(4): 45-54.   DOI   ScienceOn
3 Stewart, T. J., E. Roberts, P. Eleazer, R. Boland, and D. Widland. 2006. "Reliability and validity issues for two common measures of medical students' attitudes toward older adults." Educational Gerontology, 32: 409-421.   DOI   ScienceOn
4 Weems, G. H., A. J. Onwuegbuzie, J. B. Schreiber, and S. J. Eggers. 2003. "Characteristics of respondents who respond differently to positively and negatively worded items on rating scales." Assessment and Evaluation in Higher Education, 28: 587-607.   DOI
5 Wildt, A. R. and M. B. Mazis. 1978. "Determinants of scale responses: Label versus position." Journal of Marketing Research, XV: 261-267.
6 Worcester, R. M. and T. R. Burns. 1975. "A statistical examination of the relative precision of verbal scales." Journal of Market Research Society, 17(3): 181-197.   DOI
7 Wyatt, R. C. and L. S. Meyers. 1987. "Psychometric properties of four 5-point Likert-type scales." Educational and Phychologicla Measurement, 47: 27-35.   DOI
8 Yammarino, F. J., S. J. Skinner, and T. L. Childers. 1991. "Understanding mail survey response behavior: A meta-analysis." Public Opinion Quarterly, 55: 613-639.   DOI   ScienceOn
9 Yun, G. W. and C. W. Trumbo. 2000. "Comparative response to a survey executed by post, e-mail, & web form." Journal of Market Research Society, 6(1): 235-239.
10 Dillman, D. A. and J. Tarnai. 1988. "Administrative issues in mixed mode surveys." In R. M. Groves, P. P. Biemer, L. E. Lyberg, J. T. Massey, W. L. Nicholls, II & J. Wakesberg (Eds.), Telephone survey methodology (pp. 509-528). New York: Wiley.   DOI   ScienceOn
11 Noh, Y. 2010a. "A study on developing evaluation criteria for electronic resources in evaluation indicators of libraries." Journal of Academic Librarianship, 36(1): 41-52. .   DOI   ScienceOn
12 Noh, Y. 2010b. "A study on designing of metadata for constructing the Library Map Information System." Journal of the Korean Society for Information Management, 27(3): 241-264.   과학기술학회마을   DOI   ScienceOn
13 Quinn, S. B. and W. A. Belson. 1969. The Effects of Reversing of the Order of Presentation of Verbal Rating Scales in Survey Interviews. London: Survey Research Centre.
14 Noh, Y. 2010c. "A Study on Metadata Elements for Web-based Reference Resources System Developed through Usability Testing." Library Hi Tech, 29(2): 242-265. .
15 Payne, J. D. 1971. "The effects of reversing the order of verbal rating scales in a postal survey." Journal of the Market Research Society, 14: 30-44.
16 Payne, S. L. 1951. The Art of Asking Questions. Princeton: Princeton University Press.
17 Ruggg, D. and H. Cantril. 1944. "The wording of questions." In H. Cantril (Ed.) Gauging public opinion. Princeton: Princeton University Press   DOI   ScienceOn
18 Schwarz, N. B., H. J. Knauper, E. Hippler, E. Noelle- Neumann, and L. Clark. 1991. "Rating scales: Numeric values may change the meaning of scale labels." Public Opinion Quarterly, 55: 570-582.   DOI   ScienceOn
19 Schwarz, N., H. J. Hippler, and E. Noelle-Neumann. 1992. "A cognitive model of response-order effects in survey measurement." In N. Schwarz& S. Sudman (Eds.), Context effects in social and psychological research (pp. 187-199). New York: Springer-Verlag.   DOI   ScienceOn
20 Sedlmeier, P. 2006. "The role of scales in student ratings." Learning and Instruction, 16: 401- 415.   DOI   ScienceOn
21 Jacoby, J. and M. S. Matell. 1971. "Three-point Likert scales are good enough." Journal of Marketing Research, 8: 495-500.   DOI   ScienceOn
22 James, J. and R. Bolstein. 1992. "Large monetary incentives and their effect on mail survey response rates." Public Opinion Quarterly, 56: 442-53.   DOI   ScienceOn
23 Likert, R. 1932. "A technique for the measurement of attitudes." Archives in Psychology, 140: 1-55.   DOI
24 Komorita, S. S. 1963. "Attitude content, intensity and the neutral point on a Likert scale." Journal of Social Psychology, 61: 327-334.   DOI
25 Krosnick, J. A. and D. F. Alwin. 1987. "An evaluation of a cognitive theory of response-order effects in survey measurement." Public Opinion Quarterly, 51: 201-209.   DOI   ScienceOn
26 Kulas, J. T., A. A. Stachowski, and B. A. Haynes. 2008. "Middle response function in Likert-responses to personality items." Journal of Business Psychology, 22: 251-259.   DOI
27 Matell, M. S. and J. Jacoby. 1972. "Is there an optimal number of alternatives for Likert scale items? Effects of testing time and scale properties." Journal of Applied Psychology, 56(6): 506- 509.   DOI
28 McFadden, L. S. and S. E. Krug. 1984. "Psychometric function of the "neutral" response option in clinical personality scales." Multivariate Experimental Clinical Research, 7: 25-33.   DOI   ScienceOn
29 Mueller, J. E. 1970. "Choosing among 133 candidates." Public Opinion Quarterly, 34: 395-402.   DOI   ScienceOn
30 Nicholls, M. R., C. A. Orr, M. Okubo, and A. Loftus. 2006. "Satisfaction guaranteed: The effect of spatial biases on responses to Likert scales." Psychological Science, 17: 1027-1028.   DOI   ScienceOn
31 De Leeuw, E. D. 1992. Data Quality in Mail, Telephone, and Face-to-face Surveys. Amsterdam: TT Publications.
32 Dillman, D. A. 1999. Mail and internet surveys: The tailored design method, (2nd Ed.). New York: John Wiley and Sons.
33 Garland, R. 1991. "The mid-point on a rating scale: Is it desirable?" Marketing Bulletin, 2: 66-70, Research Note 3.   DOI
34 Everett, S. A., J. H. Price, A. Bedell, and S. K. Telljohann. 1997. "The effect of a monetary incentive in increasing the return rate of a survey to family physicians." Eval Health Professions, 20: 207-4.   DOI   ScienceOn
35 Fox, R. J., M. R. Crask, and J. Kim. 1988. "Mail survey response rate: A meta-analysis of selected techniques for introducing response." Public Opinion Quarterly, 52: 467-491.   DOI   ScienceOn
36 Friedman, H. H., Y. Wilamowsky, and L. W. Friedman.1981. "A comparison of balanced and unbalanced rating scales." The Mid-Atlantic Journal of Business, 19(2): 1-7.
37 Goldberg, L. R. 1981. "Unconfounding situational attributions from uncertain, neutral, and ambiguous ones: A psychometric analysis of descriptions of oneself and various types of others." Journal of Personality and Social Psychology, 41(3): 517-552.   DOI
38 Green, B. F. 1969. "Attitude measurement." In G. Lindzey (Ed.), Handbook of Social Psychology. Cambridge, Mass.: Addison-Wesley.   DOI
39 Guy, R. F. and M. Norvell. 1977. "The neutral point on a Likert scale." Journal of Psychology: Interdisciplinary and Applied, 95(2): 199-204.   DOI
40 Hartley, J. and L. R. Betts. 2010. "Four layouts and a finding: The effects of changes in the order of the verbal labels and numerical values on Likert-type scales." International Journal of Social Research Methodology, 13(1): 17-27.   DOI   ScienceOn
41 Bignami-Van Assche, S. G., G. Reniers, and A. A. Weinreb. 2003. "An assessment of the KDICP and MDICP data quality." Demographic Research, S1(2): 31-76.   DOI
42 Beck, J. 1996. Career anchors, organizational commitment, and job plateaus: An analysis of hotel executive operating committee members' career development. (Doctoral Dissertation, Purdue University, 1996). Dissertation Abstracts International, 58(03). (Publication Number: AAT 9725508).
43 Becker, S. L. 1954. "Why an order effect." Public Opinion Quarterly, 18: 271-278.   DOI   ScienceOn
44 Belson, W. A. 1966. "The effects of reversing the presentation order of verbal rating scales." Journal of Advertising Research, 6: 30-376.
45 Brook, D. and G. J. Upton. 1974. "Primacy effects in single trial free recall." Applied Statistics, 23: 414-419.   DOI   ScienceOn
46 Carp, F. M. 1974. "Position effects on interview response." Journal of Gerontology, 29: 581-587.   DOI   ScienceOn
47 Church, A. H. 1993. "Estimating the effect of incentive on mail survey response rates: A meta-analysis." Public Opinion Quarterly, 57: 62-79.   DOI   ScienceOn
48 Cobanoglu, C. B. Warde, and P. J. Moreo. 2001. "A comparison of mail, fax and web-based survey methods." International Journal of Market Research, 43(4): 441-452.
49 Cummins, R. A. and E. Gullone. 2000. "Why we should not use 5-point Likert scales: The case for subjective quality of life measurement." Proceedings, Second International Conference on Quality of Life in Cities, Singapore: National University of Singapore, 74-93.
50 Dawes, J. 2008. "Do data characteristics change according to the number of points used? An experiment using 5-point, 7-point and 10-point scales." International Journal of Market Research, 50: 61-77.   DOI
51 Armstong, R. L. 1987. "The midpoint on a five-point Likert-type scale." Perceptual and Motor Skills, 64: 359-362.   DOI
52 Amoo, T. and H. H. Friedman. 2001. "Do numeric values influence subjects' responses to rating scales?" Journal of International Marketing and Marketing Research, 26: 41-46.
53 Armitage, C. and C. Deeprose. 2004. "Changing student evaluations by means of the numeric values of rating scales." Psychology Learning and Teaching, 3: 122-125.   DOI