For autobiographical information, what is the
longest reference period one can use in a
survey?
Nancy Mathiowetz, Associate Professor, Sociology and Urban Studies,
University of Wisconsin-Milwaukee
Answer:
Survey research questions concerning behaviors are often dependent upon the
retrospective recall ability of respondents. Empirical evidence indicates that
the accuracy for temporal information lessens with time; the design tradeoff is
often one in which researchers wish to ask about short reference periods so as
to reduce response error vs. the need to ask about long reference periods, so as
to capture rare events (e.g., hospitalizations; major purchases). Given this,
what can survey designers use as guidelines for determining the ideal reference
period for capturing accurate autobiographical information? Unfortunately, there
is no simply answer to this question, since reference period is but one factor
that impacts the quality of data based on retrospective recall. In addition to
reference period, questionnaire designers need to consider the distinctiveness
of the behavior of interest, the saliency of the behavior, and the nature of the
task one is asking the respondent to perform. As the length of the reference
period increases, the likelihood of multiple, similar events also increases.
Although the occurrence of multiple, similar events may make the task of
retrieving information concerning any one event more difficult (lack of
distinctiveness), responses to a yes/no or ever/never question may be improved
due to the repeated behavior within the reference period. Important or salient
events or behaviors tend to be well reported; in part, such events may benefit
from more elaborate encoding and more frequent retrieval and reporting of the
event (rehearsal).
One of the difficulties facing questionnaire
designers is the lack of information concerning the behavior experience of
different respondents. The theoretical and empirical literature suggests that
simple experience structures are simple to report, even over long periods of
time, and that difficult experience structures are quite difficult to report,
regardless of the length of the reference period. Hence, questionnaire designers
may wish to tailor question sequences, based on questions which sort respondents
according to the complexity of his or her behavioral experience.
What are the practical implications for the
questionnaire designers with respect to the length of the reference period?
Researchers must consider both the characteristics of the behavior or event and
the characteristics of the response task. With respect to the characteristics of
the behavior, the length of the reference period can vary as a function of the
distinctiveness and saliency of the behavior or event of interest. However, one
must also consider the nature of the task facing the respondent in making the
determination of the reference period. The retrieval of detailed episodic
information will most likely require the use of a different reference period
than ever/never occurrence questions. For example, the retrieval of detailed
dietary information may require the use of a reference period no longer than 24
hours whereas the quality of reports of a purchase of a new automobile may be
quite high for reference periods of a year or more. In addition, questionnaire
designers can improve the quality of retrospective reports of behavior
(regardless of reference period) through the use of multiple cues which take
advantage of the way in which memories are organized (including cues which focus
on details other than when the event occurred) and by allowing respondents
sufficient time to adequately search his or her memory.
For a review of the theoretical and empirical
literature related to the effects of the length of reference period, the
following volumes provide an excellent review as well as a number of references
for additional reading:
S. Sudman, N. Bradburn, and N. Schwarz (1996).
Thinking about Answers: The Application of Cognitive Processes to Survey
Methodology San Francisco: Jossey-Bass. See Chapters 7 and 8.
Tourangeau, R., Rips, L, and Rasinski, K. (2000)
The Psychology of Survey Response. Cambridge University Press. See
Chapters 3 and 4.
The Survey Statistician, no. 50, pages 14-15, July 2004
|