International Association of Survey Statisticians (IASS)

Ask the Experts: Concepts

For autobiographical information, what is the longest reference period one can use in a survey?
 

 

For autobiographical information, what is the longest reference period one can use in a survey?
Nancy Mathiowetz, Associate Professor, Sociology and Urban Studies, University of Wisconsin-Milwaukee

Answer: 
Survey research questions concerning behaviors are often dependent upon the retrospective recall ability of respondents. Empirical evidence indicates that the accuracy for temporal information lessens with time; the design tradeoff is often one in which researchers wish to ask about short reference periods so as to reduce response error vs. the need to ask about long reference periods, so as to capture rare events (e.g., hospitalizations; major purchases). Given this, what can survey designers use as guidelines for determining the ideal reference period for capturing accurate autobiographical information? Unfortunately, there is no simply answer to this question, since reference period is but one factor that impacts the quality of data based on retrospective recall. In addition to reference period, questionnaire designers need to consider the distinctiveness of the behavior of interest, the saliency of the behavior, and the nature of the task one is asking the respondent to perform. As the length of the reference period increases, the likelihood of multiple, similar events also increases. Although the occurrence of multiple, similar events may make the task of retrieving information concerning any one event more difficult (lack of distinctiveness), responses to a yes/no or ever/never question may be improved due to the repeated behavior within the reference period. Important or salient events or behaviors tend to be well reported; in part, such events may benefit from more elaborate encoding and more frequent retrieval and reporting of the event (rehearsal).

One of the difficulties facing questionnaire designers is the lack of information concerning the behavior experience of different respondents. The theoretical and empirical literature suggests that simple experience structures are simple to report, even over long periods of time, and that difficult experience structures are quite difficult to report, regardless of the length of the reference period. Hence, questionnaire designers may wish to tailor question sequences, based on questions which sort respondents according to the complexity of his or her behavioral experience.

What are the practical implications for the questionnaire designers with respect to the length of the reference period? Researchers must consider both the characteristics of the behavior or event and the characteristics of the response task. With respect to the characteristics of the behavior, the length of the reference period can vary as a function of the distinctiveness and saliency of the behavior or event of interest. However, one must also consider the nature of the task facing the respondent in making the determination of the reference period. The retrieval of detailed episodic information will most likely require the use of a different reference period than ever/never occurrence questions. For example, the retrieval of detailed dietary information may require the use of a reference period no longer than 24 hours whereas the quality of reports of a purchase of a new automobile may be quite high for reference periods of a year or more. In addition, questionnaire designers can improve the quality of retrospective reports of behavior (regardless of reference period) through the use of multiple cues which take advantage of the way in which memories are organized (including cues which focus on details other than when the event occurred) and by allowing respondents sufficient time to adequately search his or her memory. 

For a review of the theoretical and empirical literature related to the effects of the length of reference period, the following volumes provide an excellent review as well as a number of references for additional reading: 

S. Sudman, N. Bradburn, and N. Schwarz (1996). Thinking about Answers: The Application of Cognitive Processes to Survey Methodology San Francisco: Jossey-Bass. See Chapters 7 and 8. 

Tourangeau, R., Rips, L, and Rasinski, K. (2000) The Psychology of Survey Response. Cambridge University Press. See Chapters 3 and 4.

The Survey Statistician, no. 50, pages 14-15, July 2004

 


 

Version française