Errors in self-reports of health services use: impact on alzheimer disease clinical trial designs

Abstract

BACKGROUND: Most Alzheimer disease clinical trials that compare the use of health services rely on reports of caregivers. The goal of this study was to assess the accuracy of self-reports among older adults with Alzheimer disease and their caregiver proxy respondents. This issue is particularly relevant to Alzheimer disease clinical trials because inaccuracy can lead both to loss of power and increased bias in study outcomes. METHODS: We compared respondent accuracy in reporting any use and in reporting the frequency of use with actual utilization data as documented in a comprehensive database. We next simulated the impact of underreporting and overreporting on sample size estimates and treatment effect bias for clinical trials comparing utilization between experimental groups. RESULTS: Respondents self-reports have a poor level of accuracy with κ-values often below 0.5. Respondents tend to underreport use even for rare events such as hospitalizations and nursing home stays. In analyses simulating underreporting and overreporting of varying magnitude, we found that errors in self-reports can increase the required sample size by 15% to 30%. In addition, bias in the reported treatment effect ranged from 3% to 18% due to both underreporting and overreporting errors. CONCLUSIONS: Use of self-report data in clinical trials of Alzheimer disease treatments may inflate sample size needs. Even when adequate power is achieved by increasing sample size, reporting errors can result in a biased estimate of the true effect size of the intervention.