A comparison of techniques for deriving clustering and switching scores from verbal fluency word lists

Abstract

Objective: To compare techniques for computing clustering and switching scores in terms of agreement, correlation, and empirical value as predictors of incident cognitive impairment (ICI). Methods: We transcribed animal and letter F fluency recordings on 640 cases of ICI and matched controls from a national epidemiological study, amending each transcription with word timings. We then calculated clustering and switching scores, as well as scores indexing speed of responses, using techniques described in the literature. We evaluated agreement among the techniques with Cohen's κ and calculated correlations among the scores. After fitting a base model with raw scores, repetitions, and intrusions, we fit a series of Bayesian logistic regression models adding either clustering and switching scores or speed scores, comparing the models in terms of several metrics. We partitioned the ICI cases into acute and progressive cases and repeated the regression analysis for each group. Results: For animal fluency, we found that models with speed scores derived using the slope difference algorithm achieved the best values of the Watanabe-Akaike Information Criterion (WAIC), but with good net reclassification improvement (NRI) only for the progressive group (8.2%). For letter fluency, different models excelled for prediction of acute and progressive cases. For acute cases, NRI was best for speed scores derived from a network model (3.4%), while for progressive cases, the best model used clustering and switching scores derived from the same network model (5.1%). Combining variables from the best animal and letter F models led to marginal improvements in model fit and NRI only for the all-cases and acute-cases analyses. Conclusion: Speed scores improve a base model for predicting progressive cognitive impairment from animal fluency. Letter fluency scores may provide complementary information.