Browsing by Author "Clark, David Glenn"

Now showing 1 - 3 of 3
  • Thumbnail Image
    Item
    A comparison of techniques for deriving clustering and switching scores from verbal fluency word lists
    (Frontiers Media, 2022-09-14) Bushnell, Justin; Svaldi, Diana; Ayers, Matthew R.; Gao, Sujuan; Unverzagt, Frederick; Del Gaizo, John; Wadley, Virginia G.; Kennedy, Richard; Goñi, Joaquín; Clark, David Glenn; Neurology, School of Medicine
    Objective: To compare techniques for computing clustering and switching scores in terms of agreement, correlation, and empirical value as predictors of incident cognitive impairment (ICI). Methods: We transcribed animal and letter F fluency recordings on 640 cases of ICI and matched controls from a national epidemiological study, amending each transcription with word timings. We then calculated clustering and switching scores, as well as scores indexing speed of responses, using techniques described in the literature. We evaluated agreement among the techniques with Cohen's κ and calculated correlations among the scores. After fitting a base model with raw scores, repetitions, and intrusions, we fit a series of Bayesian logistic regression models adding either clustering and switching scores or speed scores, comparing the models in terms of several metrics. We partitioned the ICI cases into acute and progressive cases and repeated the regression analysis for each group. Results: For animal fluency, we found that models with speed scores derived using the slope difference algorithm achieved the best values of the Watanabe-Akaike Information Criterion (WAIC), but with good net reclassification improvement (NRI) only for the progressive group (8.2%). For letter fluency, different models excelled for prediction of acute and progressive cases. For acute cases, NRI was best for speed scores derived from a network model (3.4%), while for progressive cases, the best model used clustering and switching scores derived from the same network model (5.1%). Combining variables from the best animal and letter F models led to marginal improvements in model fit and NRI only for the all-cases and acute-cases analyses. Conclusion: Speed scores improve a base model for predicting progressive cognitive impairment from animal fluency. Letter fluency scores may provide complementary information.
  • Thumbnail Image
    Item
    Post-Processing Automatic Transcriptions with Machine Learning for Verbal Fluency Scoring
    (Elsevier, 2023) Bushnell, Justin; Unverzagt, Frederick; Wadley, Virginia G.; Kennedy, Richard; Del Gaizo, John; Clark, David Glenn; Neurology, School of Medicine
    Objective: To compare verbal fluency scores derived from manual transcriptions to those obtained using automatic speech recognition enhanced with machine learning classifiers. Methods: Using Amazon Web Services, we automatically transcribed verbal fluency recordings from 1400 individuals who performed both animal and letter F verbal fluency tasks. We manually adjusted timings and contents of the automatic transcriptions to obtain "gold standard" transcriptions. To make automatic scoring possible, we trained machine learning classifiers to discern between valid and invalid utterances. We then calculated and compared verbal fluency scores from the manual and automatic transcriptions. Results: For both animal and letter fluency tasks, we achieved good separation of valid versus invalid utterances. Verbal fluency scores calculated based on automatic transcriptions showed high correlation with those calculated after manual correction. Conclusion: Many techniques for scoring verbal fluency word lists require accurate transcriptions with word timings. We show that machine learning methods can be applied to improve off-the-shelf ASR for this purpose. These automatically derived scores may be satisfactory for some applications. Low correlations among some of the scores indicate the need for improvement in automatic speech recognition before a fully automatic approach can be reliably implemented.
  • Thumbnail Image
    Item
    Verbal fluency response times predict incident cognitive impairment
    (Alzheimer’s Association, 2022-04-06) Ayers, Matthew R.; Bushnell, Justin; Gao, Sujuan; Unverzagt, Frederick; Del Gaizo, John; Wadley, Virginia G.; Kennedy, Richard; Clark, David Glenn; Neurology, School of Medicine
    Introduction: In recent decades, researchers have defined novel methods for scoring verbal fluency tasks. In this work, we evaluate novel scores based on speed of word responses. Methods: We transcribed verbal fluency recordings from 641 cases of incident cognitive impairment (ICI) and matched controls, all participants in a large national epidemiological study. Timing measurements of utterances were used to calculate a speed score for each recording. Traditional raw and speed scores were entered into Cox proportional hazards (CPH) regression models predicting time to ICI. Results: Concordance of the CPH model with speed scores was 0.599, an improvement of 3.4% over a model with only raw scores and demographics. Scores with significant effects included animals raw and speed scores, and letter F speed score. Discussion: Novel verbal fluency scores based on response times could enable use of remotely administered fluency tasks for early detection of cognitive decline. Highlights: The current work evaluates prognostication with verbal fluency speed scores. These speed scores improve survival models predicting cognitive decline. Cases with progressive decline have some characteristics suggestive of Alzheimer's disease. The subset of acute decliners is probably pathologically heterogeneous.