CMS Practice Assessment Tool Validity for Alternative Payment Models

Abstract

Objectives: To study the predictive validity of the CMS Practice Assessment Tool (PAT) among 632 primary care practices. Study Design: Retrospective observational study. Methods: The study included primary care physician practices recruited by the Great Lakes Practice Transformation Network (GLPTN), 1 of 29 CMS-awarded networks, and used data from 2015 to 2019. At enrollment, trained quality improvement advisers scored each of the PAT’s 27 milestones by its degree of implementation based on interviews with staff, review of documents, direct observation of practice activity, and professional judgment. The GLPTN also tracked each practice’s status regarding alternative payment model (APM) enrollment. Exploratory factor analysis (EFA) was used to identify summary scores; mixed-effects logistic regression was used to assess the relationship between derived scores with APM participation. Results: EFA revealed that the PAT’s 27 milestones could be summed into 1 overall score and 5 secondary scores. By the end of the 4-year project, 38% of practices were enrolled in an APM. A baseline overall score and 3 secondary scores were associated with increased odds of joining an APM (overall score: odds ratio [OR], 1.06; 95% CI, 0.99-1.12; P = .061; data-driven care quality score: OR, 1.11; 95% CI, 1.00-1.22; P = .040; efficient care delivery score: OR, 1.08; 95% CI, 1.03-1.13; P = .003; collaborative engagement score: OR, 0.88; 95% CI, 0.80-0.96; P = .005). Conclusions: These results demonstrate that the PAT has adequate predictive validity for APM participation.