Sword Health recently commissioned Milliman to conduct an independent review of the methodological approach used to estimate return on investment (ROI) for Sword’s Thrive AI-guided musculoskeletal (MSK) program. The review focused on whether the methodology aligns with approaches commonly used in observational evaluations of digital health interventions and whether refinements could strengthen transparency and reproducibility. Thrive delivers one-on-one AI-guided MSK care supported by computer-vision technology and oversight by a doctor of physical therapy. The ROI methodology uses a retrospective observational analysis—with propensity score matching and a difference-in-differences estimator—to measure changes in medical spending for program users relative to a matched comparison group.

Discussion points:

• Sword’s methodology is consistent with approaches commonly used in digital health and care management evaluations where randomized trials are not feasible and administrative claims data are used to measure financial outcomes.
• The use of therapy claims for comparison group selection promotes clinical comparability, but narrows external generalizability to populations with similar care-seeking profiles.
• Matching relies on observed claims-based characteristics, and residual confounding related to unobserved clinical or behavioral factors remain—and may materially influence the reported savings estimates.
• The fixed 80/20 paid-to-allowed claims conversion factor may not reflect all benefit designs, and converted amounts should be identified separately from results based on observed allowed claims data.

This report was commissioned by Sword Health.