Machine Learning Outperforms ACC / AHA CVD Risk Calculator in MESA.

https://www.ncbi.nlm.nih.gov/pubmed/30571498

https://www.ahajournals.org/doi/full/10.1161/JAHA.118.009476?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed

J Am Heart Assoc. 2018 Nov 20;7(22):e009476. doi: 10.1161/JAHA.118.009476.

Machine Learning Outperforms ACC / AHA CVD Risk Calculator in MESA.

Kakadiaris IA1, Vrigkas M1, Yen AA2, Kuznetsova T3, Budoff M4, Naghavi M2.

Author information

1

1 Computational Biomedicine Lab University of Houston TX.

2

2 Society for Heart Attack Prevention and Eradication Palo Alto CA.

3

3 Research Unit Hypertension and Cardiovascular Epidemiology KU Leuven Department of Cardiovascular Sciences University of Leuven Belgium.

4

4 Division of Cardiology Los Angeles Biomedical Research at Harbor-UCLA Medical Center Torrance CA.

Abstract

Background Studies have demonstrated that the current US guidelines based on American College of Cardiology/American Heart Association (ACC/AHA) Pooled Cohort Equations Risk Calculator may underestimate risk of atherosclerotic cardiovascular disease ( CVD ) in certain high-risk individuals, therefore missing opportunities for intensive therapy and preventing CVD events. Similarly, the guidelines may overestimate risk in low risk populations resulting in unnecessary statin therapy. We used Machine Learning ( ML ) to tackle this problem. Methods and Results We developed a ML Risk Calculator based on Support Vector Machines ( SVM s) using a 13-year follow up data set from MESA (the Multi-Ethnic Study of Atherosclerosis) of 6459 participants who were atherosclerotic CVD-free at baseline. We provided identical input to both risk calculators and compared their performance. We then used the FLEMENGHO study (the Flemish Study of Environment, Genes and Health Outcomes) to validate the model in an external cohort. ACC / AHA Risk Calculator, based on 7.5% 10-year risk threshold, recommended statin to 46.0%. Despite this high proportion, 23.8% of the 480 "Hard CVD " events occurred in those not recommended statin, resulting in sensitivity 0.76, specificity 0.56, and AUC 0.71. In contrast, ML Risk Calculator recommended only 11.4% to take statin, and only 14.4% of "Hard CVD " events occurred in those not recommended statin, resulting in sensitivity 0.86, specificity 0.95, and AUC 0.92. Similar results were found for prediction of "All CVD " events. Conclusions The ML Risk Calculator outperformed the ACC/AHA Risk Calculator by recommending less drug therapy, yet missing fewer events. Additional studies are underway to validate the ML model in other cohorts and to explore its ability in short-term CVD risk prediction.

KEYWORDS:

Artificial intelligence; Machine learning; atherosclerosis; cardiovascular disease prevention; cardiovascular disease risk factors; cardiovascular risk; clinical decision support; prediction statistics; statin

PMID:

 

30571498

 

DOI:

 

10.1161/JAHA.118.009476

현재의 ACC/AHA CVD risk calculator 보다,

머신러닝 CVD risk calculator 가 더 낫다.

SVM 알고리즘.

13년 MESA data, 6459 명.

불필요한 statin 추천을 덜 하면서

CVD event 놓치는 것도 덜하다.

External validation 시행함.

ROC 커브는 당연 좋게 나오고.

우리는 과도하게 statin 을 처방해 오고 있던 것일까?    

(제약회사의 입김 때문에?,  미국을 중심으로 한 소위 의료선진국의 과도한 가이드라인 때문에?? )

(아직까지 알고리즘은, 탐욕이나 욕심을 부리진 않을 테니까... / 또는 짐짓 먼저 CVD 발생할까 우려를 한다던지 하지 않을테니...) 

Figure 1 Overview of ML approach. For each ML model, we divided the study population 50/50 into training and prediction subset cohorts. Next, we augmented the training subset using NEATER and trained the SVM prediction model. During prediction, each sample in the prediction cohort was analyzed and classified. Then, the cohorts switched places (ie, prediction becomes training, and vice versa) and the process was repeated. The overall iterative process was repeated 10 times for each ML model, and the results were averaged. CVD indicates cardiovascular disease; FLEMENGHO study, the Flemish Study of Environment, Genes and Health Outcomes; HDL, high‐density lipoprotein; MESA, the Multi‐Ethnic Study of Atherosclerosis; NEATER, a method for the filtering of oversampled data using non‐cooperative game theory; SVM, Support Vector Machine.

Figure 2 Receiver operating characteristic (ROC) curves for prediction of (A) “Hard CVD” events and (B) “All CVD” events comparing the ML Risk Calculator (blue) with the American College of Cardiology/American Heart Association (ACC/AHA) Risk Calculator (red). AUC indicates area under the curve; CVD, cardiovascular disease; ML, Machine Learning.

Figure 2 Receiver operating characteristic (ROC) curves for prediction of (A) “Hard CVD” events and (B) “All CVD” events comparing the ML Risk Calculator (blue) with the American College of Cardiology/American Heart Association (ACC/AHA) Risk Calculator (red). AUC indicates area under the curve; CVD, cardiovascular disease; ML, Machine Learning.