Journal
EPILEPSIA
Volume 61, Issue 1, Pages 39-48Publisher
WILEY
DOI: 10.1111/epi.16398
Keywords
clinical decision support; epilepsy surgery; machine learning; natural language processing
Categories
Funding
- National Heart, Lung, and Blood Institute [K25 HL12595]
- Agency for Healthcare Research and Quality [R21 HS024977]
Ask authors/readers for more resources
Objective Delay to resective epilepsy surgery results in avoidable disease burden and increased risk of mortality. The objective was to prospectively validate a natural language processing (NLP) application that uses provider notes to assign epilepsy surgery candidacy scores. Methods The application was trained on notes from (1) patients with a diagnosis of epilepsy and a history of resective epilepsy surgery and (2) patients who were seizure-free without surgery. The testing set included all patients with unknown surgical candidacy status and an upcoming neurology visit. Training and testing sets were updated weekly for 1 year. One- to three-word phrases contained in patients' notes were used as features. Patients prospectively identified by the application as candidates for surgery were manually reviewed by two epileptologists. Performance metrics were defined by comparing NLP-derived surgical candidacy scores with surgical candidacy status from expert chart review. Results The training set was updated weekly and included notes from a mean of 519 +/- 67 patients. The area under the receiver operating characteristic curve (AUC) from 10-fold cross-validation was 0.90 +/- 0.04 (range = 0.83-0.96) and improved by 0.002 per week (P < .001) as new patients were added to the training set. Of the 6395 patients who visited the neurology clinic, 4211 (67%) were evaluated by the model. The prospective AUC on this test set was 0.79 (95% confidence interval [CI] = 0.62-0.96). Using the optimal surgical candidacy score threshold, sensitivity was 0.80 (95% CI = 0.29-0.99), specificity was 0.77 (95% CI = 0.64-0.88), positive predictive value was 0.25 (95% CI = 0.07-0.52), and negative predictive value was 0.98 (95% CI = 0.87-1.00). The number needed to screen was 5.6. Significance An electronic health record-integrated NLP application can accurately assign surgical candidacy scores to patients in a clinical setting.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available