Calibration of computational tools for missense variant pathogenicity classification and ClinGen recommendations for PP3/BP4 criteria

Recommendations from the American College of Medical Genetics and Genomics and the Association for Molecular Pathology (ACMG/ AMP) for interpreting sequence variants specify the use of computational predictors as supportinglevel of evidence for pathogenicity or benignity using criteria PP3 and BP4, respectively. However, score intervals defined by tool developers, and ACMG/AMP recommenda-tions that require the consensus of multiple predictors, lack quantitative support. Previously, we described a probabilistic framework that quantified the strengths of evidence (supporting, moderate, strong, very strong) within ACMG/AMP recommendations. We have extended this framework to computational predictors and introduce a new standard that converts a tool's scores to PP3 and BP4 evidence strengths. Our approach is based on estimating the local positive predictive value and can calibrate any computational tool or other continuous-scale evidence on any variant type. We estimate thresholds (score intervals) corresponding to each strength of evidence for pathogenicity and benignity for thirteen missense variant interpretation tools, using carefully assembled independent data sets. Most tools achieved supporting evidence level for both pathogenic and benign classification using newly established thresholds. Multi-ple tools reached score thresholds justifying moderate and several reached strong evidence levels. One tool reached very strong evidence level for benign classification on some variants. Based on these findings, we provide recommendations for evidence-based revisions of the PP3 and BP4 ACMG/AMP criteria using individual tools and future assessment of computational methods for clinical interpretation.

Automated summary

The American College of Medical Genetics and Genomics and the Association for Molecular Pathology (ACMG/AMP) provide recommendations for interpreting sequence variants, specifying the use of computational predictors as evidence for pathogenicity or benignity. However, the lack of quantitative support in the score intervals defined by tool developers and ACMG/AMP recommendations that require consensus of multiple predictors is addressed by a probabilistic framework proposed by the researchers. This framework is extended to computational predictors and introduces a new standard for converting tool scores to evidence strengths.