Beyond independent error assumptions in large GNN atomistic models

The calculation of relative energy difference has significant practical applications, such as determining adsorption energy, screening for optimal catalysts with volcano plots, and calculating reaction energies. Although Density Functional Theory (DFT) is effective in calculating relative energies through systematic error cancellation, the accuracy of Graph Neural Networks (GNNs) in this regard remains uncertain. To address this, we analyzed similar to 483 x 10(6) pairs of energy differences predicted by DFT and GNNs using the Open Catalyst 2020-Dense dataset. Our analysis revealed that GNNs exhibit a correlated error that can be reduced through subtraction, challenging the assumption of independent errors in GNN predictions and leading to more precise energy difference predictions. To assess the magnitude of error cancellation in chemically similar pairs, we introduced a new metric, the subgroup error cancellation ratio. Our findings suggest that state-of-the-art GNN models can achieve error reduction of up to 77% in these subgroups, which is comparable to the error cancellation observed with DFT. This significant error cancellation allows GNNs to achieve higher accuracy than individual energy predictions and distinguish subtle energy differences. We propose the marginal correct sign ratio as a metric to evaluate this performance. Additionally, our results show that the similarity in local embeddings is related to the magnitude of error cancellation, indicating the need for a proper training method that can augment the embedding similarity for chemically similar adsorbate-catalyst systems.

Automated summary

The calculation of relative energy difference is important in various practical applications. This study analyzed the performance of Graph Neural Networks (GNNs) in predicting energy differences and found that GNNs exhibit a correlated error that can be reduced through subtraction, leading to more precise predictions. The results show that GNNs can achieve comparable error reduction to Density Functional Theory (DFT) and can distinguish subtle energy differences. The similarity in local embeddings is related to the magnitude of error cancellation, highlighting the need for proper training methods.