PDBench: evaluating computational methods for protein-sequence design

Ever increasing amounts of protein structure data, combined with advances in machine learning, have led to the rapid proliferation of methods available for protein-sequence design. In order to utilize a design method effectively, it is important to understand the nuances of its performance and how it varies by design target. Here, we present PDBench, a set of proteins and a number of standard tests for assessing the performance of sequence-design methods. PDBench aims to maximize the structural diversity of the benchmark, compared with previous benchmarking sets, in order to provide useful biological insight into the behaviour of sequence-design methods, which is essential for evaluating their performance and practical utility. We believe that these tools are useful for guiding the development of novel sequence design algorithms and will enable users to choose a method that best suits their design target.

Automated summary

With the increasing amount of protein structure data and advances in machine learning, the number of available protein sequence design methods is rapidly expanding. To effectively utilize a design method, understanding its performance nuances and how it varies by design target is crucial. PDBench is introduced as a collection of proteins and standard tests to assess the performance of sequence design methods. Compared to previous benchmarking sets, PDBench aims to maximize the structural diversity of the benchmark to provide valuable biological insights into the behavior of sequence design methods, which is essential for evaluating their performance and practical utility. We believe these tools are valuable for guiding the development of novel sequence design algorithms and assisting users in choosing a method that best suits their design target.