PaccMannRL: De novo generation of hit-like anticancer molecules from transcriptomic data via reinforcement learning

With the advent of deep generative models in computational chemistry, in-silico drug design is undergoing an unprecedented transformation. Although deep learning approaches have shown potential in generating compounds with desired chemical properties, they disregard the cellular environment of target diseases. Bridging systems biology and drug design, we present a reinforcement learning method for de novo molecular design from gene expression profiles. We construct a hybrid Variational Autoencoder that tailors molecules to target-specific transcriptomic profiles, using an anticancer drug sensitivity prediction model (PaccMann) as reward function. Without incorporating information about anticancer drugs, the molecule generation is biased toward compounds with high predicted efficacy against cell lines or cancer types. The generation can be further refined by subsidiary constraints such as toxicity. Our cancer-type-specific candidate drugs are similar to cancer drugs in drug-likeness, synthesizability, and solubility and frequently exhibit the highest structural similarity to compounds with known efficacy against these cancer types.

Automated summary

This study presents a reinforcement learning method for de novo molecular design from gene expression profiles, utilizing an anticancer drug sensitivity prediction model as a reward function to generate cancer-type-specific candidate drugs with high structural similarity to compounds with known efficacy against these cancer types. The generated molecules can be further refined by subsidiary constraints like toxicity.