GuacaMol: Benchmarking Models for De Novo Molecular Design

De novo design seeks to generate molecules with required property profiles by virtual design-make-test cycles. With the emergence of deep learning and neural generative models in many application areas, models for molecular design based on neural networks appeared recently and show promising results.
However, the new models have not been profiled on consistent tasks, and comparative studies to well-established algorithms have only seldom been performed. To standardise the assessment of both classical and neural models for de novo molecular design, we propose an evaluation framework, GuacaMol, based on a suite of standardised benchmarks.
The benchmark tasks encompass measuring the fidelity of the models to reproduce the property distribution of the training sets, the ability to generate novel molecules, the exploration and exploitation of chemical space, and a variety of single and multi-objective optimisation tasks.  The benchmarking framework is available as an open-source Python package.
Leaderboard
DISTRIBUTION BENCHMARKS
benchmark
AAE
Graph MCTS
Random Sampler
SMILES LSTM
VAE
ORGAN
Validity
0.822
1.000
1.000
0.959
0.870
0.379
Uniqueness
1.000
1.000
0.997
1.000
0.999
0.841
Novelty
0.998
0.994
0.000
0.912
0.974
0.686
KL divergence
0.886
0.522
0.998
0.991
0.982
0.267
Frechet ChemNet Distance
0.529
0.015
0.929
0.913
0.863
0.000
GOAL DIRECTED BENCHMARKS
benchmark
Best of Dataset
SMILES GA
Graph MCTS
Graph GA
SMILES LSTM
Celecoxib rediscovery
Troglitazone rediscovery
Thiothixene rediscovery
Aripiprazole similarity
Albuterol similarity
Mestranol similarity
Median molecules 1
Median molecules 2
Submit your work
If you developed a model for de novo molecular design, you can evaluate it with the GuacaMol benchmarks with the open-source GuacaMol framework. If you would like your model to appear in the leaderboard, please send your JSON files (generated by GuacaMol) to guacamol@benevolent.ai.

Also, please reach out if you want assistance with the GuacaMol framework or if you have any questions about its usage.
More research
Our people have the freedom to investigate new scientific fields, learn new things and explore a multitude of areas of artificial intelligence, machine learning and drug discovery. Read BenevolentAI’s latest research publications here.
ALL PUBLICATIONS