GuacaMol: Benchmarking Models for De Novo Molecular Design

De novo design seeks to generate molecules with required property profiles by virtual design-make-test cycles. With the emergence of deep learning and neural generative models in many application areas, models for molecular design based on neural networks appeared recently and show promising results.

However, the new models have not been profiled on consistent tasks, and comparative studies to well-established algorithms have only seldom been performed. To standardise the assessment of both classical and neural models for de novo molecular design, we propose an evaluation framework, GuacaMol, based on a suite of standardised benchmarks.

The benchmark tasks encompass measuring the fidelity of the models to reproduce the property distribution of the training sets, the ability to generate novel molecules, the exploration and exploitation of chemical space, and a variety of single and multi-objective optimisation tasks. The benchmarking framework is available as an open-source Python package.

BLOG

PUBLICATIONS

GITHUB

Leaderboard

DISTRIBUTION BENCHMARKS

benchmark

AAE

Graph MCTS

Random Sampler

SMILES LSTM

VAE

ORGAN

Validity

0.822

1.000

0.959

0.870

0.379

Uniqueness

1.000

0.997

1.000

0.999

0.841

Novelty

0.998

0.994

0.000

0.912

0.974

0.686

KL divergence

0.886

0.522

0.998

0.991

0.982

0.267

Frechet ChemNet Distance

0.529

0.015

0.929

0.913

0.863

0.000

see molecules

GOAL DIRECTED BENCHMARKS

benchmark

Best of Dataset

SMILES GA

Graph MCTS

Graph GA

SMILES LSTM

Celecoxib rediscovery

Troglitazone rediscovery

Thiothixene rediscovery

Aripiprazole similarity

Albuterol similarity

Mestranol similarity

C11H24

C9H10N2O2PF2Cl

Median molecules 1

Median molecules 2

Osimertinib MPO

Fexofenadine MPO

Ranolazine MPO

Perindopril MPO

Amlodipine MPO

Sitagliptin MPO

Zaleplon MPO

Valsartan SMARTS

Scaffold Hop

Deco Hop

Submit your work

If you developed a model for de novo molecular design, you can evaluate it with the GuacaMol benchmarks with the open-source GuacaMol framework. If you would like your model to appear in the leaderboard, please send your JSON files (generated by GuacaMol) to guacamol@benevolent.ai.

Also, please reach out if you want assistance with the GuacaMol framework or if you have any questions about its usage.

More research

Our people have the freedom to investigate new scientific fields, learn new things and explore a multitude of areas of artificial intelligence, machine learning and drug discovery. Read BenevolentAI’s latest research publications here.

ALL PUBLICATIONS