Generative Models

VAEs, diffusion models, and flow-based models for molecular generation — from SMILES-RNNs to equivariant 3-D structure generation.

Courses

3 resources

What are Diffusion Models? — Lilian Weng

Detailed mathematical walkthrough of DDPMs, score matching, and SDEs. The best single reference for diffusion model theory.

lilianweng.github.io · Free

From Autoencoder to Beta-VAE — Lilian Weng

Clear derivation of the ELBO, reparameterisation trick, and disentangled VAE variants. Essential background for molecular VAEs.

lilianweng.github.io · Free

Generative Models — Stanford CS236

Stanford's graduate course on generative models: VAEs, normalizing flows, GANs, diffusion, and autoregressive models. Slides and notes free.

Stanford CS236 · Free

Tutorials

2 resources

DDPM from Scratch — Hugging Face

Annotated implementation of DDPM in PyTorch, reproducing the original paper step by step.

HuggingFace Blog · Colab notebook

MOSES Generative Models Examples

Reference implementations of VAE, JT-VAE, CharRNN, and ORGAN on the MOSES benchmark. Ready-to-run code for molecular generation.

Papers

4 resources

Kingma & Welling — "Auto-Encoding Variational Bayes" (2014)

Original VAE paper. Essential reference for the ELBO objective and reparameterisation trick.

ICLR 2014 · arXiv:1312.6114

Ho et al. — "Denoising Diffusion Probabilistic Models" (2020)

Foundational DDPM paper connecting denoising score matching to a variational bound.

NeurIPS 2020 · arXiv:2006.11239

Hoogeboom et al. — "Equivariant Diffusion for Molecule Generation in 3D" (2022)

EDM: diffusion in continuous atom-coordinate space using SE(3)-equivariant networks for valid 3-D molecule generation.

ICML 2022 · arXiv:2203.17003

Gómez-Bombarelli et al. — "Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules" (2018)

Proposes the molecular VAE latent space for Bayesian optimisation of drug-like properties. Highly cited.

ACS Central Science 2018

Tools

4 resources

MOSES Benchmark

Standardised evaluation framework computing validity, uniqueness, novelty, FCD, and scaffold similarity for molecular generative models.

BenevolentAI's benchmark for goal-directed molecular generation. Includes distribution learning and goal-directed benchmarks.

REINVENT 4 (AstraZeneca)

Production-grade molecular design tool using RL over a generative SMILES model, supporting multi-parameter optimisation.

230M+ commercially available compounds in ready-to-dock format. ZINC-250k is the standard training corpus for molecular generation.

Datasets

3 resources

250k drug-like molecules from ZINC — the standard training corpus for molecular VAEs, graph-based generators, and flow models.

Aspuru-Guzik group · Free

2.4M bioactive molecules with SMILES and assay data. Used as both a training corpus and a reference distribution for generative model evaluation.

ChEMBL-derived benchmark dataset and evaluation suite for distribution learning and goal-directed molecular generation.

BenevolentAI · Free