Lecture 4 — Neural Networks & Deep Learning · AI4Chemical Sciences Bootcamp

Recording

Recording will be available after the bootcamp.

August 2026

Learning Objectives

Implement a multi-layer perceptron (MLP) in PyTorch from first principles
Derive the backpropagation algorithm and explain how automatic differentiation (autograd) implements it
Apply common regularisation techniques — dropout, batch normalisation, weight decay — and know when to use each
Select a learning-rate scheduler and understand the role of momentum in Adam

Key Takeaways

Takeaway 1. Backpropagation is just the chain rule applied recursively. Understanding this demystifies "black box" neural networks and helps diagnose vanishing/exploding gradient issues.
Takeaway 2. Adam is the default optimiser for most tasks, but its adaptive learning rates can cause poor generalisation; SGD with momentum often wins on large, well-tuned experiments.
Takeaway 3. Batch normalisation and dropout are complementary: BatchNorm stabilises training dynamics; dropout prevents co-adaptation of neurons. Use both carefully, not by default.
Takeaway 4. The training loop — forward pass, loss, backward pass, step — is the same for every architecture. Mastering it in PyTorch lets you customise any model in the literature.