Diffusion Models: An overview

Diffusion modelling framework have been applied for image synthesis and form the basis of current SOTA models such as DALL-E from openAI and Imagen from google. With these successful demonstrations there as huge interest in the diffusion models to generate synthesis data for potential applications of these models in text to image generation, video generation, AR, VR, medical imaging, and digital twins and beyond.

In a previous post we covered the optimization objective (loss function) for diffusion models, and it relationships to variational inference. In this post we cover the implementation of diffusion models, in particular, understanding explain key component of the code for openAI’s implementation of the guided diffusion model a port for Ho et al’s tensorflow code based. It is assumed the reader has some prior knowledge of diffusion models but we cover an overview of the key elements of diffusion models which are as follows.

1. Forward process

2. Reverse process

3. Training objective

4. Guided Diffusion

5. Combined with other generative frameworks

6. SOTA: DALLE-2-OpenAI and Imagen-Google

Forward process

The forward process q(x1:T|x0) is a markov chain of latent variable xt at time t obtained by progressively adding gaussian noise to the previous time, model has fixed parameters βt, and noise is added via a schedule (1-βt) such that the noise added at each time t is small…