Curriculum

1. Diffusion Theory

1. Generative Modeling (What is generative model?)
2. Stochastic differential equation
3. Score matching (+flow matching)
4. Diffusion formulation

2. Diffusion techniques

1. Sampling (DDPM, DDIM..)
2. Conditioning, Guidance
3. Noise scheduling
4. Efficiency (distill, consistency..)

3. Diffusion language model

1. Continuous diffusion (Embedding, Latent)
2. Discrete diffusion (mdlm, score matching)
3. AR VS. NAR
4. Scaling

   

Contents

Week 1: Introduction to Diffusion Model

This seminar video is the introduction to the Diffusion Seminar series. This session introduces the foundational concepts of generative modeling, with a focus on the basics of diffusion models. We then explore how these models extend to language, highlighting recent developments in diffusion language models. Finally, we outline future directions and research opportunities covered in the full seminar series.

Presenter: Woojin Kim

 

Week 2: Generative Modeling I

This seminar video is the second week's session of the Diffusion Seminar series. In this session, we cover the definition and taxonomy of generative modeling, and provide an overview of three major categories derived from this taxonomy: autoregressive modeling, variational autoencoders (VAE), and generative adversarial networks (GANs).

Presenter: Jihwan Hong

 

Week 3: Generative Modeling Ⅱ

This seminar video is the third session of the Diffusion Seminar series. In this session, we cover the basic mathematical and statistical concepts needed to understand upcoming topics such as VAEs and diffusion models. We also explore the ELBO optimization in VAEs in detail, examine the concept of reparameterization, and provide an overview of DDPMs.

Presenter: Jaeik Kim

 

Week 4: Denoising Diffusion Probabilistic Model

In this session, we cover Denoising Diffusion Probabilistic Model(DDPM), which is largely acknowledged as the starting place of modern diffusion models. We cover the technical details of DDPM, ranging from forward and reverse process to training and sampling of DDPMs.

Presenter: Yejoon Lee

 

Week 5: DDIM and Score-based Modeling

This seminar video is the fifth week's session of the Diffusion Seminar series. In this session, we begin with a detailed introduction to DDIM and then explore score-based generative modeling and its connection to the diffusion framework. We present the probability flow ODE and demonstrate how DDIM can be derived as a discretized Euler approximation of this continuous process.

Presenter: Woojin Kim

 

Week 6: Various Diffusion Space

In this seminar, we will explore the various diffusion spaces used in diffusion models. Building on the pixel-space diffusion we have discussed so far, we will extend our focus to diffusion in latent space and discrete space, examining their characteristics.

Presenter: Jihwan Hong

 

Week 7: Technical components

Week 8: Diffusion Language Models I

Week 9: Diffusion Language Models Ⅱ

Week 10: Diffusion Language Models Ⅲ

Week 11: Flow Matching & Token Ordering