Curriculum

1. Diffusion Theory

1. Generative Modeling (What is generative model?)
2. Stochastic differential equation
3. Score matching (+flow matching)
4. Diffusion formulation

2. Diffusion techniques

1. Sampling (DDPM, DDIM..)
2. Conditioning, Guidance
3. Noise scheduling
4. Efficiency (distill, consistency..)

3. Diffusion language model

1. Continuous diffusion (Embedding, Latent)
2. Discrete diffusion (mdlm, score matching)
3. AR VS. NAR
4. Scaling

   

Contents

Week 1: Introduction to Diffusion Model

This seminar video is the introduction to the Diffusion Seminar series. This session introduces the foundational concepts of generative modeling, with a focus on the basics of diffusion models. We then explore how these models extend to language, highlighting recent developments in diffusion language models. Finally, we outline future directions and research opportunities covered in the full seminar series.

Presenter: Woojin Kim

 

Week 2: Generative Modeling I

This seminar video is the second week's session of the Diffusion Seminar series. In this session, we cover the definition and taxonomy of generative modeling, and provide an overview of three major categories derived from this taxonomy: autoregressive modeling, variational autoencoders (VAE), and generative adversarial networks (GANs).

Presenter: Jihwan Hong

 

Week 3: Generative Modeling Ⅱ

This seminar video is the third session of the Diffusion Seminar series. In this session, we cover the basic mathematical and statistical concepts needed to understand upcoming topics such as VAEs and diffusion models. We also explore the ELBO optimization in VAEs in detail, examine the concept of reparameterization, and provide an overview of DDPMs.

Presenter: Jaeik Kim

 

Week 4: Denoising Diffusion Probabilistic Model

In this session, we cover Denoising Diffusion Probabilistic Model(DDPM), which is largely acknowledged as the starting place of modern diffusion models. We cover the technical details of DDPM, ranging from forward and reverse process to training and sampling of DDPMs.

Presenter: Yejoon Lee

 

Week 5: DDIM and Score-based Modeling

This seminar video is the fifth week's session of the Diffusion Seminar series. In this session, we begin with a detailed introduction to DDIM and then explore score-based generative modeling and its connection to the diffusion framework. We present the probability flow ODE and demonstrate how DDIM can be derived as a discretized Euler approximation of this continuous process.

Presenter: Woojin Kim

 

Week 6: Various Diffusion Space

In this seminar, we will explore the various diffusion spaces used in diffusion models. Building on the pixel-space diffusion we have discussed so far, we will extend our focus to diffusion in latent space and discrete space, examining their characteristics.

Presenter: Jihwan Hong

 

Week 7: Technical components

This seminar explored key advancements in diffusion models, focusing on conditioning mechanisms and diffusion distillation techniques. We began by reviewing the mathematical background of diffusion processes and the role of conditioning, emphasizing how class and other auxiliary information can be incorporated into the denoising network to guide generation. We then examined diffusion distillation methods aimed at accelerating sampling without compromising image quality. We also discussed consistency distillation, derived from the Consistency Models framework, which enforces a self-consistency property and a boundary condition along probability flow ODE trajectories.

Presenter: Jaeik Kim

 

Week 8: Diffusion Language Models I

This seminar video is the eighth week's session of the Diffusion Seminar series. In this session, we introduce the basics of diffusion language models, discuss how diffusion has been adapted from the image to the text domain with discrete token spaces, and cover continuous diffusion language model formulations.

Presenter: Woojin Kim

 

Week 9: Diffusion Language Models Ⅱ

This seminar video is the ninth week's session of the Diffusion Seminar series. In this session, we focus on discrete diffusion language models, explore their continuous-time extensions, and present masked diffusion objectives as a bridge to established NLP training paradigms.

Presenter: Woojin Kim

 

Week 10: Diffusion Language Models Ⅲ

[The video will be released Soon.]

Presenter: Yejoon Lee

 

Week 11: Flow Matching & Token Ordering

This presentation introduces the concept of token ordering in large language models (LLMs), examining whether the conventional left-to-right autoregressive generation is truly optimal for diverse downstream tasks. The historical development of left-to-right ordering, from early NLP research to modern transformers, is outlined, followed by discussion of seminal works such as Sigma GPT and Any-Order GPT, which explore arbitrary permutations and demonstrate both the challenges and potential advantages of moving beyond identity ordering. Theoretical connections to combinatorial optimization and group theory are highlighted, along with findings showing that any-order autoregression is more difficult to train yet can outperform left-to-right ordering under curriculum learning or task-specific structures. Future directions are suggested, including balancing efficiency and expressivity, meta-learning order strategies, advancing KV caching for encoder-based masked diffusion models, and extending token ordering research to modalities beyond text.

Presenter: Jaeik Kim